Your SlideShare is downloading. ×
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
2010 10 Sottilare AIIDE Keynote Presentation
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

2010 10 Sottilare AIIDE Keynote Presentation

615

Published on

Bob\'s presentation AI and Interactive Digital Entertainment (AIIDE) Conference 2010

Bob\'s presentation AI and Interactive Digital Entertainment (AIIDE) Conference 2010

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
615
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. U.S. Army Research, Development and Engineering Command A perspective on the role of AI and interactive digital media in military training Presentation to the AI and Interactive Digital Entertainment Conference (AIIDE) October 2010 Stanford University Robert A. Sottilare, Ph.D. Chief Technology Officer SFC Paul Ray Smith Simulation & Training Technology Center robert.sottilare@us.army.mil
  • 2. Disclaimer • I will be talking about commercial products in addition to government-developed technology • What I say should not be construed as an endorsement of any commercial products • Much of what will be covered in this talk has application outside of a military context • Challenge is the word of the day… SFC Paul Ray Smith Simulation & Training Technology Center
  • 3. Agenda • A little about my organization and my background… • Motivation to exploit AI and IDM for military training • AI and IDM in serious games and military training • Opportunities to exploit AI and IDM • Challenges and Next Steps SFC Paul Ray Smith Simulation & Training Technology Center
  • 4. My Organization SFC Paul Ray Smith Simulation & Training Technology Center part of the Army Research Laboratory SFC Paul Ray Smith Simulation & Training Technology Center
  • 5. SFC Paul Ray Smith Simulation & Training Technology Center
  • 6. A little about me… • you can find me on • 25+ years in military training & simulation research, development and acquisition • My research interests are in adaptive tutoring • my research is conducted in the Learning in Intelligent Tutoring Environments (LITE) Lab • and yes… I like the Simpsons LITE Lab SFC Paul Ray Smith Simulation & Training Technology Center
  • 7. Motivation to exploit AI and Interactive Media for Military Training SFC Paul Ray Smith Simulation & Training Technology Center
  • 8. Motivation to exploit AI and Interactive Media for Military Training • U.S. Army is deployed worldwide on a consistent basis so training technology needs to be: – accessible wherever the Soldier is located – interactive regardless of infrastructure available – intuitive so it can be used in the absence of instructors • The time allotted for training is limited so training needs to be: – efficient as well as effective – engaging, challenging and relevant to the Soldier’s mission – adaptive to the trainee’s needs and capabilities Today, Soldiers do more than their traditional missions… SFC Paul Ray Smith Simulation & Training Technology Center
  • 9. Learning & Entertainment Learning Objectives1 • Gain attention • Inform Stimulate immersion • Stimulate recall Gain/maintain attention • Present stimulus material Motivate • Provide learner LEARNING FUN Guide guidance • Elicit performance Stimulate interaction • Provide feedback Inform • Assess performance • Enhance retention transfer [1] Gagne, R. M. (1965). The conditions of learning. New York: Holt, Rinehart and Winston. SFC Paul Ray Smith Simulation & Training Technology Center
  • 10. Training the process of bringing a person, team or organization to an agreed standard of proficiency by practice and instruction • relevant task and conditions • standards • measures of performance • constructive and timely feedback SFC Paul Ray Smith Simulation & Training Technology Center
  • 11. Exploit AI and Interactive Digital Media to improve Soldier… • learning: acquisition of knowledge and/or skill • performance: accomplishment of a given task measured against some standard • retention: ability to perform a learned behavior (task) over time without the reinforcement of practice • competence: demonstrated level of ability through performance over time; potential for future performance SFC Paul Ray Smith Simulation & Training Technology Center
  • 12. Available and emerging technologies • intelligent technologies for training – adaptive tutoring systems – virtual humans • serious games • virtual worlds • mobile learning SFC Paul Ray Smith Simulation & Training Technology Center
  • 13. Intelligent Technologies for Training PURPOSE: Design, develop, apply and evaluate intelligent agent technologies (tools and methods) to enhance training, leader development and education; and to reduce associated support costs OBJECTIVE: Use intelligent agents to supplement one-to-one and one-to-many TLE experiences where human support is Adaptive Tutors limited, impractical or completely unavailable TECHNICAL CHALLENGES: Application of intelligent agents in ill-defined domains; understanding natural language in multi- sided conversations; interrupted conversations team tutoring; rapid authoring; and virtual human cognition Virtual Humans SFC Paul Ray Smith Simulation & Training Technology Center
  • 14. Intelligent Technologies for Training use real behaviors and physiological use simulated measures to model and cognitive states to predict trainee drive virtual human cognitive states behaviors Adaptive Tutors Virtual Humans SFC Paul Ray Smith Simulation & Training Technology Center
  • 15. Adaptive Tutoring Research PURPOSE: Enable computer-based intelligent tutors to adapt to trainee “readiness-to-learn” by evaluating the influence of performance variables in the trainee model OBJECTIVE: Experimentally evaluate trainee state variables (self-reported, observed, physiological and behavioral) to accurately/ passively assess the trainee’s cognitive state & adapt content/interaction to meet trainee needs TECHNICAL CHALLENGES: Determining optimal sensor suite to accurately/passively identify cognitive state; development of machine learning models to adapt instruction sensors  predicted trainee state  instructional strategy selection SFC Paul Ray Smith Simulation & Training Technology Center
  • 16. Individual Tutoring Systems Sottilare, R. (2010). Toward the Development of an Intelligent Tutoring System for Distributed Team Training through Passive Sensing. In Proceedings of the 10th Intelligent Tutoring Systems Conference, Pittsburgh, June 2010. SFC Paul Ray Smith Simulation & Training Technology Center
  • 17. Team Tutoring Systems Sottilare, R. (2010). Toward the Development of an Intelligent Tutoring System for Distributed Team Training through Passive Sensing. In Proceedings of the 10th Intelligent Tutoring Systems Conference, Pittsburgh, June 2010. SFC Paul Ray Smith Simulation & Training Technology Center
  • 18. Adaptive Tutoring Systems • Few military training simulations today implement intelligent tutoring… • In most of those cases, the trainee model is mainly focused on performance goals and know little or nothing about: – trainee cognition (including affect) – trainee past performance (competence level) • Many of today’s tutors focus in well-defined domains (e.g. physics, math) and the world is messy… • The difficult work is ahead of us… SFC Paul Ray Smith Simulation & Training Technology Center
  • 19. Virtual Humans SFC Paul Ray Smith Simulation & Training Technology Center
  • 20. Virtual Humans • What parts of real people are represented in virtual humans? – physical appearance, behaviors and interaction – limited cognition (including affect) – culture, values? – goals, beliefs – social interaction – trust – persistence… Arnold, Lazarus, Frijda, Scherer, Ortony et al. SFC Paul Ray Smith Simulation & Training Technology Center
  • 21. Appraisal Theory courtesy of Stacy Marsella, Institute for Creative Technologies Arnold, Lazarus, Frijda, Scherer, Ortony et al. SFC Paul Ray Smith Simulation & Training Technology Center
  • 22. Virtual Humans • Many current and potential applications: – training partners, adversaries or neutral characters – tutoring, mentoring, coaching – marketing – social support – entertainment SFC Paul Ray Smith Simulation & Training Technology Center
  • 23. Virtual Humans courtesy of Jon Gratch, Institute for Creative Technologies SFC Paul Ray Smith Simulation & Training Technology Center
  • 24. Virtual Patients/Virtual Sick Call • Create medically accurate virtual patients with advanced interaction capabilities • Train Combat Medics on the proper procedures for sick call TECHNICAL CHALLENGES: • Identifying/creating appropriate and meaningful medical dialog/content • Developing adaptive, realistic human behaviors SFC Paul Ray Smith Simulation & Training Technology Center
  • 25. Serious Games SFC Paul Ray Smith Simulation & Training Technology Center
  • 26. Tactical Combat Casualty Care SFC Paul Ray Smith Simulation & Training Technology Center
  • 27. Tactical Questioning System PURPOSE: Develop a capability for sustainment training that included interrogations of detainees, civilians and persons of interest OBJECTIVE: • Develop 3D Interactive environment • Soldier interacts with virtual humans to sustain their interrogation skills in realistic manner • Utilize translator and foreign languages TECHNICAL CHALLENGES: • Speech and natural language processing • Question and Answer type of conversations • Authoring capability to create new characters SFC Paul Ray Smith Simulation & Training Technology Center
  • 28. Multi-national Training Experiment using Virtual Battlespace 2 • uses commercial game • primarily kinetic operations • distributed shared environment SFC Paul Ray Smith Simulation & Training Technology Center
  • 29. UrbanSim PC-based game that allows Battalion and Brigade Commanders to practice situational understanding and operations in complex, contemporary environments. Key Learning Objectives: – Develop and maintain situational understanding – Develop Commander’s intent – Develop Commander’s critical information requirements – Direct operations – Anticipate 2nd and 3rd order effects – Command outside the chain of command SFC Paul Ray Smith Simulation & Training Technology Center
  • 30. Bi-lateral Negotiation Simulation (BiLAT) PC, game engine-based, cognitive training tool used for developing skills in how to plan for and conduct bi-lateral meetings and/or negotiations in different cultural settings Key Learning Objectives: Negotiation Strategy: Planning and executing tactics that help achieve mission objectives Cultural Awareness: Applying culturally- appropriate interpersonal skills Trust-Building Strategy: Developing/applying tactics to gain negotiating partners’ trust Challenge – combine pedagogy, game play design, interactive research technologies, and software development to address key objectives SFC Paul Ray Smith Simulation & Training Technology Center
  • 31. Bi-lateral Negotiation Simulation (BiLAT) Social and Cultural Modeling SFC Paul Ray Smith Simulation & Training Technology Center
  • 32. Virtual Worlds • Next Generation VW Platforms must be: – scalable – work across operating systems and devices – easily deployed – secure – open – interoperable – future proof - scale dynamically – economically sound – good business model David Smith, Chief Innovation Officer, Lockheed Martin Global Training and Logistics A Virtual Worlds Architecture Framework: A Shorter Path to Mastery for All DOD Personnel SFC Paul Ray Smith Simulation & Training Technology Center
  • 33. OnLine Interactive Virtual Environment (OLIVE) SFC Paul Ray Smith Simulation & Training Technology Center
  • 34. Multinational Training Experiment using OLIVE SFC Paul Ray Smith Simulation & Training Technology Center
  • 35. Enhanced Dynamic Geo-social Environment (EDGE) Blends traits from MMO, serious games and virtual world technology to replicate the operational environment as accurately as possible. SFC Paul Ray Smith Simulation & Training Technology Center 3
  • 36. Federal Virtual Worlds Challenge • What is the Federal Virtual Worlds Challenge? • It is an open, global Challenge to the world for the best of the best implementations within virtual environments. • The event is conducted to explore innovative and interactive solutions in virtual environments. The criteria are intentionally unbounded to encourage creative results. • This year the focus is on artificial intelligence! • http://www.fvwc.army.mil/index.php SFC Paul Ray Smith Simulation & Training Technology Center
  • 37. Mobile Learning SFC Paul Ray Smith Simulation & Training Technology Center
  • 38. Combat Medic Card Games SFC Paul Ray Smith Simulation & Training Technology Center
  • 39. SFC Paul Ray Smith Simulation & Training Technology Center
  • 40. Combat Hunter Mobile SFC Paul Ray Smith Simulation & Training Technology Center
  • 41. Challenges and Next Steps • Economic challenges • AI for training challenges • IDM for training challenges • Interaction design challenges • Grand challenges for educational technology • Next steps for educational technology SFC Paul Ray Smith Simulation & Training Technology Center
  • 42. Economic Challenges • The U.S. Army is a large population distributed worldwide and covering a variety of occupational specialties – Maintaining the proficiency of this population is challenging and expensive – New missions = new training content – New personnel and equipment = continuous training • AI technology (tools and methods) are needed to: – reduce training costs – while maintaining training effectiveness – and improving availability and adaptability SFC Paul Ray Smith Simulation & Training Technology Center
  • 43. AI for Training Challenges • Adaptable tutoring systems – improve perception/prediction of trainee’s state – improve assessment of learning, performance, competence and retention – improve selection of appropriate instructional strategies based on the trainee’s state (e.g. competency level) • content, tempo, challenge, flow and feedback – automate authoring of content and expert models SFC Paul Ray Smith Simulation & Training Technology Center
  • 44. IDM for Training Challenges • Interactive mixed reality training environments – everyone wants the holodeck, but its not economically or technically feasible now – explore alternative evolutionary steps to get holodeck-like capabilities now – natural locomotion – no adverse effects (e.g. simulator sickness) – real-time, distributed effects SFC Paul Ray Smith Simulation & Training Technology Center
  • 45. Interaction Design Challenges for AI and IDM • Trainee situational awareness comparable to “live” environments • Passive machine perception of trainee and “live” environment • Natural Language Understanding & Generation • Natural locomotion in virtual environments SFC Paul Ray Smith Simulation & Training Technology Center
  • 46. Grand Challenges for Educational Technology • Personalize Education • Assess Student Learning • Support Social Learning • Diminish Boundaries • Develop Alternative Teaching Methods • Enhance the Role of Stakeholders • Address Policy Changes Woolf, B. P. (2010). A Roadmap for Education Technology. National Science Foundation # 0637190 SFC Paul Ray Smith Simulation & Training Technology Center
  • 47. Next Steps for Educational Technology • User Modeling • Mobile Training – Sensing trainee state during individual training – Interactive team training and performance assessment • Networking Tools • Serious Games • Intelligent Environments • Educational Data Mining (authoring tools) – Content – Models (student models, expert models…) • Rich Interfaces Adapted from: Woolf, B. P. (2010). A Roadmap for Education Technology. National Science Foundation # 0637190 SFC Paul Ray Smith Simulation & Training Technology Center
  • 48. Questions SFC Paul Ray Smith Simulation & Training Technology Center

×