SlideShare a Scribd company logo
Lessons Learned
Original Statement
The U.S. needs a way, given
a representative simulation, to
rapidly explore a strategy for
possible novel uses of
existing platforms and
weapons.
David Song
B.S. CS
Yujing Zhang
M.A. East Asian Studies
Shashank Rammoorthy
M.S. CS
Jonathan Sepulveda
M.S. Materials Science
Project Wargames
30+
interviews
Current Statement
Strategic wargames stand to
benefit from a stronger integration
of AI+ML but are struggling to find
adoption and usage. How can this
be addressed?
1
Lessons Learned
Defining a Problem: Research
Weeks 2 3 4 5 6 7 8 9
Scavenging the Solution Space: Testing
Future
Exploring the Problem
2
Lessons Learned 3
Lessons Learned
But what is a wargame?
4
1. Physics-based simulators
2. TTXs for campaigns
3. High-level diplomacy wargames
Lessons Learned
Then we played with wargames and talked to experts...
5
Lessons Learned 6
So we went ‘back to the drawing board’...
Picking one case study of interest, we
focused on the Cross-Straits scenario…
● We visited talks on Taiwan’s
Economic Security to understand
variables often overlooked
● The sad reality is, wargame players
often have very little time to
anticipate or prepare for wargames,
and are not concerned with non-
combatant variables
● The most important element of
wargames ought to be to teach
people to think creatively
Lessons Learned
We realized wargaming was not easy
Then we started learning and asking…
● What are CoAs?
● Differences between wargaming vs.
M&S?
● Why does the DoD want innovation
within such traditional processes?
● What hasn’t been attempted
already?
7
Excerpt from Battle! Practical Wargaming by Charles Grant
Lessons Learned
Problems we found with wargaming
8
We immediately found out that there are…
● Many moving parts & elements:
researchers → designers →
wargamers → Red and Blue teams
● Little data is shared amongst
designers and team members
● There are multiple types of
wargames
● Most games remain TTXs
● Too difficult to build de novo a
robust, AI-capable Red adversary
That left us thinking:
● Virtual wargames are often used for
training, but seldom as formal
proceedings; why?
● Should we seek to replace
wargamers altogether with AI+ML
‘magic’?
Lessons Learned
What some of our interviewees told us…
9
Sebastian Bae
CNA Corp. | Wargamer
Yuna Wong
IDA | Defense Analyst
Jacquelyn Schneider
Hoover, CISAC | Researcher
Pete Pellegrino
U.S. NWC | Game Design Lead
"There are a lot of reasons for
why AI/ML hasn't been used in
wargaming in the DoD…technical
illiteracy within the DoD…policies
are inflexible, and systems are
difficult to change"
Lessons Learned
What some of our interviewees told us…
10
Sebastian Bae
CNA Corp. | Wargamer
Yuna Wong
IDA | Defense Analyst
Jacquelyn Schneider
Hoover, CISAC | Researcher
Pete Pellegrino
U.S. NWC | Game Design Lead
"We should never use new
wargames, but rather well-studied
wargames and already-studied
ones in predicting our outcomes &
enhancing our adjudications”
Lessons Learned
What some of our interviewees told us…
11
Sebastian Bae
CNA Corp. | Wargamer
Yuna Wong
IDA | Defense Analyst
Jacquelyn Schneider
Hoover, CISAC | Researcher
Pete Pellegrino
U.S. NWC | Game Design Lead
"Wargames need data to eat from.
You need a volume of data that
AI+ML can learn from and be helpful;
where is that data coming from a
wargame? Wargames don’t generate
simulation data in copious amounts”
Lessons Learned
What some of our interviewees told us…
12
Sebastian Bae
CNA Corp. | Wargamer
Yuna Wong
IDA | Defense Analyst
Jacquelyn Schneider
Hoover, CISAC | Researcher
Pete Pellegrino
U.S. NWC | Game Design Lead
"We put too much emphasis on the
Red...players are generally not from
Russia or China, and so you struggle to
extrapolate specific country foreign
policy decisions versus general human
reactions to international crises.”
Many different objections to
AI+ML!
Lessons Learned
We got out of the building, went to an NPS Conference…
13
● Existing AI+ML capabilities have to be
better leveraged
● AI+ML does not need to make drastic
technical advances to be feasible for
wargaming uses
● There’s no readily accessible central
repository of past wargame data
outcomes to help future teams
● Ideally, forecast decision spaces as
opposed to having AI+ML play moves
Lessons Learned
It’s really hard to build Alphastar
with the simulations so far
14
Lessons Learned
Defining a Problem: Research
Weeks 2 3 4 5 6 7 8 9
Scavenging the Solution Space: Testing
Future
15
Pivot: Let’s help humans
play wargames better with AI
tools instead of replacing
humans completely with AI
Lessons Learned
Defining a Problem: Research
Weeks 2 3 4 5 6 7 8 9
Scavenging the Solution Space: Testing
Future
16
Identifying
Recommendations
Lessons Learned 17
1. Digitize wargames to increase iterations
"Accelerating and making
wargames more efficient will
come down to a series of
technologies that could very
well be unclassified in
development”
Major defense startup
company
Lessons Learned 18
2. Create AI tools for digitized wargames
“Not many people in the U.S.
have access to the decision-
making teams you find in Red
teams [the PLA and the CCP].”
Elizabeth Bartels | RAND
Lessons Learned 19
3. Implement organizational changes
We read-up on traditional approaches to
building wargames, noting its strengths..
● We found more support that cross-talk
between wargamers & designers and
players is important
● Academics do not differ that sharply
from the assessment of wargame players
insofar as AI+ML integration is concerned
● Information asymmetry predominates
within wargames–not an attribute of UCC
commanders
● Need more collaborations among
wargamers, designers, combat &
command
Lessons Learned
Defining a Problem: Research
Weeks 2 3 4 5 6 7 8 9
Scavenging the Solution Space: Testing
Future
20
Next steps
1. Validate our recommendations
2. Shadow a wargame
3. Continue working with our sponsor

More Related Content

What's hot

Team Short Circuit - 2021 Technology, Innovation & Great Power Competition
Team Short Circuit - 2021 Technology, Innovation & Great Power CompetitionTeam Short Circuit - 2021 Technology, Innovation & Great Power Competition
Team Short Circuit - 2021 Technology, Innovation & Great Power CompetitionStanford University
 
Team Army venture capital - 2021 Technology, Innovation & Great Power Competi...
Team Army venture capital - 2021 Technology, Innovation & Great Power Competi...Team Army venture capital - 2021 Technology, Innovation & Great Power Competi...
Team Army venture capital - 2021 Technology, Innovation & Great Power Competi...Stanford University
 
Neurosmart H4D 2021 Lessons Learned
Neurosmart H4D 2021 Lessons LearnedNeurosmart H4D 2021 Lessons Learned
Neurosmart H4D 2021 Lessons LearnedStanford University
 
Project Agrippa H4D 2021 Lessons Learned
Project Agrippa H4D 2021 Lessons LearnedProject Agrippa H4D 2021 Lessons Learned
Project Agrippa H4D 2021 Lessons LearnedStanford University
 
Invisa Engr245 2022 Lessons Learned
Invisa Engr245 2022 Lessons LearnedInvisa Engr245 2022 Lessons Learned
Invisa Engr245 2022 Lessons LearnedStanford University
 
Gordian Knot Center Roundtable w/Depty SecDef
Gordian Knot Center Roundtable w/Depty SecDef Gordian Knot Center Roundtable w/Depty SecDef
Gordian Knot Center Roundtable w/Depty SecDef Stanford University
 
Lecture 4- Technology, Innovation and Great Power Competition
Lecture 4- Technology, Innovation and Great Power CompetitionLecture 4- Technology, Innovation and Great Power Competition
Lecture 4- Technology, Innovation and Great Power CompetitionStanford University
 
Steve Blank's Secret History of Silicon Valley Computer History Museum 120708...
Steve Blank's Secret History of Silicon Valley Computer History Museum 120708...Steve Blank's Secret History of Silicon Valley Computer History Museum 120708...
Steve Blank's Secret History of Silicon Valley Computer History Museum 120708...Stanford University
 
Launch your own: Corporate innovation accelerator
Launch your own: Corporate innovation acceleratorLaunch your own: Corporate innovation accelerator
Launch your own: Corporate innovation acceleratorBoard of Innovation
 
Lecture 5- Technology, Innovation and Great Power Competition
Lecture 5- Technology, Innovation and Great Power CompetitionLecture 5- Technology, Innovation and Great Power Competition
Lecture 5- Technology, Innovation and Great Power CompetitionStanford University
 
Lean Startups Steve Blank Eric Ries
Lean Startups Steve Blank Eric RiesLean Startups Steve Blank Eric Ries
Lean Startups Steve Blank Eric RiesStanford University
 
580827935-Mckinsey-Training-1-3.pdf
580827935-Mckinsey-Training-1-3.pdf580827935-Mckinsey-Training-1-3.pdf
580827935-Mckinsey-Training-1-3.pdfdavidscribddavidscri
 
Cirque du Soleil's BLUE OCEAN STRATEGY: One-page Story of How Cirque du Solei...
Cirque du Soleil's BLUE OCEAN STRATEGY: One-page Story of How Cirque du Solei...Cirque du Soleil's BLUE OCEAN STRATEGY: One-page Story of How Cirque du Solei...
Cirque du Soleil's BLUE OCEAN STRATEGY: One-page Story of How Cirque du Solei...Rod King, Ph.D.
 

What's hot (20)

Team Short Circuit - 2021 Technology, Innovation & Great Power Competition
Team Short Circuit - 2021 Technology, Innovation & Great Power CompetitionTeam Short Circuit - 2021 Technology, Innovation & Great Power Competition
Team Short Circuit - 2021 Technology, Innovation & Great Power Competition
 
Team Army venture capital - 2021 Technology, Innovation & Great Power Competi...
Team Army venture capital - 2021 Technology, Innovation & Great Power Competi...Team Army venture capital - 2021 Technology, Innovation & Great Power Competi...
Team Army venture capital - 2021 Technology, Innovation & Great Power Competi...
 
Project Agrippa TIGPC
Project Agrippa TIGPCProject Agrippa TIGPC
Project Agrippa TIGPC
 
Neurosmart H4D 2021 Lessons Learned
Neurosmart H4D 2021 Lessons LearnedNeurosmart H4D 2021 Lessons Learned
Neurosmart H4D 2021 Lessons Learned
 
Project Agrippa H4D 2021 Lessons Learned
Project Agrippa H4D 2021 Lessons LearnedProject Agrippa H4D 2021 Lessons Learned
Project Agrippa H4D 2021 Lessons Learned
 
Invisa Engr245 2022 Lessons Learned
Invisa Engr245 2022 Lessons LearnedInvisa Engr245 2022 Lessons Learned
Invisa Engr245 2022 Lessons Learned
 
Gordian Knot Center Roundtable w/Depty SecDef
Gordian Knot Center Roundtable w/Depty SecDef Gordian Knot Center Roundtable w/Depty SecDef
Gordian Knot Center Roundtable w/Depty SecDef
 
Customer Discovery Skills
Customer Discovery Skills Customer Discovery Skills
Customer Discovery Skills
 
Lecture 4- Technology, Innovation and Great Power Competition
Lecture 4- Technology, Innovation and Great Power CompetitionLecture 4- Technology, Innovation and Great Power Competition
Lecture 4- Technology, Innovation and Great Power Competition
 
Steve Blank's Secret History of Silicon Valley Computer History Museum 120708...
Steve Blank's Secret History of Silicon Valley Computer History Museum 120708...Steve Blank's Secret History of Silicon Valley Computer History Museum 120708...
Steve Blank's Secret History of Silicon Valley Computer History Museum 120708...
 
Educators guide jan 2014
Educators guide jan 2014Educators guide jan 2014
Educators guide jan 2014
 
Launch your own: Corporate innovation accelerator
Launch your own: Corporate innovation acceleratorLaunch your own: Corporate innovation accelerator
Launch your own: Corporate innovation accelerator
 
Learn 2 Win H4D Stanford 2019
Learn 2 Win H4D Stanford 2019Learn 2 Win H4D Stanford 2019
Learn 2 Win H4D Stanford 2019
 
Lecture 5- Technology, Innovation and Great Power Competition
Lecture 5- Technology, Innovation and Great Power CompetitionLecture 5- Technology, Innovation and Great Power Competition
Lecture 5- Technology, Innovation and Great Power Competition
 
E245 syllabus rev15
E245 syllabus rev15E245 syllabus rev15
E245 syllabus rev15
 
Lean Startups Steve Blank Eric Ries
Lean Startups Steve Blank Eric RiesLean Startups Steve Blank Eric Ries
Lean Startups Steve Blank Eric Ries
 
580827935-Mckinsey-Training-1-3.pdf
580827935-Mckinsey-Training-1-3.pdf580827935-Mckinsey-Training-1-3.pdf
580827935-Mckinsey-Training-1-3.pdf
 
Hacking for Recovery syllabus
Hacking for Recovery syllabusHacking for Recovery syllabus
Hacking for Recovery syllabus
 
Drive with Dan Pink
Drive with Dan PinkDrive with Dan Pink
Drive with Dan Pink
 
Cirque du Soleil's BLUE OCEAN STRATEGY: One-page Story of How Cirque du Solei...
Cirque du Soleil's BLUE OCEAN STRATEGY: One-page Story of How Cirque du Solei...Cirque du Soleil's BLUE OCEAN STRATEGY: One-page Story of How Cirque du Solei...
Cirque du Soleil's BLUE OCEAN STRATEGY: One-page Story of How Cirque du Solei...
 

Similar to Team Wargames - 2022 Technology, Innovation & Great Power Competition

A survival guide for UX in complex environments
A survival guide for UX in complex environmentsA survival guide for UX in complex environments
A survival guide for UX in complex environmentsPaula de Matos
 
Military Flight Training - Digital Technology Disruption Ahead?
Military Flight Training - Digital Technology Disruption Ahead?Military Flight Training - Digital Technology Disruption Ahead?
Military Flight Training - Digital Technology Disruption Ahead?Andy Fawkes
 
What's Trending in NATO Commercial Technologies and Games
What's Trending in NATO Commercial Technologies and GamesWhat's Trending in NATO Commercial Technologies and Games
What's Trending in NATO Commercial Technologies and GamesJay Gendron
 
Immersive Learning Simulations Astd Final2
Immersive Learning Simulations Astd Final2Immersive Learning Simulations Astd Final2
Immersive Learning Simulations Astd Final2Mark Oehlert
 
What is Gamification?
What is Gamification? What is Gamification?
What is Gamification? Karl Kapp
 
Security Operations as a Video Game (Bsides Vancouver 2019)
Security Operations as a Video Game (Bsides Vancouver 2019)Security Operations as a Video Game (Bsides Vancouver 2019)
Security Operations as a Video Game (Bsides Vancouver 2019)Rob Fry
 
Introduction to reinforcement learning
Introduction to reinforcement learningIntroduction to reinforcement learning
Introduction to reinforcement learningXander Steenbrugge
 
Geek Night Manchester
Geek Night ManchesterGeek Night Manchester
Geek Night Manchesterfrankmt
 
Pixel-Lab / Games:EDU / Matt Southern / Graduating Games
Pixel-Lab / Games:EDU / Matt Southern / Graduating GamesPixel-Lab / Games:EDU / Matt Southern / Graduating Games
Pixel-Lab / Games:EDU / Matt Southern / Graduating Gamespixellab
 
Software Engineering for ML/AI, keynote at FAS*/ICAC/SASO 2019
Software Engineering for ML/AI, keynote at FAS*/ICAC/SASO 2019Software Engineering for ML/AI, keynote at FAS*/ICAC/SASO 2019
Software Engineering for ML/AI, keynote at FAS*/ICAC/SASO 2019Patrizio Pelliccione
 
Small talks about AI, and new opportunity in IC design industry
Small talks about AI, and new opportunity in IC design industrySmall talks about AI, and new opportunity in IC design industry
Small talks about AI, and new opportunity in IC design industryMutai Lin
 
Red vs. Blue Why we’ve been getting it wrong for 25 years
Red vs. Blue Why we’ve been getting it wrong for 25 yearsRed vs. Blue Why we’ve been getting it wrong for 25 years
Red vs. Blue Why we’ve been getting it wrong for 25 yearsEC-Council
 
GAMIFIN 2019 Conference Keynote: How to fail at #gamification research
GAMIFIN 2019 Conference Keynote: How to fail at #gamification researchGAMIFIN 2019 Conference Keynote: How to fail at #gamification research
GAMIFIN 2019 Conference Keynote: How to fail at #gamification researchLennart Nacke
 
All in AI: LLM Landscape & RAG in 2024 with Mark Ryan (Google) & Jerry Liu (L...
All in AI: LLM Landscape & RAG in 2024 with Mark Ryan (Google) & Jerry Liu (L...All in AI: LLM Landscape & RAG in 2024 with Mark Ryan (Google) & Jerry Liu (L...
All in AI: LLM Landscape & RAG in 2024 with Mark Ryan (Google) & Jerry Liu (L...Daniel Zivkovic
 
Session 1 AI literacy What is AI and how do we use it (video).pptx
Session 1 AI literacy What is AI and how do we use it (video).pptxSession 1 AI literacy What is AI and how do we use it (video).pptx
Session 1 AI literacy What is AI and how do we use it (video).pptxjameshodgkinson9
 
The Future of AI (September 2019)
The Future of AI (September 2019)The Future of AI (September 2019)
The Future of AI (September 2019)Julien SIMON
 
Global Game Jam Overview
Global Game Jam OverviewGlobal Game Jam Overview
Global Game Jam OverviewSusan Gold
 
Machine Learning for Computer Games
Machine Learning for Computer GamesMachine Learning for Computer Games
Machine Learning for Computer Gamesbutest
 
AI01_1_Introduction.pptx
AI01_1_Introduction.pptxAI01_1_Introduction.pptx
AI01_1_Introduction.pptxNguyễn Tiến
 
Scaling SlideShare to the World - An Asian Perpective
Scaling SlideShare to the World - An Asian PerpectiveScaling SlideShare to the World - An Asian Perpective
Scaling SlideShare to the World - An Asian PerpectiveAmit Ranjan
 

Similar to Team Wargames - 2022 Technology, Innovation & Great Power Competition (20)

A survival guide for UX in complex environments
A survival guide for UX in complex environmentsA survival guide for UX in complex environments
A survival guide for UX in complex environments
 
Military Flight Training - Digital Technology Disruption Ahead?
Military Flight Training - Digital Technology Disruption Ahead?Military Flight Training - Digital Technology Disruption Ahead?
Military Flight Training - Digital Technology Disruption Ahead?
 
What's Trending in NATO Commercial Technologies and Games
What's Trending in NATO Commercial Technologies and GamesWhat's Trending in NATO Commercial Technologies and Games
What's Trending in NATO Commercial Technologies and Games
 
Immersive Learning Simulations Astd Final2
Immersive Learning Simulations Astd Final2Immersive Learning Simulations Astd Final2
Immersive Learning Simulations Astd Final2
 
What is Gamification?
What is Gamification? What is Gamification?
What is Gamification?
 
Security Operations as a Video Game (Bsides Vancouver 2019)
Security Operations as a Video Game (Bsides Vancouver 2019)Security Operations as a Video Game (Bsides Vancouver 2019)
Security Operations as a Video Game (Bsides Vancouver 2019)
 
Introduction to reinforcement learning
Introduction to reinforcement learningIntroduction to reinforcement learning
Introduction to reinforcement learning
 
Geek Night Manchester
Geek Night ManchesterGeek Night Manchester
Geek Night Manchester
 
Pixel-Lab / Games:EDU / Matt Southern / Graduating Games
Pixel-Lab / Games:EDU / Matt Southern / Graduating GamesPixel-Lab / Games:EDU / Matt Southern / Graduating Games
Pixel-Lab / Games:EDU / Matt Southern / Graduating Games
 
Software Engineering for ML/AI, keynote at FAS*/ICAC/SASO 2019
Software Engineering for ML/AI, keynote at FAS*/ICAC/SASO 2019Software Engineering for ML/AI, keynote at FAS*/ICAC/SASO 2019
Software Engineering for ML/AI, keynote at FAS*/ICAC/SASO 2019
 
Small talks about AI, and new opportunity in IC design industry
Small talks about AI, and new opportunity in IC design industrySmall talks about AI, and new opportunity in IC design industry
Small talks about AI, and new opportunity in IC design industry
 
Red vs. Blue Why we’ve been getting it wrong for 25 years
Red vs. Blue Why we’ve been getting it wrong for 25 yearsRed vs. Blue Why we’ve been getting it wrong for 25 years
Red vs. Blue Why we’ve been getting it wrong for 25 years
 
GAMIFIN 2019 Conference Keynote: How to fail at #gamification research
GAMIFIN 2019 Conference Keynote: How to fail at #gamification researchGAMIFIN 2019 Conference Keynote: How to fail at #gamification research
GAMIFIN 2019 Conference Keynote: How to fail at #gamification research
 
All in AI: LLM Landscape & RAG in 2024 with Mark Ryan (Google) & Jerry Liu (L...
All in AI: LLM Landscape & RAG in 2024 with Mark Ryan (Google) & Jerry Liu (L...All in AI: LLM Landscape & RAG in 2024 with Mark Ryan (Google) & Jerry Liu (L...
All in AI: LLM Landscape & RAG in 2024 with Mark Ryan (Google) & Jerry Liu (L...
 
Session 1 AI literacy What is AI and how do we use it (video).pptx
Session 1 AI literacy What is AI and how do we use it (video).pptxSession 1 AI literacy What is AI and how do we use it (video).pptx
Session 1 AI literacy What is AI and how do we use it (video).pptx
 
The Future of AI (September 2019)
The Future of AI (September 2019)The Future of AI (September 2019)
The Future of AI (September 2019)
 
Global Game Jam Overview
Global Game Jam OverviewGlobal Game Jam Overview
Global Game Jam Overview
 
Machine Learning for Computer Games
Machine Learning for Computer GamesMachine Learning for Computer Games
Machine Learning for Computer Games
 
AI01_1_Introduction.pptx
AI01_1_Introduction.pptxAI01_1_Introduction.pptx
AI01_1_Introduction.pptx
 
Scaling SlideShare to the World - An Asian Perpective
Scaling SlideShare to the World - An Asian PerpectiveScaling SlideShare to the World - An Asian Perpective
Scaling SlideShare to the World - An Asian Perpective
 

More from Stanford University

Altuna Engr245 2022 Lessons Learned
Altuna Engr245 2022 Lessons LearnedAltuna Engr245 2022 Lessons Learned
Altuna Engr245 2022 Lessons LearnedStanford University
 
ānanda Engr245 2022 Lessons Learned
ānanda Engr245 2022 Lessons Learnedānanda Engr245 2022 Lessons Learned
ānanda Engr245 2022 Lessons LearnedStanford University
 
Team Catena - 2021 Technology, Innovation & Great Power Competition
Team Catena - 2021 Technology, Innovation & Great Power CompetitionTeam Catena - 2021 Technology, Innovation & Great Power Competition
Team Catena - 2021 Technology, Innovation & Great Power CompetitionStanford University
 
Team Apollo - 2021 Technology, Innovation & Great Power Competition
Team Apollo - 2021 Technology, Innovation & Great Power CompetitionTeam Apollo - 2021 Technology, Innovation & Great Power Competition
Team Apollo - 2021 Technology, Innovation & Great Power CompetitionStanford University
 
Team Drone - 2021 Technology, Innovation & Great Power Competition
Team Drone - 2021 Technology, Innovation & Great Power CompetitionTeam Drone - 2021 Technology, Innovation & Great Power Competition
Team Drone - 2021 Technology, Innovation & Great Power CompetitionStanford University
 
Team Aurora - 2021 Technology, Innovation & Great Power Competition
Team Aurora - 2021 Technology, Innovation & Great Power CompetitionTeam Aurora - 2021 Technology, Innovation & Great Power Competition
Team Aurora - 2021 Technology, Innovation & Great Power CompetitionStanford University
 
Team Conflicted Capital Team - 2021 Technology, Innovation & Great Power Comp...
Team Conflicted Capital Team - 2021 Technology, Innovation & Great Power Comp...Team Conflicted Capital Team - 2021 Technology, Innovation & Great Power Comp...
Team Conflicted Capital Team - 2021 Technology, Innovation & Great Power Comp...Stanford University
 
Lecture 7 - Technology, Innovation and Great Power Competition - Space
Lecture 7 - Technology, Innovation and Great Power Competition - SpaceLecture 7 - Technology, Innovation and Great Power Competition - Space
Lecture 7 - Technology, Innovation and Great Power Competition - SpaceStanford University
 
Lecture 6 - Technology, Innovation and Great Power Competition - Autonomy and...
Lecture 6 - Technology, Innovation and Great Power Competition - Autonomy and...Lecture 6 - Technology, Innovation and Great Power Competition - Autonomy and...
Lecture 6 - Technology, Innovation and Great Power Competition - Autonomy and...Stanford University
 
Lecture 6- Technology, Innovation and Great Power Competition - Unmanned Syst...
Lecture 6- Technology, Innovation and Great Power Competition - Unmanned Syst...Lecture 6- Technology, Innovation and Great Power Competition - Unmanned Syst...
Lecture 6- Technology, Innovation and Great Power Competition - Unmanned Syst...Stanford University
 
DIU Presentation to Stanford TIGPC Class 10.19.21
DIU Presentation to Stanford TIGPC Class 10.19.21DIU Presentation to Stanford TIGPC Class 10.19.21
DIU Presentation to Stanford TIGPC Class 10.19.21Stanford University
 
Lecture 3 - Technology, Innovation and Great Power Competition
Lecture 3 - Technology, Innovation and Great Power CompetitionLecture 3 - Technology, Innovation and Great Power Competition
Lecture 3 - Technology, Innovation and Great Power CompetitionStanford University
 
Lecture 2 -Technology, Innovation and Great Power Competition
Lecture 2 -Technology, Innovation and Great Power CompetitionLecture 2 -Technology, Innovation and Great Power Competition
Lecture 2 -Technology, Innovation and Great Power CompetitionStanford University
 
Lecture 1 -Technology, Innovation and Great Power Competition
Lecture 1 -Technology, Innovation and Great Power CompetitionLecture 1 -Technology, Innovation and Great Power Competition
Lecture 1 -Technology, Innovation and Great Power CompetitionStanford University
 

More from Stanford University (15)

Altuna Engr245 2022 Lessons Learned
Altuna Engr245 2022 Lessons LearnedAltuna Engr245 2022 Lessons Learned
Altuna Engr245 2022 Lessons Learned
 
ānanda Engr245 2022 Lessons Learned
ānanda Engr245 2022 Lessons Learnedānanda Engr245 2022 Lessons Learned
ānanda Engr245 2022 Lessons Learned
 
Team Catena - 2021 Technology, Innovation & Great Power Competition
Team Catena - 2021 Technology, Innovation & Great Power CompetitionTeam Catena - 2021 Technology, Innovation & Great Power Competition
Team Catena - 2021 Technology, Innovation & Great Power Competition
 
Team Apollo - 2021 Technology, Innovation & Great Power Competition
Team Apollo - 2021 Technology, Innovation & Great Power CompetitionTeam Apollo - 2021 Technology, Innovation & Great Power Competition
Team Apollo - 2021 Technology, Innovation & Great Power Competition
 
Team Drone - 2021 Technology, Innovation & Great Power Competition
Team Drone - 2021 Technology, Innovation & Great Power CompetitionTeam Drone - 2021 Technology, Innovation & Great Power Competition
Team Drone - 2021 Technology, Innovation & Great Power Competition
 
Team Aurora - 2021 Technology, Innovation & Great Power Competition
Team Aurora - 2021 Technology, Innovation & Great Power CompetitionTeam Aurora - 2021 Technology, Innovation & Great Power Competition
Team Aurora - 2021 Technology, Innovation & Great Power Competition
 
Team Conflicted Capital Team - 2021 Technology, Innovation & Great Power Comp...
Team Conflicted Capital Team - 2021 Technology, Innovation & Great Power Comp...Team Conflicted Capital Team - 2021 Technology, Innovation & Great Power Comp...
Team Conflicted Capital Team - 2021 Technology, Innovation & Great Power Comp...
 
Lecture 7 - Technology, Innovation and Great Power Competition - Space
Lecture 7 - Technology, Innovation and Great Power Competition - SpaceLecture 7 - Technology, Innovation and Great Power Competition - Space
Lecture 7 - Technology, Innovation and Great Power Competition - Space
 
Lecture 6 - Technology, Innovation and Great Power Competition - Autonomy and...
Lecture 6 - Technology, Innovation and Great Power Competition - Autonomy and...Lecture 6 - Technology, Innovation and Great Power Competition - Autonomy and...
Lecture 6 - Technology, Innovation and Great Power Competition - Autonomy and...
 
Lecture 6- Technology, Innovation and Great Power Competition - Unmanned Syst...
Lecture 6- Technology, Innovation and Great Power Competition - Unmanned Syst...Lecture 6- Technology, Innovation and Great Power Competition - Unmanned Syst...
Lecture 6- Technology, Innovation and Great Power Competition - Unmanned Syst...
 
DIU Presentation to Stanford TIGPC Class 10.19.21
DIU Presentation to Stanford TIGPC Class 10.19.21DIU Presentation to Stanford TIGPC Class 10.19.21
DIU Presentation to Stanford TIGPC Class 10.19.21
 
Semiconductor Industry Tutorial
Semiconductor Industry TutorialSemiconductor Industry Tutorial
Semiconductor Industry Tutorial
 
Lecture 3 - Technology, Innovation and Great Power Competition
Lecture 3 - Technology, Innovation and Great Power CompetitionLecture 3 - Technology, Innovation and Great Power Competition
Lecture 3 - Technology, Innovation and Great Power Competition
 
Lecture 2 -Technology, Innovation and Great Power Competition
Lecture 2 -Technology, Innovation and Great Power CompetitionLecture 2 -Technology, Innovation and Great Power Competition
Lecture 2 -Technology, Innovation and Great Power Competition
 
Lecture 1 -Technology, Innovation and Great Power Competition
Lecture 1 -Technology, Innovation and Great Power CompetitionLecture 1 -Technology, Innovation and Great Power Competition
Lecture 1 -Technology, Innovation and Great Power Competition
 

Recently uploaded

Basic Civil Engg Notes_Chapter-6_Environment Pollution & Engineering
Basic Civil Engg Notes_Chapter-6_Environment Pollution & EngineeringBasic Civil Engg Notes_Chapter-6_Environment Pollution & Engineering
Basic Civil Engg Notes_Chapter-6_Environment Pollution & EngineeringDenish Jangid
 
How to the fix Attribute Error in odoo 17
How to the fix Attribute Error in odoo 17How to the fix Attribute Error in odoo 17
How to the fix Attribute Error in odoo 17Celine George
 
Basic_QTL_Marker-assisted_Selection_Sourabh.ppt
Basic_QTL_Marker-assisted_Selection_Sourabh.pptBasic_QTL_Marker-assisted_Selection_Sourabh.ppt
Basic_QTL_Marker-assisted_Selection_Sourabh.pptSourabh Kumar
 
size separation d pharm 1st year pharmaceutics
size separation d pharm 1st year pharmaceuticssize separation d pharm 1st year pharmaceutics
size separation d pharm 1st year pharmaceuticspragatimahajan3
 
Students, digital devices and success - Andreas Schleicher - 27 May 2024..pptx
Students, digital devices and success - Andreas Schleicher - 27 May 2024..pptxStudents, digital devices and success - Andreas Schleicher - 27 May 2024..pptx
Students, digital devices and success - Andreas Schleicher - 27 May 2024..pptxEduSkills OECD
 
ppt your views.ppt your views of your college in your eyes
ppt your views.ppt your views of your college in your eyesppt your views.ppt your views of your college in your eyes
ppt your views.ppt your views of your college in your eyesashishpaul799
 
IATP How-to Foreign Travel May 2024.pdff
IATP How-to Foreign Travel May 2024.pdffIATP How-to Foreign Travel May 2024.pdff
IATP How-to Foreign Travel May 2024.pdff17thcssbs2
 
The impact of social media on mental health and well-being has been a topic o...
The impact of social media on mental health and well-being has been a topic o...The impact of social media on mental health and well-being has been a topic o...
The impact of social media on mental health and well-being has been a topic o...sanghavirahi2
 
Advances in production technology of Grapes.pdf
Advances in production technology of Grapes.pdfAdvances in production technology of Grapes.pdf
Advances in production technology of Grapes.pdfDr. M. Kumaresan Hort.
 
Pragya Champions Chalice 2024 Prelims & Finals Q/A set, General Quiz
Pragya Champions Chalice 2024 Prelims & Finals Q/A set, General QuizPragya Champions Chalice 2024 Prelims & Finals Q/A set, General Quiz
Pragya Champions Chalice 2024 Prelims & Finals Q/A set, General QuizPragya - UEM Kolkata Quiz Club
 
The Art Pastor's Guide to Sabbath | Steve Thomason
The Art Pastor's Guide to Sabbath | Steve ThomasonThe Art Pastor's Guide to Sabbath | Steve Thomason
The Art Pastor's Guide to Sabbath | Steve ThomasonSteve Thomason
 
Gyanartha SciBizTech Quiz slideshare.pptx
Gyanartha SciBizTech Quiz slideshare.pptxGyanartha SciBizTech Quiz slideshare.pptx
Gyanartha SciBizTech Quiz slideshare.pptxShibin Azad
 
Basic phrases for greeting and assisting costumers
Basic phrases for greeting and assisting costumersBasic phrases for greeting and assisting costumers
Basic phrases for greeting and assisting costumersPedroFerreira53928
 
Neurulation and the formation of the neural tube
Neurulation and the formation of the neural tubeNeurulation and the formation of the neural tube
Neurulation and the formation of the neural tubeSaadHumayun7
 
Application of Matrices in real life. Presentation on application of matrices
Application of Matrices in real life. Presentation on application of matricesApplication of Matrices in real life. Presentation on application of matrices
Application of Matrices in real life. Presentation on application of matricesRased Khan
 
Industrial Training Report- AKTU Industrial Training Report
Industrial Training Report- AKTU Industrial Training ReportIndustrial Training Report- AKTU Industrial Training Report
Industrial Training Report- AKTU Industrial Training ReportAvinash Rai
 
Salient features of Environment protection Act 1986.pptx
Salient features of Environment protection Act 1986.pptxSalient features of Environment protection Act 1986.pptx
Salient features of Environment protection Act 1986.pptxakshayaramakrishnan21
 

Recently uploaded (20)

Basic Civil Engg Notes_Chapter-6_Environment Pollution & Engineering
Basic Civil Engg Notes_Chapter-6_Environment Pollution & EngineeringBasic Civil Engg Notes_Chapter-6_Environment Pollution & Engineering
Basic Civil Engg Notes_Chapter-6_Environment Pollution & Engineering
 
How to the fix Attribute Error in odoo 17
How to the fix Attribute Error in odoo 17How to the fix Attribute Error in odoo 17
How to the fix Attribute Error in odoo 17
 
NCERT Solutions Power Sharing Class 10 Notes pdf
NCERT Solutions Power Sharing Class 10 Notes pdfNCERT Solutions Power Sharing Class 10 Notes pdf
NCERT Solutions Power Sharing Class 10 Notes pdf
 
Basic_QTL_Marker-assisted_Selection_Sourabh.ppt
Basic_QTL_Marker-assisted_Selection_Sourabh.pptBasic_QTL_Marker-assisted_Selection_Sourabh.ppt
Basic_QTL_Marker-assisted_Selection_Sourabh.ppt
 
size separation d pharm 1st year pharmaceutics
size separation d pharm 1st year pharmaceuticssize separation d pharm 1st year pharmaceutics
size separation d pharm 1st year pharmaceutics
 
Students, digital devices and success - Andreas Schleicher - 27 May 2024..pptx
Students, digital devices and success - Andreas Schleicher - 27 May 2024..pptxStudents, digital devices and success - Andreas Schleicher - 27 May 2024..pptx
Students, digital devices and success - Andreas Schleicher - 27 May 2024..pptx
 
ppt your views.ppt your views of your college in your eyes
ppt your views.ppt your views of your college in your eyesppt your views.ppt your views of your college in your eyes
ppt your views.ppt your views of your college in your eyes
 
IATP How-to Foreign Travel May 2024.pdff
IATP How-to Foreign Travel May 2024.pdffIATP How-to Foreign Travel May 2024.pdff
IATP How-to Foreign Travel May 2024.pdff
 
The impact of social media on mental health and well-being has been a topic o...
The impact of social media on mental health and well-being has been a topic o...The impact of social media on mental health and well-being has been a topic o...
The impact of social media on mental health and well-being has been a topic o...
 
Advances in production technology of Grapes.pdf
Advances in production technology of Grapes.pdfAdvances in production technology of Grapes.pdf
Advances in production technology of Grapes.pdf
 
Pragya Champions Chalice 2024 Prelims & Finals Q/A set, General Quiz
Pragya Champions Chalice 2024 Prelims & Finals Q/A set, General QuizPragya Champions Chalice 2024 Prelims & Finals Q/A set, General Quiz
Pragya Champions Chalice 2024 Prelims & Finals Q/A set, General Quiz
 
The Art Pastor's Guide to Sabbath | Steve Thomason
The Art Pastor's Guide to Sabbath | Steve ThomasonThe Art Pastor's Guide to Sabbath | Steve Thomason
The Art Pastor's Guide to Sabbath | Steve Thomason
 
Gyanartha SciBizTech Quiz slideshare.pptx
Gyanartha SciBizTech Quiz slideshare.pptxGyanartha SciBizTech Quiz slideshare.pptx
Gyanartha SciBizTech Quiz slideshare.pptx
 
Operations Management - Book1.p - Dr. Abdulfatah A. Salem
Operations Management - Book1.p  - Dr. Abdulfatah A. SalemOperations Management - Book1.p  - Dr. Abdulfatah A. Salem
Operations Management - Book1.p - Dr. Abdulfatah A. Salem
 
Basic phrases for greeting and assisting costumers
Basic phrases for greeting and assisting costumersBasic phrases for greeting and assisting costumers
Basic phrases for greeting and assisting costumers
 
Neurulation and the formation of the neural tube
Neurulation and the formation of the neural tubeNeurulation and the formation of the neural tube
Neurulation and the formation of the neural tube
 
Application of Matrices in real life. Presentation on application of matrices
Application of Matrices in real life. Presentation on application of matricesApplication of Matrices in real life. Presentation on application of matrices
Application of Matrices in real life. Presentation on application of matrices
 
Post Exam Fun(da) Intra UEM General Quiz - Finals.pdf
Post Exam Fun(da) Intra UEM General Quiz - Finals.pdfPost Exam Fun(da) Intra UEM General Quiz - Finals.pdf
Post Exam Fun(da) Intra UEM General Quiz - Finals.pdf
 
Industrial Training Report- AKTU Industrial Training Report
Industrial Training Report- AKTU Industrial Training ReportIndustrial Training Report- AKTU Industrial Training Report
Industrial Training Report- AKTU Industrial Training Report
 
Salient features of Environment protection Act 1986.pptx
Salient features of Environment protection Act 1986.pptxSalient features of Environment protection Act 1986.pptx
Salient features of Environment protection Act 1986.pptx
 

Team Wargames - 2022 Technology, Innovation & Great Power Competition

  • 1. Lessons Learned Original Statement The U.S. needs a way, given a representative simulation, to rapidly explore a strategy for possible novel uses of existing platforms and weapons. David Song B.S. CS Yujing Zhang M.A. East Asian Studies Shashank Rammoorthy M.S. CS Jonathan Sepulveda M.S. Materials Science Project Wargames 30+ interviews Current Statement Strategic wargames stand to benefit from a stronger integration of AI+ML but are struggling to find adoption and usage. How can this be addressed? 1
  • 2. Lessons Learned Defining a Problem: Research Weeks 2 3 4 5 6 7 8 9 Scavenging the Solution Space: Testing Future Exploring the Problem 2
  • 4. Lessons Learned But what is a wargame? 4 1. Physics-based simulators 2. TTXs for campaigns 3. High-level diplomacy wargames
  • 5. Lessons Learned Then we played with wargames and talked to experts... 5
  • 6. Lessons Learned 6 So we went ‘back to the drawing board’... Picking one case study of interest, we focused on the Cross-Straits scenario… ● We visited talks on Taiwan’s Economic Security to understand variables often overlooked ● The sad reality is, wargame players often have very little time to anticipate or prepare for wargames, and are not concerned with non- combatant variables ● The most important element of wargames ought to be to teach people to think creatively
  • 7. Lessons Learned We realized wargaming was not easy Then we started learning and asking… ● What are CoAs? ● Differences between wargaming vs. M&S? ● Why does the DoD want innovation within such traditional processes? ● What hasn’t been attempted already? 7 Excerpt from Battle! Practical Wargaming by Charles Grant
  • 8. Lessons Learned Problems we found with wargaming 8 We immediately found out that there are… ● Many moving parts & elements: researchers → designers → wargamers → Red and Blue teams ● Little data is shared amongst designers and team members ● There are multiple types of wargames ● Most games remain TTXs ● Too difficult to build de novo a robust, AI-capable Red adversary That left us thinking: ● Virtual wargames are often used for training, but seldom as formal proceedings; why? ● Should we seek to replace wargamers altogether with AI+ML ‘magic’?
  • 9. Lessons Learned What some of our interviewees told us… 9 Sebastian Bae CNA Corp. | Wargamer Yuna Wong IDA | Defense Analyst Jacquelyn Schneider Hoover, CISAC | Researcher Pete Pellegrino U.S. NWC | Game Design Lead "There are a lot of reasons for why AI/ML hasn't been used in wargaming in the DoD…technical illiteracy within the DoD…policies are inflexible, and systems are difficult to change"
  • 10. Lessons Learned What some of our interviewees told us… 10 Sebastian Bae CNA Corp. | Wargamer Yuna Wong IDA | Defense Analyst Jacquelyn Schneider Hoover, CISAC | Researcher Pete Pellegrino U.S. NWC | Game Design Lead "We should never use new wargames, but rather well-studied wargames and already-studied ones in predicting our outcomes & enhancing our adjudications”
  • 11. Lessons Learned What some of our interviewees told us… 11 Sebastian Bae CNA Corp. | Wargamer Yuna Wong IDA | Defense Analyst Jacquelyn Schneider Hoover, CISAC | Researcher Pete Pellegrino U.S. NWC | Game Design Lead "Wargames need data to eat from. You need a volume of data that AI+ML can learn from and be helpful; where is that data coming from a wargame? Wargames don’t generate simulation data in copious amounts”
  • 12. Lessons Learned What some of our interviewees told us… 12 Sebastian Bae CNA Corp. | Wargamer Yuna Wong IDA | Defense Analyst Jacquelyn Schneider Hoover, CISAC | Researcher Pete Pellegrino U.S. NWC | Game Design Lead "We put too much emphasis on the Red...players are generally not from Russia or China, and so you struggle to extrapolate specific country foreign policy decisions versus general human reactions to international crises.” Many different objections to AI+ML!
  • 13. Lessons Learned We got out of the building, went to an NPS Conference… 13 ● Existing AI+ML capabilities have to be better leveraged ● AI+ML does not need to make drastic technical advances to be feasible for wargaming uses ● There’s no readily accessible central repository of past wargame data outcomes to help future teams ● Ideally, forecast decision spaces as opposed to having AI+ML play moves
  • 14. Lessons Learned It’s really hard to build Alphastar with the simulations so far 14
  • 15. Lessons Learned Defining a Problem: Research Weeks 2 3 4 5 6 7 8 9 Scavenging the Solution Space: Testing Future 15 Pivot: Let’s help humans play wargames better with AI tools instead of replacing humans completely with AI
  • 16. Lessons Learned Defining a Problem: Research Weeks 2 3 4 5 6 7 8 9 Scavenging the Solution Space: Testing Future 16 Identifying Recommendations
  • 17. Lessons Learned 17 1. Digitize wargames to increase iterations "Accelerating and making wargames more efficient will come down to a series of technologies that could very well be unclassified in development” Major defense startup company
  • 18. Lessons Learned 18 2. Create AI tools for digitized wargames “Not many people in the U.S. have access to the decision- making teams you find in Red teams [the PLA and the CCP].” Elizabeth Bartels | RAND
  • 19. Lessons Learned 19 3. Implement organizational changes We read-up on traditional approaches to building wargames, noting its strengths.. ● We found more support that cross-talk between wargamers & designers and players is important ● Academics do not differ that sharply from the assessment of wargame players insofar as AI+ML integration is concerned ● Information asymmetry predominates within wargames–not an attribute of UCC commanders ● Need more collaborations among wargamers, designers, combat & command
  • 20. Lessons Learned Defining a Problem: Research Weeks 2 3 4 5 6 7 8 9 Scavenging the Solution Space: Testing Future 20 Next steps 1. Validate our recommendations 2. Shadow a wargame 3. Continue working with our sponsor