Team Omniscient aims to help intelligence analysts review large quantities of imagery data more efficiently. They identified detecting North Korean AN-2 aircraft in satellite images as a specific problem their computer vision tools could help solve. This would allow analysts to more rapidly locate and identify objects of interest, focusing their time on higher-level tasks. The team outlined key partners, resources, activities, and impact factors to execute their solution and measure its success at achieving the mission.

1. Team Omniscient
Original Problem
DCGS operators need an automated way to review
a larger quantity of collected imaging data in order
to surface actionable intelligence to leadership.
Sponsor Organization: US Air Force Tactical Exploitation of National Capabilities (AF TENCAP)
108
Interviews
Supported By: Maj Rose (Sponsor), COL Smith-Heys (Military Mentor), Kevin Ray (Business Mentor), Gus Hernandez
(Advisor)
Final Problem
Analysts lack the computer vision tools to augment
their ability to rapidly locate, identify, and analyze
objects of interest, which would allow them to
focus their time on higher order analysis tasks.
Nick Mirda | GSB ‘21
Prior Army Intelligence
Officer
Summer: BCG
Jon Braatz | MS CS ‘20
Computer Vision
Research
Summer: !
Andrew Fang | BS CS ‘22
Computer Vision
Products
Summer: Anduril

2. 90+% of images never reviewed!
There’s too much data!

3. KEY PARTNERS
We will liaison with two
DCGS (Distributed Common
Ground/Surface System)
centers, located at Langley
AFB and Beale AFB
Other Potential Partners:
- Intelligence Analysts
- USAF Weapons School
- DARPA
- KesselRun
- Air Force Research Lab
- Sandia National Labs
- MIT Lincoln Lab
- DIU
- NASIC
KEY RESOURCES
VALUE PROPOSITIONSKEY ACTIVITIES
MISSION ACHIEVEMENT/IMPACT FACTORS
● We will measure mission achievement by: the usability and
accuracy of our model on data provided by the Air Force
● Our beneficiaries will measure mission achievement by: the
adoption/ease-of-use for analysts whose jobs we will simplify
and ability to surface exploitable info in collected data.
DEPLOYMENT
BUY-IN & SUPPORT
MISSION BUDGET/COST
● Budget for training costs (AWS/Azure GPU time)
● Potential costs for proprietary software
BENEFICIARIES
1. Operators:
Intelligence
analysts who parse
through images
(NASIC, CENTCOM,
etc)
2. Decision Makers:
High level decision
makers who need
actionable
intelligence quickly
and efficiently
1. Narrow problem space
2. Access Data
3. Build model by
augmenting
YOLO/existing models
4. Test model
1. Analysts who manually
identify actionable intel
(to provide insight on
what is considered
valuable data + provide
robustly labeled data)
2. AWS/Azure for training
3. Lots of data
1. Workload Reduction:
Reduce human hours
currently spent on
identifying actionable
information in images
1. Decrease Intelligence
Processing Timeline:
Capture information
from images faster
than a human analyst
1. Reduce Data
Backlogs: Parse
image database
backlogs to surface
exploitable images
1. End Users: Analysts who
parse through images of
AF pilots who are willing
to test our software
2. Leadership: Budget
authority and operational
policy experts
1. Initial deployment:
command line service
2. Future deployments:
product with a
UI/clear instructions
for analysts
Mission Model Canvas: initial expectations

4. KEY PARTNERS
We will liaison with two
DCGS (Distributed Common
Ground/Surface System)
centers, located at Langley
AFB and Beale AFB
Other Potential Partners:
- Intelligence Analysts
- USAF Weapons School
- DARPA
- KesselRun
- Air Force Research Lab
- Sandia National Labs
- MIT Lincoln Lab
- DIU
- NASIC
KEY RESOURCES
VALUE PROPOSITIONSKEY ACTIVITIES
MISSION ACHIEVEMENT/IMPACT FACTORS
● We will measure mission achievement by: the usability and
accuracy of our model on data provided by the Air Force
● Our beneficiaries will measure mission achievement by: the
adoption/ease-of-use for analysts whose jobs we will simplify
and ability to surface exploitable info in collected data.
DEPLOYMENT
BUY-IN & SUPPORT
MISSION BUDGET/COST
● Budget for training costs (AWS/Azure GPU time)
● Potential costs for proprietary software
BENEFICIARIES
1. Operators:
Intelligence
analysts who parse
through images
(NASIC, CENTCOM,
etc)
2. Decision Makers:
High level decision
makers who need
actionable
intelligence quickly
and efficiently
1. Narrow problem space
2. Access Data
3. Build model by
augmenting
YOLO/existing models
4. Test model
1. Analysts who manually
identify actionable intel
(to provide insight on
what is considered
valuable data + provide
robustly labeled data)
2. AWS/Azure for training
3. Lots of data
1. Workload Reduction:
Reduce human hours
currently spent on
identifying actionable
information in images
1. Decrease Intelligence
Processing Timeline:
Capture information
from images faster
than a human analyst
1. Reduce Data
Backlogs: Parse
image database
backlogs to surface
exploitable images
1. End Users: Analysts who
parse through images of
AF pilots who are willing
to test our software
2. Leadership: Budget
authority and operational
policy experts
1. Initial deployment:
command line service
2. Future deployments:
product with a
UI/clear instructions
for analysts
Mission Model Canvas: initial expectations
Initial Thoughts:
1. Help analysts look for
“needles in haystacks”

5. KEY PARTNERS
We will liaison with two
DCGS (Distributed Common
Ground/Surface System)
centers, located at Langley
AFB and Beale AFB
Other Potential Partners:
- Intelligence Analysts
- USAF Weapons School
- DARPA
- KesselRun
- Air Force Research Lab
- Sandia National Labs
- MIT Lincoln Lab
- DIU
- NASIC
KEY RESOURCES
VALUE PROPOSITIONSKEY ACTIVITIES
MISSION ACHIEVEMENT/IMPACT FACTORS
● We will measure mission achievement by: the usability and
accuracy of our model on data provided by the Air Force
● Our beneficiaries will measure mission achievement by: the
adoption/ease-of-use for analysts whose jobs we will simplify
and ability to surface exploitable info in collected data.
DEPLOYMENT
BUY-IN & SUPPORT
MISSION BUDGET/COST
● Budget for training costs (AWS/Azure GPU time)
● Potential costs for proprietary software
BENEFICIARIES
1. Operators:
Intelligence
analysts who parse
through images
(NASIC, CENTCOM,
etc)
2. Decision Makers:
High level decision
makers who need
actionable
intelligence quickly
and efficiently
1. Narrow problem space
2. Access Data
3. Build model by
augmenting
YOLO/existing models
4. Test model
1. Analysts who manually
identify actionable intel
(to provide insight on
what is considered
valuable data + provide
robustly labeled data)
2. AWS/Azure for training
3. Lots of data
1. Workload Reduction:
Reduce human hours
currently spent on
identifying actionable
information in images
1. Decrease Intelligence
Processing Timeline:
Capture information
from images faster
than a human analyst
1. Reduce Data
Backlogs: Parse
image database
backlogs to surface
exploitable images
1. End Users: Analysts who
parse through images of
AF pilots who are willing
to test our software
2. Leadership: Budget
authority and operational
policy experts
1. Initial deployment:
command line service
2. Future deployments:
product with a
UI/clear instructions
for analysts
Mission Model Canvas: initial expectations
Initial Thoughts:
1. Help analysts look for
“needles in haystacks”
2. What do those
“needles” look like?

6. KEY PARTNERS
We will liaison with two
DCGS (Distributed Common
Ground/Surface System)
centers, located at Langley
AFB and Beale AFB
Other Potential Partners:
- Intelligence Analysts
- USAF Weapons School
- DARPA
- KesselRun
- Air Force Research Lab
- Sandia National Labs
- MIT Lincoln Lab
- DIU
- NASIC
KEY RESOURCES
VALUE PROPOSITIONSKEY ACTIVITIES
MISSION ACHIEVEMENT/IMPACT FACTORS
● We will measure mission achievement by: the usability and
accuracy of our model on data provided by the Air Force
● Our beneficiaries will measure mission achievement by: the
adoption/ease-of-use for analysts whose jobs we will simplify
and ability to surface exploitable info in collected data.
DEPLOYMENT
BUY-IN & SUPPORT
MISSION BUDGET/COST
● Budget for training costs (AWS/Azure GPU time)
● Potential costs for proprietary software
BENEFICIARIES
1. Operators:
Intelligence
analysts who parse
through images
(NASIC, CENTCOM,
etc)
2. Decision Makers:
High level decision
makers who need
actionable
intelligence quickly
and efficiently
1. Narrow problem space
2. Access Data
3. Build model by
augmenting
YOLO/existing models
4. Test model
1. Analysts who manually
identify actionable intel
(to provide insight on
what is considered
valuable data + provide
robustly labeled data)
2. AWS/Azure for training
3. Lots of data
1. Workload Reduction:
Reduce human hours
currently spent on
identifying actionable
information in images
1. Decrease Intelligence
Processing Timeline:
Capture information
from images faster
than a human analyst
1. Reduce Data
Backlogs: Parse
image database
backlogs to surface
exploitable images
1. End Users: Analysts who
parse through images of
AF pilots who are willing
to test our software
2. Leadership: Budget
authority and operational
policy experts
1. Initial deployment:
command line service
2. Future deployments:
product with a
UI/clear instructions
for analysts
Mission Model Canvas: initial expectations
Initial Thoughts:
1. Help analysts look for
“needles in haystacks”
2. What do those
“needles” look like?
3. Get our hands on
imagery data to build
a solution

7. BENEFICIARIES BUYERS PARTNERS
We interviewed 108 people all holding a
different piece of the puzzle.
EXPERTS

8. How we’re feeling: we got this!

9. Everyone we talked to had a different problem.
Detect Changes
We were overwhelmed.

10. Everyone we talked to had a different problem.
Image Clarity Rating
(NIIRS)
Detect Changes
We were overwhelmed.

11. Everyone we talked to had a different problem.
Image Clarity Rating
(NIIRS)
North Korean MissilesDetect Changes
We were overwhelmed.

12. Everyone we talked to had a different problem.
Full-Motion Video
(Maven)
Image Clarity Rating
(NIIRS)
North Korean MissilesDetect Changes
We were overwhelmed.

13. Everyone we talked to had a different problem.
Full-Motion Video
(Maven)
Image Clarity Rating
(NIIRS)
North Korean MissilesDetect Changes
And there were a lot of imagery options.
We were overwhelmed.

14. Crisis!

15. We lost two teammates!

16. How we’re feeling: yikes

17. Air Force is shifting to higher level analysis.
Imagery Analysis
● Recording object position
● Annotating observations
Imagery Understanding
● Situational analysis
● Deep understanding

18. “Instead of counting objects that can be automatically detected, my analysts
can ask why those vehicles are there, really unleashes analytic horsepower.”
-Director of Operations @ 13th Intel Squadron
Machine learning can automate the drudge work.

19. ● Tracking all aircraft in flight (NRO).
○ Unsuccessful.
● Project Maven: automatic full-motion video analysis.
○ Mixed results.
● Automatic airfield layout change detection (NGA).
○ Ongoing.
● Identify groupings of tanks (NGA).
○ Ongoing.
Previous efforts stumbled due to overambitious goals and improperly labelled data.
Many DoD programs to automate imagery
analysis, but most are still work-in-progress.
?
?
?

20. First MVP: a generic computer vision tool.
● it processes analyst imagery to detect objects.
● it runs in the background.
● it uses computer vision.
Feedback:
1) “I’ve heard this dozens of times.”
2) “I care less about innovation, more about
integration.”
We need a specific use case & a way to get in.

21. A bad pivot: we jumped on the first computer vision
solution we saw (computer vision to help bandwidth).
• We thought RQ-4 Global Hawks had significant bandwidth limitations that
hampered SAR imagery delivery to the base, after speaking with a pilot.
We pivoted too early, deviated from beneficiary insights, and
were invalidated with further interviews.

22. How we’re feeling: demoralized :/

23. Breakthrough!

24. Insight: The AN-2 is a proxy for what our
solution can deliver.
● This is a discrete, strategically relevant problem we
can sink our teeth into
● If we can track these, we expand to tracking other
equipment
● it’s not easy, but we think we could build it.
How do we get in? How do we execute?

25. 3 meter resolution 50 centimeter resolution
Our algorithms would require a lot of good data.
We scraped a few thousand images. For free!

26. Taechon Airfield AN-2
Image #B4993
Date: 05MAY2020
Asset Quantity: 9
Asset Coordinates:
1. 51SYE1354920036
2. 51SYE1347120046
...
9. 51SYE1319020263c
First validated MVP automates AN-2 detection &
feeds into analyst workflow. .KML Outputs

27. KEY PARTNERS
Air Force DGS-3 AETs that
will be our customers/users.
Commercial satellite imagery
companies to acquire data to
train on. (Maxar, Planet).
Innovation and Research
organizations to accelerate
classified data/system access.
(DIU, CRADA, SBIR).
KEY RESOURCES
VALUE PROPOSITIONSKEY ACTIVITIES
MISSION ACHIEVEMENT/IMPACT FACTORS
● We will measure mission achievement by: accuracy of AN-2
classifications from still imagery and anomaly alerts.
● Our beneficiaries will measure mission achievement by: the ability
to identify meaningful activity/objects of interest from large data
sets and adoption/ease-of-use for analysts.
DEPLOYMENT
BUY-IN & SUPPORT
MISSION BUDGET/COST
● Budget for training costs (AWS/Azure GPU time).
● Unclassified EO data/labels from commercial companies.
● Potential costs for proprietary software.
BENEFICIARIES
1. Operators: DGS-3
AET Analysts (Phase
1), ISR Pilots, Sensor
Operators, Collection
Managers
1. Decision Makers: US
Forces and Korea
leadership need
actionable intel fast.
1. Analysts who manually
analyze still imagery (to
provide insight on what is
data + provide robustly
labeled data).
2. Lots of commercial
satellite data (Maxar).
1. Decrease AN-2 Imagery
Process Time: Quickly
scan large images to
extract quantity/location
of assets of interest.
1. Get AN-2 Anomalies to
Analysts/Leaders Fast:
Alert analysts to enable
them to verify suspicious
activity ASAP.
1. Up-to-Date AN-2
Activity/Locations:
Know where all AN-2s
are and what they do
with reliable information.
1. End Users: Imagery
analysts & AETs at DGS-3.
1. Leadership: DGS-3
Collections Managers,
Combatant Commanders
setting ISR priorities.
1. Image Processing on
unclass computer.
2. Future deployments:
plugin for existing
analysis software
3. Continuously-running
anomaly notification
System
1. Access commercial
satellite data of NK
airfields.
2. Build and test model
on unclassified satellite
imagery.
3. Identify integration
pathway.
Mission Model Canvas: use case identified!
Allowed us to:
1. Articulate specific
value proposition

28. KEY PARTNERS
Air Force DGS-3 AETs that
will be our customers/users.
Commercial satellite imagery
companies to acquire data to
train on. (Maxar, Planet).
Innovation and Research
organizations to accelerate
classified data/system access.
(DIU, CRADA, SBIR).
KEY RESOURCES
VALUE PROPOSITIONSKEY ACTIVITIES
MISSION ACHIEVEMENT/IMPACT FACTORS
● We will measure mission achievement by: accuracy of AN-2
classifications from still imagery and anomaly alerts.
● Our beneficiaries will measure mission achievement by: the ability
to identify meaningful activity/objects of interest from large data
sets and adoption/ease-of-use for analysts.
DEPLOYMENT
BUY-IN & SUPPORT
MISSION BUDGET/COST
● Budget for training costs (AWS/Azure GPU time).
● Unclassified EO data/labels from commercial companies.
● Potential costs for proprietary software.
BENEFICIARIES
1. Operators: DGS-3
AET Analysts (Phase
1), ISR Pilots, Sensor
Operators, Collection
Managers
1. Decision Makers: US
Forces and Korea
leadership need
actionable intel fast.
1. Analysts who manually
analyze still imagery (to
provide insight on what is
data + provide robustly
labeled data).
2. Lots of commercial
satellite data (Maxar).
1. Decrease AN-2 Imagery
Process Time: Quickly
scan large images to
extract quantity/location
of assets of interest.
1. Get AN-2 Anomalies to
Analysts/Leaders Fast:
Alert analysts to enable
them to verify suspicious
activity ASAP.
1. Up-to-Date AN-2
Activity/Locations:
Know where all AN-2s
are and what they do
with reliable information.
1. End Users: Imagery
analysts & AETs at DGS-3.
1. Leadership: DGS-3
Collections Managers,
Combatant Commanders
setting ISR priorities.
1. Image Processing on
unclass computer.
2. Future deployments:
plugin for existing
analysis software
3. Continuously-running
anomaly notification
System
1. Access commercial
satellite data of NK
airfields.
2. Build and test model
on unclassified satellite
imagery.
3. Identify integration
pathway.
Mission Model Canvas: use case identified!
Allowed us to:
1. Articulate specific
value proposition
2. Focus on the
appropriate end users

29. KEY PARTNERS
Air Force DGS-3 AETs that
will be our customers/users.
Commercial satellite imagery
companies to acquire data to
train on. (Maxar, Planet).
Innovation and Research
organizations to accelerate
classified data/system access.
(DIU, CRADA, SBIR).
KEY RESOURCES
VALUE PROPOSITIONSKEY ACTIVITIES
MISSION ACHIEVEMENT/IMPACT FACTORS
● We will measure mission achievement by: accuracy of AN-2
classifications from still imagery and anomaly alerts.
● Our beneficiaries will measure mission achievement by: the ability
to identify meaningful activity/objects of interest from large data
sets and adoption/ease-of-use for analysts.
DEPLOYMENT
BUY-IN & SUPPORT
MISSION BUDGET/COST
● Budget for training costs (AWS/Azure GPU time).
● Unclassified EO data/labels from commercial companies.
● Potential costs for proprietary software.
BENEFICIARIES
1. Operators: DGS-3
AET Analysts (Phase
1), ISR Pilots, Sensor
Operators, Collection
Managers
1. Decision Makers: US
Forces and Korea
leadership need
actionable intel fast.
1. Analysts who manually
analyze still imagery (to
provide insight on what is
data + provide robustly
labeled data).
2. Lots of commercial
satellite data (Maxar).
1. Decrease AN-2 Imagery
Process Time: Quickly
scan large images to
extract quantity/location
of assets of interest.
1. Get AN-2 Anomalies to
Analysts/Leaders Fast:
Alert analysts to enable
them to verify suspicious
activity ASAP.
1. Up-to-Date AN-2
Activity/Locations:
Know where all AN-2s
are and what they do
with reliable information.
1. End Users: Imagery
analysts & AETs at DGS-3.
1. Leadership: DGS-3
Collections Managers,
Combatant Commanders
setting ISR priorities.
1. Image Processing on
unclass computer.
2. Future deployments:
plugin for existing
analysis software
3. Continuously-running
anomaly notification
System
1. Access commercial
satellite data of NK
airfields.
2. Build and test model
on unclassified satellite
imagery.
3. Identify integration
pathway.
Mission Model Canvas: use case identified!
Allowed us to:
1. Articulate specific
value proposition
2. Focus on the
appropriate end users
3. Outline a specific
integration pathway

30. How we’re feeling: Yes! On the right track!

31. Unclassified
Satellite Imagery
NIPRnet (Unclassified)
Unclassified Tool, Integrates with Classified Systems

32. Unclassified
Satellite Imagery
Entity
Detection
Algorithms
NIPRnet (Unclassified)
Unclassified Integration into Existing Analyst Workflow

33. Unclassified
Satellite Imagery
Entity
Detection
Algorithms
NIPRnet (Unclassified)
Unclassified Integration into Existing Analyst Workflow
Historical
Baseline
Comparison

34. Unclassified
Satellite Imagery
Entity
Detection
Algorithms
NIPRnet (Unclassified)
Unclassified Integration into Existing Analyst Workflow
Historical
Baseline
Comparison
.KML
Output Files

35. Unclassified
Satellite Imagery
Entity
Detection
Algorithms
NIPRnet (Unclassified)
Unclassified Integration into Existing Analyst Workflow
Historical
Baseline
Comparison
Date
Location
Quantity
SIPRnet (Classified)
.KML
Output Files

36. Data Uploaded
to SIPRnet
Unclassified
Satellite Imagery
Entity
Detection
Algorithms
NIPRnet (Unclassified)
Unclassified Integration into Existing Analyst Workflow
Historical
Baseline
Comparison
Date
Location
Quantity
SIPRnet (Classified)
.KML
Output Files
.KMLs
Uploaded

37. Data Uploaded
to SIPRnet
Unclassified
Satellite Imagery
Entity
Detection
Algorithms
Data integrates with analyst tools
NIPRnet (Unclassified)
Unclassified Integration into Existing Analyst Workflow
Historical
Baseline
Comparison
Date
Location
Quantity
SIPRnet (Classified)
.KML
Output Files
.KMLs
Uploaded

38. Data Uploaded
to SIPRnet
Unclassified
Satellite Imagery
Entity
Detection
Algorithms
Data integrates with analyst tools
Intel products
are built more
quickly and
effectively.
NIPRnet (Unclassified)
Unclassified Integration into Existing Analyst Workflow
Historical
Baseline
Comparison
Date
Location
Quantity
SIPRnet (Classified)
.KML
Output Files
.KMLs
Uploaded

39. Credibility:
Computer Vision:
Working Towards
Unofficial Demo
with Air Combat
Command
(In 2-3 Weeks)
TENCAP
Mentorship
to TENCAP Letter
of Endorsement

41. Help us build omniscience.
MILITARY: Collaboration- how can we help you?
PRIMEs: Platforms and Partnerships- how can we work together?
NEXT STEPS:
COMMERCIAL: Dual use | Venture Funding- let’s chat!
Email: hello@omniscientlabs.io
$20k | Now $50k | 6 Months
Working over summer:
● Develop AN-2 algorithms
● Scope out dual use
applications