The Narrative Mind team seeks to develop tools to optimize discovery and investigation of communication trends on social media. They have conducted 66 interviews total with experts, users, and potential buyers. The team hypothesizes that their narrative detection units may overlap with other commercial tools, and plan to do a demo day to compare capabilities. They have proposed an MVP approach of outputting 3 types of narratives to analysts based on frequency of units associated with real-world narratives.

1. Team
Narrative
Mind

2. Team Info
The Narrative Mind team contains experts in software engineering, social media design, and
web-based information operations (IO). We seek to develop tools that will optimize discovery
and investigation of communication trends on social media.
Weekly Total: 10 interviews
Users: 16
Experts: 45
Buyers: 5
Cumulative Total: 66 interviews

3. OTA Process -- Next Steps
OTA Summary
● Prototype projects only.
● Small businesses or non-traditional
defense contractors participate to a
significant extent.
● More negotiable IP terms
● Initial Award ~$5M
● Multiple phases
Phases Post Initial Award
● Iterative process (detailed view, next slide)
● Number of phases is flexible.
● Amount of money is flexible.
● Color of money is flexible.
● Scale is bounded by “production” levels.
Can’t be used to “reduce competition” or
“increase production.
Alternatives Briefly Compared
● SBRIs are smaller in scale and require
more administrative overhead.
● DARPA Cyber FastTrack is deprecated
but had a similar idea to OTA.
● CRADA is designed for research sharing,
not really prototype efforts.

4. OTA Iterative Evaluation Process
No overarching regulations or timelines.
“Get back to me in a year and I’ll let you how it goes.”
Example: Parsons, Critical Stack & Wolf Den
Innovation Challenge Initial Award Testing Evaluation
~$5M
1. Industry Day
2. Reqs. Synopsis
3. Request White Papers
4. Received 13 papers
5. Four proposals selected.
6. “Demo” replaced by
Technical Discussion.
Original
requirements
synopsis
modified.
1. ACT office has
personnel working with
testers from all ranks.
2. Army Cyber Battle Lab
involved for
integration/concepts.
Budget Adoption
??????????
Final acquisition may
be expedited if
requirement money is
already set aside in
budgets, e.g. RDT&E
1. Problem
requirements are
changed iteratively.
2. Phase objectives
and timeline are
flexible.

5. Variants of OTA Processes
OTA processes can vary significantly.
● Type of Material/Resource
○ Different PEO
○ Different Consortium
● Different Funding Model
○ Prototypes?
○ Investment?
● Evaluation Style
○ Dependent on funding?
Example: Army Battery Technology circa 2003 (?)
OTA process similar to In-Q-Tel model.
● Army Venture Capital Initiative created non-
profit venture fund OnPoint
● No prototype
● Investment based
● RDT&E money and S&T money
○ Relatively larger budgets and lasts longer.
● ~$25M initial fund?
● Future customer privileges?
https://www.ciaonet.org/attachments/11954/uploads
http://www.businesswire.com/news/home/20041111005159/en/PowerGenix-Announces-Closing-Series-Financing-OnPoint-
Technologies

6. Hypotheses ❏ Our narrative units may exist in other commercial tools
❏ If not, make the introduction between ARCYBER and relevant tools
❏ Communication links between problem awareness and problem response are weak
Experiments ❏ Talk to commercial tools about their tools, direct comparison to our narrative units
Results There is overlap between our narrative units and other commercial tools on social network
analysis
Actions 1. Set up “demo” day to show ARCYBER a selection of commercial tools with capabilities that
seem to fit their social media network analysis needs
2. Use the commercial tools demo’d as a foil to understand what ARCYBER needs and doesn’t
need
Customer Discovery

7. Commercial Tools’ Value
- “Intelligent” content filtration and
categorization systems
- Powerful data visualization
- Geospatial integrations
- Cross-platform information integrity
MVP - “Demo Day”
Research/Academic Tools’ Value
- Novel scraping and aggregation
techniques
- Domain expertise in counterterrorism
- Human intuition for rooting out
narratives
Wednesday, May 4 with ARCYBER sponsors in person

8. MVP - 3 Types of “Narratives” Outputted by Our System
A list of hashtags that
appear with the same
message with extreme
frequency.
Positive or negative
opinion of geographic or
demographic population.
Narrative Type #1 Narrative Type #2
Narrative Type #3
Displayed
to Analysts
Analysts associate
each “narrative unit”
with a real-world,
human-readable
description of a
narrative.
Gain creator: Software automatically
tracks activity of a real-world narratives by
tracking the activity of associated “units”.
Real World Narrative
Long-form, unstructured human description or
collection of info.
A ranked list of’ most
popular multi-media
content over time
Pain reliever: Social media firehoses
scanned for list of narrative units that match
predefined “schemas”, ranked by frequency.
Problems with this MVP:
● What are the right “units”?
● How is a narrative’s overall growth
calculated from the multiple units with
different sizes? (e.g. relative
“weights?)
Benefits of this MVP:
● Aside from schema, specific units
don’t need to be defined a priori.
● Model allows retroactive reindexing of
a single narrative’s growth based on
new associations.
● Builds a “training set” for more
sophisticated efforts.

9. Responses to Narrative Units Presented Previously
A ranked list most
popular content
over time
A list of
hashtags that
appear with the
same message
with extreme
frequency.
Positive or negative opinion among
geographic or demographic
population.
Narrative Type #1
Narrative Type #2
Narrative Type #3
E.g
Popularitywithusers
Sentiment analysis is thought
to be too crude to provide
useful input. May pollute
signal-noise.
Unit #2 “popular content”
(commonly described as
media like photos or videos)
seemed to be the most
credible. Unit #1 co-occurring
hashtags might provide more
granularity and context than
just single hashtag trends.

10. All the Hypotheses
Global
Tweet-level
Awareness Response
Adversary IO Org Chart
No baseline for monitoring/aggregating
use of tech; cyber targeting
Automatic Narrative Detection
Language/culture experts aren’t able
to work at scale
Virality Predictor
Understand which narratives are the most salient
Important Event Predictor
Preempt events
Counter-Narrative Generator
DoD/Gvt. social media
presence is weak
Persistent ID-Alias tracker
Account bans and multiple aliases across
different networks make IDs hard to track
Filesharing Site Scraper
Need more cached information
access
Expedited Content Categorization
Scale of social media makes manual efforts
painful
Bot Detector
Can’t determine actual scale of
organization’s following and support.

11. Mission Model Canvas
- Categorize social media posts
and users by content for
monitoring and tactical purposes.- Gnip/Twitter/Facebook
- CrowdFlower,
Samasource, or
Mechanical Turk
- Pre-existing social
media service and micro-
labor aggregators
- Third-party access
platforms for social
media,
-Data visualization
-Content analysis
platforms (Sens.ai,
Leidos)
Primary
ARCYBER
-Bg. General (decision
maker)
-MAJ/LTC/COL
(operational plan)
-Analysts/Operators
(actionable insights)
COCOMs
-General (decision
maker)
-MAJ/LTC/COL
-Analyst/Operator
Secondary
Political Campaigns
-Campaign managers
-Supporters
Consumer Brands
-CMO
-Public Relations Team
- Optimize workflow for
social media analysts.
-Deliver insights to
commanders about
online environment.
-Insights into tactical
responses against
specific targets
broadcasting narratives
-Detect in real time
-Early warning - Architecture that can
support massive
concurrent data
aggregation and
analysis.
- Customized UI
- Testing with analysts
- MechanicalTurk or crowdsourcing labor (microtasks)
- UI Development/Testing with ARCYBER analysts.
- Software Development
- Access to Twitter
firehose or API
- Local language
speaking crowdsourcing
staff.
- Accurate testing for
intercoder reliability
- ARCYBER: Bg. General,
LTC, Strategic Initiatives
Group, OTA, Purchasing PMs,
End-User operator)
-COCOMs: OTA, Operators,
Purchasing PMs
-Political Campaign:
Opposition research team, ???
-Private Sector: CMO, ???
Beneficiaries
Mission AchievementMission Budget/Costs
Buy-In/Support
Deployment
Value
Proposition
Key Activities
Key Resources
Key Partners

12. Value Proposition Canvas - ARCYBER
Products
& Services
Web/Desktop
Application Provide high-level
overview of groups,
update research
Customer
Jobs
- New mission priorities.
- Little familiarity with overall
environment.
- No repository for high-level
learnings
Gains
Pains
Gain
Creators
Pain
Relievers
- New opportunities
-Illustrates evolution of info
dissemination networks
- Provides insight for
mapping “online terrain”
Visualize information about
how organizations are
using social media
ARCYBER - Operator/Analyst (Actionable Insights)

13. Value Proposition Canvas - ARCYBER
Products
& Services
Web/Desktop
Application
Aggregate insights
from analysts,
compile into plan
passed to decision
maker
- New mission priorities.
- Proliferating/viral social
media platforms
- Urgency of issues ability to
prioritize
Customer
Jobs
Gains
Pains
Gain
Creators
Pain
Relievers
-Faster turnaround time with
better data gained from
information awareness software
and technology
ARCYBER - MAJ/LTC/COL (Operational Plans)
-Better data in informing
operational plans provided
to decision makers
-Reducing uncertainty in
operational planning

14. Value Proposition Canvas - ARCYBER
Products
& Services
Web/Desktop
Application
Act on reports and
plans generated
about various cyber
threats across globe
- Little understanding of
ground-level nuances
- Little understanding of
proliferating platforms
Customer
Jobs
Gains
Pains
Gain
Creators
Pain
Relievers
- Organizational chart of
technology broken down by
functional purpose
- Timeline-based viewer
-Reduce uncertainty of
decision making
-Add methodological rigor
to ARCYBER’s
offensive/defensive
operations
ARCYBER - Brigadier General (Decision Maker)

15. Value Proposition Canvas - COCOM
Products
& Services
Web/Desktop
Application
Target audience
analysis, finding
opportunities to
optimize influence
operations
Customer
Jobs
-Loss of visibility on key
communicators (when
accounts shut down/opened)
- No repository for high-level
learnings
Gains
Pains
Gain
Creators
Pain
Relievers
-Auto-populated list & real-time
visualization of key accounts,
dissemination networks
-Allows more accurate
-Illustrates evolution of info
dissemination networks
- Provides insight for mapping
“online terrain”
Closure of visibility gap on
key accounts
COCOM - Operator/Analyst (Actionable Insights)

16. Value Proposition Canvas - COCOM
Products
& Services
Web/Desktop
Application
-Long term
operational planning
- Measuring
effectiveness
Customer
Jobs
- Unclear picture of how
some networks evolve,
adapt over time
Gains
Pains
Gain
Creators
Pain
Relievers
- Organizational chart of
technology broken down by
functional purpose
- Visualization of key accounts,
dissemination networks over
time
-Illustrates evolution of info
dissemination networks
- Provides insight for
measuring operations,
mapping “online terrain”
Greater awareness of
network evolution improves
planning, indicators of
operational effectiveness
COCOM - Maj/LTC/Col. (O4-O6) (Operational Plans)

17. Value Proposition Canvas - COCOM
Products
& Services
Web/Desktop
Application
- Judge potential
effect and risk of
operations
- Approve/reject
plans
Customer
Jobs
- Less familiarity with
internet as an operational
environment
- Needs high level summary
of how users use technology
Gains
Pains
Gain
Creators
Pain
Relievers
- Organizational chart of
technology broken down by
functional purpose
- Timeline-based viewer
-Illustrates evolution of info
dissemination networks
- Provides insight for
mapping “online terrain”
Visualize high-level
information about how
regional users use
technology.
COCOM - Brigadier General (O7) (Decision Maker)

18. Value Proposition Canvas - Political Campaign
Products
& Services
Web/Desktop
Application
-Identify opposition
narratives on SM
channels
-Craft responses to
respond w/short
turnaround
Customer
Jobs
-Limited lead time in
anticipating changes from
opposition campaigns
-New stories/themes can
derail campaign momentum
without a targeted response
Gains
Pains
Gain
Creators
Pain
Relievers
-Faster response time to
opposition’s campaigns
-Maintain the initiative in media
environment by rapidly
responding to threats
-Early detection of
emerging opposition
narratives on SM
-Insight into evolving
environment
Counter opposition
themes quickly, identify
newly spreading
rumors that could
undermine campaign.
Political Campaigns - Opposition Analysis/Campaign Director

19. Value Proposition Canvas - Consumer Product Company
Products
& Services
Web/Desktop
Application
-Monitor brand
integrity and
customer themes
-Identify and address
emerging problems
Customer
Jobs
-Identify themes about
corporate brand, pinpoint
problem areas early
-Avoid negative narratives
from becoming costly PR or
marketing problems
Gains
Pains
Gain
Creators
Pain
Relievers
-List of customer narratives
about brand that are being
posted online
-Proactive response to
emerging brand issues, better
customer satisfaction
-Early warning about
negative themes
-Identify problem areas,
protect corporate brand
from costly problems
-Protect brand integrity,
create greater
shareholder value by
addressing customer
concerns
Consumer Product Company - CMO/Marketing Organization