Web & Social Media Analytics Previous Year Question Paper.pdf
Aggregate db Lessons Learned H4Dip Stanford 2016
1. 87Interviews
AggregateDB
Sponsor: Conflict and Stabilization Operations (CSO)
Point of Contact: Santiago Stocker
Original Challenge:
CSO needs to
acquire more
information about
informal leaders.
Final Problem Solved:
American Diplomats
have this information,
but it is unstructured,
inefficient and difficult
to access.
2. MS Computer
Science
Software Engineer,
Machine Learning
NLP & Data Mining
Meet Our Team
Shutong Zhang
BS Computer Science
Front-End, UI/UX
Software Engineer
Christian Valadez
BA History
Minor Arabic
Regional Expert
Nathan Hausman
4. A Case Study : Crisis in Algeria
21 Days until shutdown....
Helicopter
Company
5. A Case Study : Crisis in Algeria
Embassy in
Algeria
18 Days until shutdown....
Helicopter
Company
6. A Case Study : Crisis in Algeria
Embassy in
Algeria
16 Days until shutdown....
Diplomat: “Who
Do I Talk To?”
Helicopter
Company
7. A Case Study : Crisis in Algeria
Embassy in
Algeria
0 Days until shutdown....
Diplomat: “Who
Do I Talk To?”
Helicopter
Compayny
Policy Maker:
Informed Diplomat
8. A Case Study : Crisis in Algeria
Embassy in
Algeria
0 Days until shutdown....
Diplomat: “Who
Do I Talk To?”
Helicopter
Company
Policy Maker:
Informed Diplomat
“Real people and real companies suffer
when diplomats can’t find the right people
at the right time.”
11. MVP 1: Connecting CSO to Embassy
● Reviewed With 2 policy makers, 8 diplomats
Uniform feedback from Diplomats in Africa:
“I do not have time to reupload this information.”
x10
“Just make the policy makers read my cables.”
13. MVP 2: Summarizing Classified Cables
Use Natural Language
Processing to “scrape”
key
- Names
- Locations
- Dates
14. MVP 2: Summarizing Classified Cables
● Reviewed with 10 diplomats, 5 desk officers and 2
technologists
Diplomat: “Foggy Bottom is a cable monster.”
x10
Desk Officer : “Cables are classified,
unaggregated and unstructured.”
Policy Maker: “Cables don’t show
relationships I need to know.”
19. MVP 3: Contact Network Map
● Reviewed With 8 diplomats
Diplomats (Political/Economic):
“I really only need this to map
my 10 closest contacts.”
“Finally, a product that addresses the
needs of diplomats on the ground!
22. Beneficiaries
Mission AchievementMission Budget/Costs
Buy-In/Support
Deployment
Value PropositionKey Activities
Key Resources
Key Partners
CSO:
Santiago - Writing report about Boko Haram
Tyler - Analyzing relationship between known
members of BH and unclassified actors
Will Meeker - Travels to Nigeria for 2 years to
generate contacts for CSO, get Santiago data
Embassy Staff:
LES: Locally engaged staff
Caroline: Economic/ Political Officer at small
Embassy in stable country
James/Steve: Economic/ Political Officer at large
Embassy in an unstable country
Nick: Public Diplomacy Officer at busy Embassy
S/ES& Special Projects
Kapil Gupta: Commitment to show off our
demonstrate prototype
John Janek:Will send our prototype to
Mauritania Embassy friends w/DCM
Foreign Nationals:(OMS): Office
Management Specialists and LES
To implement and upkeep
program at each embassy with
updated information about
contacts from events/meetings
Individual FSO buy-in
DCM to okay program at 1
embassy as test case and
delegate time for FSO’s and LES
to test/use
Kapil Gupta: Commitment to
show off our demonstrate
prototype
CSO Sponsor: Santiago Stocker
IDEO team leader for QDDR
A Dev Garden Consulting: TAP,
SBIR, Accelerator/Combinator
and Angel Funding consulting,
connections, proposal writing
Beta starts in Nigeria
Embassy, with Vanguard
newspaper(English) and
twitter hexagon of Abuja
Target 8 Countries in MENA-
Saudi, Israel, Qatar... when
NLP+AI perfected, final
contract from DOS when
product 3.0 ready for
world-wide deployment
Need CSO to continue
connecting us upstream w/
former clients and
downstream with data
sources in field
DCM initial support, for 1 or
more POL/ECON sections
FSO to implement first 10
names that come to mind,
LES long term key holders
Source
Aggregation:
Gives organizations the ability to
identify most important actors in
rapidly evolving situations using
data mined from a breadth of
sources.
Hierarchy
visualization:ensure
people from all backgrounds can
analyze and filter relevant data
depending on their needs.
Extraction/
Suggestion: Perform
natural language processing
on news articles, twitter and
etc to extract categories and
information for informal
leaders. Suggest novel
contacts for each FSO based
on known network.
Improve CSO’s ability to write relevant reports by giving them tools for
passive “granular” data collection w/o Embassy involvement
Provide long-term contact management, node mapping software,
hierarchical ranking software, news feed of recent activity of known contacts
new contacts discovery
Contact suggestion that cuts reliable informant acquisition time in half
Current contact
management and
information collection
New contact discovery
and understanding
Previously published unclassified
reports commissioned by DOS
Nigerian news papers
gurdian, sahara Times, Punch,
Business day, Vanguard
Polis Backbone
SV companies /DOS projects who
are working in a similar space
aka SalesForce and Transparency
team
(partnerships)
Immediate: MVP--> Prototype that integrates w/ DOS Blackberry
Classic and Windows XP... sweat equity
FY1: Deployment to 1 Embassy: Travel to Target Embassy, Legal
Fees for DOS Cloud Review, 200 contract hours with AI & UIXUX
FY2-4: NLP Non-Latin Script- LinguaSys GlobalNLP, Full Time AI
expert to perfect contact suggestion, travel for sales/lobbying
23. Beneficiaries
Mission AchievementMission Budget/Costs
Buy-In/Support
Deployment
Value PropositionKey Activities
Key Resources
Key Partners
CSO:
Santiago - Writing report about Boko Haram
Tyler - Analyzing relationship between known
members of BH and unclassified actors
Will Meeker - Travels to Nigeria for 2 years to
generate contacts for CSO, get Santiago data
Embassy Staff:
LES: Locally engaged staff
Caroline: Economic/ Political Officer at small
Embassy in stable country
James/Steve: Economic/ Political Officer at large
Embassy in an unstable country
Nick: Public Diplomacy Officer at busy Embassy
S/ES& Special Projects
Kapil Gupta: Commitment to show off our
demonstrate prototype
John Janek:Will send our prototype to
Mauritania Embassy friends w/DCM
Foreign Nationals:(OMS): Office
Management Specialists and LES
To implement and upkeep
program at each embassy with
updated information about
contacts from events/meetings
Individual FSO buy-in
DCM to okay program at 1
embassy as test case and
delegate time for FSO’s and LES
to test/use
Kapil Gupta: Commitment to
show off our demonstrate
prototype
CSO Sponsor: Santiago Stocker
IDEO team leader for QDDR
A Dev Garden Consulting: TAP,
SBIR, Accelerator/Combinator
and Angel Funding consulting,
connections, proposal writing
Beta starts in Nigeria
Embassy, with Vanguard
newspaper(English) and
twitter hexagon of Abuja
Target 8 Countries in MENA-
Saudi, Israel, Qatar... when
NLP+AI perfected, final
contract from DOS when
product 3.0 ready for
world-wide deployment
Need CSO to continue
connecting us upstream w/
former clients and
downstream with data
sources in field
DCM initial support, for 1 or
more POL/ECON sections
FSO to implement first 10
names that come to mind,
LES long term key holders
Source
Aggregation:
Gives organizations the ability to
identify most important actors in
rapidly evolving situations using
data mined from a breadth of
sources.
Hierarchy
visualization:ensure
people from all backgrounds can
analyze and filter relevant data
depending on their needs.
Extraction/
Suggestion: Perform
natural language processing
on news articles, twitter and
etc to extract categories and
information for informal
leaders. Suggest novel
contacts for each FSO based
on known network.
Improve CSO’s ability to write relevant reports by giving them tools for
passive “granular” data collection w/o Embassy involvement
Provide long-term contact management, node mapping software,
hierarchical ranking software, news feed of recent activity of known contacts
new contacts discovery
Contact suggestion that cuts reliable informant acquisition time in half
Current contact
management and
information collection
New contact discovery
and understanding
Previously published unclassified
reports commissioned by DOS
Nigerian news papers
gurdian, sahara Times, Punch,
Business day, Vanguard
Polis Backbone
SV companies /DOS projects who
are working in a similar space
aka SalesForce and Transparency
team
(partnerships)
Immediate: MVP--> Prototype that integrates w/ DOS Blackberry
Classic and Windows XP... sweat equity
FY1: Deployment to 1 Embassy: Travel to Target Embassy, Legal
Fees for DOS Cloud Review, 200 contract hours with AI & UIXUX
FY2-4: NLP Non-Latin Script- LinguaSys GlobalNLP, Full Time AI
expert to perfect contact suggestion, travel for sales/lobbying
Value Proposition
Extraction/Suggestions
Perform natural language processing on
twitter and RSS feeds to extract categories
and information for informal leaders.
Suggest novel contacts for each FSO based
on known network.
26. IDEO
Same conclusion
State Department is in need of contact management
solution.
Did IDEO overreach?
IDEO wanted to gut and replace entire workflow, instead of
addressing specific pain points.
What we learned
Do not interrupt people’s existing workflow.
1
2
3
28. Final MVP: Contact Management & Suggestion
Twitter
Known
Contacts
People You Should Connect
With
Connect
Rotimi Akeredolu, Governor-
Elect of Ondo State
Connect
Betty Akeredolu, founder and
president of BRECAN, married
to Rotimi Akeredolu
Connect
Simon Lalong, Nigerian lawyer
and politician
See More ->
https://github.com/USStateDept/Polis
29. Deploying AggregateDB to Fix Crisis in Algeria
Embassy in
Algeria
Shorten to 7 Days using AggregateDB
Diplomat: “Who
Do I Talk To?”
Helicopter
Companies
Policy Maker:
Informed Diplomat
30. Deploying AggregateDB to Fix Crisis in Algeria
Embassy in
Algeria
Shorten to 7 Days using AggregateDB
Diplomat: “Who
Do I Talk To?”
Helicopter
Companies
Policy Maker:
Informed Diplomat
Deploying AggregateDB To Fix
Diplomacy Crises
● Deployment to target embassy -
Mauritania
● Work w/ IIP for 8 embassy
deployment