Your SlideShare is downloading. ×
0
22 May 2014 CDE competition: Information processing and sensemaking presentation
22 May 2014 CDE competition: Information processing and sensemaking presentation
22 May 2014 CDE competition: Information processing and sensemaking presentation
22 May 2014 CDE competition: Information processing and sensemaking presentation
22 May 2014 CDE competition: Information processing and sensemaking presentation
22 May 2014 CDE competition: Information processing and sensemaking presentation
22 May 2014 CDE competition: Information processing and sensemaking presentation
22 May 2014 CDE competition: Information processing and sensemaking presentation
22 May 2014 CDE competition: Information processing and sensemaking presentation
22 May 2014 CDE competition: Information processing and sensemaking presentation
22 May 2014 CDE competition: Information processing and sensemaking presentation
22 May 2014 CDE competition: Information processing and sensemaking presentation
22 May 2014 CDE competition: Information processing and sensemaking presentation
22 May 2014 CDE competition: Information processing and sensemaking presentation
22 May 2014 CDE competition: Information processing and sensemaking presentation
22 May 2014 CDE competition: Information processing and sensemaking presentation
22 May 2014 CDE competition: Information processing and sensemaking presentation
22 May 2014 CDE competition: Information processing and sensemaking presentation
22 May 2014 CDE competition: Information processing and sensemaking presentation
22 May 2014 CDE competition: Information processing and sensemaking presentation
22 May 2014 CDE competition: Information processing and sensemaking presentation
22 May 2014 CDE competition: Information processing and sensemaking presentation
22 May 2014 CDE competition: Information processing and sensemaking presentation
22 May 2014 CDE competition: Information processing and sensemaking presentation
22 May 2014 CDE competition: Information processing and sensemaking presentation
22 May 2014 CDE competition: Information processing and sensemaking presentation
22 May 2014 CDE competition: Information processing and sensemaking presentation
22 May 2014 CDE competition: Information processing and sensemaking presentation
22 May 2014 CDE competition: Information processing and sensemaking presentation
22 May 2014 CDE competition: Information processing and sensemaking presentation
22 May 2014 CDE competition: Information processing and sensemaking presentation
22 May 2014 CDE competition: Information processing and sensemaking presentation
22 May 2014 CDE competition: Information processing and sensemaking presentation
22 May 2014 CDE competition: Information processing and sensemaking presentation
22 May 2014 CDE competition: Information processing and sensemaking presentation
22 May 2014 CDE competition: Information processing and sensemaking presentation
22 May 2014 CDE competition: Information processing and sensemaking presentation
22 May 2014 CDE competition: Information processing and sensemaking presentation
22 May 2014 CDE competition: Information processing and sensemaking presentation
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

22 May 2014 CDE competition: Information processing and sensemaking presentation

761

Published on

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
761
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
39
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Information Processing and Sensemaking C4ISR Concepts and Solutions Tranche 4 22 May 2014 © Crown copyright 2013 Dstl 27 May 2014 UK OFFICIAL dstlsensors@dstl.gov.uk
  • 2. Contents • Military Advisor Brief: Setting the scene • Technical Brief: The technical approach, challenges and some of our work • Customer Brief: A technology transition model and a view from the customer © Crown copyright 2013 Dstl 27 May 2014 UK OFFICIAL
  • 3. Military Advisor Brief The Problem Space 22 May 2014 © Crown copyright 2013 Dstl 27 May 2014 UK OFFICIAL
  • 4. Setting the Scene • Information – Context + Analysis • Intelligence • Situational Understanding • Decision Making – Human endeavour – “More” data is not necessarily better – Observe, Orient, Decide, Act © Crown copyright 2013 Dstl 27 May 2014 UK OFFICIAL Observe Orient Decide Act
  • 5. • Target based approach – E.g. Detect enemy tanks The Past © Crown copyright 2013 Dstl 27 May 2014 UK OFFICIAL *Decision*
  • 6. The Present • Target-based approach does not work – Complex problems – Problem-based approach is required – Huge volume of disparate data available © Crown copyright 2013 Dstl 27 May 2014 UK OFFICIAL
  • 7. The Future © Crown copyright 2013 Dstl 27 May 2014 UK OFFICIAL
  • 8. The Future © Crown copyright 2013 Dstl 27 May 2014 UK OFFICIAL
  • 9. The Future © Crown copyright 2013 Dstl 27 May 2014 UK OFFICIAL
  • 10. The Challenge © Crown copyright 2013 Dstl 27 May 2014 Time Amount Capacity of decision maker Data UK OFFICIAL
  • 11. Information processing and sensemaking Technical brief 22 May 2014 © Crown copyright 2013 Dstl 27 May 2014 UK OFFICIAL
  • 12. Technical challenges • Data association and correlation of both unstructured and structured data • Uncertainty propagation and management across multiple data representations • Automated hypothesis generation • Automated learning to understand complex relationships • Autonomous model generation • Techniques that cope with large-scale data © Crown copyright 2013 Dstl 27 May 2014 UK OFFICIAL
  • 13. CDE themed competition Key dates – Launch event: 22 May 2014 – Webinar: 3 June 2014 – Competition close: 26 June 2014 at 5pm – Proof-of-concept research complete: 31 August 2015 © Crown copyright 2013 Dstl 27 May 2014 UK OFFICIAL
  • 14. CDE themed competition • Funding – £600k of funding for this phase 1 CDE competition – £50-100k range per proposal – up to £1M for phase 2 funding • Duration and delivery – up to 12 months duration from September 2014 to September 2015 – ideally with close technical partnering for delivery early and often focussing on prototype code and software, not lengthy literature reviews – if there is background intellectual property (IP), there should be 2 deliverables of a full rights and limited version with background information clearly identified – Final deliverable for phase 1 should be a phase 2 proposal © Crown copyright 2013 Dstl 27 May 2014
  • 15. CDE competition funding • A joint C4ISR concepts and solutions (CCS) and intelligence collection and exploitation (ICE) project competition • Joint MOD funding under: – decision support and experimentation programme • CCS project • Project technical lead – Steven Meers • CDE point of contact – Paul Thomas – command, control, information and intelligence (C2I2) programme • ICE project • project technical lead and CDE point of contact – Warren Marks © Crown copyright 2013 Dstl 27 May 2014 UK OFFICIAL
  • 16. ICE project multi-intelligence work and ethos • The ICE project delivers against the C2I2 programme • The work builds on recent multi-intelligence efforts structured around applied near-term research and development (R&D) and more basic R&D in text analytics, spatio-temporal correlation and data association and: – maintain an operational focus – work in a data-rich environment – don’t lose sight of the art of the possible – use open-source technology where possible – transition technology quickly to operations bearing in mind defence lines of development (DLOD) considerations © Crown copyright 2013 Dstl 27 May 2014 UK OFFICIAL
  • 17. Some of our current multi-intelligence activities © Crown copyright 2013 Dstl 27 May 2014 Network analytics • MAMBA • MAMBA web services • Log, path, web analytics Text analytics • Baleen • Tag Crowd • Dhugal Spatio-temporal correlation • Maritime data association • Event correlation • ENVI services engine Sense-making • BANISH • Virtual toolbox Exploitation • Technology transition • User feedback • B-S-G model UK OFFICIAL
  • 18. Baleen – text processor Datasources Model generation Tag Crowd Baleen – UIMA pipeline Graph store Sesame / Apache Jena Document store MongoDB Search index ElasticSearch Visualisation eg Mamba Search Dhugal Document ingest Entity extraction Relationship extraction © Crown copyright 2013 Dstl 27 May 2014 UK OFFICIAL
  • 19. © Crown copyright 2013 Dstl 27 May 2014 UK OFFICIAL
  • 20. © Crown copyright 2013 Dstl 27 May 2014 UK OFFICIAL
  • 21. Home Office exploitation: © Crown copyright 2013 Dstl 27 May 2014 UK OFFICIAL
  • 22. Home Office exploitation: © Crown copyright 2013 Dstl 27 May 2014 UK OFFICIAL
  • 23. Home Office exploitation: © Crown copyright 2013 Dstl 27 May 2014 UK OFFICIAL
  • 24. Home Office exploitation: © Crown copyright 2013 Dstl 27 May 2014 UK OFFICIAL
  • 25. Log, path and web analytics © Crown copyright 2013 Dstl 27 May 2014 UK OFFICIAL
  • 26. © Crown copyright 2013 Dstl 27 May 2014 UK OFFICIAL ©
  • 27. © Crown copyright 2013 Dstl 27 May 2014 UK OFFICIAL
  • 28. © Crown copyright 2013 Dstl 27 May 2014 UK OFFICIAL
  • 29. © Crown copyright 2013 Dstl 27 May 2014 UK OFFICIAL MAMBA framework encourages light and agile builds from the shared framework with client builds as needed
  • 30. Technical challenges • Data association and correlation of both unstructured and structured data • Uncertainty propagation and management across multiple data representations • Automated hypothesis generation • Automated learning to understand complex relationships • Autonomous model generation • Techniques that cope with large-scale data © Crown copyright 2013 Dstl 27 May 2014 UK OFFICIAL
  • 31. Technical problems • There are MOD joint user-agreed FY2014- 2015 research requirements – data association and spatio-temporal correlation – text analytics – error propagation – activity-based intelligence • understanding the world in terms of scenarios • exploitation of multi-intelligence • fuse the multi-intelligence to identify scenarios of interest – improved sensemaking – problem decomposition © Crown copyright 2013 Dstl 27 May 2014 UK OFFICIAL
  • 32. Example: BANISH – Bayes Net tool What is the probability that it is raining, given the grass is wet? © Crown copyright 2013 Dstl 27 May 2014 UK OFFICIAL • Simple, user-centric tool, for forming Bayes Nets applied to defence intelligence (DI) • Create conceptual models and add variables, dependency, states with values (also using DI uncertainty yardstick) • Example opportunities for further development: – developments in data API – crowd-sourced categorical distributions or PDF – identification of knowledge gaps supporting ‘collect’ – fused graph with other users and data – judgement representation
  • 33. Example: event correlation • Process already collected data and implement correlation metrics producing probability of association between events © Crown copyright 2013 Dstl 27 May 2014 UK OFFICIAL Process adapted from: D Wang, D Pedreschi, C Song, F Giannotti, Human mobility, social ties, and link prediction, KDD ‘11, San Diego, 2011
  • 34. Example: event correlation © Crown copyright 2013 Dstl 27 May 2014 UK OFFICIAL • Example opportunities for development: – global geohashing – different data association and information level fusion algorithms eg weighting – currently tf-idf – probability of association linked to pattern-of-life work and eg links to instantaneous entropy • www.orchid.ac.uk/eprints/69/1/paper_extended_past2.pdf – front-end spatially focussed dashboards and alerting opportunities – fusion of derived information and real data within graphical models
  • 35. Other examples and ideas • From CCS advanced intelligence exploitation academic workshop: – human and computer co-working on uncertainty representation leading to suggested collection parameters for the system and prediction of uncertainty – machine learning techniques applied to already collected and analysed data for future use and model development • Application of other domain knowledge eg biologically inspired algorithms or financial applications, such as used in algo trading, for predicting and forecasting • Deep learning algorithms, association analysis applied to various sources – eg association rules produced with Home Office work • Apply lessons learned from commercial sensemaking – eg NetFlix prize, Amazon analytics, Google Knowledge Graph © Crown copyright 2013 Dstl 27 May 2014 UK OFFICIAL
  • 36. Potential data sources and frameworks • Encourage use of free and open- source software • Encourage use of open standards • Encourage good data Application Programming Interfaces (API) • Authority from MAMBA partnership to provide end-user license agreements (EULA) to successful parties • Apache Unstructured Information Management architecture • Ozone Widget Framework • VAST 2014 data – Previous VAST data sets • Collected CCS datasets from previous trials • Open-source datasets – Wikimapia – DBpedia – Freebase – Twitter – Transport for London Datasets – www.gov.uk – WW1 War Diaries © Crown copyright 2013 Dstl 27 May 2014 UK OFFICIAL
  • 37. The exploitation challenge • There is a lot of brain power and effort already from the community (MOD, open source, academia, industry) • Together we could translate that into capability over the proposal length (Sep 2014 to Sep 2015) • Iterative delivery with a mechanism for exploitation and verification and validation • Opportunity for real quantitative and qualitative feedback • Quickly move from academic papers and low technology readiness level (TRL) work to medium TRL applications with exploitation on operational systems © Crown copyright 2013 Dstl 27 May 2014 UK OFFICIAL
  • 38. Joint Forces Intelligence Group brief 22 May 2014 © Crown copyright 2013 Dstl 27 May 2014 UK OFFICIAL
  • 39. A technology transition model © Crown copyright 2013 Dstl 27 May 2014 UK OFFICIAL

×