HunchWorks: Combining Human Expertise and Big Data
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

HunchWorks: Combining Human Expertise and Big Data

on

  • 2,762 views

O'Reilly Strata Conference ...

O'Reilly Strata Conference
New York City
September 23, 2011

Slides from our talk at Strata about U.N. Global Pulse's HunchWorks initiative, a system designed to detect and mitigate emerging global crises before they occur.

Adaptive Path helped Global Pulse work through the messy human challenges of the HunchWorks experience, including establishing trust with the system and fostering a community of experts with complementary skills.

Chris van der Walt (U.N. Global Pulse)
Dane Petersen (Adaptive Path)
Sara Farmer (U.N. Global Pulse)

Statistics

Views

Total Views
2,762
Views on SlideShare
2,762
Embed Views
0

Actions

Likes
0
Downloads
16
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

HunchWorks: Combining Human Expertise and Big Data Presentation Transcript

  • 1. HunchWorks:Combining human expertise and big dataSeptember 23rd, 2011 www.unglobalpulse.org @unglobalpulse #hunchworks
  • 2. Chris van der Walt,Project Lead, HunchWorks, Global Pulse@cvanderwaltDane Petersen,Experience Designer, Adaptive Path@thegreatsunraSara Farmer,Chief Platform Architect, Global Pulse@bodaceacat
  • 3. Harnessing innovationto protect the vulnerable
  • 4. Vulnerability:The risk of households falling in or remaining inpoverty due to idiosyncratic or aggregate hazards.Source: Professor Wim Naudé, United Nations University
  • 5. 1.373billion peopleliving on less than $1.25 per dayand billions more at riskof falling into extreme povertySource: The World Bank
  • 6. Information Gap 1 YEAR 2 YEARS 3 YEARS 4 YEARS HUNGER AND POVERTY EDUCATION GENDER EQUALITY CHILD MORTALITY MATERNAL HEALTH HIV AND MALARIA Source: Millennium Development Goals Report 2011
  • 7. Ok, so now what?
  • 8. OPPORTUNITY 1:Proliferation of sensors‣ millions of networked sensors are being embedded in the physical world in devices such as mobile phones and automobiles, sensing, creating, and communicating data
  • 9. OPPORTUNITY 2:Big data‣ 30 billion pieces of content shared on Facebook every month.‣ 40% projected growth in global data generated per year vs 5% in global IT spending.‣ increasingly massive amount of information out there, problem is separating signal from noiseSource: McKinsey Global Institute (2011) Big data: The next frontier for innovation, competition, and productivity
  • 10. OPPORTUNITY 3:Social & collaboration technologies‣ we can engage human beings all over the world and leverage their situational awareness, expertise and capacity.
  • 11. OPPORTUNITY 4:Assistive reasoning‣ human capacity to sift through data to detect and understand meaningful connections can be vastly magnified‣ combine human understanding and computational reasoning capacity to take advantage of the strengths of both
  • 12. OPPORTUNITY 5:People‣ 5 billion mobile phones‣ the international community has staff on the ground in many developing countries‣ the reach of the organization can be increased through engaging others outside of the organization via social technologies‣ challenge is communication
  • 13. Weak signals:Advanced, noisy and socially situatedindicators of change in trends and systemsthat constitute raw informational materialfor enabling anticipatory action.Wikipedia (2011) Future studies. Available at http://en.wikipedia.org/wiki/Future_studies
  • 14. HunchWorks Overview
  • 15. 17What is HunchWorks?‣ a hypothesis management system to detect weak signals‣ both humans and data mining tools (wrapped as intelligent agents) can post ‘hunches’ about anomalous or interesting data and observations that might indicate that a social crisis is starting to unfold.‣ a tool to engage and connect the global network of UN and international development field staff with the greater institution to rapidly collect information about emerging situations
  • 16. 18The hunch mechanism1. Detect an interesting “weak signal”2. Hypothesize (make a hunch) about the nature of the signal3. Share the hunch with the network4. Hunch acts as an attractor for related signals and other evidence5. Engage communities of practice to support rapid verification through the social graph6. Use confirmed hunch as a basis for further action and to refine trust within the system
  • 17. HunchWorks: a use case
  • 18. 20Kissa: analystKissa works in Pulse Lab, Uganda.She notices an increase in populationmovements from Kenya and suspects thatit is in response to the unfolding droughtin the Horn of Africa but lacks concreteevidence.She decides to post to HunchWorks togather further evidence.
  • 19. 21
  • 20. 22
  • 21. 23
  • 22. 24
  • 23. 25
  • 24. 26
  • 25. 27
  • 26. 28
  • 27. 29
  • 28. 30
  • 29. 31
  • 30. 32
  • 31. 33Resolving a hunchOnce Kissa has determined that enough credible evidence isamassed, she may opt to change the status of the hunch toreflect the hunch being:• proven• disproven• closedWe are also considering allowing contributors to vote on huncheswhere the hunch owner allows it or for whatever reason is no
  • 32. Designing the HunchWorks Experience
  • 33. People are complicated.
  • 34. Saving the world is complicated.
  • 35. Trying to save the world with people?
  • 36. photo credit: flickr.com/photos/johnmcnab/4248698746
  • 37. photo credit: flickr.com/photos/johnmcnab/4248698746
  • 38. 7 billion people.
  • 39. 7 billion people.5 billion mobile phones.
  • 40. People are more than tailpipes.
  • 41. People are more than tailpipes.7 billion HunchWorks users?
  • 42. From 7 billion to 2,000.
  • 43. HunchWorks Users
  • 44. How?
  • 45. How?
  • 46. How?Why?
  • 47. Tools alone will not be enoughto sustain HunchWorks.
  • 48. Tools alone will not be enoughto sustain HunchWorks.We must build a strong communityaround these tools.
  • 49. Leverage existing connections…
  • 50. Leverage existing connections…and suggest new ones.
  • 51. Connections depend on trust.
  • 52. A picture of a hunch.
  • 53. A picture of a hunch.
  • 54. A hunch is the discussion surrounding it.
  • 55. HunchWorks is an action engine.
  • 56. HunchWorks is an action engine.“Is this hunch true enough todo something about it?”
  • 57. Making that action happen.
  • 58. Open Source and the UN
  • 59. Open Source and the UN Code Jams Hackathons Blue Hacks Licensing Open Innovation Models Git Facilitation Repositories
  • 60. Integration
  • 61. Algorithms‣ Similarity and complementarity metrics‣ Relevance‣ Credibility‣ Hunch splitting, merging and clustering‣ Mixing human insight with big data results
  • 62. System Issues‣ Security‣ Localisation‣ Federation
  • 63. Federation ? DATA FEEDHUNCHWORKS ? HUNCHWORKS ? Intermittent connection HUNCHWORKS ? DATA SITES ? (OPEN DATA, MAPS, GAZETTEERS ETC) HUNCHWORKSHUNCHWORKS ? No connection/sneakerware HUNCHWORKS
  • 64. Timeline
  • 65. Roadmap‣ to November 2011 (prototype delivery) Hunchworks is in the Proof of Concept stage.‣ From November 2011 til January 2012 (first release) Hunchworks is in seed stage (early development), looking for angel-style funding through UN Foundation.‣ From Feb 2012 till Jan 2013 (first full-featured release) HunchWorks is in early stage development
  • 66. We need your help!‣ Funding‣ Beta Communities‣ Development partners‣ Did I mention funding...?
  • 67. Thank youChris van der Walt@cvanderwaltDane Petersen@thegreatsunra www.unglobalpulse.orgSara Farmer @unglobalpulse@bodaceacat info@unglobalpulse.org