Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Swift Update May 6

1,414 views

Published on

This was prepared for a presentation of http://swiftapp.org at InSTEDD. It gives an overview of the latest designs (which are changing quickly) and a backgrounder on how we got to this point.

Published in: Self Improvement, Technology
  • Be the first to comment

Swift Update May 6

  1. 1. SWIFT FOSS data triage platform Crowdsourced Crisis Data Media Analytics (Africa) (Middle East)
  2. 2. May 6, 2009 Team Update Where we are today? Reeling from an awesome conference. Enjoying the good company of InSTEDD. International coworkers in person for the first time. Swift is going live after months of collaborative design work. Numerous deployments are scheduled in very near term. Currently iterating “live” for votereport.in *gulp :)
  3. 3. Huh?
  4. 4. the deployment problem
  5. 5. the deployment problem If you have to deploy, you have a problem
  6. 6. How might we imagine a better crisis reporting tool? By designing a better crisis listening tool.
  7. 7. why quot;At present, early warning units within the UN ... use manual labor to collect relevant information from online sources. Most units employ full-time staff for this, often meaning that 80% of an analyst’s time is actually used to collect pertinent articles and reports, leaving only 20% of the time for actual analysis, interpretation and policy recommendations. We can do better. Analysts ought to be spending 80% of their time analyzing.quot; Patrick Meier, “Crimson Hexagon: Early Warning 2.0?” February 17, 2009 http://earlywarning.wordpress.com/2009/02/17/crimson-hexagon-early-warning-20
  8. 8. We need new listening tools. Ability to report eyewitness stories Ability to act on them 40000 30000 20000 10000 0 1910 1940 1980 2010
  9. 9. Signal is Overwhelmed by Noise
  10. 10. How much do you need to know to act?
  11. 11. Well, how much data can you take?
  12. 12. What can we build when we focus on the user interface needs of citizen editors? How does this changes our idea of an “analyst”? Can data entry actually make an impact? Could it actually be ... enjoyable to use?
  13. 13. What if we assume that citizen editors deserve the best?
  14. 14. What if we treat their data with the same respect?
  15. 15. STRUCTURED PUBLISHING The revolution will not be transmitted via giant .xls files.
  16. 16. Output: The Crisis API The resulting crisis database is open, with distributed storage and standards-based portability, based on: ICAL / FOAF / Dublin Core / Etc. via RSS / CSV / JSON / SQL / Etc. and all of it is handled by Freebase.com
  17. 17. Swift the method has evolved into a platform: 1. data gathering engine (aggregator) 2. data structuring tool (wiki) 3. most importantly, an API for crisis data
  18. 18. Swift is an aggregator with entity extraction By “roping together” relevant feeds, then parsing their content, we can get a rich database of people, places and organizations in real time. We are working with Freebase.com and Calais. Swift is mostly a Rails app, Twitter Vote Report.
  19. 19. the sweeper
  20. 20. CITIZEN EDITORS Good with Data? Now you can help.
  21. 21. Swift is designed for Improving information findability in a crisis Making it easier to find things that you didn't know you were looking for Better understanding media from other parts of the world Making urgent data more sharable (structured, published and accessible) Making it more obvious what information is missing about an crisis Promoting the work of eyewitnesses with prepared crisis editors Expanding the grassroots reporting network Preserving information across crises
  22. 22. Al Jazeera deployment
  23. 23. Indian Elections coverage
  24. 24. Meedan.net
  25. 25. swiftapp.org
  26. 26. Tracker InSTEDD is good at listening.
  27. 27. Evolve
  28. 28. Vine
  29. 29. Real-timelines
  30. 30. Realtime Aggregators
  31. 31. Advanced analytics are mainstreaming
  32. 32. Our FOSS Toolkit is Growing Really Fast
  33. 33. But tending data is not new
  34. 34. Mapping is not new
  35. 35. From the Swift perspective, if you want to find the new opportunities for change ... Focus on speed and people. We can work together faster than ever before.
  36. 36. Human-curated data can happen in realtime with the help of entity recognition and reconciliation. Really, they just help us work faster.
  37. 37. Recognition saves time
  38. 38. Reconciliation Saves Time
  39. 39. WHO ARE THE EDITORS?
  40. 40. CITIZEN EDITORS Text
  41. 41. GEEK OUT A SEC Every incoming is parsed into an object with attributes: 1. URI.body (the text of the url) 2. URI.rating (anyone can rate through a web UI) 3. URI.submitters (anyone who linked to it) 4. URI.history (every revision preserved) 5. URI.tags (added by humans and machines)
  42. 42. PFIF people finder interface format used for 90,000 entries after Katrina
  43. 43. Grassroots reporting: Database-driven journalism and data curation.
  44. 44. TWITCH GAMES
  45. 45. NEWS API, OH MY
  46. 46. Weather Related Disasters Urban and Rural Fire Earthquake Economic Crisis Homelessness War Possibilities with speed: Internal Displacement & Refugee Environmental Crisis Aviation Disasters Highway Accidents What to do in a ___ based on my location? Volcano Report a ____ location Missing Person Report a ___ accident. Structure Collapse ___ detection and reporting. Railroad Disasters Health Disasters “There was a ____. Are you ok?” alerts. Power Outage Neighborhood-level ____ warnings. Explosions Floods AIDS Famine Landslide Stampede Foot and Mouth Avalanche Fatal Accident Tsunami Disaster Bird Flu
  47. 47. Swift’s realtime strengthens the Ushahidi alert cycle Subscribe to disaster keyword based alerts about an emergency eg: “san francisco, earthquake”
  48. 48. Possible with speed:
  49. 49. See Also Crimson Hexagon state of the art NLP. VRA's GeoMonitor, a natural language parser that reads the headlines of Reuters and AFP news wires and codes to state quot;who did what, to who, where and when?quot; Rated quot;virtually identicalquot; to human event summarizing. JRC's European Media Monitor, which can parse thousands of different news sources but faces limitations since analysts still need to read each article to understand the nature of the terrorist event. http://emm.jrc.it/NewsBrief/clusteredition/en/latest.html Tabari text parsing engine, quot;event data coder that has been used in at least five NSF-sponsored projects and produced data used in a number of refereed articles in political science.quot; FORECITE Forecasting of Crises and Instability Using Text-Based Events, developed by the US Center for Army Analysis (CAA) UN's HEWSweb A relatively new early warning system Biowarn Textual analysis with infectious disease focus Cewarn FAST Comprehensive system by Swisspeace
  50. 50. A minimal implementation: an analogy We’ll put a card table at the public library and work on reports with highlighters, put everything into a card catalog and leave it in the library.

×