Ubiquitous Angels; ambient sensor networks to crowd source crisis response and community awareness

Ubiquitous Angels; ambient sensor networks to crowd source crisis response and community awareness - Presentation Transcript

Ubiquitous Angels Open Source Bridge Portland Oregon June 2009 http://opensourcebridge.org/sessions/246 twitter.com/anselm blog.makerlab.org anselm hook

What I’ll cover • A look at other similar sensor efforts • A little bit of technicals • Critical perspectives

If • we could see our neighborhoods better • There might be an opportunity to participate • We could better help people in need • We could connect people together with similar needs • We could crowd-source social problems • We all know this.

A big use case • #iranelection • where are the protests? • are there speciﬁc instances of vote fraud? • who are the key twitterers? • are there speciﬁc things we can do? • are there local people who can help?

A smaller use case • Restaurants often dump food every night • Gleaners try to collect food • How can we connect dumpers with gleaners?

Other use cases • My child is missing. • Somebody just stole my bike. • Can somebody give me a ride? • Does anybody have a place where I can stay? • Car pooling? • I’d like somebody to share my war stories with.

Crisis at many levels

Where is the real time map of us?

Hard to Automate • ‘Automated maps of feeds look like database barf on a map’ - Chris Blow

Ushahidi Swift Initiative • blog.ushahidi.com/index.php/tag/swift/ • a crisis listening tool • the Mumbai Crisis for example • electoral fraud events such as in Iran • the upcoming quake in San Francisco • crowd source filtering to eliminate noise • crisis editors mark posts up with metadata • first responders can see filtered views

Ushahidi Swift

Swift Control Panel

Code and Praxis

A software stack for a civic lens • search - to help restrict the firehose that is twitter and co. • aggregation - as tuned by search • reaping - throwing away old data as an ongoing process • actual fetching content from twitter and co. • geolocation of content by any means possible • natural language parsing for semantic content and meaning • language translation from foreign languages to english • presentation layer for the sit-at-home-view and in-the-field views • curatorial role in marking data up or changing metadata • first responder views, feedback and loop closure

Roles • Participant - somebody involved in an issue • Reporter - as an on the ground witness • Curator - acting as an editor to ﬁlter reports • Matchmaker - matching reports with potential responders • Responder - going into the ﬁeld to help resolve a situation.

John Nunemaker’s Twitter Gem • http://github.com/jnunemaker/twitter • http://apiwiki.twitter.com • h = Twitter::HTTPAuth.new(name,pass) • twitter = Twitter::Base.new(h) • results = twitter.user_timeline(:id =>‘joi’) • results.each { |twit| puts twit.location }

Social Graph is easy • twitter.friends(:id => ‘joi’ ) • Try to rate limit queries • Build your own model of the social graph • Easy to do, crucial to provide subjectivity • An alternative is to go through YQL • Social graph issues underpin many projects

Fetching social graph

YQL for the win • yql = “http://query.yahooapis.com/v1/public/yql?q=” • query = “select * from party where id = ‘joi’ or id = ‘meedan’ or id = ‘bob’” ... • schema = “use ‘http://angel.makerlab.org/yql/ twitter.user.timeline.xml’ as party;” • url = “#{yql}#{url_escape(schema+query)} • url = url + “format=json”; • data = JSON.parse(open(url).read) • data[“query”][“results”].each { |i| puts i.title } • a higher rate limit ~may~ allow more data access

Metacarta Geolocation • request = Net::HTTP::Get.new(‘/webservices/ GeoTagger/JSON/basic?version=1.0.0&doc=banff!’) • request.basic_auth name,pass • h = Net::HTTP.start(‘ondemand.metacarta.com’) • response = h.request(request) • data = JSON.parse(response.body) • latitude = data[“locations”][0][“Centroid”][“Latitude”]

Yahoo Placemaker • geolocating posts by brute force • url = URI.parse(‘http://wherein.yahooapis.com/v1/document’) • args = {‘documentContent’ => ‘banff’, ‘documentType’ => ‘text/plain’, ‘appid’=>mysecretappkey } • r = Net::HTTP.post_form(url,args) • doc = Hpricot::XML(response.body) • latitude = (node/:latitude).innerHTML • longitude = (node/:longitude).innerHTML

acts_as_solr for search • picky and brittle • doesn’t play well with others • can geo-constrain searches more or less • query = “volunteer lat:[42 TO 60]” • Model.rebuild_solr_index() • results = Model.ﬁnd_by_solr(query) • results.docs { |data| puts data.name }

engtagger for meaning extraction • engtagger.rubyforge.org • phrase = “is anyone awake out there? i just got mugged in front of my house and could really use help calling the police” • tagger = EngTagger.new • tagged = tgr.add_tags(phrase) • nouns = tagger.get_nouns(tagged) • {“police”=>1,”help”=>1, ... }

Clustering Approaches to clump activity • Carrot2 • Amazon Turk • Human ‘volunteer turk’ approach • Contextual Network Graphs • Latent Semantic Indexing

Carrot2 • project.carrot2.org • Submit your requests directly via http • Can only cluster 1000 documents at a time • Used as a way to get even more metadata • Forms clusters dynamically

Brief Comments on Visualization • Views are political • Subjectivity is crucial • Speed of user interface; probably gamelike • Tactical; showing immediate context • http://map.cartagen.org for example

Still looking for similar efforts - any advice? • I call this class of services ‘real time brokerages’. But I don’t see many of them. • Craigslist? • Shizzow? • IceCondor? • Other?

Criticism • Side effects often dominate over intended consequences of any project or endeavor. • Abstract views may make us callous or may badly reflect ground truth. • Ignorance is bliss. The world is filled with sob stories. Best to not dwell? • Such services may be used solely for the most banal aspirations and goals. • Struggle may be important - making things too easy may harm fitness and lower diversity of skills and abilities over long term. • What about poor people who are outside of any implied technological social network? • Why not just help people around you? • Any technology should go hand in hand with day to day personal practice that is unmediated? • Feedback loops may be created that accelerate and disrupt society. • Virtual and visual only for curators; not tactile; uses only one sense. A concern?

links http://swiftapp.org/ http://crisismapping.ning.com/ http://ushahidi.com http://ncfoodnet.com/ http://www.flickr.com/photos/unthinkingly/3507131912/ http://cartagen.net http://blog.twittervotereport.com/ http://blog.ushahidi.com/?s=swift http://www.qapture.net/lists/ http://www.marketingpilgrim.com/2008/12/social-media-monitoring-tools.html http://tinker.com/users/find/?findName=&findEmail=&findLocation=banff&profType= http://www.crunchgear.com/2007/04/02/patent-monkey-geographic-cell-messaging-with-squareloop/ http://www.dantynan.com/2009/04/02/twitter-added-to-federal-emergency-response-network/ ( April Fools Gag ) http://stephensonstrategies.com/directory-of-major-blog-posts/boy-did-i-underestimate-twitters-value-in-a-disaster/ http://www.comminit.com/en/node/290987/307 http://yvoschaap.com/twitterthoughts/?map=1 http://mashable.com/2009/03/16/twitter-visualizations/ http://theeword.blogspot.com/2009/06/best-twitter-applications-for-social.html http://monitter.com/ http://maphawk.blogspot.com/2009/01/geographic-search-engine-for-news.html http://muti.co.za/static/newsmap.html http://www.planetizen.com/node/38227 http://www.localtweeps.com/ http://keithhopper.com/taxonomy/term/28 http://keithhopper.com/blog/finding-twitterstars http://pipes.yahoo.com/pipes/pipe.run?Location=portland%2C+oregon&radius=15&_render=rss&_id=2fdf06974be88f13a63937fb4d36e499 http://delicious.com/tag/twitter+map http://www.twitip.com/twellowhood-find-local-twitter-users/ http://www.tweepz.com/feed?q=loc%3Abanff&C= http://www.twitterlocal.net/ ( old ) http://www.frontlinesms.com/

thanks http://github.com/ anselm/angel http://github.com/ anselm/SpinnyGlobe