SFUSGS

952 views

Published on

SF USGS Earthquakes via Twitter: How to make a simple Twitter bot to aggregate news.

Published in: Technology, Travel
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
952
On SlideShare
0
From Embeds
0
Number of Embeds
87
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Use d to be you could look out the window…
  • I’m from Y!R MESS IEG.
  • Toms first quake.
  • SFUSGS

    1. 1. @sfusgs<br />Geo-tweet my quake…<br />@ayman • yahoo! research<br />http://en.wikipedia.org/wiki/File:San_francisco_1906_earthquake.jpg<br />
    2. 2. Zync • Y! Messenger<br />
    3. 3. FireEagle<br />
    4. 4. FireEagle<br />@ayman<br />
    5. 5. http://bit.ly/statler<br />
    6. 6. shake and rumble in 2GA<br />
    7. 7. http://earthquake.usgs.gov/earthquakes/recenteqsus/Quakes/nc71336726.php<br />
    8. 8. Click through maps…<br />
    9. 9. A Want Ad<br />
    10. 10. USGS lists every quake<br />
    11. 11. http://earthquake.usgs.gov/earthquakes/catalogs/<br />
    12. 12. http://earthquake.usgs.gov/earthquakes/catalogs/<br />
    13. 13. Trust me, it is just XML<br />
    14. 14. Yahoo! Search goes to YDN<br />
    15. 15. Google Search goes to YDN<br />
    16. 16. Bing goes to YDN<br />
    17. 17. Using ElementTree<br />
    18. 18. The flow was simple<br />CRON this::<br />cUrl XML feed from USGS<br />find local quakes<br />bit.ly the quake permalink<br />if not duplicate<br />record this quake<br />tweet this quake<br />
    19. 19. def fetch_quakes():<br />def fetch_quakes():<br />api = bitly.Api(login=’ABCDEF&apos;, apikey=’12345678901234567890&apos;)<br />rss = parse(urllib.urlopen(USGS_URL)).getroot()<br /> quakes = []<br /> for element in rss.findall(&apos;{%s}entry&apos; % USGS_NS):<br /> title = element.findtext(&apos;{%s}title&apos; % USGS_NS)<br /> if (title.find(&quot;San Francisco Bay&quot;) &gt; -1):<br /> point = element.findtext(&apos;{%s}point&apos; % GEORSS_NS)<br /> points = point.split(None, 1)<br /> link = element.find(&apos;{%s}link&apos; % USGS_NS)<br />href = &apos;http://earthquakes.usgs.gov%s&apos; % link.get(&apos;href&apos;)<br />href = api.shorten(href)<br /> if (is_duplicate(href) is False):<br />f = open(‘urls.txt&apos;, &apos;a&apos;)<br />f.write(&apos;%s &apos; % href)<br />f.close()<br /> summary = element.findtext(&apos;{%s}summary&apos; % USGS_NS)<br /> summary = summary.split(&quot;&lt;br&gt;&quot;)[1]<br /> summary = summary.replace(&apos; at epicenter&lt;/p&gt;&lt;p&gt;&apos;, &apos; &apos;)<br /> summary = strip_tags(summary)<br />quakes.append({&apos;title&apos;: element.findtext(&apos;{%s}title&apos; % USGS_NS),<br /> &apos;link&apos; : href,<br /> &apos;updated&apos; : element.findtext(&apos;{%s}updated&apos; % USGS_NS),<br /> &apos;long&apos; : points[1],<br /> &apos;lat&apos; : points[0],<br /> &apos;elev&apos; : element.findtext(&apos;{%s}elev&apos; % GEORSS_NS),<br /> &apos;summary&apos; : summary})<br /> return quakes<br />
    20. 20. def is_duplicate(url):<br />def is_duplicate(url):<br /> from subprocess import call<br />if(os.path.exists(‘urls.txt&apos;) is False):<br /> call([&apos;/usr/bin/touch&apos;, ‘urls.txt&apos;])<br />r = call([&apos;/bin/fgrep&apos;,<br />url,<br /> ‘urls.txt&apos;])<br /> if (r == 0):<br /> return True<br /> else:<br /> return False<br />
    21. 21. def send_tweet(t):<br />def send_tweet(t):<br /> import httplib2<br /> username = ’ABCDEFG&apos;<br /> password = ’12345678901234567890&apos;<br />msg = (&apos;%s %s %s&apos; % (t[&apos;title&apos;], t[&apos;link&apos;], t[&apos;summary&apos;]))<br /> http = httplib2.Http()<br />http.add_credentials(username, password)<br /> response = http.request(<br /> &quot;http://twitter.com/statuses/update.xml&quot;,<br /> &quot;POST&quot;,<br />urllib.urlencode({&quot;status&quot;: msg, <br /> &quot;lat&quot;: t[&apos;lat&apos;], <br /> &quot;long&quot;: t[&apos;long&apos;]})<br /> )<br />SEE: http://code.google.com/p/httplib2/<br />ps: you shouldn’t store your passwords like that…this is just an illustration<br />
    22. 22. fin<br />
    23. 23. Crunched<br />
    24. 24. USGS on Open Gov’t<br />
    25. 25. Open Questions?<br />How can other local sources be disseminated?<br />How can mass media be filtered so it can be consumed in a similar manner?<br />Can we collect information around the news? For example, find all the tweets that relate to a particular quake.<br />Thanks, @ayman.<br />

    ×