Consuming the Twitter Streaming API                           with Ruby and MongoDB                                      J...
Goals                  • Watch for any                          tweet that contains                          certain keywo...
Demo (@aplusk)Friday, August 12, 2011
Twitter Streaming API+                         Mongo DB                     • Twitter Streaming API                     • ...
The many APIs of                              Twitter                     • Twitter Streaming API                     • Us...
Twitter Streaming API                     • Keywords                     • Users                     • Locations          ...
TweetStream gem                          by Michael Bleigh                          https://github.com/intridea/tweetstrea...
Connecting to the                          Twitter Streaming API                     • JSON responses                     ...
Limitations of Twitter                             Streaming API                     • 400 Keywords                     • ...
A Tweet, in JSONFriday, August 12, 2011
Intro to MongoDB                     • NoSQL - what does that mean?                     • Great fit for JSON-oriented appli...
Storing data in MongoDB                     • Native format of MongoDB is BSON,                          similar to JSON  ...
MongoDB + Ruby                     • mongo gem                     • bson_ext gem                     • http://www.mongodb...
Considerations for                                MongoDB                     • Tweets - very verbose JSON                ...
Here’s the code                     • Okay, the whole thing is really done in                          about three lines.F...
Where do you run this?                     • Command line                     • Your own server                     • Hero...
MongoLabFriday, August 12, 2011
Now what?                     • Step 1: Collect Tweets                     • Step 2: ????                     • Step 3: Pr...
To Learn More                     • https://dev.twitter.com/docs/streaming-api/                          concepts         ...
Upcoming SlideShare
Loading in …5
×

Twitter streamingapi rubymongodbv2

906
-1

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
906
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
8
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Twitter streamingapi rubymongodbv2

  1. 1. Consuming the Twitter Streaming API with Ruby and MongoDB Jeff Linwood Lone Star Ruby Conference V August 12, 2011 http://www.jefflinwood.com @jefflinwoodFriday, August 12, 2011
  2. 2. Goals • Watch for any tweet that contains certain keywords • Store those tweets into the MongoDB databaseFriday, August 12, 2011
  3. 3. Demo (@aplusk)Friday, August 12, 2011
  4. 4. Twitter Streaming API+ Mongo DB • Twitter Streaming API • MongoDB • Not a web applicationFriday, August 12, 2011
  5. 5. The many APIs of Twitter • Twitter Streaming API • User Streams • Site Streams (for the big boys) • REST API • Search APIFriday, August 12, 2011
  6. 6. Twitter Streaming API • Keywords • Users • Locations Great photo is by rachel_thecat http://www.flickr.com/photos/23209605@N00/2786126623/Friday, August 12, 2011
  7. 7. TweetStream gem by Michael Bleigh https://github.com/intridea/tweetstreamFriday, August 12, 2011
  8. 8. Connecting to the Twitter Streaming API • JSON responses • HTTP Basic Authentication • One stream per account (dev/prod) • Leave it open! • Don’t constantly reconnect, and if you do, back offFriday, August 12, 2011
  9. 9. Limitations of Twitter Streaming API • 400 Keywords • 5,000 User Ids • 25 Location Boxes • Can ask Twitter for increased accessFriday, August 12, 2011
  10. 10. A Tweet, in JSONFriday, August 12, 2011
  11. 11. Intro to MongoDB • NoSQL - what does that mean? • Great fit for JSON-oriented applications • If you don’t know your schema in advance • Query language • Map ReduceFriday, August 12, 2011
  12. 12. Storing data in MongoDB • Native format of MongoDB is BSON, similar to JSON • Connect to a database (similar to MySQL) • Connect to a collection (created if it doesn’t exist) • Insert JSON (in our case, a tweet)Friday, August 12, 2011
  13. 13. MongoDB + Ruby • mongo gem • bson_ext gem • http://www.mongodb.org/display/DOCS/ Ruby+Language+CenterFriday, August 12, 2011
  14. 14. Considerations for MongoDB • Tweets - very verbose JSON • Date format in Tweets not same as MongoDB • May want to pre-process Tweets • Can use both MongoDB and MySQL in same app if you wantFriday, August 12, 2011
  15. 15. Here’s the code • Okay, the whole thing is really done in about three lines.Friday, August 12, 2011
  16. 16. Where do you run this? • Command line • Your own server • Heroku + MongoLab • Other cloud servicesFriday, August 12, 2011
  17. 17. MongoLabFriday, August 12, 2011
  18. 18. Now what? • Step 1: Collect Tweets • Step 2: ???? • Step 3: Profit!Friday, August 12, 2011
  19. 19. To Learn More • https://dev.twitter.com/docs/streaming-api/ concepts • http://www.mongodb.org/ • https://github.com/jefflinwood/Tweeter- Keeper • http://www.jefflinwood.com/ • @jefflinwood on TwitterFriday, August 12, 2011
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×