Successfully reported this slideshow.
Your SlideShare is downloading. ×

Inside Buzztter

More Related Content

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

Inside Buzztter

  1. 1. RubyConf 2009 2009-11-19 Inside buzztter 設樂 洋爾 (Yoji Shidara) yoji.shidara@enishi-tech.com
  2. 2. My name is Yoji SHIDARA.
  3. 3. Enishi Tech Inc. Small Company. Only 2 staffs.
  4. 4. Small Office. (CEO’s flat)
  5. 5. I’m from Sapporo! @darashi (me) Sapporo 1.4hrs 9.5hrs Tokyo San Fransisco!
  6. 6. Sapporo is a beautiful provincial city of Japan.
  7. 7. Delicious foods are waiting for you!
  8. 8. Do you know buzztter.com?
  9. 9. RubyConf 2009 2009-11-19 LT Outside buzztter 設樂 洋爾 (Yoji Shidara) yoji.shidara@enishi-tech.com
  10. 10. http://buzztter.com
  11. 11. http://buzztter.com buzz-phrases (auto update) Full-text Search Results (auto update)
  12. 12. Servers are in my room! next to my bed
  13. 13. We crawls: 1,932,823 tweets/day (Japanese) 607,749 tweets/day (English) (We’re now focusing on Japanese users due to API limits) We have: 1,336,444 PVs / Month
  14. 14. Server 1: Phenom 9950 Memory 8GB SSD 80GB + HDD 80GB Server 2: PhenomII x4 955 Memory 8GB HDD 500GB
  15. 15. @buzztter
  16. 16. Bot for English was banned because it tweeted spam as hot :-<
  17. 17. Buzztter:
  18. 18. buzz-phrase detection + real time full text search
  19. 19. ... like official “trending topics”
  20. 20. buzztter released before the official one launched!
  21. 21. buzztter supports English and Japanese!
  22. 22. 1-2 minutes for buzz analysis par a language.
  23. 23. HOT notification bot is available.
  24. 24. RubyConf 2009 2009-11-19 LT Inside buzztter 設樂 洋爾 (Yoji Shidara) yoji.shidara@enishi-tech.com
  25. 25. System Diagram: Rough Sketch Web Notification Frontend module crawler Fulltext Analyzer module Search module BuzzDAS: Buzz Data Analysis System
  26. 26. System Diagram: Crawler Typhoeus Twitter crawler EventMachine RabbitMQ MQ crawl crawl scheduling controller libtextcat Users DB langugage language, guesser PostgreSQL post frequency
  27. 27. notifier pagecache crawler bot varnish Twitter4R Web Frontend net/irc Rails MQ Web Service API Sinatra memcached groonga fulltext importer search analyzer engine BuzzDAS: Buzz Data Analysis System
  28. 28. analyzer tokenizer phrase extractor detects change of phrase occurrence Reference Recent Index Index keeps post in 24 hours keeps recent 1 hour
  29. 29. Try http://buzztter.com and give me your feedback please! If you Interested in these keywords: groonga, AMQP, RabbitMQ, Typhoeus, EventMachine, PostgreSQL, libtextcat, Sinatra, Rails, Twitter4R, net/irc, memcached, PrefixSpan ,MeCab, TF-IDF, ... or our BuzzDAS Engine, please contact me! My name is Yoji Shidara.

×