Your SlideShare is downloading. ×
Treasure Data and Heroku
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

Treasure Data and Heroku

4,789
views

Published on

http://herokujp.doorkeeper.jp/events/3405

http://herokujp.doorkeeper.jp/events/3405

Published in: Technology

0 Comments
12 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
4,789
On Slideshare
0
From Embeds
0
Number of Embeds
17
Actions
Shares
0
Downloads
18
Comments
0
Likes
12
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Treasure Data Treasure Data and Heroku Masahiro Nakagawa Heroku Meetup #8 TreasureData + Waza Report!! Thu, 04 Apr 2013Friday, April 5, 13
  • 2. Who are you?  Masahiro Nakagawa • @repeatedly / masa@treasure-data.com  Treasure Data, Inc. • Senior Software Engineer, since 2012/11  Open Source projects • D Programming Language • MessagePack: D, Python, etc... • Fluentd: Core, mongo, etc... • etc... 2Friday, April 5, 13
  • 3. Introduction to Treasure DataFriday, April 5, 13
  • 4. Company Overview  Silicon Valley-based Company • All Founders are Japanese • Hironobu Yoshikawa • Kazuki Ohta • Sadayuki Furuhashi  OSS Enthusiasts • MessagePack, Fluentd, etc. 4Friday, April 5, 13
  • 5. Investors  Bill Tai  Naren Gupta - Nexus Ventures, Director of Redhat, TIBCO  Othman Laraki - Former VP Growth at Twitter  James Lindenbaum, Adam Wiggins, Orion Henry - Heroku Founders  Anand Babu Periasamy, Hitesh Chellani - Gluster Founders  Yukihiro “Matz” Matsumoto - Creator of Ruby  Dan Scheinman - Director of Arista Networks  Jerry Yang - Founder of Yahoo!  + 10 more people • and.... 5Friday, April 5, 13
  • 6. Treasure Data = Cloud + Big Data Cloud Big Data-as-a-Service Database-as-a-service Enterprise Lightweight RDBMS Traditional RDBMS Data Warehouse DB2 On-Premise $34B $10B market market 1Bil entry Data Volume Or 10TB © 2012 Forrester Research, Inc. Reproduction Prohibited 6Friday, April 5, 13
  • 7. Why Cloud? ‘Time’ is Money Ideal Customer Expectation Value Maintain Obsolete RedShift over time EMR AWS (or hosted Hadoops) Reality EC2 Step-by-step manual integrations (On-Premise) S3 Upgrade HW/SW Selection, PoC, Deploy... Time Sign-up or PO 7Friday, April 5, 13
  • 8. Full Stack Support for Big Data Reporting Our best-in-class architecture Data from almost any source and operations team ensure the can be securely and reliably integrity and availability of your uploaded using td-agent in data. streaming or batch mode. Our SQL, REST, JDBC, ODBC You can store gigabytes to and command-line interfaces petabytes of data efficiently and support all major query tools securely in our cloud-based and approaches. columnar datastore. 8Friday, April 5, 13
  • 9. 9 Product Data Collection Data Warehouse Data Analysis Web Log BI Tools App Log Streaming Upload REST Tableau, QlickView Open-Source JDBC / ODBC Log Collector >60billion / month Excel, etc. Sensor Columnar Storage SQL 2,000+ companies (HiveQL) (incl. LinkedIn, etc) + or Hadoop Pig Bulk Loader MapReduce RDBMS Dashboard CSV / TSV Bulk Upload CRM MySQL, Parallel Upload Result push Custom App, 250bil+ records Postgres RDBMS, FTP, etc. 2mil+ jobs ERP Oracle, etc. Value Proposition: “Time-to-Answer” 20bil+, 2 weeks, 2 weeks, 3 weeks, 3bil+, 3 weeks 2 weeks, UK/Austria US Japan Singapore US Multi-Tenant: Single Code for Everyone - no code modification, Improving the Platform Faster.Friday, April 5, 13
  • 10. Customer Use CasesFriday, April 5, 13
  • 11. 11 Our Customers – Fortune Global 500 leaders and start-ups including:Friday, April 5, 13
  • 12. Example in AdTech: MobFox 1. Europe’s largest independent mobile ad exchange. 2. 20 billion imps/month (circa Jan. 2013) 3. Serving ads for 15,000+ mobile apps (circa Jan. 2013) 4. Needed Big Data Analytics infrastructure ASAP. 12Friday, April 5, 13
  • 13. Two Weeks From Start to Finish! 13Friday, April 5, 13
  • 14. Viki.com: “Global Hulu”Friday, April 5, 13
  • 15. Viki.com Before  Hard to manage Hadoop  Complicated data collectionFriday, April 5, 13
  • 16. Viki.com After  No more Hadoop maintenance  Versatile data collector, td-agentFriday, April 5, 13
  • 17. Our UsageFriday, April 5, 13
  • 18. https://console.treasure-data.com/ 18Friday, April 5, 13
  • 19. http://fluentd.org/ 19Friday, April 5, 13
  • 20. Other usage  Staging environment  Internal testing application  Proxy server for our used services 20Friday, April 5, 13
  • 21. Heroku integrationFriday, April 5, 13
  • 22. http://blog.treasure-data.com/post/44003014921/treasure- data-is-sponsoring-heroku-waza-2013 22Friday, April 5, 13
  • 23. Matzhttp://www.wired.com/business/2013/03/heroku-waza/ http://instagram.com/p/WTIEwpA_9-/# 23Friday, April 5, 13
  • 24. Heroku addons https://addons.heroku.com/provider/resources/technical/how/overview 24Friday, April 5, 13
  • 25. 25Friday, April 5, 13
  • 26. https://addons.heroku.com/treasure-data 26Friday, April 5, 13
  • 27. Using Heroku addon  Setup “td” command • Install via td-toolbelt or rubygems  Setup “td” heroku plugin • heroku plugins:install https://github.com/treasure-data/ heroku-td.git  Add ‘td’ gem to your Gemfile • or STDOUT log collecting  “heroku td” is now available for Treasure Data • “heroku td xxx”: xxx is the same as “td” command https://devcenter.heroku.com/articles/treasure-data 27Friday, April 5, 13
  • 28. Just STDOUT  Use STDOUT to collect event logs • No need libraries • log forward via Heroku syslog drain  Format • @[db_name.table_name] json_in_one_line • Ruby: puts @[service.users] {"name":"D", "via":"Phobos"} http://blog.treasure-data.com/post/41886298790/just- stdout-the-simplest-most-flexible-way-to-collect 28Friday, April 5, 13
  • 29. 29Friday, April 5, 13
  • 30. Conclusion  Treasure Data • Cloud based Big-data analytics platform • Provide Machete for Big data reporting  Heroku and Treasure Data • Treasure Data addon • easy to integrate with your Heroku app • STDOUT log collecting with Heroku syslog drain 30Friday, April 5, 13
  • 31. Big Data for the Rest of Us www.treasure-data.com | @TreasureDataFriday, April 5, 13