Treasure Data                              Treasure Data and Heroku                                   Masahiro Nakagawa   ...
Who are you?          Masahiro Nakagawa              • @repeatedly / masa@treasure-data.com          Treasure Data, Inc....
Introduction to          Treasure DataFriday, April 5, 13
Company Overview          Silicon Valley-based Company              • All Founders are Japanese                      • Hi...
Investors             Bill Tai             Naren Gupta - Nexus Ventures, Director of Redhat, TIBCO             Othman L...
Treasure Data = Cloud + Big Data     Cloud                                                                            Big ...
Why Cloud? ‘Time’ is Money                               Ideal    Customer                Expectation     Value           ...
Full Stack Support for Big Data Reporting        Our best-in-class architecture       Data from almost any source        a...
9   Product                 Data Collection                                    Data Warehouse                             ...
Customer Use CasesFriday, April 5, 13
11         Our Customers – Fortune Global 500 leaders and         start-ups including:Friday, April 5, 13
Example in AdTech: MobFox           1. Europe’s largest independent mobile ad exchange.           2. 20 billion imps/month...
Two Weeks From Start to Finish!                                                        13Friday, April 5, 13
Viki.com: “Global Hulu”Friday, April 5, 13
Viki.com Before          Hard to manage Hadoop          Complicated data collectionFriday, April 5, 13
Viki.com After          No more Hadoop maintenance          Versatile data collector, td-agentFriday, April 5, 13
Our UsageFriday, April 5, 13
https://console.treasure-data.com/                                                           18Friday, April 5, 13
http://fluentd.org/                                           19Friday, April 5, 13
Other usage          Staging environment          Internal testing application          Proxy server for our used servi...
Heroku integrationFriday, April 5, 13
http://blog.treasure-data.com/post/44003014921/treasure-           data-is-sponsoring-heroku-waza-2013                    ...
Matzhttp://www.wired.com/business/2013/03/heroku-waza/   http://instagram.com/p/WTIEwpA_9-/#                              ...
Heroku addons                      https://addons.heroku.com/provider/resources/technical/how/overview                    ...
25Friday, April 5, 13
https://addons.heroku.com/treasure-data                                                                26Friday, April 5, 13
Using Heroku addon          Setup “td” command              • Install via td-toolbelt or rubygems          Setup “td” he...
Just STDOUT          Use STDOUT to collect event logs              • No need libraries              • log forward via Her...
29Friday, April 5, 13
Conclusion          Treasure Data              • Cloud based Big-data analytics platform              • Provide Machete f...
Big Data for the Rest of Us                      www.treasure-data.com | @TreasureDataFriday, April 5, 13
Upcoming SlideShare
Loading in …5
×

Treasure Data and Heroku

6,249 views

Published on

http://herokujp.doorkeeper.jp/events/3405

Published in: Technology

Treasure Data and Heroku

  1. 1. Treasure Data Treasure Data and Heroku Masahiro Nakagawa Heroku Meetup #8 TreasureData + Waza Report!! Thu, 04 Apr 2013Friday, April 5, 13
  2. 2. Who are you?  Masahiro Nakagawa • @repeatedly / masa@treasure-data.com  Treasure Data, Inc. • Senior Software Engineer, since 2012/11  Open Source projects • D Programming Language • MessagePack: D, Python, etc... • Fluentd: Core, mongo, etc... • etc... 2Friday, April 5, 13
  3. 3. Introduction to Treasure DataFriday, April 5, 13
  4. 4. Company Overview  Silicon Valley-based Company • All Founders are Japanese • Hironobu Yoshikawa • Kazuki Ohta • Sadayuki Furuhashi  OSS Enthusiasts • MessagePack, Fluentd, etc. 4Friday, April 5, 13
  5. 5. Investors  Bill Tai  Naren Gupta - Nexus Ventures, Director of Redhat, TIBCO  Othman Laraki - Former VP Growth at Twitter  James Lindenbaum, Adam Wiggins, Orion Henry - Heroku Founders  Anand Babu Periasamy, Hitesh Chellani - Gluster Founders  Yukihiro “Matz” Matsumoto - Creator of Ruby  Dan Scheinman - Director of Arista Networks  Jerry Yang - Founder of Yahoo!  + 10 more people • and.... 5Friday, April 5, 13
  6. 6. Treasure Data = Cloud + Big Data Cloud Big Data-as-a-Service Database-as-a-service Enterprise Lightweight RDBMS Traditional RDBMS Data Warehouse DB2 On-Premise $34B $10B market market 1Bil entry Data Volume Or 10TB © 2012 Forrester Research, Inc. Reproduction Prohibited 6Friday, April 5, 13
  7. 7. Why Cloud? ‘Time’ is Money Ideal Customer Expectation Value Maintain Obsolete RedShift over time EMR AWS (or hosted Hadoops) Reality EC2 Step-by-step manual integrations (On-Premise) S3 Upgrade HW/SW Selection, PoC, Deploy... Time Sign-up or PO 7Friday, April 5, 13
  8. 8. Full Stack Support for Big Data Reporting Our best-in-class architecture Data from almost any source and operations team ensure the can be securely and reliably integrity and availability of your uploaded using td-agent in data. streaming or batch mode. Our SQL, REST, JDBC, ODBC You can store gigabytes to and command-line interfaces petabytes of data efficiently and support all major query tools securely in our cloud-based and approaches. columnar datastore. 8Friday, April 5, 13
  9. 9. 9 Product Data Collection Data Warehouse Data Analysis Web Log BI Tools App Log Streaming Upload REST Tableau, QlickView Open-Source JDBC / ODBC Log Collector >60billion / month Excel, etc. Sensor Columnar Storage SQL 2,000+ companies (HiveQL) (incl. LinkedIn, etc) + or Hadoop Pig Bulk Loader MapReduce RDBMS Dashboard CSV / TSV Bulk Upload CRM MySQL, Parallel Upload Result push Custom App, 250bil+ records Postgres RDBMS, FTP, etc. 2mil+ jobs ERP Oracle, etc. Value Proposition: “Time-to-Answer” 20bil+, 2 weeks, 2 weeks, 3 weeks, 3bil+, 3 weeks 2 weeks, UK/Austria US Japan Singapore US Multi-Tenant: Single Code for Everyone - no code modification, Improving the Platform Faster.Friday, April 5, 13
  10. 10. Customer Use CasesFriday, April 5, 13
  11. 11. 11 Our Customers – Fortune Global 500 leaders and start-ups including:Friday, April 5, 13
  12. 12. Example in AdTech: MobFox 1. Europe’s largest independent mobile ad exchange. 2. 20 billion imps/month (circa Jan. 2013) 3. Serving ads for 15,000+ mobile apps (circa Jan. 2013) 4. Needed Big Data Analytics infrastructure ASAP. 12Friday, April 5, 13
  13. 13. Two Weeks From Start to Finish! 13Friday, April 5, 13
  14. 14. Viki.com: “Global Hulu”Friday, April 5, 13
  15. 15. Viki.com Before  Hard to manage Hadoop  Complicated data collectionFriday, April 5, 13
  16. 16. Viki.com After  No more Hadoop maintenance  Versatile data collector, td-agentFriday, April 5, 13
  17. 17. Our UsageFriday, April 5, 13
  18. 18. https://console.treasure-data.com/ 18Friday, April 5, 13
  19. 19. http://fluentd.org/ 19Friday, April 5, 13
  20. 20. Other usage  Staging environment  Internal testing application  Proxy server for our used services 20Friday, April 5, 13
  21. 21. Heroku integrationFriday, April 5, 13
  22. 22. http://blog.treasure-data.com/post/44003014921/treasure- data-is-sponsoring-heroku-waza-2013 22Friday, April 5, 13
  23. 23. Matzhttp://www.wired.com/business/2013/03/heroku-waza/ http://instagram.com/p/WTIEwpA_9-/# 23Friday, April 5, 13
  24. 24. Heroku addons https://addons.heroku.com/provider/resources/technical/how/overview 24Friday, April 5, 13
  25. 25. 25Friday, April 5, 13
  26. 26. https://addons.heroku.com/treasure-data 26Friday, April 5, 13
  27. 27. Using Heroku addon  Setup “td” command • Install via td-toolbelt or rubygems  Setup “td” heroku plugin • heroku plugins:install https://github.com/treasure-data/ heroku-td.git  Add ‘td’ gem to your Gemfile • or STDOUT log collecting  “heroku td” is now available for Treasure Data • “heroku td xxx”: xxx is the same as “td” command https://devcenter.heroku.com/articles/treasure-data 27Friday, April 5, 13
  28. 28. Just STDOUT  Use STDOUT to collect event logs • No need libraries • log forward via Heroku syslog drain  Format • @[db_name.table_name] json_in_one_line • Ruby: puts @[service.users] {"name":"D", "via":"Phobos"} http://blog.treasure-data.com/post/41886298790/just- stdout-the-simplest-most-flexible-way-to-collect 28Friday, April 5, 13
  29. 29. 29Friday, April 5, 13
  30. 30. Conclusion  Treasure Data • Cloud based Big-data analytics platform • Provide Machete for Big data reporting  Heroku and Treasure Data • Treasure Data addon • easy to integrate with your Heroku app • STDOUT log collecting with Heroku syslog drain 30Friday, April 5, 13
  31. 31. Big Data for the Rest of Us www.treasure-data.com | @TreasureDataFriday, April 5, 13

×