Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Presto in my_use_case

5,523 views

Published on

presto meetup

Published in: Technology
  • Be the first to comment

Presto in my_use_case

  1. 1. Presto in my use case Presto Meetup 2015/01/20 http://eventdots.jp/event/276987 @wyukawa
  2. 2. Agenda •  Log Analysis System Overview •  Why do I select Presto? •  How do I use Presto? •  How do I manage Presto? •  My impression for Presto
  3. 3. Log Analysis System Overview Hadoop, Hive of HDP2.1 Azkaban 2.6.4 Presto 0.89 Cognos 10.2.1 Prestogres 0.4.8 DB MySQL 5.5 DBDB ETL with Python 2.7.7 InfiniDB Pentaho Saiku Shib
  4. 4. Why do I select Presto? •  my familiar(on Twitter) people use •  comparison with other is a bother •  Let s try!
  5. 5. How do I use Presto? •  batch with Hive on MR2(not Tez) •  select only with Presto •  execute adhoc query •  report by Cognos with Prestogres •  create presto view http://d.hatena.ne.jp/wyukawa/20140824/1408881620
  6. 6. What is Cognos? •  Commercial BI tool by IBM •  Pros •  authorization management •  flexible reporting(but not easy) •  Cons •  slow rendering speed •  no permanent link •  difficult to understand error message •  web site(especially download site) is bad •  Windows and IE are necessary
  7. 7. What is Prestogres? PostgreSQL pgpool-II (patched) BI tool Presto Prestogres
  8. 8. Why do I use Prestogres? •  Interesting •  MySQL is possible to be a bottleneck •  Hadoop is easier to scale than MySQL •  reduce maintenance cost for multi storages •  but not achieve
  9. 9. Prestogres in my use case Presto 0.89 Cognos 10.2.1 Prestogres 0.4.8 Prestogres ODBC Driver It is not easy to connect Cognos to Presto. Thanks! > @frsyuki http://d.hatena.ne.jp/wyukawa/20140623/1403521909
  10. 10. Problem in my use case •  Cognos don t issue where cause with Presgtogres+ODBC •  select … from … where yyyymmdd= 20150120 … •  slow rendering because of no predicate pushdown •  workaround is to use bigint to where cause, not string •  solution(not deploy in production. BTW, Thanks! > @frsyuki ) •  Cognos 10.2.2 •  patched(protocolVersion=2) PostgreSQL JDBC Driver •  Prestogres 0.6.3 •  Presto 0.86 •  increase dentry cache http://d.hatena.ne.jp/wyukawa/20141216/1418706186
  11. 11. How do I manage Presto? •  How to deploy Presto? •  use Ansible •  Presto Setting •  task.cpu-timer-enabled=false •  How to monitor Presto? •  GrowthForecast + jstat2gf, JMX •  slow query visualization by nata2 http://d.hatena.ne.jp/wyukawa/20140803/1407052542 http://d.hatena.ne.jp/wyukawa/20141218/1418882574
  12. 12. My impression for Presto •  stable •  frequent version up •  easy to install •  easy to upgrade •  but failed at 0.80, 0.87 •  leverage effect

×