SlideShare a Scribd company logo
Presto in my use case
Presto Meetup 2015/01/20
http://eventdots.jp/event/276987
@wyukawa
Agenda
•  Log Analysis System Overview
•  Why do I select Presto?
•  How do I use Presto?
•  How do I manage Presto?
•  My impression for Presto
Log Analysis System Overview
Hadoop, Hive of HDP2.1
Azkaban
2.6.4
Presto 0.89
Cognos
10.2.1
Prestogres
0.4.8
DB
MySQL 5.5
DBDB
ETL with
Python 2.7.7
InfiniDB
Pentaho
Saiku
Shib
Why do I select Presto?
•  my familiar(on Twitter) people use
•  comparison with other is a bother
•  Let s try!
How do I use Presto?
•  batch with Hive on MR2(not Tez)
•  select only with Presto
•  execute adhoc query
•  report by Cognos with Prestogres
•  create presto view
http://d.hatena.ne.jp/wyukawa/20140824/1408881620
What is Cognos?
•  Commercial BI tool by IBM
•  Pros
•  authorization management
•  flexible reporting(but not easy)
•  Cons
•  slow rendering speed
•  no permanent link
•  difficult to understand error message
•  web site(especially download site) is bad
•  Windows and IE are necessary
What is Prestogres?
PostgreSQL
pgpool-II
(patched)
BI tool
Presto
Prestogres
Why do I use Prestogres?
•  Interesting
•  MySQL is possible to be a bottleneck
•  Hadoop is easier to scale than MySQL
•  reduce maintenance cost for multi storages
•  but not achieve
Prestogres in my use case
Presto 0.89
Cognos
10.2.1
Prestogres
0.4.8
Prestogres
ODBC Driver
It is not easy to connect Cognos to Presto.
Thanks! > @frsyuki
http://d.hatena.ne.jp/wyukawa/20140623/1403521909
Problem in my use case
•  Cognos don t issue where cause with Presgtogres+ODBC
•  select … from … where yyyymmdd= 20150120 …
•  slow rendering because of no predicate pushdown
•  workaround is to use bigint to where cause, not string
•  solution(not deploy in production. BTW, Thanks! > @frsyuki )
•  Cognos 10.2.2
•  patched(protocolVersion=2) PostgreSQL JDBC Driver
•  Prestogres 0.6.3
•  Presto 0.86
•  increase dentry cache
http://d.hatena.ne.jp/wyukawa/20141216/1418706186
How do I manage Presto?
•  How to deploy Presto?
•  use Ansible
•  Presto Setting
•  task.cpu-timer-enabled=false
•  How to monitor Presto?
•  GrowthForecast + jstat2gf, JMX
•  slow query visualization by nata2
http://d.hatena.ne.jp/wyukawa/20140803/1407052542
http://d.hatena.ne.jp/wyukawa/20141218/1418882574
My impression for Presto
•  stable
•  frequent version up
•  easy to install
•  easy to upgrade
•  but failed at 0.80, 0.87
•  leverage effect

More Related Content

What's hot

Treasure Data and AWS - Developers.io 2015
Treasure Data and AWS - Developers.io 2015Treasure Data and AWS - Developers.io 2015
Treasure Data and AWS - Developers.io 2015
N Masahiro
 
Presto - Analytical Database. Overview and use cases.
Presto - Analytical Database. Overview and use cases.Presto - Analytical Database. Overview and use cases.
Presto - Analytical Database. Overview and use cases.
Wojciech Biela
 
Presto - Hadoop Conference Japan 2014
Presto - Hadoop Conference Japan 2014Presto - Hadoop Conference Japan 2014
Presto - Hadoop Conference Japan 2014Sadayuki Furuhashi
 
Boston Hadoop Meetup: Presto for the Enterprise
Boston Hadoop Meetup: Presto for the EnterpriseBoston Hadoop Meetup: Presto for the Enterprise
Boston Hadoop Meetup: Presto for the Enterprise
Matt Fuller
 
Presto meetup 2015-03-19 @Facebook
Presto meetup 2015-03-19 @FacebookPresto meetup 2015-03-19 @Facebook
Presto meetup 2015-03-19 @Facebook
Treasure Data, Inc.
 
Presto at Twitter
Presto at TwitterPresto at Twitter
Presto at Twitter
Bill Graham
 
Presto Strata Hadoop SJ 2016 short talk
Presto Strata Hadoop SJ 2016 short talkPresto Strata Hadoop SJ 2016 short talk
Presto Strata Hadoop SJ 2016 short talk
kbajda
 
How to ensure Presto scalability 
in multi use case
How to ensure Presto scalability 
in multi use case How to ensure Presto scalability 
in multi use case
How to ensure Presto scalability 
in multi use case
Kai Sasaki
 
Prestogres internals
Prestogres internalsPrestogres internals
Prestogres internals
Sadayuki Furuhashi
 
Presto @ Treasure Data - Presto Meetup Boston 2015
Presto @ Treasure Data - Presto Meetup Boston 2015Presto @ Treasure Data - Presto Meetup Boston 2015
Presto @ Treasure Data - Presto Meetup Boston 2015
Taro L. Saito
 
Presto Testing Tools: Benchto & Tempto (Presto Boston Meetup 10062015)
Presto Testing Tools: Benchto & Tempto (Presto Boston Meetup 10062015)Presto Testing Tools: Benchto & Tempto (Presto Boston Meetup 10062015)
Presto Testing Tools: Benchto & Tempto (Presto Boston Meetup 10062015)
Matt Fuller
 
Facebook Presto presentation
Facebook Presto presentationFacebook Presto presentation
Facebook Presto presentation
Cyanny LIANG
 
Tale of ISUCON and Its Bench Tools
Tale of ISUCON and Its Bench ToolsTale of ISUCON and Its Bench Tools
Tale of ISUCON and Its Bench Tools
SATOSHI TAGOMORI
 
tdtechtalk20160330johan
tdtechtalk20160330johantdtechtalk20160330johan
tdtechtalk20160330johan
Johan Gustavsson
 
Lessons PostgreSQL learned from commercial databases, and didn’t
Lessons PostgreSQL learned from commercial databases, and didn’tLessons PostgreSQL learned from commercial databases, and didn’t
Lessons PostgreSQL learned from commercial databases, and didn’t
PGConf APAC
 
Logging for Production Systems in The Container Era
Logging for Production Systems in The Container EraLogging for Production Systems in The Container Era
Logging for Production Systems in The Container Era
Sadayuki Furuhashi
 
Hello, Enterprise! Meet Presto. (Presto Boston Meetup 10062015)
Hello, Enterprise! Meet Presto. (Presto Boston Meetup 10062015)Hello, Enterprise! Meet Presto. (Presto Boston Meetup 10062015)
Hello, Enterprise! Meet Presto. (Presto Boston Meetup 10062015)
Matt Fuller
 
Presto in my_use_case2
Presto in my_use_case2Presto in my_use_case2
Presto in my_use_case2
wyukawa
 
Planet-scale Data Ingestion Pipeline: Bigdam
Planet-scale Data Ingestion Pipeline: BigdamPlanet-scale Data Ingestion Pipeline: Bigdam
Planet-scale Data Ingestion Pipeline: Bigdam
SATOSHI TAGOMORI
 
PostgreSQL Enterprise Class Features and Capabilities
PostgreSQL Enterprise Class Features and CapabilitiesPostgreSQL Enterprise Class Features and Capabilities
PostgreSQL Enterprise Class Features and Capabilities
PGConf APAC
 

What's hot (20)

Treasure Data and AWS - Developers.io 2015
Treasure Data and AWS - Developers.io 2015Treasure Data and AWS - Developers.io 2015
Treasure Data and AWS - Developers.io 2015
 
Presto - Analytical Database. Overview and use cases.
Presto - Analytical Database. Overview and use cases.Presto - Analytical Database. Overview and use cases.
Presto - Analytical Database. Overview and use cases.
 
Presto - Hadoop Conference Japan 2014
Presto - Hadoop Conference Japan 2014Presto - Hadoop Conference Japan 2014
Presto - Hadoop Conference Japan 2014
 
Boston Hadoop Meetup: Presto for the Enterprise
Boston Hadoop Meetup: Presto for the EnterpriseBoston Hadoop Meetup: Presto for the Enterprise
Boston Hadoop Meetup: Presto for the Enterprise
 
Presto meetup 2015-03-19 @Facebook
Presto meetup 2015-03-19 @FacebookPresto meetup 2015-03-19 @Facebook
Presto meetup 2015-03-19 @Facebook
 
Presto at Twitter
Presto at TwitterPresto at Twitter
Presto at Twitter
 
Presto Strata Hadoop SJ 2016 short talk
Presto Strata Hadoop SJ 2016 short talkPresto Strata Hadoop SJ 2016 short talk
Presto Strata Hadoop SJ 2016 short talk
 
How to ensure Presto scalability 
in multi use case
How to ensure Presto scalability 
in multi use case How to ensure Presto scalability 
in multi use case
How to ensure Presto scalability 
in multi use case
 
Prestogres internals
Prestogres internalsPrestogres internals
Prestogres internals
 
Presto @ Treasure Data - Presto Meetup Boston 2015
Presto @ Treasure Data - Presto Meetup Boston 2015Presto @ Treasure Data - Presto Meetup Boston 2015
Presto @ Treasure Data - Presto Meetup Boston 2015
 
Presto Testing Tools: Benchto & Tempto (Presto Boston Meetup 10062015)
Presto Testing Tools: Benchto & Tempto (Presto Boston Meetup 10062015)Presto Testing Tools: Benchto & Tempto (Presto Boston Meetup 10062015)
Presto Testing Tools: Benchto & Tempto (Presto Boston Meetup 10062015)
 
Facebook Presto presentation
Facebook Presto presentationFacebook Presto presentation
Facebook Presto presentation
 
Tale of ISUCON and Its Bench Tools
Tale of ISUCON and Its Bench ToolsTale of ISUCON and Its Bench Tools
Tale of ISUCON and Its Bench Tools
 
tdtechtalk20160330johan
tdtechtalk20160330johantdtechtalk20160330johan
tdtechtalk20160330johan
 
Lessons PostgreSQL learned from commercial databases, and didn’t
Lessons PostgreSQL learned from commercial databases, and didn’tLessons PostgreSQL learned from commercial databases, and didn’t
Lessons PostgreSQL learned from commercial databases, and didn’t
 
Logging for Production Systems in The Container Era
Logging for Production Systems in The Container EraLogging for Production Systems in The Container Era
Logging for Production Systems in The Container Era
 
Hello, Enterprise! Meet Presto. (Presto Boston Meetup 10062015)
Hello, Enterprise! Meet Presto. (Presto Boston Meetup 10062015)Hello, Enterprise! Meet Presto. (Presto Boston Meetup 10062015)
Hello, Enterprise! Meet Presto. (Presto Boston Meetup 10062015)
 
Presto in my_use_case2
Presto in my_use_case2Presto in my_use_case2
Presto in my_use_case2
 
Planet-scale Data Ingestion Pipeline: Bigdam
Planet-scale Data Ingestion Pipeline: BigdamPlanet-scale Data Ingestion Pipeline: Bigdam
Planet-scale Data Ingestion Pipeline: Bigdam
 
PostgreSQL Enterprise Class Features and Capabilities
PostgreSQL Enterprise Class Features and CapabilitiesPostgreSQL Enterprise Class Features and Capabilities
PostgreSQL Enterprise Class Features and Capabilities
 

Viewers also liked

PYNQで○○してみた!
PYNQで○○してみた!PYNQで○○してみた!
PYNQで○○してみた!
aster_ism
 
PYNQ 祭り: Pmod のプログラミング
PYNQ 祭り: Pmod のプログラミングPYNQ 祭り: Pmod のプログラミング
PYNQ 祭り: Pmod のプログラミング
ryos36
 
APACHE TOREE: A JUPYTER KERNEL FOR SPARK by Marius van Niekerk
APACHE TOREE: A JUPYTER KERNEL FOR SPARK by Marius van NiekerkAPACHE TOREE: A JUPYTER KERNEL FOR SPARK by Marius van Niekerk
APACHE TOREE: A JUPYTER KERNEL FOR SPARK by Marius van Niekerk
Spark Summit
 
20171012 found IT #9 PySparkの勘所
20171012 found  IT #9 PySparkの勘所20171012 found  IT #9 PySparkの勘所
20171012 found IT #9 PySparkの勘所
Ryuji Tamagawa
 
[db analytics showcase Sapporo 2017] A15: Pythonでの分散処理再入門 by 株式会社HPCソリューションズ ...
[db analytics showcase Sapporo 2017] A15: Pythonでの分散処理再入門 by 株式会社HPCソリューションズ ...[db analytics showcase Sapporo 2017] A15: Pythonでの分散処理再入門 by 株式会社HPCソリューションズ ...
[db analytics showcase Sapporo 2017] A15: Pythonでの分散処理再入門 by 株式会社HPCソリューションズ ...
Insight Technology, Inc.
 
Pynq祭り資料
Pynq祭り資料Pynq祭り資料
Pynq祭り資料
一路 川染
 
PYNQ祭りLT todotani
PYNQ祭りLT todotaniPYNQ祭りLT todotani
PYNQ祭りLT todotani
Kenshi Kamiya
 
PYNQ単体でUIを表示してみる(PYNQまつり)
PYNQ単体でUIを表示してみる(PYNQまつり)PYNQ単体でUIを表示してみる(PYNQまつり)
PYNQ単体でUIを表示してみる(PYNQまつり)
Kenta IDA
 
PYNQ祭り
PYNQ祭りPYNQ祭り
PYNQ祭り
Mr. Vengineer
 
Pynqでカメラ画像をリアルタイムfastx コーナー検出
Pynqでカメラ画像をリアルタイムfastx コーナー検出Pynqでカメラ画像をリアルタイムfastx コーナー検出
Pynqでカメラ画像をリアルタイムfastx コーナー検出
marsee101
 
20170927 pydata tokyo データサイエンスな皆様に送る分散処理の基礎の基礎、そしてPySparkの勘所
20170927 pydata tokyo データサイエンスな皆様に送る分散処理の基礎の基礎、そしてPySparkの勘所20170927 pydata tokyo データサイエンスな皆様に送る分散処理の基礎の基礎、そしてPySparkの勘所
20170927 pydata tokyo データサイエンスな皆様に送る分散処理の基礎の基礎、そしてPySparkの勘所
Ryuji Tamagawa
 
20170210 sapporotechbar7
20170210 sapporotechbar720170210 sapporotechbar7
20170210 sapporotechbar7
Ryuji Tamagawa
 
コンピュータエンジニアへのFPGAのすすめ
コンピュータエンジニアへのFPGAのすすめコンピュータエンジニアへのFPGAのすすめ
コンピュータエンジニアへのFPGAのすすめ
Takeshi HASEGAWA
 

Viewers also liked (13)

PYNQで○○してみた!
PYNQで○○してみた!PYNQで○○してみた!
PYNQで○○してみた!
 
PYNQ 祭り: Pmod のプログラミング
PYNQ 祭り: Pmod のプログラミングPYNQ 祭り: Pmod のプログラミング
PYNQ 祭り: Pmod のプログラミング
 
APACHE TOREE: A JUPYTER KERNEL FOR SPARK by Marius van Niekerk
APACHE TOREE: A JUPYTER KERNEL FOR SPARK by Marius van NiekerkAPACHE TOREE: A JUPYTER KERNEL FOR SPARK by Marius van Niekerk
APACHE TOREE: A JUPYTER KERNEL FOR SPARK by Marius van Niekerk
 
20171012 found IT #9 PySparkの勘所
20171012 found  IT #9 PySparkの勘所20171012 found  IT #9 PySparkの勘所
20171012 found IT #9 PySparkの勘所
 
[db analytics showcase Sapporo 2017] A15: Pythonでの分散処理再入門 by 株式会社HPCソリューションズ ...
[db analytics showcase Sapporo 2017] A15: Pythonでの分散処理再入門 by 株式会社HPCソリューションズ ...[db analytics showcase Sapporo 2017] A15: Pythonでの分散処理再入門 by 株式会社HPCソリューションズ ...
[db analytics showcase Sapporo 2017] A15: Pythonでの分散処理再入門 by 株式会社HPCソリューションズ ...
 
Pynq祭り資料
Pynq祭り資料Pynq祭り資料
Pynq祭り資料
 
PYNQ祭りLT todotani
PYNQ祭りLT todotaniPYNQ祭りLT todotani
PYNQ祭りLT todotani
 
PYNQ単体でUIを表示してみる(PYNQまつり)
PYNQ単体でUIを表示してみる(PYNQまつり)PYNQ単体でUIを表示してみる(PYNQまつり)
PYNQ単体でUIを表示してみる(PYNQまつり)
 
PYNQ祭り
PYNQ祭りPYNQ祭り
PYNQ祭り
 
Pynqでカメラ画像をリアルタイムfastx コーナー検出
Pynqでカメラ画像をリアルタイムfastx コーナー検出Pynqでカメラ画像をリアルタイムfastx コーナー検出
Pynqでカメラ画像をリアルタイムfastx コーナー検出
 
20170927 pydata tokyo データサイエンスな皆様に送る分散処理の基礎の基礎、そしてPySparkの勘所
20170927 pydata tokyo データサイエンスな皆様に送る分散処理の基礎の基礎、そしてPySparkの勘所20170927 pydata tokyo データサイエンスな皆様に送る分散処理の基礎の基礎、そしてPySparkの勘所
20170927 pydata tokyo データサイエンスな皆様に送る分散処理の基礎の基礎、そしてPySparkの勘所
 
20170210 sapporotechbar7
20170210 sapporotechbar720170210 sapporotechbar7
20170210 sapporotechbar7
 
コンピュータエンジニアへのFPGAのすすめ
コンピュータエンジニアへのFPGAのすすめコンピュータエンジニアへのFPGAのすすめ
コンピュータエンジニアへのFPGAのすすめ
 

Similar to Presto in my_use_case

kranonit S06E01 Игорь Цинько: High load
kranonit S06E01 Игорь Цинько: High loadkranonit S06E01 Игорь Цинько: High load
kranonit S06E01 Игорь Цинько: High loadKrivoy Rog IT Community
 
Silverstripe at scale - design & architecture for silverstripe applications
Silverstripe at scale - design & architecture for silverstripe applicationsSilverstripe at scale - design & architecture for silverstripe applications
Silverstripe at scale - design & architecture for silverstripe applications
BrettTasker
 
Optimizing Access with SQL Server
Optimizing Access with SQL ServerOptimizing Access with SQL Server
Optimizing Access with SQL Server
PRPASS Chapter
 
PGConf.ASIA 2019 Bali - Keynote Speech 2 - Ivan Pachenko
PGConf.ASIA 2019 Bali - Keynote Speech 2 - Ivan PachenkoPGConf.ASIA 2019 Bali - Keynote Speech 2 - Ivan Pachenko
PGConf.ASIA 2019 Bali - Keynote Speech 2 - Ivan Pachenko
Equnix Business Solutions
 
Release Management with Visual Studio Team Services and Office Dev PnP
Release Management with Visual Studio Team Services and Office Dev PnPRelease Management with Visual Studio Team Services and Office Dev PnP
Release Management with Visual Studio Team Services and Office Dev PnP
Petter Skodvin-Hvammen
 
Serverless java
Serverless   javaServerless   java
Serverless java
Vishwas N
 
pandas.(to/from)_sql is simple but not fast
pandas.(to/from)_sql is simple but not fastpandas.(to/from)_sql is simple but not fast
pandas.(to/from)_sql is simple but not fast
Uwe Korn
 
Vincent biret azure functions and flow (ottawa)
Vincent biret azure functions and flow (ottawa)Vincent biret azure functions and flow (ottawa)
Vincent biret azure functions and flow (ottawa)
Vincent Biret
 
Vincent biret azure functions and flow (toronto)
Vincent biret azure functions and flow (toronto)Vincent biret azure functions and flow (toronto)
Vincent biret azure functions and flow (toronto)
Vincent Biret
 
What SQL DBAs need to know about SharePoint-Kansas City, Sept 2013
What SQL DBAs need to know about SharePoint-Kansas City, Sept 2013What SQL DBAs need to know about SharePoint-Kansas City, Sept 2013
What SQL DBAs need to know about SharePoint-Kansas City, Sept 2013
J.D. Wade
 
Presto – Today and Beyond – The Open Source SQL Engine for Querying all Data...
Presto – Today and Beyond – The Open Source SQL Engine for Querying all Data...Presto – Today and Beyond – The Open Source SQL Engine for Querying all Data...
Presto – Today and Beyond – The Open Source SQL Engine for Querying all Data...
Dipti Borkar
 
AdminCamp 2017 - IBM Connections Adminblast
AdminCamp 2017 - IBM Connections AdminblastAdminCamp 2017 - IBM Connections Adminblast
AdminCamp 2017 - IBM Connections Adminblast
Nico Meisenzahl
 
SharePoint Performance Monitoring with Sean P. McDonough
SharePoint Performance Monitoring with Sean P. McDonoughSharePoint Performance Monitoring with Sean P. McDonough
SharePoint Performance Monitoring with Sean P. McDonough
Gabrijela Orsag
 
How to Monitor Postgres Like a Pro!
How to Monitor Postgres Like a Pro!How to Monitor Postgres Like a Pro!
How to Monitor Postgres Like a Pro!
EDB
 
Microservices: The Best Practices
Microservices: The Best PracticesMicroservices: The Best Practices
Microservices: The Best Practices
Pavel Mička
 
Speed up Interactive Analytic Queries over Existing Big Data on Hadoop with P...
Speed up Interactive Analytic Queries over Existing Big Data on Hadoop with P...Speed up Interactive Analytic Queries over Existing Big Data on Hadoop with P...
Speed up Interactive Analytic Queries over Existing Big Data on Hadoop with P...
viirya
 
PostgreSQL as a Strategic Tool
PostgreSQL as a Strategic ToolPostgreSQL as a Strategic Tool
PostgreSQL as a Strategic Tool
EDB
 
Client Side Performance for Back End Developers - Cambridge .NET User Group -...
Client Side Performance for Back End Developers - Cambridge .NET User Group -...Client Side Performance for Back End Developers - Cambridge .NET User Group -...
Client Side Performance for Back End Developers - Cambridge .NET User Group -...
Bart Read
 
SPS calgary 2017 introduction to azure functions microsoft flow
SPS calgary 2017 introduction to azure functions microsoft flowSPS calgary 2017 introduction to azure functions microsoft flow
SPS calgary 2017 introduction to azure functions microsoft flow
Vincent Biret
 
Sps philly 2015 create your own dev vm
Sps philly 2015   create your own dev vmSps philly 2015   create your own dev vm
Sps philly 2015 create your own dev vm
Gaurav Mahajan
 

Similar to Presto in my_use_case (20)

kranonit S06E01 Игорь Цинько: High load
kranonit S06E01 Игорь Цинько: High loadkranonit S06E01 Игорь Цинько: High load
kranonit S06E01 Игорь Цинько: High load
 
Silverstripe at scale - design & architecture for silverstripe applications
Silverstripe at scale - design & architecture for silverstripe applicationsSilverstripe at scale - design & architecture for silverstripe applications
Silverstripe at scale - design & architecture for silverstripe applications
 
Optimizing Access with SQL Server
Optimizing Access with SQL ServerOptimizing Access with SQL Server
Optimizing Access with SQL Server
 
PGConf.ASIA 2019 Bali - Keynote Speech 2 - Ivan Pachenko
PGConf.ASIA 2019 Bali - Keynote Speech 2 - Ivan PachenkoPGConf.ASIA 2019 Bali - Keynote Speech 2 - Ivan Pachenko
PGConf.ASIA 2019 Bali - Keynote Speech 2 - Ivan Pachenko
 
Release Management with Visual Studio Team Services and Office Dev PnP
Release Management with Visual Studio Team Services and Office Dev PnPRelease Management with Visual Studio Team Services and Office Dev PnP
Release Management with Visual Studio Team Services and Office Dev PnP
 
Serverless java
Serverless   javaServerless   java
Serverless java
 
pandas.(to/from)_sql is simple but not fast
pandas.(to/from)_sql is simple but not fastpandas.(to/from)_sql is simple but not fast
pandas.(to/from)_sql is simple but not fast
 
Vincent biret azure functions and flow (ottawa)
Vincent biret azure functions and flow (ottawa)Vincent biret azure functions and flow (ottawa)
Vincent biret azure functions and flow (ottawa)
 
Vincent biret azure functions and flow (toronto)
Vincent biret azure functions and flow (toronto)Vincent biret azure functions and flow (toronto)
Vincent biret azure functions and flow (toronto)
 
What SQL DBAs need to know about SharePoint-Kansas City, Sept 2013
What SQL DBAs need to know about SharePoint-Kansas City, Sept 2013What SQL DBAs need to know about SharePoint-Kansas City, Sept 2013
What SQL DBAs need to know about SharePoint-Kansas City, Sept 2013
 
Presto – Today and Beyond – The Open Source SQL Engine for Querying all Data...
Presto – Today and Beyond – The Open Source SQL Engine for Querying all Data...Presto – Today and Beyond – The Open Source SQL Engine for Querying all Data...
Presto – Today and Beyond – The Open Source SQL Engine for Querying all Data...
 
AdminCamp 2017 - IBM Connections Adminblast
AdminCamp 2017 - IBM Connections AdminblastAdminCamp 2017 - IBM Connections Adminblast
AdminCamp 2017 - IBM Connections Adminblast
 
SharePoint Performance Monitoring with Sean P. McDonough
SharePoint Performance Monitoring with Sean P. McDonoughSharePoint Performance Monitoring with Sean P. McDonough
SharePoint Performance Monitoring with Sean P. McDonough
 
How to Monitor Postgres Like a Pro!
How to Monitor Postgres Like a Pro!How to Monitor Postgres Like a Pro!
How to Monitor Postgres Like a Pro!
 
Microservices: The Best Practices
Microservices: The Best PracticesMicroservices: The Best Practices
Microservices: The Best Practices
 
Speed up Interactive Analytic Queries over Existing Big Data on Hadoop with P...
Speed up Interactive Analytic Queries over Existing Big Data on Hadoop with P...Speed up Interactive Analytic Queries over Existing Big Data on Hadoop with P...
Speed up Interactive Analytic Queries over Existing Big Data on Hadoop with P...
 
PostgreSQL as a Strategic Tool
PostgreSQL as a Strategic ToolPostgreSQL as a Strategic Tool
PostgreSQL as a Strategic Tool
 
Client Side Performance for Back End Developers - Cambridge .NET User Group -...
Client Side Performance for Back End Developers - Cambridge .NET User Group -...Client Side Performance for Back End Developers - Cambridge .NET User Group -...
Client Side Performance for Back End Developers - Cambridge .NET User Group -...
 
SPS calgary 2017 introduction to azure functions microsoft flow
SPS calgary 2017 introduction to azure functions microsoft flowSPS calgary 2017 introduction to azure functions microsoft flow
SPS calgary 2017 introduction to azure functions microsoft flow
 
Sps philly 2015 create your own dev vm
Sps philly 2015   create your own dev vmSps philly 2015   create your own dev vm
Sps philly 2015 create your own dev vm
 

More from wyukawa

Presto conferencetokyo2019
Presto conferencetokyo2019Presto conferencetokyo2019
Presto conferencetokyo2019
wyukawa
 
Strata2017 sg
Strata2017 sgStrata2017 sg
Strata2017 sg
wyukawa
 
Azkaban-en
Azkaban-enAzkaban-en
Azkaban-en
wyukawa
 
Azkaban
AzkabanAzkaban
Azkaban
wyukawa
 
Upgrading from-hdp-21-to-hdp-25
Upgrading from-hdp-21-to-hdp-25Upgrading from-hdp-21-to-hdp-25
Upgrading from-hdp-21-to-hdp-25
wyukawa
 
Promcon2016
Promcon2016Promcon2016
Promcon2016
wyukawa
 
Prometheus london
Prometheus londonPrometheus london
Prometheus london
wyukawa
 
Prometheus casual talk1
Prometheus casual talk1Prometheus casual talk1
Prometheus casual talk1
wyukawa
 
My ambariexperience
My ambariexperienceMy ambariexperience
My ambariexperience
wyukawa
 
Prometheus
PrometheusPrometheus
Prometheus
wyukawa
 
Upgrading from-hdp-21-to-hdp-24
Upgrading from-hdp-21-to-hdp-24Upgrading from-hdp-21-to-hdp-24
Upgrading from-hdp-21-to-hdp-24
wyukawa
 
Hive sourcecodereading
Hive sourcecodereadingHive sourcecodereading
Hive sourcecodereadingwyukawa
 
Hdfs write
Hdfs writeHdfs write
Hdfs writewyukawa
 
Osc mercurial-public
Osc mercurial-publicOsc mercurial-public
Osc mercurial-publicwyukawa
 
Dvcs study
Dvcs studyDvcs study
Dvcs studywyukawa
 
Hudson study-zen
Hudson study-zenHudson study-zen
Hudson study-zenwyukawa
 
Shibuya.trac.8
Shibuya.trac.8Shibuya.trac.8
Shibuya.trac.8wyukawa
 
Hudson tanabata.trac
Hudson tanabata.tracHudson tanabata.trac
Hudson tanabata.trac
wyukawa
 

More from wyukawa (18)

Presto conferencetokyo2019
Presto conferencetokyo2019Presto conferencetokyo2019
Presto conferencetokyo2019
 
Strata2017 sg
Strata2017 sgStrata2017 sg
Strata2017 sg
 
Azkaban-en
Azkaban-enAzkaban-en
Azkaban-en
 
Azkaban
AzkabanAzkaban
Azkaban
 
Upgrading from-hdp-21-to-hdp-25
Upgrading from-hdp-21-to-hdp-25Upgrading from-hdp-21-to-hdp-25
Upgrading from-hdp-21-to-hdp-25
 
Promcon2016
Promcon2016Promcon2016
Promcon2016
 
Prometheus london
Prometheus londonPrometheus london
Prometheus london
 
Prometheus casual talk1
Prometheus casual talk1Prometheus casual talk1
Prometheus casual talk1
 
My ambariexperience
My ambariexperienceMy ambariexperience
My ambariexperience
 
Prometheus
PrometheusPrometheus
Prometheus
 
Upgrading from-hdp-21-to-hdp-24
Upgrading from-hdp-21-to-hdp-24Upgrading from-hdp-21-to-hdp-24
Upgrading from-hdp-21-to-hdp-24
 
Hive sourcecodereading
Hive sourcecodereadingHive sourcecodereading
Hive sourcecodereading
 
Hdfs write
Hdfs writeHdfs write
Hdfs write
 
Osc mercurial-public
Osc mercurial-publicOsc mercurial-public
Osc mercurial-public
 
Dvcs study
Dvcs studyDvcs study
Dvcs study
 
Hudson study-zen
Hudson study-zenHudson study-zen
Hudson study-zen
 
Shibuya.trac.8
Shibuya.trac.8Shibuya.trac.8
Shibuya.trac.8
 
Hudson tanabata.trac
Hudson tanabata.tracHudson tanabata.trac
Hudson tanabata.trac
 

Recently uploaded

Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
Safe Software
 
PCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase TeamPCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase Team
ControlCase
 
UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
DianaGray10
 
Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !
KatiaHIMEUR1
 
Assuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyesAssuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyes
ThousandEyes
 
PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)
Ralf Eggert
 
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Ramesh Iyer
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
James Anderson
 
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Thierry Lestable
 
Elevating Tactical DDD Patterns Through Object Calisthenics
Elevating Tactical DDD Patterns Through Object CalisthenicsElevating Tactical DDD Patterns Through Object Calisthenics
Elevating Tactical DDD Patterns Through Object Calisthenics
Dorra BARTAGUIZ
 
Key Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfKey Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdf
Cheryl Hung
 
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdf
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdfSAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdf
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdf
Peter Spielvogel
 
The Future of Platform Engineering
The Future of Platform EngineeringThe Future of Platform Engineering
The Future of Platform Engineering
Jemma Hussein Allen
 
UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4
DianaGray10
 
GraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge GraphGraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge Graph
Guy Korland
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
Kari Kakkonen
 
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
Product School
 
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
DanBrown980551
 
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdfFIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance
 
Quantum Computing: Current Landscape and the Future Role of APIs
Quantum Computing: Current Landscape and the Future Role of APIsQuantum Computing: Current Landscape and the Future Role of APIs
Quantum Computing: Current Landscape and the Future Role of APIs
Vlad Stirbu
 

Recently uploaded (20)

Essentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with ParametersEssentials of Automations: Optimizing FME Workflows with Parameters
Essentials of Automations: Optimizing FME Workflows with Parameters
 
PCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase TeamPCI PIN Basics Webinar from the Controlcase Team
PCI PIN Basics Webinar from the Controlcase Team
 
UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
 
Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !
 
Assuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyesAssuring Contact Center Experiences for Your Customers With ThousandEyes
Assuring Contact Center Experiences for Your Customers With ThousandEyes
 
PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)PHP Frameworks: I want to break free (IPC Berlin 2024)
PHP Frameworks: I want to break free (IPC Berlin 2024)
 
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...
 
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
 
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
Empowering NextGen Mobility via Large Action Model Infrastructure (LAMI): pav...
 
Elevating Tactical DDD Patterns Through Object Calisthenics
Elevating Tactical DDD Patterns Through Object CalisthenicsElevating Tactical DDD Patterns Through Object Calisthenics
Elevating Tactical DDD Patterns Through Object Calisthenics
 
Key Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdfKey Trends Shaping the Future of Infrastructure.pdf
Key Trends Shaping the Future of Infrastructure.pdf
 
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdf
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdfSAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdf
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdf
 
The Future of Platform Engineering
The Future of Platform EngineeringThe Future of Platform Engineering
The Future of Platform Engineering
 
UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4UiPath Test Automation using UiPath Test Suite series, part 4
UiPath Test Automation using UiPath Test Suite series, part 4
 
GraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge GraphGraphRAG is All You need? LLM & Knowledge Graph
GraphRAG is All You need? LLM & Knowledge Graph
 
DevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA ConnectDevOps and Testing slides at DASA Connect
DevOps and Testing slides at DASA Connect
 
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
De-mystifying Zero to One: Design Informed Techniques for Greenfield Innovati...
 
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
 
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdfFIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf
 
Quantum Computing: Current Landscape and the Future Role of APIs
Quantum Computing: Current Landscape and the Future Role of APIsQuantum Computing: Current Landscape and the Future Role of APIs
Quantum Computing: Current Landscape and the Future Role of APIs
 

Presto in my_use_case

  • 1. Presto in my use case Presto Meetup 2015/01/20 http://eventdots.jp/event/276987 @wyukawa
  • 2. Agenda •  Log Analysis System Overview •  Why do I select Presto? •  How do I use Presto? •  How do I manage Presto? •  My impression for Presto
  • 3. Log Analysis System Overview Hadoop, Hive of HDP2.1 Azkaban 2.6.4 Presto 0.89 Cognos 10.2.1 Prestogres 0.4.8 DB MySQL 5.5 DBDB ETL with Python 2.7.7 InfiniDB Pentaho Saiku Shib
  • 4. Why do I select Presto? •  my familiar(on Twitter) people use •  comparison with other is a bother •  Let s try!
  • 5. How do I use Presto? •  batch with Hive on MR2(not Tez) •  select only with Presto •  execute adhoc query •  report by Cognos with Prestogres •  create presto view http://d.hatena.ne.jp/wyukawa/20140824/1408881620
  • 6. What is Cognos? •  Commercial BI tool by IBM •  Pros •  authorization management •  flexible reporting(but not easy) •  Cons •  slow rendering speed •  no permanent link •  difficult to understand error message •  web site(especially download site) is bad •  Windows and IE are necessary
  • 8. Why do I use Prestogres? •  Interesting •  MySQL is possible to be a bottleneck •  Hadoop is easier to scale than MySQL •  reduce maintenance cost for multi storages •  but not achieve
  • 9. Prestogres in my use case Presto 0.89 Cognos 10.2.1 Prestogres 0.4.8 Prestogres ODBC Driver It is not easy to connect Cognos to Presto. Thanks! > @frsyuki http://d.hatena.ne.jp/wyukawa/20140623/1403521909
  • 10. Problem in my use case •  Cognos don t issue where cause with Presgtogres+ODBC •  select … from … where yyyymmdd= 20150120 … •  slow rendering because of no predicate pushdown •  workaround is to use bigint to where cause, not string •  solution(not deploy in production. BTW, Thanks! > @frsyuki ) •  Cognos 10.2.2 •  patched(protocolVersion=2) PostgreSQL JDBC Driver •  Prestogres 0.6.3 •  Presto 0.86 •  increase dentry cache http://d.hatena.ne.jp/wyukawa/20141216/1418706186
  • 11. How do I manage Presto? •  How to deploy Presto? •  use Ansible •  Presto Setting •  task.cpu-timer-enabled=false •  How to monitor Presto? •  GrowthForecast + jstat2gf, JMX •  slow query visualization by nata2 http://d.hatena.ne.jp/wyukawa/20140803/1407052542 http://d.hatena.ne.jp/wyukawa/20141218/1418882574
  • 12. My impression for Presto •  stable •  frequent version up •  easy to install •  easy to upgrade •  but failed at 0.80, 0.87 •  leverage effect