Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Commercial
Fast and
Expensive
ClickHouse
Fast and Free
OpenSource
Free, but…
Data is represented as small single-dimensional arrays (vectors), easily accessible for CPUs.
The percentage of instructio...
X
Y
*
Scalar
operation
X1
Y1
*
SIMD
(Single instruction, multiple data)
X2
Y2
*
X3
Y3
*
Xn
Yn
*
* The image taken from [1]
Referer CounterID Date Username Country
Read only needed columns
Compression,Vectorizedprocessing
SELECT Referer, count(*)...
Query 1. Speedup vs CPU Threads
CPU Threads
Query 3. Speedup vs CPU Threads
CPU Threads
SELECT foo FROM distributed_table
SELECT foo FROM local_table GROUP BY col1
• Server 1
SELECT foo FROM local_table GROUP B...
N Servers 1 3 140
Time, sec 1.224 0.438 0.043
Speedup x2.8 x28.5
seconds
query
• Facts
• Dimensions
• Metrics
• Projections
ClickHouse on MemCloud
Kodiak Data and Altinity now Offer a Cloud Version of ClickHouse
45
1. FASTEST MPP Open Source DBMS...
MySQL CH
ProxySQL
binlog consumer
10B * 2K * [90-120] = [1.8-2.4]PB
ClickHouse
Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko
Upcoming SlideShare
Loading in …5
×

Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko

780 views

Published on

Supercharge your analytics with ClickHouse. v.2 updated, with additional slides. By Vadim Tkachenko

Published in: Technology
  • Login to see the comments

Supercharge your Analytics with ClickHouse, v.2. By Vadim Tkachenko

  1. 1. Commercial Fast and Expensive ClickHouse Fast and Free OpenSource Free, but…
  2. 2. Data is represented as small single-dimensional arrays (vectors), easily accessible for CPUs. The percentage of instructions spent in interpretation logic is reduced by a factor equal to the vector-size The functions that perform work now typically process an array of values in a tight loop Tight loops can be optimized well by compilers, enable compilers to generate SIMD instructions automatically. Modern CPUs also do well on such loops, out-of-order execution in CPUs often takes multiple loop iterations into execution concurrently, exploiting the deeply pipelined resources of modern CPUs. It was shown that vectorized execution can improve data-intensive (OLAP) queries by a factor 50.
  3. 3. X Y * Scalar operation X1 Y1 * SIMD (Single instruction, multiple data) X2 Y2 * X3 Y3 * Xn Yn *
  4. 4. * The image taken from [1]
  5. 5. Referer CounterID Date Username Country Read only needed columns Compression,Vectorizedprocessing SELECT Referer, count(*) AS count FROM hits WHERE CounterID = 1234 AND Date >= today() - 7 GROUP BY Referer ORDER BY count DESC LIMIT 10 Discard blocks by index (CounterID, Date) Parallel processing
  6. 6. Query 1. Speedup vs CPU Threads CPU Threads
  7. 7. Query 3. Speedup vs CPU Threads CPU Threads
  8. 8. SELECT foo FROM distributed_table SELECT foo FROM local_table GROUP BY col1 • Server 1 SELECT foo FROM local_table GROUP BY col1 • Server 2 SELECT foo FROM local_table GROUP BY col1 • Server 3
  9. 9. N Servers 1 3 140 Time, sec 1.224 0.438 0.043 Speedup x2.8 x28.5
  10. 10. seconds query
  11. 11. • Facts • Dimensions • Metrics • Projections
  12. 12. ClickHouse on MemCloud Kodiak Data and Altinity now Offer a Cloud Version of ClickHouse 45 1. FASTEST MPP Open Source DBMS 2. Cutting Edge Cloud for Big Data Apps and Processing 3. World-class ClickHouse Expertise Try the ClickHouse on MemCloud demo here http://clickhouse-demo.memcloud.works/
  13. 13. MySQL CH ProxySQL binlog consumer
  14. 14. 10B * 2K * [90-120] = [1.8-2.4]PB
  15. 15. ClickHouse

×