-If combinators on aggregate functions are one of those amazing features that makes you wish SQL would just standardize on the ClickHouse dialect. This 10 minute talk shows why you want to use them. Altinity: https://altinity.com Meetup: https://www.meetup.com/San-Francisco-Bay-Area-ClickHouse-Meetup/events/285271332/

1. © 2022 Altinity, Inc.
ClickHouse -If
Combinators for Fun and
Profit
Robert Hodges, Altinity
SF Bay Area ClickHouse Meetup
May 4, 2022
1

2. © 2022 Altinity, Inc.
Let’s make some introductions
ClickHouse support and services including Altinity.Cloud
Authors of Altinity Kubernetes Operator for ClickHouse
and other open source projects
Robert Hodges
Database geek with 30+ years
on DBMS systems. Day job:
Altinity CEO
Altinity Engineering
Database geeks with centuries
of experience in DBMS and
applications
2

3. © 2022 Altinity, Inc.
What are -If Combinators?
ClickHouse Aggregation Functions
● count()
● min(value)
● max(value)
● sum(value)
● avg(value)
● uniq(value)
● uniqExact(value)
● …
3
-If versions of same functions
● countIf(condition)
● minIf(value, condition)
● maxIf(value, condition)
● sumIf(value, condition)
● avgIf(value, condition)
● uniqIf(value, condition)
● uniqExactIf(value, condition)
● …

4. © 2022 Altinity, Inc.
Finding dirty data using a histogram
– NYC Taxi data: fare amounts
WITH histogram(5)(fare_amount) AS hist
SELECT
arrayJoin(hist) AS hist1,
bar(hist1.3, 0, 1000, 20) AS bar
FROM tripdata
┌─hist1──────────────────────────────────────────────────┬─bar──────────────────┐
│ (-21474808,-12079576.375,1) │ │
│ (-12079576.375,-1342166.6614073187,163862993.25) │ ████████████████████ │
│ (-1342166.6614073187,177236.82575785994,983177956.125) │ ████████████████████ │
│ (177236.82575785994,502005.5770089285,163863003.875) │ ████████████████████ │
│ (502005.5770089285,825998.625,8.75) │ ▏ │
└────────────────────────────────────────────────────────┴──────────────────────┘
5 rows in set. Elapsed: 6.853 sec. Processed 1.31 billion rows, 5.24 GB
(191.28 million rows/s., 765.11 MB/s.)
4

5. © 2022 Altinity, Inc.
Finding dirty data with countIf()
– NYC Taxi data: fare amounts
SELECT
countIf(fare_amount < 0) AS fare_less_than_zero,
countIf((fare_amount >= 0) AND (fare_amount < 100)) AS fare_0_to_100,
countIf((fare_amount >= 100) AND (fare_amount < 1000)) AS fare_100_to_1000,
countIf(fare_amount > 1000) AS fare_1000_or_greater
FROM tripdata
┌─fare_less_than_zero─┬─fare_0_to_100─┬─fare_100_to_1000─┬─fare_1000_or_greater─┐
│ 128932 │ 1310237048 │ 537609 │ 373 │
└─────────────────────┴───────────────┴──────────────────┴──────────────────────┘
1 rows in set. Elapsed: 0.898 sec. Processed 1.31 billion rows, 5.24 GB
(1.46 billion rows/s., 5.84 GB/s.)
5
Simpler output and faster too!

6. © 2022 Altinity, Inc.
Searching for GitHub event counts using GROUP BY
-- Github event data by year.
SELECT
toYear(created_at) AS year, event_type, count()
FROM github_events
WHERE event_type IN ('PullRequestEvent', 'WatchEvent')
GROUP BY year, event_type
ORDER BY year ASC, event_type ASC
┌─year─┬─event_type───────┬──count()─┐
│ 2011 │ PullRequestEvent │ 476818 │
│ 2011 │ WatchEvent │ 1831742 │
│ 2012 │ PullRequestEvent │ 1807044 │
│ 2012 │ WatchEvent │ 4048676 │
6

7. © 2022 Altinity, Inc.
Pivoting event data using countIf()
-- Github event data by year.
SELECT
toYear(created_at) AS year,
countIf(event_type = 'PullRequestEvent') AS PRs,
countIf(event_type = 'WatchEvent') AS Stars
FROM github_events
WHERE event_type IN ('PullRequestEvent', 'WatchEvent')
GROUP BY year
┌─year─┬──────PRs─┬────Stars─┐
│ 2011 │ 476818 │ 1831742 │
│ 2012 │ 1807044 │ 4048676 │
│ 2013 │ 3103759 │ 7432800 │
│ 2014 │ 5923843 │ 11952935 │
7
Way slower if you
omit the IN clause

8. © 2022 Altinity, Inc.
Cleanup up multiple inputs on the fly with avgIf()
SELECT
count() AS rides,
avgIf(passenger_count, passenger_count < 10)
AS passengers_clean,
avgIf(fare_amount, (fare_amount >= 0) AND (fare_amount < 500))
AS fares_clean
FROM tripdata
┌──────rides─┬──passengers_clean─┬────────fares_clean─┐
│ 1310903963 │ 1.681498872092554 │ 11.424149328643022 │
└────────────┴───────────────────┴────────────────────┘
1 rows in set. Elapsed: 1.092 sec. Processed 1.31 billion rows, 6.55 GB
(1.20 billion rows/s., 6.00 GB/s.)
8
Multiple conditions
in a single scan!

9. © 2022 Altinity, Inc.
Finding out more tricks for using -If combinators
9
● ClickHouse docs - Very limited
● Altinity Knowledge Base - Examples using system tables and clever tricks
○ Example: Simple aggregate functions & combinators
● Tests in ClickHouse codebase – Examples of what’s possible but not why
cd ~/git/ClickHouse/src/tests
grep -r countIf .
grep -r avgIf .
grep -r weightedSumIf .

10. © 2022 Altinity, Inc.
© 2022 Altinity, Inc.
Thank you!
Questions?
https://altinity.com
rhodges at altinity.com
10
Altinity.Cloud
Software and support for
ClickHouse
We’re hiring!