The document discusses using the Hyperloglog algorithm and Redis to approximately count unique items in a dataset in a space-efficient way. Some key points: - Hyperloglog is an algorithm that estimates the size of a set in a space-efficient manner, sacrificing some accuracy. - It provides a similar interface to a set but uses much less space (around 12kb) while maintaining the same time complexity. - Redis implements Hyperloglog which allows approximating uniques across programming languages and persisting counts to the Redis key-value store in a space-efficient way.

1. Lightning Talk: Counting with
Hyperloglog, Node.js & Redis
SIMON PRICKETT, REDIS UNIVERSITY.
TWITTER: @SIMON_PRICKETT
WEB: HTTPS://SIMONPRICKETT.DEV

2. Hyperloglog?
• An algorithm for the "count distinct" problem.
• What's the approximate size (cardinality) of a set?
• Optimizes space efficiency over absolute accuracy.

3. Counting Uniques with a Set – Accurate, but...

4. Counting Uniques with a Set – Using a data store...

5. Counting Uniques with the Hyperloglog Algorithm
Benefits
• Similar interface to a Set
• Same performance (time complexity) as using a Set
• Much more space efficient than a Set (~ 12Kb)
• Can't retrieve items, unlike a Set
Trade-Offs
• Absolute accuracy
• Can't retrieve items, unlike a Set
• Not built into JS, need an implementation...

6. Hyperloglog Implementation
Add Function Count Function
TL;DR Don't make your own, use a library or other implementation!

7. Redis Hyperloglogs
• Fast, open source
• Uses maximum 12kb
• Count up to 2^64 uniques
(18,446,744,073,709,552,000)
• Persists to Redis key/value store
• Support for most programming languages
• Count estimates have 0.81% standard error

8. Approximating Uniques with Redis Hyperloglog

9. Resources: Redis University on YouTube

10. Thank you!
university.redislabs.com
simonprickett.dev
Twitter: @simon_prickett