RedisConf19

1. The Rise of DataOps - SQL on Redis
Andrew Stevenson
Lenses, CTO

2. Speaker – Andrew Stevenson
CTO at lenses.io
C++, Data Warehousing, Big/Fast Data
Always realtime
Clearing & Settlement
HFT
Investment Banking
Energy
Netherlands
lenses.io

3. lenses.io
lenses.io/careers
We hiring! Location doesn’t matter

4. The real agenda
DevOps to DataOps
SQL on Redis with Lenses
lenses.io

5. Technology focused
DevOps
lenses.io
$

6. Streaming
Data Flows
Data Visibility
Data
Governance
DataOps by Lenses
Enable everyone
Data focused
Includes DevOps
$

7. What does a Data Platform look
like?
lenses.io
Sources StorageTransport
Processing

8. DataOps Platform with Lenses?
lenses.io
Visibility and Flows
Operations
Security and Auditing
Data Policies

9. Lenses DataOps Platform
lenses.io

10. Lenses DataOps Platform
lenses.io

11. Lenses DataOps Platform
lenses.io

12. lenses.io
SQL

13. lenses.io
SQL
Yes, we know, not everything can
be done in SQL

14. lenses.io
SQL
Yes, we know, not everything can be done in
SQL
But a huge amount can

15. Lenses SQL Engine
lenses.io
Table Query
Continuous Query
SQL Processors
Connect Query
} Pluggable

16. 50 1 2 3 4 6 7 8 9 10
11
query start time
GBPGBP GBP GBPGBP GBP
SELECT eventId
FROM trades
WHERE currency = ‘GBP’
Events { 1, 3, 5 }
12
SELECT eventId
FROM trades
WHERE currency = ‘GBP’
Events { 7, 8, 9, 11, 12, … }
Continuous QueryTable Query

17. What’s supported?
Aggregates
Functions
User defined functions
Joins
Predicates including pushdowns
Complex types
lenses.io
Client libraries

18. Table Query
lenses.io

19. Continuous Queries
lenses.io

20. Table & Continuous Queries
lenses.io
Authenticate
Authorize
Apply data policies
Start Akka Stream

21. SQL Processors
Long running processor
T in ETL or ELT
SQL to Kafka Streams
On Redis Streams?

22. Config as Code
lenses.io
SQL +
One docker image different SQL
+

23. Demo!
lenses.io

24. Thank you!

25. PRESENTED BY