Talk given at LuaConf 2017 in Rio de Janerio about the Cassandra driver for Lua and OpenResty.

1. A Cassandra driver from and
for the Lua community
LuaConf 2017 - Rio de Janeiro
Thibault Charbonnier
Kong lead engineer
OpenResty contributor
https://github.com/thibaultcha
1

2. Cassandra - History & Use Cases
2

3. Apache Cassandra - History
- Free and open-source NoSQL database
- Published by Facebook in 2008
- Joined the Apache Foundation in 2010
3

4. Apache Cassandra - Use cases
- Decentralized
- Multi datacenter replication
- Horizontal scalability (linear)
- “Key/Value meets column-oriented”
- CQL
A typical Cassandra cluster, often
represented as a ring
4

5. Apache Cassandra - Use cases
- Leans towards AP
- Tunable Consistency
5

6. Kong - History & Use Cases
6

7. Kong - History & Use Cases
- API Gateway/Microservices orchestration tool
- Started in 2014
- Free and open-source (Community Edition)
More on Kong during Thijs Schreijer’s lightning talk at 4:45 PM
https://github.com/Mashape/kong
7

8. Kong - Use Case
8

9. Why Cassandra for Kong?
Long story short: business requirements!
- Multi data center support “out of the box”
- Ease to scale (linear + multi DC support)
- Counters! Great for rate-limiting APIs
- Great for time series data (logging & monitoring)
One drawback... We need a pure Lua implementation!
9

10. Enters: lua-resty-cassandra
10

11. lua-resty-cassandra - History
- Written by Juarez Bochi (@jbochi) in 2014
- Pure Lua (uses OpenResty’s cosockets)
- Used by globo.com for its video streaming platform
- Released under MIT License
- Support for binary protocol v2 (Cassandra 2.x)
- Single node
https://github.com/jbochi/lua-resty-cassandra
11

12. lua-resty-cassandra - First Improvements
Better support for Lua/OpenResty interoperability
“Hmmm this is going to be a problem…”
oops
12

13. lua-resty-cassandra - First Improvements
Batch support, yay!
“Thanks Juarez!”
Batch support is necessary to provide Atomicity when updating
several partition keys at once.
13

14. lua-resty-cassandra - First Improvements
Pagination Support!
14

15. lua-resty-cassandra - A fork?
- No love for our binary protocol v3 contribution :’(
- Lack of responses from @jbochi :’(
- Business requirements!
Cassandra 3.x
Binary protocol v3
Load balancing
policies
Retry policies
Cluster awareness
Multi data center
Client-to-node SSL
Client
authentication
Schema consensus
15

16. Introducing: lua-cassandra
16

17. Introducing: lua-cassandra
Datastax-like driver for Lua:
- Support for cluster awareness
- Load balancing/retry/reconnection policies
- Support for SSL connections
- Cassandra 3.x - binary protocol v3
- Good documentation with concrete examples
- Interoperability between OpenResty/Lua for
community usage
Forked in May 2015
We keep the lua-resty-cassandra
marshallers!
17

18. lua-cassandra - Two Modules
Single node:
Cluster aware:
OpenResty only :’(
- Cluster topology discovery
- Auto-prepared statements
- Schema consensus wait on
DDL queries
- Retries
- Policies
- Automatic binary protocol
selection (v2/v3/v4)
- Support for SSL connections
- Support for authentication
18

19. lua-cassandra - CQL queries
Example of a DDL query
19

20. lua-cassandra - CQL queries
Example of a DML query
20

21. lua-cassandra - CQL queries
Example of a DML query with
Cluster module
21

22. lua-cassandra - Authentication
Support for PasswordAuthenticator:
22

23. lua-cassandra - Policies
Load balancing - round-robin or datacenter-aware round-robin
Reconnection - simple or exponential
23

24. lua-cassandra - Policies
24

25. leafo/pgmoon has the original Lua/OpenResty interoperability module
https://github.com/leafo/pgmoon
lua-resty-socket: An improved version released as a separate library
https://github.com/thibaultcha/lua-resty-socket
- Fallback on LuaSocket when cosockets are not available
- Fallback on LuaSec when SSL is enabled
lua-cassandra - Interoperability
25

26. lua-cassandra - Testing
LuaCheck - https://github.com/mpeterv/luacheck
busted - https://github.com/Olivine-Labs/busted
Test::Nginx - https://github.com/openresty/test-nginx
LuaCov - https://github.com/keplerproject/luacov
26

27. lua-cassandra - Documentation
Built with LDoc - https://github.com/stevedonovan/LDoc
Hosted on GitHub - https://thibaultcha.github.io/lua-cassandra/
27

28. Reflexions
28

29. Maintaining code for Lua/LuaJIT/OpenResty
- Sometimes hard because of disparities:
- Language: (table.unpack, “table.new”…)
- Environment: cosockets vs LuaSocket (or other…) & embedding
application
- Requires a heavy test suite
- Modularization really helped
- Consider the needs for the clients of your library, to avoid fragmentation
Dropping support for PUC-Lua?
More modularization to support cluster-awareness in PUC-Lua
29

30. We are hiring!
https://mashape.com/jobs
And contributors are welcome!
30

31. Thank you!
Questions time!
31