Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
A Cassandra driver from and
for the Lua community
LuaConf 2017 - Rio de Janeiro
Thibault Charbonnier
Kong lead engineer
Op...
Cassandra - History & Use Cases
2
Apache Cassandra - History
- Free and open-source NoSQL database
- Published by Facebook in 2008
- Joined the Apache Found...
Apache Cassandra - Use cases
- Decentralized
- Multi datacenter replication
- Horizontal scalability (linear)
- “Key/Value...
Apache Cassandra - Use cases
- Leans towards AP
- Tunable Consistency
5
Kong - History & Use Cases
6
Kong - History & Use Cases
- API Gateway/Microservices orchestration tool
- Started in 2014
- Free and open-source (Commun...
Kong - Use Case
8
Why Cassandra for Kong?
Long story short: business requirements!
- Multi data center support “out of the box”
- Ease to sc...
Enters: lua-resty-cassandra
10
lua-resty-cassandra - History
- Written by Juarez Bochi (@jbochi) in 2014
- Pure Lua (uses OpenResty’s cosockets)
- Used b...
lua-resty-cassandra - First Improvements
Better support for Lua/OpenResty interoperability
“Hmmm this is going to be a pro...
lua-resty-cassandra - First Improvements
Batch support, yay!
“Thanks Juarez!”
Batch support is necessary to provide Atomic...
lua-resty-cassandra - First Improvements
Pagination Support!
14
lua-resty-cassandra - A fork?
- No love for our binary protocol v3 contribution :’(
- Lack of responses from @jbochi :’(
-...
Introducing: lua-cassandra
16
Introducing: lua-cassandra
Datastax-like driver for Lua:
- Support for cluster awareness
- Load balancing/retry/reconnecti...
lua-cassandra - Two Modules
Single node:
Cluster aware:
OpenResty only :’(
- Cluster topology discovery
- Auto-prepared st...
lua-cassandra - CQL queries
Example of a DDL query
19
lua-cassandra - CQL queries
Example of a DML query
20
lua-cassandra - CQL queries
Example of a DML query with
Cluster module
21
lua-cassandra - Authentication
Support for PasswordAuthenticator:
22
lua-cassandra - Policies
Load balancing - round-robin or datacenter-aware round-robin
Reconnection - simple or exponential...
lua-cassandra - Policies
24
leafo/pgmoon has the original Lua/OpenResty interoperability module
https://github.com/leafo/pgmoon
lua-resty-socket: An i...
lua-cassandra - Testing
LuaCheck - https://github.com/mpeterv/luacheck
busted - https://github.com/Olivine-Labs/busted
Tes...
lua-cassandra - Documentation
Built with LDoc - https://github.com/stevedonovan/LDoc
Hosted on GitHub - https://thibaultch...
Reflexions
28
Maintaining code for Lua/LuaJIT/OpenResty
- Sometimes hard because of disparities:
- Language: (table.unpack, “table.new”…...
We are hiring!
https://mashape.com/jobs
And contributors are welcome!
30
Thank you!
Questions time!
31
You’ve finished this document.
Download and read it offline.
Upcoming SlideShare
What to Upload to SlideShare
Next
Upcoming SlideShare
What to Upload to SlideShare
Next
Download to read offline and view in fullscreen.

Share

A Cassandra driver from and for the Lua community

Download to read offline

Talk given at LuaConf 2017 in Rio de Janerio about the Cassandra driver for Lua and OpenResty.

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

A Cassandra driver from and for the Lua community

  1. 1. A Cassandra driver from and for the Lua community LuaConf 2017 - Rio de Janeiro Thibault Charbonnier Kong lead engineer OpenResty contributor https://github.com/thibaultcha 1
  2. 2. Cassandra - History & Use Cases 2
  3. 3. Apache Cassandra - History - Free and open-source NoSQL database - Published by Facebook in 2008 - Joined the Apache Foundation in 2010 3
  4. 4. Apache Cassandra - Use cases - Decentralized - Multi datacenter replication - Horizontal scalability (linear) - “Key/Value meets column-oriented” - CQL A typical Cassandra cluster, often represented as a ring 4
  5. 5. Apache Cassandra - Use cases - Leans towards AP - Tunable Consistency 5
  6. 6. Kong - History & Use Cases 6
  7. 7. Kong - History & Use Cases - API Gateway/Microservices orchestration tool - Started in 2014 - Free and open-source (Community Edition) More on Kong during Thijs Schreijer’s lightning talk at 4:45 PM https://github.com/Mashape/kong 7
  8. 8. Kong - Use Case 8
  9. 9. Why Cassandra for Kong? Long story short: business requirements! - Multi data center support “out of the box” - Ease to scale (linear + multi DC support) - Counters! Great for rate-limiting APIs - Great for time series data (logging & monitoring) One drawback... We need a pure Lua implementation! 9
  10. 10. Enters: lua-resty-cassandra 10
  11. 11. lua-resty-cassandra - History - Written by Juarez Bochi (@jbochi) in 2014 - Pure Lua (uses OpenResty’s cosockets) - Used by globo.com for its video streaming platform - Released under MIT License - Support for binary protocol v2 (Cassandra 2.x) - Single node https://github.com/jbochi/lua-resty-cassandra 11
  12. 12. lua-resty-cassandra - First Improvements Better support for Lua/OpenResty interoperability “Hmmm this is going to be a problem…” oops 12
  13. 13. lua-resty-cassandra - First Improvements Batch support, yay! “Thanks Juarez!” Batch support is necessary to provide Atomicity when updating several partition keys at once. 13
  14. 14. lua-resty-cassandra - First Improvements Pagination Support! 14
  15. 15. lua-resty-cassandra - A fork? - No love for our binary protocol v3 contribution :’( - Lack of responses from @jbochi :’( - Business requirements! Cassandra 3.x Binary protocol v3 Load balancing policies Retry policies Cluster awareness Multi data center Client-to-node SSL Client authentication Schema consensus 15
  16. 16. Introducing: lua-cassandra 16
  17. 17. Introducing: lua-cassandra Datastax-like driver for Lua: - Support for cluster awareness - Load balancing/retry/reconnection policies - Support for SSL connections - Cassandra 3.x - binary protocol v3 - Good documentation with concrete examples - Interoperability between OpenResty/Lua for community usage Forked in May 2015 We keep the lua-resty-cassandra marshallers! 17
  18. 18. lua-cassandra - Two Modules Single node: Cluster aware: OpenResty only :’( - Cluster topology discovery - Auto-prepared statements - Schema consensus wait on DDL queries - Retries - Policies - Automatic binary protocol selection (v2/v3/v4) - Support for SSL connections - Support for authentication 18
  19. 19. lua-cassandra - CQL queries Example of a DDL query 19
  20. 20. lua-cassandra - CQL queries Example of a DML query 20
  21. 21. lua-cassandra - CQL queries Example of a DML query with Cluster module 21
  22. 22. lua-cassandra - Authentication Support for PasswordAuthenticator: 22
  23. 23. lua-cassandra - Policies Load balancing - round-robin or datacenter-aware round-robin Reconnection - simple or exponential 23
  24. 24. lua-cassandra - Policies 24
  25. 25. leafo/pgmoon has the original Lua/OpenResty interoperability module https://github.com/leafo/pgmoon lua-resty-socket: An improved version released as a separate library https://github.com/thibaultcha/lua-resty-socket - Fallback on LuaSocket when cosockets are not available - Fallback on LuaSec when SSL is enabled lua-cassandra - Interoperability 25
  26. 26. lua-cassandra - Testing LuaCheck - https://github.com/mpeterv/luacheck busted - https://github.com/Olivine-Labs/busted Test::Nginx - https://github.com/openresty/test-nginx LuaCov - https://github.com/keplerproject/luacov 26
  27. 27. lua-cassandra - Documentation Built with LDoc - https://github.com/stevedonovan/LDoc Hosted on GitHub - https://thibaultcha.github.io/lua-cassandra/ 27
  28. 28. Reflexions 28
  29. 29. Maintaining code for Lua/LuaJIT/OpenResty - Sometimes hard because of disparities: - Language: (table.unpack, “table.new”…) - Environment: cosockets vs LuaSocket (or other…) & embedding application - Requires a heavy test suite - Modularization really helped - Consider the needs for the clients of your library, to avoid fragmentation Dropping support for PUC-Lua? More modularization to support cluster-awareness in PUC-Lua 29
  30. 30. We are hiring! https://mashape.com/jobs And contributors are welcome! 30
  31. 31. Thank you! Questions time! 31
  • edisonqkj

    May. 6, 2020
  • wedgwood

    May. 19, 2019
  • DorLaor

    Sep. 21, 2018
  • archoncap

    Jan. 12, 2018

Talk given at LuaConf 2017 in Rio de Janerio about the Cassandra driver for Lua and OpenResty.

Views

Total views

719

On Slideshare

0

From embeds

0

Number of embeds

27

Actions

Downloads

10

Shares

0

Comments

0

Likes

4

×