What is redis ? Redis Features Replication High availability Sharding Redis Cluster Redis inside Docker containers

1. TechLabs by
A la découverte de Machine Learning, de Redis et de Spark

2. TechLabs by
2
Maturin BADO
@mccstanmbg
github.com/mccstan
REDIS

3. Redis : Introduction
3

4. ❏ What is Redis ?
❏ Get into redis
❏ Redis features
❏ Some Redis good use cases
❏ Redis not suited for ...
❏ Behind redis
4

5. NoSQL Databases :
❏ Document DBs
❏ Mongo DB, Couch DB
❏ Graph DBs
❏ Neo4j, Flock DB
❏ Column oriented DB
❏ Hbase, Cassandra, Big Table
❏ Key value DBs
❏ Memcache, Memcache DB, Redis
5

6. In-memory key-value store (DB)
6

7. Advanced In-Memory Data Structure Store
Key
Key
key
value Container
Primitive
String
7

8. Multi-Temperature Data
8

9. An external solution for your Application
9

10. 10

11. 11

12. ❏ Atomic operations
❏ Single threaded (Event loop)
❏ Publish/Subscribe
❏ Replication
❏ High availability
❏ redis sentinel
❏ Sharding
❏ redis cluster, twemproxy
❏ Auto eviction of keys
❏ Transactions support
❏ Indices
❏ Many clients/programing language
bindings
❏ Ruby, Python, Lua, C, Java, scala, Nodejs, etc
12

13. Tracking website visitors (1/2)
13

14. Tracking website visitors (2/2)
14

15. Analytics – Redis with Spark integration
15

16. Don’t use redis when …
16

17. ❏ An open source project
❏ Initially released on 2009 by @antirez (Salvatore Sanfilippo)
❏ An active community
❏ github, stackoverflow, mailing list
❏ Local meetups : London, San Francisco, New York, Tel Aviv , Paris (by SOAT)
❏ Redis Labs
❏ Commercial provider of Redis Enterprise
17

18. Replication on Redis
18

19. ❏ Distribution purposes
❏ What is Replication ?
❏ Replication on Redis
❏ Replication issues
19

20. ❏ Scalability
❏ Fault tolerance/High availability
❏ Latency
❏ Disconnected operation
20

21. Replication means keeping a copy of the same data on multiple
machines that are connected via a network. Martin Klepmann
Main approaches :
❏ Single-leader replication
❏ Multi-leader replication
❏ Leaderless replication
21

22. ❏ Asynchronous replication
❏ Multiple slaves
❏ Slaves can also be connected in a cascading-like structure
❏ Non-blocking both on master and slave side
❏ Scalability, with multiple read-only slaves
❏ Simple way configuration
slaveof <master ip> <port>
22

23. 23

24. ❏ Eventual consistency with read only slaves
❏ Window of time while some writes could be lost
❏ When writing to slaves, a master reboot could wipe slave datas
❏ No High availability at this point of configs
24

25. High availability on Redis
25

26. ❏ What is High Availability ?
❏ Redis solution for HA : Sentinel
❏ Redis Sentinel : a distributed system
❏ Configuring redis sentinel
❏ Architectural approaches
26

27. An high available system should be reliable
Reliability
“The system should continue to work correctly (performing the correct
function at the desired level of performance) even in the face of adversity
(hardware or software faults, and even human error).” M. K.
27

28. ❏ Monitoring
❏ Notification
❏ Automatic failover
❏ Configuration provider
❏ A distributed system
❏ Consensus-like algorithm
28

29. 29

30. 30

31. Partitioning (Sharding) on
Redis
31

32. ❏ What is Partitioning ?
❏ Why Partitioning ?
❏ Different implementations of partitioning
❏ Implementations of Redis partitioning
❏ Partitioning issues on redis
32

33. Splitting a big database into smaller subsets called partitions so that
different partitions can be assigned to different nodes (also known as
sharding). M. K
Partitioning is usually combined with replication
33

34. ❏ Limited amount of memory on a single computer
❏ Split Larger databases across multiple machines
❏ Using the sum of performance of many computers
❏ Allows scaling out
34

35. ❏ Approaches :
❏ range partitioning
❏ hash partitioning
❏ consistent hashing
❏ Responsibilities on a software stack :
❏ Client side partitioning
❏ Proxy assisted partitioning
❏ Query routing
35

36. ❏ Redis Cluster
❏ A mix between query routing and client side partitioning.
❏ Data is Automatically sharded across multiple Redis nodes
❏ Continue operations when a subset of the nodes fails
❏ Eventual consistency
❏ Trade-off between performance and consistency
36

37. 37

38. ❏ Twemproxy
38

39. ❏ Multiple keys operations not supported
❏ multiple keys transactions can’t be used
❏ Data handling is more complex
❏ Adding and removing capacity can be complex :
❏ Client side partitioning and proxies don't support it
❏ Pre-sharding may help
❏ Redis cluster is the easy way to deal with sharding :
❏ production-ready since April 1st, 2015
39

40. Redis inside docker
40

41. ❏ What is docker ?
❏ Motivations
❏ Begin with redis on docker
41

42. ❏ Open source project
❏ Auto deployment of applications inside software containers
❏ An additional layer : operating-system-level virtualization
42

43. ❏ Fashion ?
❏ Isolated applications with their dependencies
❏ Easy and quick deployments
❏ Easy failover management
43

44. ❏ An high availability solution
❏ Orchestration solutions : Mesos, Kubernetes, Swarm
❏ Well suited for continuous delivery and DevOps.
❏ Virtual Machines dies ?
44

45. $ docker run --name some-redis -d redis
$ docker run -it --link some-redis:redis --rm redis redis-cli
-h redis -p 6379
45

46. https://github.com/SoatGroup/redis