Salvatore Incandela Chief Architect, presented "Loyalty cashback - Scaling with MongoDB" on 25/09/2014 in Florence, at the "BIGDATATECH" an event in which they were presented some use cases of Big Data in real context.

1. Firenze,
25 Settembre
Loyalty
cashback
-­‐
Scaling
with
MongoDB
Salvatore
Incandela
1

2. 2
About Me
IT
Architecture
Agile
Open
Source
NoSql
R&D
Security
Big Data
DDD
CQRS
Change
Management
Electronic
Money
Continous
Improvement
Organization
Paybay - Copyright 2014 - All Right Reserved

3. 3
Agenda
• Context
• Problem Setting
• Problem Solving
• Problem Solved
• MongoDb Focus
• MongoDb Cluster Config
• MongoDb Cluster Status
• Our Loyalty Clients
• Next Steps
Paybay - Copyright 2014 - All Right Reserved

4. 4
Context
Loyalty Cashback Program, how it works:
Card Holder Buy Affiliated
Merchant
Acquires
Transaction
Bank
Send
Transaction
Give Rewards
Loyalty Platform
Paybay - Copyright 2014 - All Right Reserved

5. 5
Context
Big Loyalty Cashback Program
Card Holder
10Mln 50k+
Very Big!!
2+Mln per day
1,5+Mln per day
50k+ per day
1,5+Mln per day
Buy Affiliated
Merchant
Acquires
Transaction
Bank
Send
Transaction
Give Rewards
Loyalty Platform
Paybay - Copyright 2014 - All Right Reserved

6. 6
Context
Big Loyalty Cashback Program
Just for one Bank but…
10Mln 50k+
Card Holder Buy Affiliated Merchant
….the platform will be multitenant!
Give Rewards Acquires Transactions
Loyalty Platform Send Transactions Bank
Very Big!!
2+Mln per day
1,5+Mln per day
50k+ per day
1,5+Mln per day
Paybay - Copyright 2014 - All Right Reserved

7. 7
Problem Setting
• Shared DataStores causes poor performances
• FrontEnd performances
• BackEnd performances
• High scalability costs
• Lack of Smart Business and Operational Analitics
Paybay - Copyright 2014 - All Right Reserved

8. 8
Problem Setting
WebApps
RDBMS
Batch Engine
Core Services
Data Acces Layer
Performances
Budget
Mobile Web
Services
Analitics Is this solution economically sustainable?
How to Scale the Data Store?
Paybay - Copyright 2014 - All Right Reserved

9. 9
Problem Solving
Performances
WebApps Mobile Web
Budget
Batch Engine
Analitics Core Services
Data Acces Layer
RDBMS
RDBMS
Services
ß Not for Us
Paybay - Copyright 2014 - All Right Reserved

10. 10
Problem Solving
CQRS originally introduced by Greg Young
The change that CQRS introduces is to split the conceptual model into
separate models for update and display, which it refers to as Command
and Query respectively following the vocabulary of
CommandQuerySeparation.
The rationale is that for many problems, particularly in more complicated
domains, having the same conceptual model for commands and queries
leads to a more complex model that does neither well.
Martin Fowler
http://martinfowler.com/bliki/CQRS.html
Paybay - Copyright 2014 - All Right Reserved

11. 11
Problem Solving fase 1
WebApps Mobile Web
MongoDb
Performances
Budget
Analitics
Document Modeled
Fast Reads to more performances
Business Analitics collections
Operational Analitics Collections
E/R Modeled
Reliable
Strong Competencies
CQRS Principle transitions
Batch Engine
Data Acces Layer
Services
Core Services Commands Reads
ß Only for Fast Reads
RDBMS
Paybay - Copyright 2014 - All Right Reserved

12. 12
Problem Solved!
WebApps Mobile Web
MongoDb
Performances
Budget
Analitics
CQRS Principle transitions
MongoDb MongoDb
More Reliable configuration
Batch Engine
Core Services
Data Acces Layer
Services
Commands Reads
RDBMS
Still used
ß Only for Fast Reads
RDBMS
Paybay - Copyright 2014 - All Right Reserved

13. 13
MongoDb Focus
Noteworthy
• MondoDb is flexible schema
• De-normalized
• MMS: Monitoring, Backup tools
• Database Metrics, HW Statistics
• OOP Oriented no ORM Mappings
MongoDb MongoDb MongoDb
Paybay - Copyright 2014 - All Right Reserved

14. 14
MongoDb Cluster Config
Cluster Configuration
• 3 Node
• Replica Set
• 9 Collections (the biggest contains 700mln raw data)
• Compound Index
• Background indexes
• TTL Indexes
MongoDb MongoDb MongoDb
Paybay - Copyright 2014 - All Right Reserved

15. 15
MongoDb Cluster Status
Manage Tb of Data
• Up and running about 3 years without fault
• Increase Reads parformances up to 60%
• Decrease costs about 70%
• Self Learned: good documentations
MongoDb MongoDb MongoDb
Paybay - Copyright 2014 - All Right Reserved

16. 16
Our Loyalty Clients
Paybay - Copyright 2014 - All Right Reserved

17. 17
Next Steps?
We’ll adopt MongoDb as Fast Read DataStore
for the entire Blue Tiger Platform
MongoDb MongoDb MongoDb
MongoDb MongoDb MongoDb
MongoDb MongoDb MongoDb
800Mln Trx per Year
Loyalty
Couponing
Ticketing
Payments
Issuing
150k Clients
800k Trx per Year
10Mln Trx per Year
1Mln Cards per Year
MongoDb MongoDb MongoDb
MongoDb MongoDb MongoDb
MongoDb MongoDb MongoDb
MongoDb MongoDb MongoDb
Paybay - Copyright 2014 - All Right Reserved

18. 18
Contacts
{ name: “Salvatore Incandela”,
twitter: “@sincandela”,
email: “salvatore.incandela@paybay.it”,
linkedin: “https://www.linkedin.com/in/salvatoreincandela”}
Paybay - Copyright 2014 - All Right Reserved

19. Grazie

20. Q&A