Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
RAVENDB EMBEDDED AT
MASSIVE SCALES
ABOUT ME
Rodrigo Rosauro, Software Architect Manager at RDI
• Technology passionate
• Developer
• 14 years of experience i...
ABOUT RDI
• We make restaurant automation software for QSR
• Our software runs on more than 36K restaurants
• Estimated nu...
“ONE IN A MILLION IS NEXT TUESDAY”
- Gordon Letwin, Architect for DOS 4
HOW WE USED TO PERSIST DATA ON POS
... and still do (on legacy modules)
LEGACY PERSISTENCY
• Flat files
• Custom data format
• Event sourcing persistence
• Rebuild in-memory state at
every resta...
PROBLEMS WITH THAT APPROACH
•No querying – at all
•High complexity to store new data
•No standard data format
•Lack of man...
WHY WE DECIDED TO USE A DBMS?
WHY A DBMS?
• We started a full re-architecture of the POS platform
• We are definitely not database/persistence specialis...
THE SEEK FOR A DBMS
GOALS / GUIDELINES
• The new architecture is based on plug-ins, so we wanted each
individual plug-in to have its own, isol...
WHY WE CHOSE RAVENDB?
WHY RAVENDB?
• .NET
• Embedded operation mode
• In-memory mode for unit tests
• Multi-tenant
• Good transparent replicatio...
WHAT RAVENDB ALLOWED US TO DO
RAVENDB AT RDI
• Create unit tests without mocking the database
• Have advanced statistics about the data persistence
• Bo...
THE CHALLENGES
... and how we faced them
CHALLENGE #1 – MEMORY CONSTRAINTS
SPECIALIZED, OLD HARDWARE
Old hardware (sometimes 10 years old) means that we have very little
memory and processing power.
MEMORY CONSTRAINTS
• Many iterations with the RavenDB support team to fine-tune its memory
usage
• We learned that under t...
CHALLENGE #2 – ESENT
WE ARE STILL USING RAVENDB 2.5
• Our software must support Windows XPe
• .NET 4.0
• 32 bits machines
• … this means that E...
FACT: WE DON’T LIKE ESENT
(at least not for our usage scenarios)
WHY?
• Any unclean shutdown may cause a completely undetermined result
• Power outages / Application crashes / Task kill
•...
ESENTUTL.EXE
• We also hate don’t like ESENTUTL.EXE
• Many different commands to attempt to recover from different kinds o...
“SUCCESS IS STUMBLING FROM FAILURE TO
FAILURE WITH NO LOSS OF ENTHUSIASM”
― Winston S. Churchill, Ex-prime minister of UK
THE FUTURE
THE FUTURE
• Working with Hibernating Rhinos to improve Voron to our use case
• 32 bits support
• Better reliability on un...
THANK YOU VERY MUCH
Upcoming SlideShare
Loading in …5
×

RavenDB embedded at massive scales

656 views

Published on

One particular (and often forgotten) use-case for RavenDB is its usage as an embedded database. This operation mode allows application providers to abstract the complexity of database administration from their end-users while, at the same time, providing you a fully functional document store.
During this talk we will explore the challenges faced while deploying RavenDB in a massive number of machines throughout the globe (aiming at hundreds of thousands), and how RavenDB improved the capabilities of our application.

Published in: Technology
  • Login to see the comments

  • Be the first to like this

RavenDB embedded at massive scales

  1. 1. RAVENDB EMBEDDED AT MASSIVE SCALES
  2. 2. ABOUT ME Rodrigo Rosauro, Software Architect Manager at RDI • Technology passionate • Developer • 14 years of experience in software development • Working with RavenDB since 2013
  3. 3. ABOUT RDI • We make restaurant automation software for QSR • Our software runs on more than 36K restaurants • Estimated number of individual machines: around half a million • Processing almost USD $50K per second of customer payments
  4. 4. “ONE IN A MILLION IS NEXT TUESDAY” - Gordon Letwin, Architect for DOS 4
  5. 5. HOW WE USED TO PERSIST DATA ON POS ... and still do (on legacy modules)
  6. 6. LEGACY PERSISTENCY • Flat files • Custom data format • Event sourcing persistence • Rebuild in-memory state at every restart
  7. 7. PROBLEMS WITH THAT APPROACH •No querying – at all •High complexity to store new data •No standard data format •Lack of management tools
  8. 8. WHY WE DECIDED TO USE A DBMS?
  9. 9. WHY A DBMS? • We started a full re-architecture of the POS platform • We are definitely not database/persistence specialists • We wanted to remove complexity to persist data
  10. 10. THE SEEK FOR A DBMS
  11. 11. GOALS / GUIDELINES • The new architecture is based on plug-ins, so we wanted each individual plug-in to have its own, isolated, database • Zero administration • Easy schema upgrades • Transparent replication • Unit tests
  12. 12. WHY WE CHOSE RAVENDB?
  13. 13. WHY RAVENDB? • .NET • Embedded operation mode • In-memory mode for unit tests • Multi-tenant • Good transparent replication options
  14. 14. WHAT RAVENDB ALLOWED US TO DO
  15. 15. RAVENDB AT RDI • Create unit tests without mocking the database • Have advanced statistics about the data persistence • Both system-wide and per plug-in • Transparent replication • Hot failover, data distribution & real-time backups • Transparent encryption of data at rest
  16. 16. THE CHALLENGES ... and how we faced them
  17. 17. CHALLENGE #1 – MEMORY CONSTRAINTS
  18. 18. SPECIALIZED, OLD HARDWARE Old hardware (sometimes 10 years old) means that we have very little memory and processing power.
  19. 19. MEMORY CONSTRAINTS • Many iterations with the RavenDB support team to fine-tune its memory usage • We learned that under these constraints, caching can be evil • In the end, the solution was to completely disable caching on RavenDB and do some cache at the application side, only for the “hot” data.
  20. 20. CHALLENGE #2 – ESENT
  21. 21. WE ARE STILL USING RAVENDB 2.5 • Our software must support Windows XPe • .NET 4.0 • 32 bits machines • … this means that ESENT is our only option for now
  22. 22. FACT: WE DON’T LIKE ESENT (at least not for our usage scenarios)
  23. 23. WHY? • Any unclean shutdown may cause a completely undetermined result • Power outages / Application crashes / Task kill • The possible results are not easy to find out during regular testing • Crappy hardware doesn’t help • We still face sporadic full database losses with ESENT • Copying data between OS versions is awful • Recovering from most unclean shutdowns requires using ESENTUTL.EXE
  24. 24. ESENTUTL.EXE • We also hate don’t like ESENTUTL.EXE • Many different commands to attempt to recover from different kinds of unclean shutdowns. Some have different syntax on different OS versions • We had to automate all that (zero maintenance, remember?) • Almost 500 lines of code
  25. 25. “SUCCESS IS STUMBLING FROM FAILURE TO FAILURE WITH NO LOSS OF ENTHUSIASM” ― Winston S. Churchill, Ex-prime minister of UK
  26. 26. THE FUTURE
  27. 27. THE FUTURE • Working with Hibernating Rhinos to improve Voron to our use case • 32 bits support • Better reliability on unclean shutdowns • We may jump straight to RavenDB 4.0 & CoreCLR • Long-term target: Around 250K individual POS nodes running RavenDB embedded
  28. 28. THANK YOU VERY MUCH

×