Where Does Apache Geode Fit in CQRS Architectures?

410 views

Published on

SpringOne Platform 2016
Speaker: Eitan Suez; Software Engineer, Pivotal

In this talk, Eitan Suez will explore the question: Where does Geode fit in an organization's system architecture? Geode is a unique and feature-rich product that perhaps hasn't seen as much adoption as it deserves. Today's apps are no longer the straightforward, database-backed web applications we used to build a few years ago. Applications have become more sophisticated, as they've had to meet the need to scale, to be reliable, fault-tolerant, and to integrate with other systems. In this talk, Eitan will suggest one particular fit for Geode in the context of a CQRS architecture, and welcomes you to attend, and to contribute by sharing how you've put Geode to use in your organization.

Published in: Technology
0 Comments
5 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
410
On SlideShare
0
From Embeds
0
Number of Embeds
11
Actions
Shares
0
Downloads
15
Comments
0
Likes
5
Embeds 0
No embeds

No notes for slide

Where Does Apache Geode Fit in CQRS Architectures?

  1. 1. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Where Does Apache Geode Fit in CQRS Architectures? Eitan Suez @eitan_suez
  2. 2. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ About Me • Eitan Suez • Pivotal Consultant Instructor • Teach GemFire, Cloud Native, PCF • Prior to joining Pivotal, was Principal Consultant with ThoughtWorks • Long-time software developer, based in Austin, TX 2
  3. 3. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Relationship with Apache Geode • Over the years have worked on many enterprise projects for a number of customers • First hands-on experience with Geode when consulting at SouthWest Airlines.. • ..in the role of technical lead on a multi-team project, where Geode played a prominent role in the system architecture 3
  4. 4. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ My Journey • gfsh • OQL and the data browser • PDX serialization • Spring Data GemFire • Learn how to do automated functional testing with it 4
  5. 5. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Don’t immediately realize what you’ve got • At first, we were so focused on building features • Regions were already defined by solutions architects, treated them as tables • Didn’t pay too close attention to the fact that we had: • near-linear scale-out capabilities built-in with partitioned regions • fault-tolerance with redundant data copies • locators adding indirection, clients isolated from cluster specifics 5
  6. 6. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Example: Queries against partitioned regions 6 Can go further with server-side functions Client Geode Distributed System Server Query Executor Partitioned Region Server Partitioned Region Server Partitioned Region
  7. 7. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Unique Combination of Features • A Database, but in-memory? • Can also double as a simple cache? • A key-value store, but supports queries? • Supports transactions • Events? 7
  8. 8. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Impressive feature set • Briefly reviewed the traits of Apache Geode • It takes time to “wrap one’s head around” the whole of this product 8 So, what can you do with it?
  9. 9. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Use Cases “in the Small” • Specific to Java stack: O/RM and Hibernate • can plug in as Hibernate L2 Cache • Peer-to-peer configuration 9
  10. 10. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Use Cases “in the Small” Can be an out-of-process cache server, like Redis, or memcached gemcached These are fine, but does not take advantage of the full feature set 10
  11. 11. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Canonical Architecture 11 Geode Distributed System RegionsFunctions Locator Backing Store Client Client Events, Continuous Queries RegionsFunctions CacheLoader AsyncEventListener Server RegionsFunctions Client Queries, Transactions, Function Executions
  12. 12. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Switching Gears • On a couple of projects over the last couple of years, have been exposed to CQRS • At first it seemed strange, or overly complex. Didn't get it • Kept asking myself: • Why not start out simpler? • Seems rather complicated • It’s more work 12
  13. 13. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ What is CQRS? • Stands for Command Query Responsibility Segregation • A Pattern • deliberately not prescriptive regarding how you implement this separation • Separation all the way down to the database • Germ of the idea came from Bertrand Mayer (of Eiffel fame), with concept of CQS • Introduced, proposed by Greg Young • Active .NET community, Udi Dahan (among others) 13
  14. 14. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ CQRS.. 14 ..tells you what, not how, but to answer why, we are asked to look at what happens when you “go there”
  15. 15. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 15 When reads and writes are separate..
  16. 16. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ ..can optimize reads and writes • With a single schema, you’re forced to optimize for one at the expense of the other • With two schemas, one can be optimized for reads and the other for writes (have your cake and eat it too) • relational model for writes • denormalized views for reads 16 Read-Optimized Write-Optimized 3rd normal formdenormalized views Data-Representation Spectrum
  17. 17. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Reading when your data is normalized 17 Controller Services Repositories Relational Database Queries w/Joins Request transformations compositions Result Sets views Constantly reassembling views
  18. 18. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ With a denormalized schema.. • no joins necessary • no transformations • no need to reconstruct a view model for each request 18 Apache Geode Region: Customers Region: Orders Region: Products . . .
  19. 19. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ ..more benefits • Can scale reads and writes independently • many systems have a profile where reads outnumber writes at 100:1 ratio • Read and write sides can be implemented with entirely different tools and technologies • Read-side can stay up when write-side is temporarily down 19
  20. 20. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Command Side • Commands are semantic, in the language of the business, not REST CRUD: AddToCart, AddPaymentMethod, ChangeAddress • Command handling can be asynchronous • enqueue commands • can scale command handling 20 See: Udi Dahan Clarified CQRS
  21. 21. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Event Sourcing • The LOG • Append-only, no mutation • Immutable storage, doesn't destroy history • Activity just a stream of events • tables are projections, can be derived entirely from log • views can be recreated at will • multiple views 21 See: Martin Kleppmann Stream processing, Event sourcing, Reactive, CEP … and making sense of it all
  22. 22. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ The Log / Table Duality • Data in Motion vs Data at Rest • Entire History vs Snapshot in time • Source of truth vs derived information • materialized views, caches, indexes, aggregations 22 See: Jay Kreps The Log: What every software engineer should know about real-time data's unifying abstraction See: Martin Kleppmann Stream processing, Event sourcing, Reactive, CEP … and making sense of it all
  23. 23. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Replaying the Log.. • ..in test environments to reproduce bugs • ..in dev environments to test an upcoming release • ..in production to “undo” a bug • ..in production for blue-green type deployments Can transition to a new schema/representation of data in your regions because you've come up with a radically different user interface for navigating that information. 23 See: Greg Young CQRS and Event Sourcing
  24. 24. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 24 Diagram by
 “Exploring CQRS and Event Sourcing”, msdn
  25. 25. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ 25 ✓ update caches when new events come in ✓ invalidate caches proactively - ensure data in caches remain fresh ✓ inverts the cache loader concept ✓ serving data from fast, in-memory caches ✓ regions contain “ViewModel” objectsEvents Projection Updates Views in Regions Geode Distributed System Regions containing View Models Read Side - relay view models to ui - little to no transformations Events, Continuous Queries Queries
  26. 26. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Summary Apache Geode as the read store in a CQRS system is a particularly good fit: • eager cache invalidation • scalable and fast reads via.. • regions store denormalized views • partitioned regions enable linear scale-out • in-memory data supports low-latency reads 26
  27. 27. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ References & Attributions • Martin Kleppmann
 Stream processing, Event sourcing, Reactive, CEP … and making sense of it all • Rx, Erik Meijer
 Your Mouse is a Database • Greg Young
 CQRS and Event Sourcing • Jay Kreps
 The Log: What every software engineer should know about real-time data's unifying abstraction • Udi Dahan
 Clarified CQRS • Dominic Betts, Julian Dominguez, Grigori Melnik, Fernando Simonazzi, Mani Subramanian
 CQRS Journey • Dannielle Burrow
 Four Real World Use Cases For An In-Memory Data Grid 27
  28. 28. Unless otherwise indicated, these slides are © 2013-2016 Pivotal Software, Inc. and licensed under a Creative Commons Attribution- NonCommercial license: http://creativecommons.org/licenses/by-nc/3.0/ Learn More. Stay Connected. @springcentral spring.io/blog @pivotal pivotal.io/blog @pivotalcf http://engineering.pivotal.io

×