Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

#GeodeSummit - Where Does Geode Fit in Modern System Architectures

2,922 views

Published on

In this talk, Eitan Suez explores the question: Where does Geode fit in an organization's system architecture? Geode is a unique and feature-rich product that perhaps hasn't seen as much adoption as it deserves. Today's apps are no longer the straightforward, database-backed web applications we used to build a few years ago. Applications have become more sophisticated, as they've had to meet the need to scale, to be reliable, fault-tolerant, and to integrate with other systems. In this talk, Eitan will suggest one particular fit for Geode in the context of a CQRS architecture, and welcomes you to attend, and to contribute by sharing how you've put Geode to use in your organization.

Published in: Technology

#GeodeSummit - Where Does Geode Fit in Modern System Architectures

  1. 1. 2016.03.09
  2. 2. Where does Geode fit in modern system architectures? 2 Eitan Suez
  3. 3. • Eitan Suez • Pivotal Consultant Instructor • Teach GemFire, Cloud Native, PCF • Prior to joining Pivotal, was Principal Consultant with ThoughtWorks • Long-time software developer, based in Austin, TX 3 About Me
  4. 4. • Over the years have worked on many enterprise projects for a number of customers • First hands-on experience with Geode when consulting at SouthWest Airlines.. • ..in the role of technical lead on a multi-team project, where Geode played a prominent role in the system architecture 4 Relationship with Geode
  5. 5. • gfsh • OQL and the data browser • PDX serialization • Spring Data GemFire • Learn how to do automated functional testing with it 5 My Journey
  6. 6. • At first, we were so focused on building features • Regions were already defined by solutions architects, treated them as tables • Didn’t pay too close attention to the fact that we had: • near-linear scale-out capabilities built-in with partitioned regions • fault-tolerance with redundant data copies • locators adding indirection, clients isolated from cluster specifics 6 Don’t immediately realize what you’ve got
  7. 7. 7 Example: Queries against partitioned regions Client Geode Distributed System Query against partitioned region Server Query Executor Partitioned Region Server Partitioned Region Server Partitioned Region Can go further with server-side functions
  8. 8. • A Database, but in-memory? • Can also double as a simple cache? • A key-value store, but supports queries? • Supports transactions • Events? 8 Unique Combination of Features
  9. 9. • Briefly reviewed the traits of Apache Geode • It takes time to “wrap one’s head around” the whole of this product 9 Impressive feature set So, what can you do with it?
  10. 10. • Specific to Java stack: O/RM and Hibernate • can plug in as Hibernate L2 Cache • Peer-to-peer configuration 10 Use Cases “in the Small”
  11. 11. • Can be an out-of-process cache server, like Redis, or memcached ➡ gemcached • These are fine, but does not take advantage of the full feature set 11 Use Cases “in the Small”
  12. 12. 12 Canonical Architecture Geode Distributed System RegionsFunctions Locator Backing Store Client Client Events, Continuous Queries RegionsFunctions CacheLoader AsyncEventListener Server RegionsFunctions Client Queries, Transactions, Function Executions
  13. 13. • On a couple of projects over the last couple of years, have been exposed to CQRS • At first it seemed strange, or overly complex. Didn't get it • Kept asking myself: • Why not start out simpler? • Seems rather complicated • It’s more work 13 Switching Gears
  14. 14. • Stands for Command Query Responsibility Segregation • A Pattern • deliberately not prescriptive regarding how you implement this separation • Separation all the way down to the database • Germ of the idea came from Bertrand Mayer (of Eiffel fame), with concept of CQS • Introduced, proposed by Greg Young • Active .NET community, Udi Dahan among others 14 What is CQRS?
  15. 15. 15 CQRS.. ..tells you what, not how, but to answer why, we are asked to look at what happens when you “go there”
  16. 16. 16 When reads and writes are separate..
  17. 17. • With a single schema, you’re forced to optimize for one at the expense of the other • With two schemas, one can be optimized for reads and the other for writes (have your cake and eat it too) • relational model for writes • denormalized views for reads 17 ..can optimize reads and writes Read-Optimized Write-Optimized Data-Representation Spectrum 3rd normal formdenormalized views
  18. 18. 18 Reading when your data is normalized Request Services Repositories multiple queries, lots of joins results transformations views Relational Database Controller compositions Constantly reassembling views
  19. 19. • no joins necessary • no transformations • no need to reconstruct a view model for each request 19 With a denormalized schema Apache Geode Region: Customers Region: Orders Region: Products . . .
  20. 20. • Can scale reads and writes independently • many systems have a profile where reads outnumber writes at 100:1 ratio • Read and write sides can be implemented with entirely different tools and technologies • Read-side can stay up when write-side is temporarily down 20 ..more benefits
  21. 21. • Commands are semantic, in the language of the business, not REST CRUD: AddToCart, AddPaymentMethod, ChangeAddress • Command handling can be asynchronous • enqueue commands • can scale command handling 21 Command Side See: Udi Dahan Clarified CQRS
  22. 22. • The LOG • Append-only, no mutation • Immutable storage, doesn't destroy history • Activity just a stream of events • tables are projections, can be derived entirely from log • views can be recreated at will • multiple views 22 Event Sourcing See: Martin Kleppmann Stream processing, Event sourcing, Reactive, CEP … and making sense of it all
  23. 23. • Data in Motion vs Data at Rest • Entire History vs Snapshot in time • Source of truth vs derived information • materialized views, caches, indexes, aggregations 23 The Log / Table Duality See: Jay Kreps The Log: What every software engineer should know about real-time data's unifying abstraction See: Martin Kleppmann Stream processing, Event sourcing, Reactive, CEP … and making sense of it all
  24. 24. • ..in test environments to reproduce bugs • ..in dev environments to test an upcoming release • ..in production to “undo” a bug • ..in production for blue-green type deployments • Can transition to a new schema/representation of data in your regions because you've come up with a radically different user interface for navigating that information. 24 Replaying the Log See: Greg Young CQRS and Event Sourcing
  25. 25. 25 Diagram by
 “Exploring CQRS and Event Sourcing”, msdn
  26. 26. 26 ✓ update caches when new events come in ✓ invalidate caches proactively - ensure data in caches remain fresh ✓ inverts the cache loader concept ✓ serving data from fast, in-memory caches ✓ regions contain “ViewModel” objects Events Projection Updates Views in Regions Geode Distributed System Regions containing View Models Read Side - relay view models to ui - little to no transformations Events, Continuous Queries Queries
  27. 27. • Apache Geode as the read store in a CQRS system is a particularly good fit: • eager cache invalidation • scalable and fast reads via.. • regions store denormalized views • partitioned regions enable linear scale-out • in-memory data supports low-latency reads • Curious to learn how Geode is being applied in your work 27 Summary
  28. 28. • Martin Kleppmann Stream processing, Event sourcing, Reactive, CEP … and making sense of it all • Rx, Erik Meijer Your Mouse is a Database • Greg Young CQRS and Event Sourcing • Jay Kreps The Log: What every software engineer should know about real-time data's unifying abstraction • Udi Dahan Clarified CQRS • Dominic Betts, Julian Dominguez, Grigori Melnik, Fernando Simonazzi, Mani Subramanian CQRS Journey • Dannielle Burrow Four Real World Use Cases For An In-Memory Data Grid 28 References & Attributions
  29. 29. 29 Join the Apache Geode Community! • Check out http://geode.incubator.apache.org • Subscribe: user-subscribe@geode.incubator.apache.org • Download: http://geode.incubator.apache.org/releases/
  30. 30. Thank you!

×