Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Architecture patterns using In-memory data systems

247 views

Published on

You know about in-memory data systems like Redis, Infinispan, etc. But while they can do lots of things, it's hard to know when to use them.
This presentation is here to describe when to use them and in which circumstance.

Published in: Technology
  • Be the first to comment

  • Be the first to like this

Architecture patterns using In-memory data systems

  1. 1. In-memory data systems What they can do for you Emmanuel Bernard Red Hat
  2. 2. Emmanuel Bernard @emmanuelbernard
  3. 3. Data is coming Data is flowing
  4. 4. Insightful application Observe - collect data Orient - Put in context Decide - select course of action Act - Do it Continuous improvement loop Data services Event stream Data analysis Iterative storage Reasoning, decision models and optimization Execution engine
  5. 5. In-memory systems an case
  6. 6. Key foundation key/value store Distributed Elastic Optional schema ASL 2.0
  7. 7. in-memory systems in an architecture
  8. 8. Caching Caching service data Get faster Hot cache App 1 App 2 Infinispan Datastore
  9. 9. Shared memory Stateless apps in a stateful universe Microservice “Stateless" “cluster” Infinispan SharedMapSharedMapSharedMapSharedMap Load balancer
  10. 10. Primary store Microservice “local” storage Derived store (Fast) Search capabilities Analytics Distributed tasks App 1 App 2 Infinispan
  11. 11. Distributed stream Move computation to data map(λ) λ λ
  12. 12. In action!
  13. 13. Station board stop station train
  14. 14. What is the time of the day when there is the biggest ratio of delayed trains?
  15. 15. Mixing stores Mix traditional datastores and in-memory Temporary data Heavy computation App JPA OGM ORM App JPA OGM ORM App JPA OGM ORM Infinispan RDBMS
  16. 16. Data virtualisation duct taping Microservice 1 Big old Datasource Dedicated Schema 1 Dedicated Schema 2 Dedicated Schema 3 Data Virtualisation Microservice 2 Microservice 3 Old app
  17. 17. Analytics and Spark Computation out of Spark Source of data Source of events Immutable RDD Fast RDD caching Spark cluster Infinispan Cassandra Hive RDBMS App 1
  18. 18. Syncing grids Active / Active Data synced between sites One grid per usage OLTP Analytics App 1 App 2 Infinispan Infinispan Active / active cross site replication Datacenter 1 Datacenter 2 Load balancer
  19. 19. Pay attention, be reactive Reactive application Reactive microservice Also look at Debezium App 1 “Reactive app” 2 Infinispan Change event “Reactive app” 1 Change event
  20. 20. Continuous query Reactive to data changes Index the query, not the data from User u where u.age >18 and u.country = ‘France’ from User u where u.age >18 and u.hasSnapshat = true from User u where u.email contains ‘yahoo.com’ { “country”: “France”, “age”: 17, “email”: “e@gmail.com”, “hasSnapshat”: true } { “country”: “France”, “age”: 19, “email”: “e@gmail.com”, “hasSnapshat”: true } =>
  21. 21. In action!
  22. 22. CentralTrain Delays Dashboard
  23. 23. Debezium and CDC Change Data Capture Transforms datastores into data change events No change to existing apps and datastores Add features as separate apps Decorrelate event consumers (each at its own pace)
  24. 24. Data are like Gremlins but good Observe - collect data Orient - Put in context Decide - select course of action Act - Do it Continuous improvement loop
  25. 25. Thanks http://infinispan.org http://blog.infinispan.org @infinispan DEMOs: https://github.com/ galderz/datagrid-patterns http://debezium.io http://teiid.org Explore your data Take calculated risks Any use case?
  26. 26. Credits engineer by Wilson Joseph from the Noun Project panel by gira Park from the Noun Project Approve by Aha-Soft from the Noun Project Database sharing byYuguDesign from the Noun Project ram by Andrea Rizzato from the Noun Project Database Search by Nimal Raj from the Noun Project Cloud Analytics by Kevin Augustine LO from the Noun Project Broken Computer by Dan Hetteix from the Noun Project data search by Gregor Črešnar from the Noun Project Server by Creative Stall from the Noun Project Network by Creative Stall from the Noun Project transformation by Felipe Perucho from the Noun Project analytics by Roman Kovbasyuk from the Noun Project Server by Designify.me from the Noun Project

×