Your SlideShare is downloading. ×
0
Battery Framework
Battery Framework
Battery Framework
Battery Framework
Battery Framework
Battery Framework
Battery Framework
Battery Framework
Battery Framework
Battery Framework
Battery Framework
Battery Framework
Battery Framework
Battery Framework
Battery Framework
Battery Framework
Battery Framework
Battery Framework
Battery Framework
Battery Framework
Battery Framework
Battery Framework
Battery Framework
Battery Framework
Battery Framework
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Battery Framework

551

Published on

eXtremene Transaction Processing Framework description

eXtremene Transaction Processing Framework description

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
551
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. © 2009 Grid Dynamics — Scaling Mission Critical Systems<br />Battery Frameworkextreme transaction processing<br />Eugene Steinberg<br />CTO<br />
  • 2. Extreme transaction processing<br />November 12, 2009<br />© 2009 Grid Dynamics — Scaling Mission Critical Systems<br />2<br />
  • 3. XTP: reducing latency Strategies<br />November 12, 2009<br />© 2009 Grid Dynamics — Scaling Mission Critical Systems<br />3<br />
  • 4. Data management approaches<br />November 12, 2009<br />© 2009 Grid Dynamics — Scaling Mission Critical Systems<br />4<br />
  • 5. MANAGING DATA IN RAM: Products<br />November 12, 2009<br />© 2009 Grid Dynamics — Scaling Mission Critical Systems<br />5<br />RDBMS features<br />Both products support:<br />Reliability (via redundancy)<br />Failover<br />Data durability (via async persistence)<br />Oracle TimesTen<br />SQL and analytic queries<br />JDBC<br />Oracle Coherence<br />Selects, filters, aggregators<br />SMP<br />Cluster<br />Grid<br />Horizontal scalability<br />
  • 6. XTP: scalability and reliability<br />November 12, 2009<br />© 2009 Grid Dynamics — Scaling Mission Critical Systems<br />6<br />
  • 7. High level architecture<br />November 12, 2009<br />© 2009 Grid Dynamics — Scaling Mission Critical Systems<br />7<br /><ul><li> Data management in RAM
  • 8. Data partitioning
  • 9. Load balancing
  • 10. In-place data processing
  • 11. Asynchronous persistence
  • 12. Horizontal scalability</li></ul>RDBMS<br />
  • 13. Physics of distributed DATA<br />CAP theorem (Eric Brewer, 2000)<br />You may have at most two of Consistency, Availability, and Partition Tolerancein any shared-data system<br />November 12, 2009<br />© 2009 Grid Dynamics — Scaling Mission Critical Systems<br />8<br />C<br />Data grids<br />Quorum-based consistency<br />P<br />A<br />Eventually consistent databases<br />
  • 14. In Memory data grid technology<br />Storing and managing data in main memory (RAM)<br />Automated data partitioning<br />Automated clustering and dynamic node membership<br />Built-in data redundancy<br />Strong data consistency and availability<br />Multiple ways to access data: Map interface, Data Filters, Queries, Template queries, etc…<br />Designed to scale out<br />Proven and reliable technology (10 years on market, major players)<br />November 12, 2009<br />© 2009 Grid Dynamics — Scaling Mission Critical Systems<br />9<br />
  • 15. IMDG benefits for xtp<br />Fastest possible access to data (in-memory)<br />Optimized network interactions (peer-to-peer)<br />Built-in load balancing<br />Consistent data management<br />Reliable in-memory transactions <br />Transparent scale out<br />Continuity of operations (redundancy of data)<br />Automated failover and re-balancing<br />November 12, 2009<br />© 2009 Grid Dynamics — Scaling Mission Critical Systems<br />10<br />
  • 16. COHERENCE PLATFORM<br />November 12, 2009<br />© 2009 Grid Dynamics — Scaling Mission Critical Systems<br />11<br />
  • 17. STAGED EVENT DRIVEN ARCHITECTURE<br />November 12, 2009<br />© 2009 Grid Dynamics — Scaling Mission Critical Systems<br />12<br />
  • 18. PROCESSING UNITS<br />November 12, 2009<br />© 2009 Grid Dynamics — Scaling Mission Critical Systems<br />13<br />Generic<br />Generic processing unit, base for all hierarchy<br /> Transformer<br />Consumes inbound message and produces outbound message <br />Router<br />Routes inbound message to one of several outbound queues<br />Duplicates inbound message to several outbound queues<br />Fork<br />Consumes several inbound messages and produces outbound message. Synchronization point<br />Junction<br />
  • 19. processing network<br />Application model: network of processing units interconnected with queues<br />November 12, 2009<br />© 2009 Grid Dynamics — Scaling Mission Critical Systems<br />14<br />Fraud detection<br />Payment<br />Validation<br />Receipt<br />Processing<br />Order<br />service<br />Validated<br />Execution<br />Shipping<br />Product availability<br />Failed order<br />Legend<br />Junction<br />Transformator<br />Generic<br />Fork<br />Router<br />
  • 20. Bringing processing closer to data<br />IMDG + SEDA forms uniform processing fabric<br />Data and algorithms are in the same processes.<br />Leveraging data affinity and data aware routing allows to bring algorithms close to the data<br />Same server or even same process algorithm execution<br />reduce latency,<br />reduce network traffic,<br />reduce CPU utilization (no encoding/decoding of binary data format)<br />We do not need to change business logic to achieve this!<br />Done on distributed system configuration level.<br />November 12, 2009<br />© 2009 Grid Dynamics — Scaling Mission Critical Systems<br />15<br />
  • 21. Managing transactions<br />Low latency distributed transactions are challenging<br />What is provided by our basic technologies:<br />IMDG provides as fast and reliable in-memory data transactions<br />SEDA simplifies modeling of complex business transactions<br />Strategies to reduce transaction latency without compromising consistency:<br />optimistic locking(multi version concurrency control)<br />coalescence of transactions,<br />exploiting idempotent operations.<br />November 12, 2009<br />© 2009 Grid Dynamics — Scaling Mission Critical Systems<br />16<br />
  • 22. Maintainability and integration<br />November 12, 2009<br />© 2009 Grid Dynamics — Scaling Mission Critical Systems<br />17<br />
  • 23. November 12, 2009<br />© 2009 Grid Dynamics — Scaling Mission Critical Systems<br />18<br />Application of Battery Framework:core of high performance billing system<br />
  • 24. BILLING SYSTEM<br />November 12, 2009<br />© 2009 Grid Dynamics — Scaling Mission Critical Systems<br />19<br />Telecommunication services<br />Payment system<br />Billing System<br />Events<br />Events<br />Service provided logs<br />Payments<br />User data<br />Tariffing of services<br />Billing rules<br />CRM<br />Balance management<br />Balance<br />Users<br />Rules<br />Service enable/disable<br />Balance data<br />Balance<br />...<br />
  • 25. BILLING SYSTEM EVENTS<br />November 12, 2009<br />© 2009 Grid Dynamics — Scaling Mission Critical Systems<br />20<br />Payments<br />Events<br /><ul><li>Service consumption
  • 26. Payments</li></ul>Balance access<br /><ul><li>Tariffing
  • 27. Write-off
  • 28. Queries from network
  • 29. Queries from CRM</li></ul>Issues<br /><ul><li>Reading load
  • 30. Writing load
  • 31. Transactions</li></ul>Billing engine<br />Tariffing<br />Balance management<br />Events<br />Events<br />Billing system data<br />Queries<br />Queries<br />Balance<br />
  • 32. BOTTLENECK localization<br />November 12, 2009<br />© 2009 Grid Dynamics — Scaling Mission Critical Systems<br />21<br />Requirements (20 millions users)<br /><ul><li> ~ 109 Objects (counters)
  • 33. 1000 events/sec
  • 34. 50 ms latency </li></ul>Data<br /><ul><li>Users
  • 35. Balance
  • 36. Rules (wholesales, group bonuses and such)</li></ul>Billing engine<br />Events<br />Events<br />Balance management<br />Tariffing<br />Write-off<br />payment<br />Balance<br />Queries<br />Batches<br />Balance management system is a bottleneck<br />
  • 37. BILLING processing network<br />November 12, 2009<br />© 2009 Grid Dynamics — Scaling Mission Critical Systems<br />22<br />Inbound event<br />Outbound event<br />Search for relevant counters<br />Write-off check<br />Tariffing<br />Junction<br />Rules processing<br />Fork<br />Changes fixation<br />Asynchronous RDBMS replication<br />Router<br />
  • 38. SCALING processing networks<br />November 12, 2009<br />© 2009 Grid Dynamics — Scaling Mission Critical Systems<br />23<br />Horizontal scaling<br />
  • 39. PERFORMANCE AND SCALABILITY<br />November 12, 2009<br />© 2009 Grid Dynamics — Scaling Mission Critical Systems<br />24<br />Data scalability<br />Users ( millions)<br />Hardware scalability<br />Servers<br />Events/sec<br />Events/sec<br />Servers characteristics<br /><ul><li>CPU 2.5 GHz Quad core
  • 40. RAM 32 GB</li></li></ul><li>November 12, 2009<br />© 2009 Grid Dynamics — Scaling Mission Critical Systems<br />25<br />Thanks!<br />Eugene Steinberg<br />CTO<br />

×