• Like
  • Save
GigaSpaces XAP - Don't Call Me Cache!
Upcoming SlideShare
Loading in...5
×
 

GigaSpaces XAP - Don't Call Me Cache!

on

  • 408 views

Learn by example how to build a scalable event driven application using XAP

Learn by example how to build a scalable event driven application using XAP

Statistics

Views

Total Views
408
Views on SlideShare
408
Embed Views
0

Actions

Likes
0
Downloads
2
Comments
0

0 Embeds 0

No embeds

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Cache == in-memory key-value storeExample Ehcache
  • Example MemcachedOnly key-value (only read-by-ID), not a full-fledged database, no query engine, no indexing, no transactions
  • Added transactionalityIndexing (incl. nested indices)Querying engine (SQL, template-based/query-by-example)Projections, inline changesComplex objects (not supported by Coherence. Style MongoDB etc.)Document-based data model (semi-structured schema support)
  • processing co-located with the data, map-reduce, scatter-gather, remote invocation/execution, embedded/remote event containers
  • Basicvalidataion – Credit card is valid etc.Checks – fraud detection style checks
  • Writing the event into the gridRouting to right partition – content-based routing by user IDReplicated synchronously to the partition’s hot backup
  • Easily indexing properties using annotations/metadata
  • co-location of the user with its cards and transactions – ensures affinity
  • Entry written to the space is handled implicitly as joining a queue, and creating a consumer to process entries from the queueConsumer is a simple Java class with some annotations which needs to define: (1) event template - which events are of interest(2) event handler - How to handle the eventsWhich entries are of interest? Easily defined as a simple method with @EventTemplate annotation and using a SQL query
  • Event handler easily created by annotating the methodIn the payment processing example – needs to validate that (a) credit card is OK and (b) create payment authorization state object for the subsequent validation phase
  • Compare current payment against user profileAll user’s card and payment info is available found with the user on same partition (thanks to our routing policy)Simple change API for inline property updatesExample of simple template-based queries
  • We’ll use remote distributed async task dispatch to perform this validationFirst the consumer fetches the VendorPaymentMsg, then it creates a task to make the validation
  • On our system architecture we create a separate vendors cluster (space) to enable vendor profiling on single partitionThe task is dispatched from our user’s node on the the payment cluster to the right node on the vendor cluster based on our content-based routing policy (based on the merchant ID)
  • In order to validate vendor all merchant’s deals are fetched and profiled, to assure this deal matches and no fraud
  • Once calculation is completed the result is asynchronously returned from the task back to the caller node (callback to the payment cluster node)Async invocation uses enhanced version of the standard Java Async API (Future).Again we use Change API to update the PaymentAuthorization.vendorCheck status flag
  • Fetch all PaymentAuthorization that completed initial validation (status=New) and that passed both userCheck and vendorCheck (the 2 ‘true’ values)And update the status to ‘Done’
  • Fetch all PaymentAuthorization that completed initial validation (status=New) and that passed both userCheck and vendorCheck (the 2 ‘true’ values)And update the status to ‘Closed’

GigaSpaces XAP - Don't Call Me Cache! GigaSpaces XAP - Don't Call Me Cache! Presentation Transcript

  • Powering Your Application with XAP (Using payment processing as an example) “Don’t call me cache”
  • Short Intro to Caching Evolution Cache In process caching of Key->Value data structure Distribute Cache Partitioned cache nodes IMDG Partitioned system of record In Memory Application Platform Collocated IMDG and Processing Cache Cache is good for repetitive data reads But it is limited in capacity It also doesn’t handle write-heavy scenarios
  • Short Intro to Caching Evolution Cache In process caching of Key->Value data structure Distribute Cache Partitioned cache nodes IMDG Partitioned system of record In Memory Application Platform Collocated IMDG and Processing Distribute Cache Allows you to distribute your cache over numerous machines so you get Increased Capacity But it doesn’t support write heavy scenarios It’s also Limited to query by Id What about the rest of your app? - Business logic & messaging??
  • Short Intro to Caching Evolution Cache In process caching of Key->Value data structure Distribute Cache Partitioned cache nodes IMDG Partitioned system of record In Memory Application Platform Collocated IMDG and Processing IMDG solves these problems! You get increased capacity IMDG is also a System of Record with: Query APIs Optimized data access Data integrity It solves your write scalability problem .
  • Short Intro to Caching Evolution Cache In process caching of Key->Value data structure Distribute Cache Partitioned cache nodes IMDG Partitioned system of record In Memory Application Platform Collocated IMDG and Processing In Memory Application Platform XAP for end to end scaling Its an IMDG that hosts your Business logic & has messaging services! It Provides Parallel processing of data You get linear scalability You get high availability How does XAP work?
  • Here’s What a Payment Authorization Process Looks Like Payment Authorization Request Basic Validation User Profile Check Merchant Profile Check Payment Authorization Approved
  • Write the Payment Object to XAP Cash Register Application Payment User payment Cluster @SpaceId() public Long getId() { return id; } @SpaceRouting public Long getUserId() { return userId; } } @SpaceIndex public Long getCardId() { return cardId; } } The primary key of this object in the grid The grid will use this attribute to route the object to a particular partition Payment object A secondary index for query optimization
  • Write the Payment Object to XAP Cash Register Application User payment Cluster Payment @SpaceId public Long getId() { return id; } @SpaceRouting public Long getUserId() { return userId; } @SpaceIndex public Long getCardId() { return cardId; } The primary key of this object in the grid The grid will use this attribute to route the object to a particular partition Payment object A Secondary index for query optimization
  • Write the Payment Object to XAP Cash Register Application User payment Cluster Payment Index @SpaceId public Long getId() { return id; } @SpaceRouting public Long getUserId() { return userId; } @SpaceIndex public Long getCardId() { return cardId; } The primary key of this object in the grid The grid will use this attribute to route the object to a particular partition Payment object A Secondary index for query optimization
  • Let's Talk About Data Model for a Second
  • User Id (Routing)| User Name|… Card Id| Card Data | UserID (Routing)… Transaction ID | CardId | User ID (Routing)… 1  * 1  * Data Model
  • Let's Get Back to the Process
  • Payment Validation @EventTemplate public SQLQuery<Payment> getNewPayment() { SQLQuery<Payment> query = new SQLQuery<Payment>(Payment.class," paymentStatus = ? "); query.setParameter(1, Payment.PaymentStatus.New); return query; } Queue Payment Validator Single threaded handler Template “queues” events for the handler
  • Payment Validation @SpaceDataEvent public Payment validatePayment(Payment payment) { … } private boolean basicPaymentValidation(Payment payment, User user, Card card) { … } Queue Payment Class PaymentAuthorization{ Status status; Boolean userCheck; Boolean vendorCheck; } Validator Payment Authorization
  • Basic Validation if (basicPaymentValidation(payment,user,card)) { gigaSpace.write(userPaymentMsg); gigaSpace.write(vendorPaymentMsg); gigaSpace.write(paymentAuthorization); payment.setPaymentStatus(Payment.PaymentStatus.Processing); } else { payment.setPaymentStatus(Payment.PaymentStatus.SuspectedFraud); } Class PaymentAuthorization { Status status; Boolean userCheck; Boolean vendorCheck; } User Validation Vendor Validation Payment Authorization Queue Queue Queue Validator
  • User Validation Class PaymentAuthorization{ Enum Status Boolean userCheck Boolean vendorCheck } Read Message Process Queue UserCreditCardPayment Querying @SpaceDataEvent public void validateUser(UserPaymentMsg event) { … space.readMultiple(new Card(userId)) space.readMultiple(new Payment(userId)) if (valid()) { IdQuery<PaymentAutjorzation> idQuery = new IdQuery<PaymentAuthorization>(PaymentAuthoirzation.class, paymentId); space.change(idQuery, new ChangeSet().set(“userCheck", true)); } else { ... } }
  • User Validation Read Message Process Queue UserCreditCardPayment Class PaymentAuthorization { Status status; Boolean userCheck; Boolean vendorCheck; } Querying @SpaceDataEvent public void validateUser(UserPaymentMsg event) { … space.readMultiple(new Card(userId)) space.readMultiple(new Payment(userId)) if (valid()) { IdQuery<PaymentAutjorzation> idQuery = new IdQuery<PaymentAuthorization>(PaymentAuthoirzation.class, paymentId); space.change(idQuery, new ChangeSet().set(“userCheck", true)); } else { ... } }
  • User Validation Read Message Process Queue UserCreditCardPayment @SpaceDataEvent public void validateUser(UserPaymentMsg event) { … space.readMultiple(new Card(userId)) space.readMultiple(new Payment(userId)) if (valid()) { IdQuery<PaymentAutjorzation> idQuery = new IdQuery<PaymentAuthorization>(PaymentAuthorization.class, paymentId); space.change(idQuery, new ChangeSet().set(“userCheck", true)); } else { ... } } Class PaymentAuthorization { Status status; Boolean userCheck; Boolean vendorCheck; } Querying
  • Vendor Validation @SpaceDataEvent public void validateVendor(VendorPaymentMsg vendorPaymentMsg) { AsyncFuture<Boolean> future = vendorGigaSpace.execute( new VendorValidationTask(vendorPaymentMsg), vendorPaymentMsg.getMerchant()); IdQuery<PaymentAuthorization> idQuery = new IdQuery<PaymentAuthorization>(PaymentAuthorization.class, vendorPaymentMsg.getPaymentId()); if (future.get()) gigaSpace.change(idQuery, new ChangeSet(). set("vendorCheck", true)); else gigaSpace.change(idQuery, new ChangeSet(). set("vendorCheck”, false)); } Read Message Process Queue Task class PaymentAuthoirzation { Status status; Boolean userCheck; Boolean vendorCheck; }
  • Vendor Validation @SpaceDataEvent public void processVendorPaymentValidation() { AsyncFuture<Boolean> result= space.execute( new DistributedTask(new VendorValidationTask(Payment))) if (result.get()) { IdQuery<PaymentAutjorzation> idQuery = new IdQuery<PaymentAuthorization>(PaymentAuthorization.class, paymentId); space.change(idQuery, new ChangeSet().set(“vendorCheck”, true)); } else { ... } } Payment Cluster Task Vendor Cluster Class PaymentAuthorization { Status status; Boolean userCheck; Boolean vendorCheck; }
  • Vendor Validation public void validateVendor(VendorPaymentMsg vendorPaymentMsg, GigaSpace gigaSpace) { AsyncFuture<Boolean> future = vendorGigaSpace.execute(new VendorValidationTask(vendorPaymentMsg), vendorPaymentMsg.getMerchant()); IdQuery<PaymentAuthorization> idQuery = new IdQuery<PaymentAuthorization>(PaymentAuthorization.class, vendorPaymentMsg.getPaymentId()); if (future.get()) gigaSpace.change(idQuery, new ChangeSet().set("vendorCheck", true)); else gigaSpace.change(idQuery, new ChangeSet().set("vendorCheck", false)); } Payment Cluster Vendor Cluster Task Class PaymentAuthorization { Status status; Boolean userCheck; Boolean vendorCheck; }
  • Vendor Validation Payment Cluster Vendor Cluster Task public void validateVendor(VendorPaymentMsg vendorPaymentMsg, GigaSpace gigaSpace) { AsyncFuture<Boolean> future = vendorGigaSpace.execute(new VendorValidationTask(vendorPaymentMsg), vendorPaymentMsg.getMerchant()); IdQuery<PaymentAuthorization> idQuery = new IdQuery<PaymentAuthorization>(PaymentAuthorization.class, vendorPaymentMsg.getPaymentId()); if (future.get()) gigaSpace.change(idQuery, new ChangeSet().set("vendorCheck", true)); else gigaSpace.change(idQuery, new ChangeSet().set("vendorCheck", false)); } Class PaymentAuthorization { Status status; Boolean userCheck; Boolean vendorCheck; }
  • Vendor Validation Check @Override public Boolean execute() throws Exception { return validatePayment(vendorPaymentMsg); } Task DealsMerchant Class PaymentAuthorization { Status status; Boolean userCheck; Boolean vendorCheck; } Querying
  • Vendor Validation Check Task DealsMerchant @Override public Boolean execute() throws Exception { return validatePayment(vendorPaymentMsg); } Class PaymentAuthorization { Status status; Boolean userCheck; Boolean vendorCheck; } Querying
  • Vendor Validation Payment Cluster Vendor Cluster Callback Task public void validateVendor(VendorPaymentMsg vendorPaymentMsg, GigaSpace gigaSpace) { AsyncFuture<Boolean> future = vendorGigaSpace.execute(new VendorValidationTask(vendorPaymentMsg), vendorPaymentMsg.getMerchant()); IdQuery<PaymentAuthorization> idQuery = new IdQuery<PaymentAuthorization>(PaymentAuthorization.class, vendorPaymentMsg.getPaymentId()); if (future.get()) gigaSpace.change(idQuery, new ChangeSet().set("vendorCheck", true)); else gigaSpace.change(idQuery, new ChangeSet().set("vendorCheck", false)); } Class PaymentAuthorization { Status status; Boolean userCheck; Boolean vendorCheck; }
  • Yey! Payment Has Been Authorized @EventTemplate public PaymentAuthorization getNewPayment() { return new PaymentAuthorization(null, true, true, PaymentAuthorizationStatus.New); } Queue Payment Authorization
  • Yey! Payment Has Been Authorized @SpaceDataEvent public void completePaymentValidation(PaymentAuthorization paymentAuthorization) { Payment payment = gigaSpace.readById(Payment.class,paymentAuthorization.getPaymentId()); payment.setPaymentStatus(Payment.PaymentStatus.Closed); gigaSpace.write(payment); paymentAuthorization.setPaymentAuthorizationStatus( PaymentAuthorization.PaymentAuthorizationStatus.Done); } Queue Payment Authorization
  • Q&A
  • THANK YOU! Get the code from Github