Your SlideShare is downloading. ×
0
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Slice for Distributed Persistence (JavaOne 2010)
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×
Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Slice for Distributed Persistence (JavaOne 2010)

1,055

Published on

OpenJPA Slice extends JPA for horizontally partitioned database environment. This session was originally presented in JavaOne 2010.

OpenJPA Slice extends JPA for horizontally partitioned database environment. This session was originally presented in JavaOne 2010.

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,055
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
16
Comments
0
Likes
2
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. Scale Java Persistence API Applications with OpenJPA Slice Pinaki Poddar [email_address]
  • 2. Agenda <ul><li>Core Features of Slice </li></ul><ul><li>Using Slice </li></ul><ul><li>Under the hood </li></ul><ul><li>Running on Slice </li></ul>Source: If applicable, describe source origin
  • 3. What is Slice? <ul><li>Slice is a OpenJPA module for horizontally, partitioned databases </li></ul>Java Persistence API Specification [Section 3.1] says: “ A persistence unit defines the set of all classes that are related or grouped by the application, and which must be colocated in their mapping to a single database.” Slice changes that… single to multiple
  • 4. Horizontal Partitioning <ul><li>A data set D is said to be horizontally partitioned into N partitions D1 , D2 ,  , Dn iff </li></ul><ul><ul><ul><li>D = D1  D2  D3   Dn </li></ul></ul></ul><ul><ul><ul><li>Di  Dj =  for any i  j </li></ul></ul></ul><ul><li>A common mathematical term is mutually disjoint sets </li></ul><ul><li>Google coined such partition operation as Shard . </li></ul>
  • 5. Horizontal Partitioning in realistic setup <ul><li>Natural partitioning scenarios </li></ul><ul><ul><li>Customer by region (Telecom Billing) </li></ul></ul><ul><ul><li>Transaction by Month (Finance) </li></ul></ul><ul><ul><li>Software as Service Platforms (Legal compliance) </li></ul></ul>
  • 6. History of Slice <ul><li>Incubated as a Apache Lab project in Jan 2008 </li></ul><ul><li>Integrated as a OpenJPA module on July 2008 (since version 1.1) </li></ul><ul><li>Available with WebSphere Application Server version 7.0 onwards </li></ul>
  • 7. OpenJPA <ul><li>An implementation of JPA Specification </li></ul><ul><li>Apache Project since May 2007 http://openjpa.apache.org </li></ul>
  • 8. Architectural tiers of a typical JPA-based application User Application OpenJPA Standard JPA API JDBC API 400 million records
  • 9. Architectural tiers of a Slice-based application User Application OpenJPA Standard JPA API JDBC API Slice OpenJPA is a plugabble platform User-defined Data Distribution Policy 4x100 million records
  • 10. Separate Persistence Unit configured to partitioned databases C A [1] D A Unit A C A [2] C B [1] D B Unit B C B [2] C C [1] D C Unit C C C [2]
  • 11. Same persistence unit switches contexts to partitioned databases C A [1] D A Persistence Unit C A [2] C C [2] C C [1] C B [2] C B [1] D B D C
  • 12. Same persistence unit connected to partitioned databases C[1] D A Persistence Unit C[2] D B D C
  • 13. Features of Slice Slice-based User Application OpenJPA Standard JPA API JDBC API Slice No changes to Application code User-defined Data Distribution Policy Flexible per-Slice Configuration Parallel Query Execution Heterogeneous Databases Master-based Sequence Targeted Query No changes to Domain Model User-defined Query Target Policy No changes to Database Schema 4x100 millon records
  • 14. Agenda <ul><li>Core Features of Slice </li></ul><ul><li>Using Slice </li></ul><ul><li>Under the hood </li></ul><ul><li>Running on Slice </li></ul>Source: If applicable, describe source origin
  • 15. Using Slice <ul><li>Decide partition policy to </li></ul><ul><ul><li>distribute data </li></ul></ul><ul><ul><li>target query </li></ul></ul><ul><li>Configure JPA persistence unit </li></ul><ul><li>No change to </li></ul><ul><ul><li>Application Code (ok, almost!) </li></ul></ul><ul><ul><li>Domain Model </li></ul></ul><ul><ul><li>Database Schema </li></ul></ul>
  • 16. Policy based configuration <ul><li>The design goal of “ no application code change ” conflicts with the user application’s ability to control </li></ul><ul><ul><li>which slice(s) will store a new instance </li></ul></ul><ul><ul><li>which slice(s) will be searched for a query </li></ul></ul><ul><li>The compromise solution is policy based callback interfaces </li></ul><ul><ul><li>DistributionPolicy </li></ul></ul><ul><ul><li>ReplicationPolicy </li></ul></ul><ul><ul><li>QueryTargetPolicy </li></ul></ul><ul><ul><li>FinderTargetPolicy </li></ul></ul><ul><li>User application may implement these policies </li></ul><ul><li>Slice runtime would call the policy method </li></ul>
  • 17. Data Distribution Policy determines where new records are stored <ul><li>Which slice stores a new data record? </li></ul><ul><ul><li>Only the user application can decide </li></ul></ul>
  • 18. How to distribute data across slices? <ul><li>01: EntityManager em = …; </li></ul><ul><li>02: em.getTransaction().begin(); </li></ul><ul><li>03: Person person = new Person(); </li></ul><ul><li>04: person.setName(“ John ”); </li></ul><ul><li>05: person.setAge(42); </li></ul><ul><li>06: Address addr = new Address(); </li></ul><ul><li>07: addr.setCity(“ New York ”); </li></ul><ul><li>08: person.setAddress(addr); </li></ul><ul><li>09: em.persist(person); </li></ul><ul><li>10: em.getTransation().commit(); </li></ul><ul><li>01: public class MyDistributionPolicy implements DistributionPolicy { </li></ul><ul><li>02: public String distribute (Object pc, List<String> slices, Object ctx) { </li></ul><ul><li>03: return slices.get((( Person )pc).getAge()/20); </li></ul><ul><li>04: } </li></ul><ul><li>05: } </li></ul>@Entity public class Person { private String name; private int age; @OneToOne (cascade=ALL) private Address address; } @Entity public class Address { private String city; } User Application Domain Classes Data Distribution Policy
  • 19. Distribution Policy decides target slice for each instance public interface DistributionPolicy { /** * Decide the name of the slice where the given persistent * instance would be stored. * * @param pc The newly persistent or to-be-merged object. * @param slices name of the configured slices. * @param context persistence context managing the given instance. * * @return identifier of the slice. This name must match one of the * configured slice names. * @see DistributedConfiguration#getSliceNames() */ String distribute ( Object pc, List<String> slices, Object context); } Slice runtime will call this method while persisting or merging a root instance. The instance and its persistent closure will be stored in the returned slice.
  • 20. Details on Distribution Policy a b em.persist(a); // or em.merge(a); persistence context MyPolicy.distribute(a,…) { return “One” ; } One One < property name=&quot; openjpa.slice.DistributionPolicy &quot; value=“ acme.org.MyPolicy &quot;/> Slice attaches moniker to managed instance as it enters a persistence context Slice runtime calls User Application CascadeType.PERSIST CascadeType.MERGE c One
  • 21. Slice enforces Collocation Constraint on persistent closure <ul><li>Persistent Closure of a managed instance x is </li></ul><ul><ul><li>The set of instances C x :{y} where y is reachable from x by traversal of a relation cascaded as PERSIST or MERGE to an unlimited depth </li></ul></ul><ul><li>Persistence Closure C x , at the time of persist() or merge(), are stored in the same slice </li></ul><ul><ul><li>Because Slice can not join across databases </li></ul></ul><ul><li>Compliant Domain Models are referred as Constrained Tree Schema </li></ul>
  • 22. An example domain model
  • 23. Instances that violate persistent closure must be replicated Stock CSCO Ask-152 Bid-153 Trader-1 Trader-2 Trade-15 Stock GS Ask-210 Bid-211 Trader-7 Trader-1 Trade-21 slice.One slice.Two Trader-1 will violate collocation constraint and must be replicated across all slices. Data partitioned by Stock sectors
  • 24. Replicate master/shared data across slices <ul><li>E numerate replicated types in configuration </li></ul><ul><li>By default, replicated entities are stored in all slices </li></ul><ul><ul><li>or implement ReplicationPolicy </li></ul></ul><ul><li>01: public class DefaultReplicationPolicy implements ReplicationPolicy { </li></ul><ul><li>02: public String[] replicate(Object pc, List<String> slices, Object ctx) { </li></ul><ul><li>03: return slices.toArray(); </li></ul><ul><li>05: } </li></ul>< property name=“ openjpa.slice.ReplicatedTypes ” value=“ domain.Trader ”/> Data Replication Policy
  • 25. Replication Policy public interface ReplicationPolicy { /** * Decide the name of the slices where the given persistent * instance would be replicated. * * @param pc The newly persistent or to-be-merged object. * @param slices name of the configured slices. * @param context persistence context managing the given instance. * * @return identifier(s) of the slice. Each name must match one of the * configured slice names. * @see DistributedConfiguration#getSliceNames() */ String[] replicate ( Object pc, List<String> slices, Object context); } Slice runtime will call this method while persisting any replicated instance.
  • 26. Distributed Query <ul><li>Each query is executed across target slices in parallel </li></ul><ul><li>Performance upper bound is the size of the largest partition not the size of the entire dataset. </li></ul><ul><ul><li>The query will execute on 100 million instead of 400 million record </li></ul></ul><ul><li>Slice attaches moniker to the selected instance based on its origin </li></ul>
  • 27. Query Target Policy decides target slice for each query public interface QueryTargetPolicy { /** * Decide the name of the slice(s) where the given query * will be executed. * * @param query The query string to be executed. * @param params the bound parameters of the query. * @param language the language of the query * @param slices name of the configured slices * @param context persistence context executing the query. * * @return identifier of the target slices. Null value implies * all configured slices. */ String[] getTargets ( String query, Map params, String language, List<String> slices, Object context); } Slice runtime will call this method for every query. Default policy targets all available slices.
  • 28. Intrusive way to control target slices for a query EntityManager em = …; String jpql = &quot; SELECT p FROM Person p where p.name=:name ”; TypedQuery <Person> query1 = em.createQuery(jpql, Person. class ); // Set a single slice as query target query1.setHint( SlicePersistence.HINT_TARGET , “ One &quot;); List < Person > result1 = query1.setParameter(“ name ”, “ XYZ ”) .getResultList(); TypedQuery <Person> query2 = em.createQuery(jpql , Person. class ); // Set multiple slices as query targets query2.setHint( SlicePersistence.HINT_TARGET , Arrays.asList( new String []{“ One “,” Two ”}); List < Person > result2 = query2.setParameter(“ name ”, “ ABC ”) .getResultList();
  • 29. Distributed Query results are appended <ul><li>Results from individual slices are appended </li></ul>slice1 slice3 slice2 String jpql = “ select e from Employee e where e.age > 30 ”; List < Employee > result = em.createQuery(jpql, Employee. class ).getResultList(); 2001 29 BILL 2005 37 LEUNG 2008 22 ROB JOIN_YEAR AGE NAME 1987 41 JOSE 1999 35 SHIVA 2002 31 HARI JOIN_YEAR AGE NAME 1975 43 SANDRA 2007 24 MARY 2001 35 JOHN JOIN_YEAR AGE NAME 2007 24 MARY 2008 22 ROB 2001 29 BILL 2008 22 ROB 2001 29 BILL 2007 24 MARY
  • 30. Distributed Query results are sorted in-memory <ul><li>Results from individual slices are sorted across target slices for ORDER BY queries in-memory </li></ul>slice1 slice3 slice2 String jpql = “ select e from Employee e where e.age < 30 order by e.name ”; List < Employee > result = em.createQuery(jpql, Emloyee. class ).getResultList(); 2001 29 BILL 2005 37 LEUNG 2008 22 ROB JOIN_YEAR AGE NAME 1987 41 JOSE 1999 35 SHIVA 2002 31 HARI JOIN_YEAR AGE NAME 1975 43 SANDRA 2007 24 MARY 2001 35 JOHN JOIN_YEAR AGE NAME 2007 24 MARY 2008 22 ROB 2001 29 BILL 2007 22 ROB 2007 24 MARY 2001 29 BILL
  • 31. Distributed Top-N Query <ul><li>Top-N Result from each slice is merged (with ordering, if any) for LIMIT BY queries </li></ul>slice1 slice3 slice2 String jpql = “ select e from Employee e order by e.age ”; List < Employee > result = em.createQuery(jpql, Emloyee. class ) .getMaxResult(2).getResultList(); 2001 29 BILL 2005 37 LEUNG 2008 22 ROB JOIN_YEAR AGE NAME 1987 41 JOSE 1999 35 SHIVA 2002 31 HARI JOIN_YEAR AGE NAME 1975 43 SANDRA 2007 24 MARY 2001 35 JOHN JOIN_YEAR AGE NAME 2001 29 BILL 2008 22 ROB 2001 35 JOHN 2007 24 MARY 1999 35 SHIVA 2002 31 HARI 2007 24 MARY 2008 22 ROB
  • 32. Distributed Top-N Query <ul><li>Top-N Results from individual slices are appended for LIMIT BY queries without an ORDER BY clause. </li></ul>slice1 slice3 slice2 List result = em.createQuery(“ select e from Employee e ”) .setMaxResult(2).getResultList(); 2001 29 BILL 2005 37 LEUNG 2008 22 ROB JOIN_YEAR AGE NAME 1987 41 JOSE 1999 35 SHIVA 2002 31 HARI JOIN_YEAR AGE NAME 1975 43 SANDRA 2007 24 MARY 2001 35 JOHN JOIN_YEAR AGE NAME 2001 29 BILL 2008 22 ROB 2001 35 JOHN 2007 24 MARY 1999 35 SHIVA 2002 31 HARI 2007 24 MARY 2008 22 ROB
  • 33. Targeted Query <ul><li>Query and find() can be targeted to a subset of slices by hints </li></ul>slice1 slice3 slice2 List result = em.createQuery (“ SELECT e FROM Employee e WHERE e.age > 34 ”) . setHint (“ openjpa.slice.Targets ”, “ slice1,slice3 ”) . getResultList (); 2001 29 BILL 2005 37 LEUNG 2008 22 ROB JOIN_YEAR AGE NAME 1987 41 JOSE 1999 35 SHIVA 2002 31 HARI JOIN_YEAR AGE NAME 1975 43 SANDRA 2007 24 MARY 2001 35 JOHN JOIN_YEAR AGE NAME 2001 35 JOHN 1975 43 SANDRA 1999 35 SHIVA 1987 41 JOSE 1999 35 SHIVA 1987 41 JOSE 2001 35 JOHN 1975 43 SANDRA
  • 34. Aggregate Query <ul><li>Aggregate results are supported when aggregate operation is commutative to partition </li></ul>slice1 slice3 slice2 Number sum = em.createQuery (“ select sum(e.age) from Employee e where e.age > 30 ”, Number.class ).getSingleResult(); 2001 29 BILL 2005 37 LEUNG 2008 22 ROB JOIN_YEAR AGE NAME 1987 41 JOSE 1999 35 SHIVA 2002 31 HARI JOIN_YEAR AGE NAME 1975 43 SANDRA 2007 24 MARY 2001 35 JOHN JOIN_YEAR AGE NAME 78 37 107 222 78 37 107
  • 35. Distributed Aggregate Query Limitations <ul><li>Commutativity </li></ul><ul><ul><li>ability to change the order of operations without changing the end result. </li></ul></ul><ul><li>SUM() or MAX() is commutative to partition </li></ul><ul><ul><li>SUM(D) = SUM(SUM(D1), SUM(D2), SUM(D3)) where Partition(D) = {D1,D2,D3} </li></ul></ul><ul><li>But AVG() is not </li></ul><ul><ul><li>AVG(D) != AVG(AVG(D1), AVG(D2), AVG(D3)) </li></ul></ul>
  • 36. Aggregate Query <ul><li>Aggregate results are not supported when aggregate operation is not commutative to partition </li></ul>slice1 slice3 slice2 Number sum = em.createQuery (“ select avg(e.age) from Employee e ”, Number.class ) .getSingleResult(); 34.0 + 30.0 32.0 = ] + 32.0 [ / 3 Wrong! 2001 29 BILL 2005 38 LEUNG 2008 23 ROB JOIN_YEAR AGE NAME 1999 35 SHIVA 2002 31 HARI JOIN_YEAR AGE NAME 1975 43 SANDRA 2007 24 MARY 2001 35 JOHN JOIN_YEAR AGE NAME 34.0 30.0 32.0
  • 37. Query for Replicated Entities <ul><li>Replicated instances are detected and queried in a single slice </li></ul>Number sum = ( Number )em.createQuery(“ SELECT COUNT(c) FROM Coutry c ”) .getSingleResult(); slice1 slice3 slice2 3 3 1200M INDIA 82M GERMANY 300M US POPULATION CODE 1200M INDIA 82M GERMANY 300M US POPULATION CODE 1200M INDIA 82M GERMANY 300M US POPULATION CODE
  • 38. META-INF/persistence.xml configures a persistence unit <? xml version=&quot; 1.0 &quot; encoding=&quot; UTF-8 &quot;?> < persistence xmlns=&quot; http://java.sun.com/xml/ns/persistence &quot; xmlns:xsi=&quot; http://www.w3.org/2001/XMLSchema-instance &quot; version=&quot; 1.0 &quot; xsi:schemaLocation=&quot; http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd &quot;> < persistence-unit name=&quot; test “ transaction=“ RESOURCE_LOCAL ”> < provider > org.apache.openjpa.persistence.PersistenceProviderImpl </ provider > < class > domain.EntityA </ class > < class > domain.EntityB </ class > < properties > < property name=&quot; openjpa.ConnectionDriverName &quot; value=&quot; com.mysql.jdbc.Driver &quot;/> < property name=&quot; openjpa.ConnectionURL &quot; value=&quot; jdbc:mysql://localhost/test &quot;/> < property name=&quot; openjpa.jdbc.SynchronizeMappings &quot; value=&quot; buildSchema &quot;/> < property name=&quot; openjpa.Log &quot; value=&quot; SQL=TRACE &quot;/> </ properties > </ persistence-unit > List of known Persistent types Vendor-specific configuration Governed by XML Schema JPA Provider is pluggable Identified by Unit Name
  • 39. Activate Slice through configuration <ul><li>< property name=&quot; openjpa.BrokerFactory &quot; value=“ slice &quot;/> </li></ul><ul><li>Mandatory configuration </li></ul><ul><li>Activates a specialized EntityManagerFactory </li></ul>
  • 40. Each slice is referred by a moniker <ul><li>< property name=“ openjpa.slice.Names ” value=“ One,Two,Three ”/> </li></ul><ul><li>Optional ( but recommended ) configuration </li></ul><ul><li>Associates mnemonics to physical slices </li></ul>
  • 41. Identify a Master slice <ul><li>< property name=“ openjpa.slice.Master ” value=“ One ”/> </li></ul><ul><li>Optional ( but recommended ) configuration </li></ul><ul><li>Identifes a master slice for identity generation </li></ul>
  • 42. <ul><li>< property name=&quot; openjpa.slice. One . ConnectionURL “ value=&quot; jdbc:mysql://localhost/slice1 &quot;/> </li></ul><ul><li>< property name=“ openjpa.slice. Two .ConnectionURL ” value=“ jdbc:mysql://localhost/slice2 ”/> </li></ul>Specify physical slice connection details <ul><li>Mandatory configuration </li></ul><ul><li>Specifies physical connection for each slice </li></ul><ul><li>Property name prfixed by the slice moniker </li></ul>Moniker for a slice
  • 43. Slices can share common properties <ul><li>< property name=“ openjpa.slice.Names ” value=“ One,Two,Three ”/> </li></ul><ul><li>< property name=&quot; openjpa.ConnectionDriverName “ </li></ul><ul><li>value=&quot; com.mysql.jdbc.Driver &quot;/> </li></ul><ul><li>< property name=“ openjpa .slice.Three. ConnectionDriverName “ </li></ul><ul><li>value=“ com.ibm.db2.jcc.DB2Driver ”/> </li></ul><ul><li>Properties can be shared </li></ul><ul><li>unless overwritten for a specific slice </li></ul>
  • 44. Ignoring unavailable slices <ul><li>< property name=“ openjpa.slice.Lenient ” value=“ true ”/> </li></ul><ul><li>Optional configuration </li></ul><ul><li>Ignores any unreachable slice </li></ul>
  • 45. Configuration Rules <ul><li>Each slice is identified by a moniker </li></ul><ul><li>All monikers should be explicitly declared in openjpa.slice.Names </li></ul><ul><ul><li>Though implicit declaration is allowed </li></ul></ul><ul><ul><ul><li>openjpa.slice.XYZ.abc declares a slice with moniker XYZ </li></ul></ul></ul><ul><li>A master slice is either configured by openjpa.slice.Master property </li></ul><ul><ul><li>Or automatically detected by convention/heuristic as the first slice </li></ul></ul><ul><li>Each slice must be configured with database URL </li></ul>
  • 46. Configuration Rules (continued) <ul><li>Other properties can be shared </li></ul><ul><li>Each slice property defaults to common configuration </li></ul><ul><ul><li>If openjpa.slice.XYZ.abc is unspecified, then abc defaults to openjpa.abc property </li></ul></ul>
  • 47. A complete example of Slice Configuration < properties > < property name=&quot; openjpa.BrokerFactory &quot; value=“ slice &quot;/> < property name=“ openjpa.slice.Names ” value=“ One,Two,Three ”/> < property name=“ openjpa.slice.Master ” value=“ One ”/> < property name=&quot; openjpa.ConnectionDriverName &quot; value=&quot; com.mysql.jdbc.Driver &quot;/> < property name=&quot; openjpa.slice.One.ConnectionURL &quot; value=&quot; jdbc:mysql://mac1:3456/slice1 &quot;/> < property name=“ openjpa.slice.Two.ConnectionURL ” value=“ jdbc:mysql://mac2:5634/slice2 ”/> < property name=“ openjpa.slice.Three.ConnectionDriverName ” value=“ com.ibm.db2.jcc.DB2Driver ”/> < property name=“ openjpa.slice.Three.ConnectionURL ” value=“ jdbc:db2 :// mac3:50000/slice3 ”/> < property name=&quot; openjpa.slice.DistributionPolicy &quot; value=“ acme.org.MyDistroPolicy &quot;/> < property name=&quot; openjpa.jdbc.SynchronizeMappings &quot; value=&quot; buildSchema &quot;/> </ properties > </ persistence-unit > META-INF/persistence.xml Activate Slice Declare slices Configure each slice Configure common behavior Define Data Distribution Policy
  • 48. Updates <ul><li>Slice remembers original slice of each instance. </li></ul><ul><ul><li>SlicePersistence .getSlice( Object pc) returns the logical slice name for the given argument. </li></ul></ul><ul><li>If an instance is modified then the update occurs in the original slice. </li></ul><ul><li>Replicated instances are updated to many slices </li></ul><ul><ul><li>SlicePersistence .isReplicated( Object pc) </li></ul></ul><ul><li>Commit will not be invoked for a slice if no update exists for that slice </li></ul>
  • 49. Database and Transaction <ul><li>Slices can be in heterogeneous database platforms </li></ul><ul><ul><li>Each slice can use its own JDBC driver </li></ul></ul><ul><li>A pseudo (weaker) 2-phase commit protocol </li></ul>
  • 50. Agenda <ul><li>Core Features of Slice </li></ul><ul><li>Using Slice </li></ul><ul><li>Under the hood </li></ul><ul><li>Running on Slice </li></ul>Source: If applicable, describe source origin
  • 51. Core Architectural constructs of OpenJPA EntityManager Factory BrokerFactory EntityManager Broker StoreManager JDBCStore Manager JDBC API OpenJPA Configuration creates creates delegates delegates configured by facade kernel storage POJO + State manager
  • 52. Slice extends OpenJPA by Distributed Template EntityManager Factory BrokerFactory EntityManager Broker DistributedStoreManager JDBCStore Manager JDBC API JDBCStore Manager JDBCStore Manager Distributed Configuration applies Distributed Template Pattern Not aware of partitioned Databases applies Distributed Template Pattern OpenJPA Configuration OpenJPA Configuration OpenJPA Configuration + Slice Moniker facade kernel storage POJO + State manager
  • 53. Distributed Template Design Pattern public class DistributedTemplate < T > implements T , Iterable < T > { protected List < T > _delegates = new ArrayList < T >(); public void add(T t) { _delegates .add(t); } public Iterator < T > iterator() { return _delegates .iterator(); } // execution requires operation-specific merge semantics public boolean execute(String arg0) { boolean ret = true ; for (T t : this ) ret = t.execute(arg0) & ret; // merge execution result return ret; } } <ul><li>Similar to Composite </li></ul>
  • 54. Slice applies Distributed Template Design Pattern on OpenJPA/JDBC <ul><li>Distributed Template Design Pattern as main metaphor </li></ul><ul><ul><li>on JDBC artifacts (Statement, ResultSet) </li></ul></ul><ul><ul><li>major OpenJPA artifacts such as StoreManager, Query. </li></ul></ul>
  • 55. Agenda <ul><li>Core Features of Slice </li></ul><ul><li>Using Slice </li></ul><ul><li>Under the hood </li></ul><ul><li>Running on Slice </li></ul>
  • 56. OpenTrader : OpenJPA/Slice and GWT
  • 57. An example data distribution policy /** * This distribution policy determines the sector of the stock and * picks the slice at ordinal index of the enumerated Sector. */ public class SectorDistributionPolicy implements DistributionPolicy { public String distribute( Object pc, List < String > slices, Object context) { Stock stock = null ; if (pc instanceof Tradable ) { stock = (( Tradable )pc).getStock(); } else if (pc instanceof Stock ) { stock = ( Stock )pc; } else if (pc instanceof Trade ) { stock = (( Trade )pc).getStock(); } else { throw new IllegalArgumentException(“ No policy for “ + pc); } return stock != null ? slices.get(stock.getSector().ordinal()) : null ; } }
  • 58. An example query target policy public static final String MATCH_BID = &quot; select new Match(a,b) from Ask a, Bid b &quot; + &quot; where b = :bid and a.stock.symbol = b.stock.symbol &quot; + &quot; and a.price <= b.price and a.volume >= b.volume &quot; + &quot; and NOT (a.seller = b.buyer) “ + “ and a.trade is NULL and b.trade is NULL &quot;; public class SectorBasedQueryTargetPolicy implements QueryTargetPolicy { public String [] getTargets( String query, Map < Object , Object > params, String language, List < String > slices, Object context) { Stock stock = null ; if (TradingService. MATCH_BID .equals(query)) { stock = ((Tradable)params.get(&quot; bid &quot;)).getStock(); return new String[]{slices.get(stock.getSector().ordinal())}; } return null ; } }
  • 59. Future Work <ul><li>Support a wider notion of heterogeneity </li></ul><ul><ul><li>Different mappings to different databases </li></ul></ul><ul><ul><li>Mixing data storage technologies </li></ul></ul>
  • 60. Future Work <ul><li>Dynamic reconfiguration </li></ul><ul><ul><li>Adding slices </li></ul></ul><ul><ul><li>Removing slices </li></ul></ul><ul><ul><li>Availability/Consistency debate </li></ul></ul><ul><li>Detection of unsupported queries </li></ul><ul><li>Stronger transaction warranty </li></ul>
  • 61. References <ul><li>Slice Documentation </li></ul><ul><li>http://openjpa.apache.org/builds/latest/docs/manual/manual.html#ref_guide_slice </li></ul><ul><li>Article on Slice </li></ul><ul><li>http://www.ibm.com/developerworks/java/library/os-openjpa/index.html?ca=drs- </li></ul><ul><li>OpenTrader: a case-study on Slice + GWT </li></ul><ul><li>http://openjpa.apache.org/samples/opentrader </li></ul><ul><li>svn co https://svn.apache.org/repos/asf/openjpa/trunk/openjpa-examples/opentrader </li></ul>
  • 62. Thank You!

×