Successfully reported this slideshow.
Scale Java Persistence API Applications with OpenJPA Slice  Pinaki Poddar [email_address]
Agenda <ul><li>Core Features of Slice </li></ul><ul><li>Using Slice </li></ul><ul><li>Under the hood </li></ul><ul><li>Run...
What is Slice? <ul><li>Slice is a OpenJPA module for horizontally, partitioned databases </li></ul>Java Persistence API Sp...
Horizontal Partitioning <ul><li>A data set  D  is said to be horizontally partitioned into N partitions  D1 ,  D2 ,   ,  ...
Horizontal Partitioning in realistic setup <ul><li>Natural partitioning scenarios </li></ul><ul><ul><li>Customer by region...
History of Slice <ul><li>Incubated as a Apache Lab project in Jan 2008 </li></ul><ul><li>Integrated as a OpenJPA module on...
OpenJPA  <ul><li>An implementation of JPA Specification </li></ul><ul><li>Apache Project since May 2007   http://openjpa.a...
Architectural tiers of a typical JPA-based application User Application OpenJPA Standard JPA API JDBC API 400 million reco...
Architectural tiers of a Slice-based application User Application OpenJPA Standard JPA API JDBC API Slice OpenJPA is a plu...
Separate Persistence Unit configured to partitioned databases  C A [1] D A Unit A C A [2] C B [1] D B Unit B C B [2] C C [...
Same persistence unit switches contexts to partitioned databases C A [1] D A Persistence Unit C A [2] C C [2] C C [1] C B ...
Same persistence unit connected to partitioned databases C[1] D A Persistence Unit C[2] D B D C
Features of Slice Slice-based User Application OpenJPA Standard JPA API JDBC API Slice No changes to Application code User...
Agenda <ul><li>Core Features of Slice </li></ul><ul><li>Using Slice </li></ul><ul><li>Under the hood </li></ul><ul><li>Run...
Using Slice <ul><li>Decide partition policy to </li></ul><ul><ul><li>distribute data  </li></ul></ul><ul><ul><li>target qu...
Policy based configuration <ul><li>The design goal of “ no application code change ” conflicts with the user application’s...
Data Distribution Policy determines where new records are stored <ul><li>Which slice stores a new data record? </li></ul><...
How to distribute data across slices? <ul><li>01:   EntityManager  em = …; </li></ul><ul><li>02:  em.getTransaction().begi...
Distribution Policy decides target slice for each instance public interface   DistributionPolicy  {  /**  * Decide the nam...
Details on Distribution Policy  a b em.persist(a); //  or em.merge(a); persistence context MyPolicy.distribute(a,…) { retu...
Slice enforces Collocation Constraint on persistent closure <ul><li>Persistent Closure  of a managed instance  x  is </li>...
An example domain model
Instances that violate persistent closure must be replicated Stock CSCO Ask-152 Bid-153 Trader-1 Trader-2 Trade-15 Stock G...
Replicate  master/shared data  across slices <ul><li>E numerate replicated types in configuration   </li></ul><ul><li>By d...
Replication Policy public   interface   ReplicationPolicy  {  /**  * Decide the name of the slices where the given persist...
Distributed Query <ul><li>Each query is executed across target slices  in   parallel </li></ul><ul><li>Performance upper b...
Query Target Policy decides target slice for each query public interface   QueryTargetPolicy  {  /**  * Decide the name of...
Intrusive way to control target slices for a query EntityManager  em = …; String   jpql = &quot; SELECT p FROM Person p wh...
Distributed Query results are appended <ul><li>Results from individual slices are appended  </li></ul>slice1 slice3 slice2...
Distributed Query results are sorted in-memory <ul><li>Results from individual slices are sorted  across target slices  fo...
Distributed Top-N Query <ul><li>Top-N Result from each slice is merged (with ordering, if any) for  LIMIT BY  queries </li...
Distributed Top-N Query <ul><li>Top-N Results from individual slices are appended for  LIMIT BY  queries without an  ORDER...
Targeted Query <ul><li>Query and  find()  can be targeted to a subset of slices by  hints </li></ul>slice1 slice3 slice2 L...
Aggregate Query <ul><li>Aggregate results are supported when aggregate operation is  commutative  to partition </li></ul>s...
Distributed Aggregate Query Limitations <ul><li>Commutativity </li></ul><ul><ul><li>ability to change the order of operati...
Aggregate Query <ul><li>Aggregate results are  not  supported when aggregate operation is  not   commutative  to partition...
Query for Replicated Entities <ul><li>Replicated instances are detected and queried in a  single  slice </li></ul>Number  ...
META-INF/persistence.xml configures a persistence unit  <? xml  version=&quot; 1.0 &quot; encoding=&quot; UTF-8 &quot;?> <...
Activate Slice through configuration <ul><li>< property   name=&quot; openjpa.BrokerFactory &quot;  value=“ slice &quot;/>...
Each slice is referred by a moniker <ul><li>< property   name=“ openjpa.slice.Names ” value=“ One,Two,Three ”/> </li></ul>...
Identify a Master slice <ul><li>< property   name=“ openjpa.slice.Master ” value=“ One ”/> </li></ul><ul><li>Optional ( bu...
<ul><li>< property   name=&quot; openjpa.slice. One . ConnectionURL “  value=&quot; jdbc:mysql://localhost/slice1 &quot;/>...
Slices can share common properties <ul><li>< property   name=“ openjpa.slice.Names ” value=“ One,Two,Three ”/> </li></ul><...
Ignoring unavailable slices  <ul><li>< property   name=“ openjpa.slice.Lenient ” value=“ true ”/> </li></ul><ul><li>Option...
Configuration Rules  <ul><li>Each slice is identified by a moniker </li></ul><ul><li>All monikers should be explicitly dec...
Configuration Rules (continued)  <ul><li>Other properties can be shared </li></ul><ul><li>Each slice property defaults to ...
A complete example of Slice Configuration  < properties > < property   name=&quot; openjpa.BrokerFactory &quot;  value=“ s...
Updates <ul><li>Slice remembers  original  slice of each instance.  </li></ul><ul><ul><li>SlicePersistence .getSlice( Obje...
Database and Transaction <ul><li>Slices can be in  heterogeneous  database platforms </li></ul><ul><ul><li>Each slice can ...
Agenda <ul><li>Core Features of Slice </li></ul><ul><li>Using Slice </li></ul><ul><li>Under the hood </li></ul><ul><li>Run...
Core Architectural constructs of OpenJPA EntityManager Factory BrokerFactory EntityManager Broker StoreManager JDBCStore M...
Slice extends OpenJPA by Distributed Template EntityManager Factory BrokerFactory EntityManager Broker DistributedStoreMan...
Distributed Template Design Pattern public class   DistributedTemplate < T >  implements   T ,  Iterable < T > { protected...
Slice applies Distributed Template Design Pattern on OpenJPA/JDBC  <ul><li>Distributed Template Design Pattern  as main me...
Agenda <ul><li>Core Features of Slice </li></ul><ul><li>Using Slice </li></ul><ul><li>Under the hood </li></ul><ul><li>Run...
OpenTrader : OpenJPA/Slice and GWT
An example data distribution policy /** * This distribution policy determines the sector of the stock and * picks the slic...
An example query target policy public   static   final  String  MATCH_BID   = &quot; select   new  Match(a,b)  from  Ask a...
Future Work <ul><li>Support a wider notion of heterogeneity </li></ul><ul><ul><li>Different mappings to different database...
Future Work <ul><li>Dynamic reconfiguration </li></ul><ul><ul><li>Adding slices </li></ul></ul><ul><ul><li>Removing slices...
References <ul><li>Slice Documentation </li></ul><ul><li>http://openjpa.apache.org/builds/latest/docs/manual/manual.html#r...
Thank You!
Upcoming SlideShare
Loading in …5
×

Slice for Distributed Persistence (JavaOne 2010)

1,371 views

Published on

OpenJPA Slice extends JPA for horizontally partitioned database environment. This session was originally presented in JavaOne 2010.

Published in: Technology
  • Be the first to comment

Slice for Distributed Persistence (JavaOne 2010)

  1. 1. Scale Java Persistence API Applications with OpenJPA Slice Pinaki Poddar [email_address]
  2. 2. Agenda <ul><li>Core Features of Slice </li></ul><ul><li>Using Slice </li></ul><ul><li>Under the hood </li></ul><ul><li>Running on Slice </li></ul>Source: If applicable, describe source origin
  3. 3. What is Slice? <ul><li>Slice is a OpenJPA module for horizontally, partitioned databases </li></ul>Java Persistence API Specification [Section 3.1] says: “ A persistence unit defines the set of all classes that are related or grouped by the application, and which must be colocated in their mapping to a single database.” Slice changes that… single to multiple
  4. 4. Horizontal Partitioning <ul><li>A data set D is said to be horizontally partitioned into N partitions D1 , D2 ,  , Dn iff </li></ul><ul><ul><ul><li>D = D1  D2  D3   Dn </li></ul></ul></ul><ul><ul><ul><li>Di  Dj =  for any i  j </li></ul></ul></ul><ul><li>A common mathematical term is mutually disjoint sets </li></ul><ul><li>Google coined such partition operation as Shard . </li></ul>
  5. 5. Horizontal Partitioning in realistic setup <ul><li>Natural partitioning scenarios </li></ul><ul><ul><li>Customer by region (Telecom Billing) </li></ul></ul><ul><ul><li>Transaction by Month (Finance) </li></ul></ul><ul><ul><li>Software as Service Platforms (Legal compliance) </li></ul></ul>
  6. 6. History of Slice <ul><li>Incubated as a Apache Lab project in Jan 2008 </li></ul><ul><li>Integrated as a OpenJPA module on July 2008 (since version 1.1) </li></ul><ul><li>Available with WebSphere Application Server version 7.0 onwards </li></ul>
  7. 7. OpenJPA <ul><li>An implementation of JPA Specification </li></ul><ul><li>Apache Project since May 2007 http://openjpa.apache.org </li></ul>
  8. 8. Architectural tiers of a typical JPA-based application User Application OpenJPA Standard JPA API JDBC API 400 million records
  9. 9. Architectural tiers of a Slice-based application User Application OpenJPA Standard JPA API JDBC API Slice OpenJPA is a plugabble platform User-defined Data Distribution Policy 4x100 million records
  10. 10. Separate Persistence Unit configured to partitioned databases C A [1] D A Unit A C A [2] C B [1] D B Unit B C B [2] C C [1] D C Unit C C C [2]
  11. 11. Same persistence unit switches contexts to partitioned databases C A [1] D A Persistence Unit C A [2] C C [2] C C [1] C B [2] C B [1] D B D C
  12. 12. Same persistence unit connected to partitioned databases C[1] D A Persistence Unit C[2] D B D C
  13. 13. Features of Slice Slice-based User Application OpenJPA Standard JPA API JDBC API Slice No changes to Application code User-defined Data Distribution Policy Flexible per-Slice Configuration Parallel Query Execution Heterogeneous Databases Master-based Sequence Targeted Query No changes to Domain Model User-defined Query Target Policy No changes to Database Schema 4x100 millon records
  14. 14. Agenda <ul><li>Core Features of Slice </li></ul><ul><li>Using Slice </li></ul><ul><li>Under the hood </li></ul><ul><li>Running on Slice </li></ul>Source: If applicable, describe source origin
  15. 15. Using Slice <ul><li>Decide partition policy to </li></ul><ul><ul><li>distribute data </li></ul></ul><ul><ul><li>target query </li></ul></ul><ul><li>Configure JPA persistence unit </li></ul><ul><li>No change to </li></ul><ul><ul><li>Application Code (ok, almost!) </li></ul></ul><ul><ul><li>Domain Model </li></ul></ul><ul><ul><li>Database Schema </li></ul></ul>
  16. 16. Policy based configuration <ul><li>The design goal of “ no application code change ” conflicts with the user application’s ability to control </li></ul><ul><ul><li>which slice(s) will store a new instance </li></ul></ul><ul><ul><li>which slice(s) will be searched for a query </li></ul></ul><ul><li>The compromise solution is policy based callback interfaces </li></ul><ul><ul><li>DistributionPolicy </li></ul></ul><ul><ul><li>ReplicationPolicy </li></ul></ul><ul><ul><li>QueryTargetPolicy </li></ul></ul><ul><ul><li>FinderTargetPolicy </li></ul></ul><ul><li>User application may implement these policies </li></ul><ul><li>Slice runtime would call the policy method </li></ul>
  17. 17. Data Distribution Policy determines where new records are stored <ul><li>Which slice stores a new data record? </li></ul><ul><ul><li>Only the user application can decide </li></ul></ul>
  18. 18. How to distribute data across slices? <ul><li>01: EntityManager em = …; </li></ul><ul><li>02: em.getTransaction().begin(); </li></ul><ul><li>03: Person person = new Person(); </li></ul><ul><li>04: person.setName(“ John ”); </li></ul><ul><li>05: person.setAge(42); </li></ul><ul><li>06: Address addr = new Address(); </li></ul><ul><li>07: addr.setCity(“ New York ”); </li></ul><ul><li>08: person.setAddress(addr); </li></ul><ul><li>09: em.persist(person); </li></ul><ul><li>10: em.getTransation().commit(); </li></ul><ul><li>01: public class MyDistributionPolicy implements DistributionPolicy { </li></ul><ul><li>02: public String distribute (Object pc, List<String> slices, Object ctx) { </li></ul><ul><li>03: return slices.get((( Person )pc).getAge()/20); </li></ul><ul><li>04: } </li></ul><ul><li>05: } </li></ul>@Entity public class Person { private String name; private int age; @OneToOne (cascade=ALL) private Address address; } @Entity public class Address { private String city; } User Application Domain Classes Data Distribution Policy
  19. 19. Distribution Policy decides target slice for each instance public interface DistributionPolicy { /** * Decide the name of the slice where the given persistent * instance would be stored. * * @param pc The newly persistent or to-be-merged object. * @param slices name of the configured slices. * @param context persistence context managing the given instance. * * @return identifier of the slice. This name must match one of the * configured slice names. * @see DistributedConfiguration#getSliceNames() */ String distribute ( Object pc, List<String> slices, Object context); } Slice runtime will call this method while persisting or merging a root instance. The instance and its persistent closure will be stored in the returned slice.
  20. 20. Details on Distribution Policy a b em.persist(a); // or em.merge(a); persistence context MyPolicy.distribute(a,…) { return “One” ; } One One < property name=&quot; openjpa.slice.DistributionPolicy &quot; value=“ acme.org.MyPolicy &quot;/> Slice attaches moniker to managed instance as it enters a persistence context Slice runtime calls User Application CascadeType.PERSIST CascadeType.MERGE c One
  21. 21. Slice enforces Collocation Constraint on persistent closure <ul><li>Persistent Closure of a managed instance x is </li></ul><ul><ul><li>The set of instances C x :{y} where y is reachable from x by traversal of a relation cascaded as PERSIST or MERGE to an unlimited depth </li></ul></ul><ul><li>Persistence Closure C x , at the time of persist() or merge(), are stored in the same slice </li></ul><ul><ul><li>Because Slice can not join across databases </li></ul></ul><ul><li>Compliant Domain Models are referred as Constrained Tree Schema </li></ul>
  22. 22. An example domain model
  23. 23. Instances that violate persistent closure must be replicated Stock CSCO Ask-152 Bid-153 Trader-1 Trader-2 Trade-15 Stock GS Ask-210 Bid-211 Trader-7 Trader-1 Trade-21 slice.One slice.Two Trader-1 will violate collocation constraint and must be replicated across all slices. Data partitioned by Stock sectors
  24. 24. Replicate master/shared data across slices <ul><li>E numerate replicated types in configuration </li></ul><ul><li>By default, replicated entities are stored in all slices </li></ul><ul><ul><li>or implement ReplicationPolicy </li></ul></ul><ul><li>01: public class DefaultReplicationPolicy implements ReplicationPolicy { </li></ul><ul><li>02: public String[] replicate(Object pc, List<String> slices, Object ctx) { </li></ul><ul><li>03: return slices.toArray(); </li></ul><ul><li>05: } </li></ul>< property name=“ openjpa.slice.ReplicatedTypes ” value=“ domain.Trader ”/> Data Replication Policy
  25. 25. Replication Policy public interface ReplicationPolicy { /** * Decide the name of the slices where the given persistent * instance would be replicated. * * @param pc The newly persistent or to-be-merged object. * @param slices name of the configured slices. * @param context persistence context managing the given instance. * * @return identifier(s) of the slice. Each name must match one of the * configured slice names. * @see DistributedConfiguration#getSliceNames() */ String[] replicate ( Object pc, List<String> slices, Object context); } Slice runtime will call this method while persisting any replicated instance.
  26. 26. Distributed Query <ul><li>Each query is executed across target slices in parallel </li></ul><ul><li>Performance upper bound is the size of the largest partition not the size of the entire dataset. </li></ul><ul><ul><li>The query will execute on 100 million instead of 400 million record </li></ul></ul><ul><li>Slice attaches moniker to the selected instance based on its origin </li></ul>
  27. 27. Query Target Policy decides target slice for each query public interface QueryTargetPolicy { /** * Decide the name of the slice(s) where the given query * will be executed. * * @param query The query string to be executed. * @param params the bound parameters of the query. * @param language the language of the query * @param slices name of the configured slices * @param context persistence context executing the query. * * @return identifier of the target slices. Null value implies * all configured slices. */ String[] getTargets ( String query, Map params, String language, List<String> slices, Object context); } Slice runtime will call this method for every query. Default policy targets all available slices.
  28. 28. Intrusive way to control target slices for a query EntityManager em = …; String jpql = &quot; SELECT p FROM Person p where p.name=:name ”; TypedQuery <Person> query1 = em.createQuery(jpql, Person. class ); // Set a single slice as query target query1.setHint( SlicePersistence.HINT_TARGET , “ One &quot;); List < Person > result1 = query1.setParameter(“ name ”, “ XYZ ”) .getResultList(); TypedQuery <Person> query2 = em.createQuery(jpql , Person. class ); // Set multiple slices as query targets query2.setHint( SlicePersistence.HINT_TARGET , Arrays.asList( new String []{“ One “,” Two ”}); List < Person > result2 = query2.setParameter(“ name ”, “ ABC ”) .getResultList();
  29. 29. Distributed Query results are appended <ul><li>Results from individual slices are appended </li></ul>slice1 slice3 slice2 String jpql = “ select e from Employee e where e.age > 30 ”; List < Employee > result = em.createQuery(jpql, Employee. class ).getResultList(); 2001 29 BILL 2005 37 LEUNG 2008 22 ROB JOIN_YEAR AGE NAME 1987 41 JOSE 1999 35 SHIVA 2002 31 HARI JOIN_YEAR AGE NAME 1975 43 SANDRA 2007 24 MARY 2001 35 JOHN JOIN_YEAR AGE NAME 2007 24 MARY 2008 22 ROB 2001 29 BILL 2008 22 ROB 2001 29 BILL 2007 24 MARY
  30. 30. Distributed Query results are sorted in-memory <ul><li>Results from individual slices are sorted across target slices for ORDER BY queries in-memory </li></ul>slice1 slice3 slice2 String jpql = “ select e from Employee e where e.age < 30 order by e.name ”; List < Employee > result = em.createQuery(jpql, Emloyee. class ).getResultList(); 2001 29 BILL 2005 37 LEUNG 2008 22 ROB JOIN_YEAR AGE NAME 1987 41 JOSE 1999 35 SHIVA 2002 31 HARI JOIN_YEAR AGE NAME 1975 43 SANDRA 2007 24 MARY 2001 35 JOHN JOIN_YEAR AGE NAME 2007 24 MARY 2008 22 ROB 2001 29 BILL 2007 22 ROB 2007 24 MARY 2001 29 BILL
  31. 31. Distributed Top-N Query <ul><li>Top-N Result from each slice is merged (with ordering, if any) for LIMIT BY queries </li></ul>slice1 slice3 slice2 String jpql = “ select e from Employee e order by e.age ”; List < Employee > result = em.createQuery(jpql, Emloyee. class ) .getMaxResult(2).getResultList(); 2001 29 BILL 2005 37 LEUNG 2008 22 ROB JOIN_YEAR AGE NAME 1987 41 JOSE 1999 35 SHIVA 2002 31 HARI JOIN_YEAR AGE NAME 1975 43 SANDRA 2007 24 MARY 2001 35 JOHN JOIN_YEAR AGE NAME 2001 29 BILL 2008 22 ROB 2001 35 JOHN 2007 24 MARY 1999 35 SHIVA 2002 31 HARI 2007 24 MARY 2008 22 ROB
  32. 32. Distributed Top-N Query <ul><li>Top-N Results from individual slices are appended for LIMIT BY queries without an ORDER BY clause. </li></ul>slice1 slice3 slice2 List result = em.createQuery(“ select e from Employee e ”) .setMaxResult(2).getResultList(); 2001 29 BILL 2005 37 LEUNG 2008 22 ROB JOIN_YEAR AGE NAME 1987 41 JOSE 1999 35 SHIVA 2002 31 HARI JOIN_YEAR AGE NAME 1975 43 SANDRA 2007 24 MARY 2001 35 JOHN JOIN_YEAR AGE NAME 2001 29 BILL 2008 22 ROB 2001 35 JOHN 2007 24 MARY 1999 35 SHIVA 2002 31 HARI 2007 24 MARY 2008 22 ROB
  33. 33. Targeted Query <ul><li>Query and find() can be targeted to a subset of slices by hints </li></ul>slice1 slice3 slice2 List result = em.createQuery (“ SELECT e FROM Employee e WHERE e.age > 34 ”) . setHint (“ openjpa.slice.Targets ”, “ slice1,slice3 ”) . getResultList (); 2001 29 BILL 2005 37 LEUNG 2008 22 ROB JOIN_YEAR AGE NAME 1987 41 JOSE 1999 35 SHIVA 2002 31 HARI JOIN_YEAR AGE NAME 1975 43 SANDRA 2007 24 MARY 2001 35 JOHN JOIN_YEAR AGE NAME 2001 35 JOHN 1975 43 SANDRA 1999 35 SHIVA 1987 41 JOSE 1999 35 SHIVA 1987 41 JOSE 2001 35 JOHN 1975 43 SANDRA
  34. 34. Aggregate Query <ul><li>Aggregate results are supported when aggregate operation is commutative to partition </li></ul>slice1 slice3 slice2 Number sum = em.createQuery (“ select sum(e.age) from Employee e where e.age > 30 ”, Number.class ).getSingleResult(); 2001 29 BILL 2005 37 LEUNG 2008 22 ROB JOIN_YEAR AGE NAME 1987 41 JOSE 1999 35 SHIVA 2002 31 HARI JOIN_YEAR AGE NAME 1975 43 SANDRA 2007 24 MARY 2001 35 JOHN JOIN_YEAR AGE NAME 78 37 107 222 78 37 107
  35. 35. Distributed Aggregate Query Limitations <ul><li>Commutativity </li></ul><ul><ul><li>ability to change the order of operations without changing the end result. </li></ul></ul><ul><li>SUM() or MAX() is commutative to partition </li></ul><ul><ul><li>SUM(D) = SUM(SUM(D1), SUM(D2), SUM(D3)) where Partition(D) = {D1,D2,D3} </li></ul></ul><ul><li>But AVG() is not </li></ul><ul><ul><li>AVG(D) != AVG(AVG(D1), AVG(D2), AVG(D3)) </li></ul></ul>
  36. 36. Aggregate Query <ul><li>Aggregate results are not supported when aggregate operation is not commutative to partition </li></ul>slice1 slice3 slice2 Number sum = em.createQuery (“ select avg(e.age) from Employee e ”, Number.class ) .getSingleResult(); 34.0 + 30.0 32.0 = ] + 32.0 [ / 3 Wrong! 2001 29 BILL 2005 38 LEUNG 2008 23 ROB JOIN_YEAR AGE NAME 1999 35 SHIVA 2002 31 HARI JOIN_YEAR AGE NAME 1975 43 SANDRA 2007 24 MARY 2001 35 JOHN JOIN_YEAR AGE NAME 34.0 30.0 32.0
  37. 37. Query for Replicated Entities <ul><li>Replicated instances are detected and queried in a single slice </li></ul>Number sum = ( Number )em.createQuery(“ SELECT COUNT(c) FROM Coutry c ”) .getSingleResult(); slice1 slice3 slice2 3 3 1200M INDIA 82M GERMANY 300M US POPULATION CODE 1200M INDIA 82M GERMANY 300M US POPULATION CODE 1200M INDIA 82M GERMANY 300M US POPULATION CODE
  38. 38. META-INF/persistence.xml configures a persistence unit <? xml version=&quot; 1.0 &quot; encoding=&quot; UTF-8 &quot;?> < persistence xmlns=&quot; http://java.sun.com/xml/ns/persistence &quot; xmlns:xsi=&quot; http://www.w3.org/2001/XMLSchema-instance &quot; version=&quot; 1.0 &quot; xsi:schemaLocation=&quot; http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd &quot;> < persistence-unit name=&quot; test “ transaction=“ RESOURCE_LOCAL ”> < provider > org.apache.openjpa.persistence.PersistenceProviderImpl </ provider > < class > domain.EntityA </ class > < class > domain.EntityB </ class > < properties > < property name=&quot; openjpa.ConnectionDriverName &quot; value=&quot; com.mysql.jdbc.Driver &quot;/> < property name=&quot; openjpa.ConnectionURL &quot; value=&quot; jdbc:mysql://localhost/test &quot;/> < property name=&quot; openjpa.jdbc.SynchronizeMappings &quot; value=&quot; buildSchema &quot;/> < property name=&quot; openjpa.Log &quot; value=&quot; SQL=TRACE &quot;/> </ properties > </ persistence-unit > List of known Persistent types Vendor-specific configuration Governed by XML Schema JPA Provider is pluggable Identified by Unit Name
  39. 39. Activate Slice through configuration <ul><li>< property name=&quot; openjpa.BrokerFactory &quot; value=“ slice &quot;/> </li></ul><ul><li>Mandatory configuration </li></ul><ul><li>Activates a specialized EntityManagerFactory </li></ul>
  40. 40. Each slice is referred by a moniker <ul><li>< property name=“ openjpa.slice.Names ” value=“ One,Two,Three ”/> </li></ul><ul><li>Optional ( but recommended ) configuration </li></ul><ul><li>Associates mnemonics to physical slices </li></ul>
  41. 41. Identify a Master slice <ul><li>< property name=“ openjpa.slice.Master ” value=“ One ”/> </li></ul><ul><li>Optional ( but recommended ) configuration </li></ul><ul><li>Identifes a master slice for identity generation </li></ul>
  42. 42. <ul><li>< property name=&quot; openjpa.slice. One . ConnectionURL “ value=&quot; jdbc:mysql://localhost/slice1 &quot;/> </li></ul><ul><li>< property name=“ openjpa.slice. Two .ConnectionURL ” value=“ jdbc:mysql://localhost/slice2 ”/> </li></ul>Specify physical slice connection details <ul><li>Mandatory configuration </li></ul><ul><li>Specifies physical connection for each slice </li></ul><ul><li>Property name prfixed by the slice moniker </li></ul>Moniker for a slice
  43. 43. Slices can share common properties <ul><li>< property name=“ openjpa.slice.Names ” value=“ One,Two,Three ”/> </li></ul><ul><li>< property name=&quot; openjpa.ConnectionDriverName “ </li></ul><ul><li>value=&quot; com.mysql.jdbc.Driver &quot;/> </li></ul><ul><li>< property name=“ openjpa .slice.Three. ConnectionDriverName “ </li></ul><ul><li>value=“ com.ibm.db2.jcc.DB2Driver ”/> </li></ul><ul><li>Properties can be shared </li></ul><ul><li>unless overwritten for a specific slice </li></ul>
  44. 44. Ignoring unavailable slices <ul><li>< property name=“ openjpa.slice.Lenient ” value=“ true ”/> </li></ul><ul><li>Optional configuration </li></ul><ul><li>Ignores any unreachable slice </li></ul>
  45. 45. Configuration Rules <ul><li>Each slice is identified by a moniker </li></ul><ul><li>All monikers should be explicitly declared in openjpa.slice.Names </li></ul><ul><ul><li>Though implicit declaration is allowed </li></ul></ul><ul><ul><ul><li>openjpa.slice.XYZ.abc declares a slice with moniker XYZ </li></ul></ul></ul><ul><li>A master slice is either configured by openjpa.slice.Master property </li></ul><ul><ul><li>Or automatically detected by convention/heuristic as the first slice </li></ul></ul><ul><li>Each slice must be configured with database URL </li></ul>
  46. 46. Configuration Rules (continued) <ul><li>Other properties can be shared </li></ul><ul><li>Each slice property defaults to common configuration </li></ul><ul><ul><li>If openjpa.slice.XYZ.abc is unspecified, then abc defaults to openjpa.abc property </li></ul></ul>
  47. 47. A complete example of Slice Configuration < properties > < property name=&quot; openjpa.BrokerFactory &quot; value=“ slice &quot;/> < property name=“ openjpa.slice.Names ” value=“ One,Two,Three ”/> < property name=“ openjpa.slice.Master ” value=“ One ”/> < property name=&quot; openjpa.ConnectionDriverName &quot; value=&quot; com.mysql.jdbc.Driver &quot;/> < property name=&quot; openjpa.slice.One.ConnectionURL &quot; value=&quot; jdbc:mysql://mac1:3456/slice1 &quot;/> < property name=“ openjpa.slice.Two.ConnectionURL ” value=“ jdbc:mysql://mac2:5634/slice2 ”/> < property name=“ openjpa.slice.Three.ConnectionDriverName ” value=“ com.ibm.db2.jcc.DB2Driver ”/> < property name=“ openjpa.slice.Three.ConnectionURL ” value=“ jdbc:db2 :// mac3:50000/slice3 ”/> < property name=&quot; openjpa.slice.DistributionPolicy &quot; value=“ acme.org.MyDistroPolicy &quot;/> < property name=&quot; openjpa.jdbc.SynchronizeMappings &quot; value=&quot; buildSchema &quot;/> </ properties > </ persistence-unit > META-INF/persistence.xml Activate Slice Declare slices Configure each slice Configure common behavior Define Data Distribution Policy
  48. 48. Updates <ul><li>Slice remembers original slice of each instance. </li></ul><ul><ul><li>SlicePersistence .getSlice( Object pc) returns the logical slice name for the given argument. </li></ul></ul><ul><li>If an instance is modified then the update occurs in the original slice. </li></ul><ul><li>Replicated instances are updated to many slices </li></ul><ul><ul><li>SlicePersistence .isReplicated( Object pc) </li></ul></ul><ul><li>Commit will not be invoked for a slice if no update exists for that slice </li></ul>
  49. 49. Database and Transaction <ul><li>Slices can be in heterogeneous database platforms </li></ul><ul><ul><li>Each slice can use its own JDBC driver </li></ul></ul><ul><li>A pseudo (weaker) 2-phase commit protocol </li></ul>
  50. 50. Agenda <ul><li>Core Features of Slice </li></ul><ul><li>Using Slice </li></ul><ul><li>Under the hood </li></ul><ul><li>Running on Slice </li></ul>Source: If applicable, describe source origin
  51. 51. Core Architectural constructs of OpenJPA EntityManager Factory BrokerFactory EntityManager Broker StoreManager JDBCStore Manager JDBC API OpenJPA Configuration creates creates delegates delegates configured by facade kernel storage POJO + State manager
  52. 52. Slice extends OpenJPA by Distributed Template EntityManager Factory BrokerFactory EntityManager Broker DistributedStoreManager JDBCStore Manager JDBC API JDBCStore Manager JDBCStore Manager Distributed Configuration applies Distributed Template Pattern Not aware of partitioned Databases applies Distributed Template Pattern OpenJPA Configuration OpenJPA Configuration OpenJPA Configuration + Slice Moniker facade kernel storage POJO + State manager
  53. 53. Distributed Template Design Pattern public class DistributedTemplate < T > implements T , Iterable < T > { protected List < T > _delegates = new ArrayList < T >(); public void add(T t) { _delegates .add(t); } public Iterator < T > iterator() { return _delegates .iterator(); } // execution requires operation-specific merge semantics public boolean execute(String arg0) { boolean ret = true ; for (T t : this ) ret = t.execute(arg0) & ret; // merge execution result return ret; } } <ul><li>Similar to Composite </li></ul>
  54. 54. Slice applies Distributed Template Design Pattern on OpenJPA/JDBC <ul><li>Distributed Template Design Pattern as main metaphor </li></ul><ul><ul><li>on JDBC artifacts (Statement, ResultSet) </li></ul></ul><ul><ul><li>major OpenJPA artifacts such as StoreManager, Query. </li></ul></ul>
  55. 55. Agenda <ul><li>Core Features of Slice </li></ul><ul><li>Using Slice </li></ul><ul><li>Under the hood </li></ul><ul><li>Running on Slice </li></ul>
  56. 56. OpenTrader : OpenJPA/Slice and GWT
  57. 57. An example data distribution policy /** * This distribution policy determines the sector of the stock and * picks the slice at ordinal index of the enumerated Sector. */ public class SectorDistributionPolicy implements DistributionPolicy { public String distribute( Object pc, List < String > slices, Object context) { Stock stock = null ; if (pc instanceof Tradable ) { stock = (( Tradable )pc).getStock(); } else if (pc instanceof Stock ) { stock = ( Stock )pc; } else if (pc instanceof Trade ) { stock = (( Trade )pc).getStock(); } else { throw new IllegalArgumentException(“ No policy for “ + pc); } return stock != null ? slices.get(stock.getSector().ordinal()) : null ; } }
  58. 58. An example query target policy public static final String MATCH_BID = &quot; select new Match(a,b) from Ask a, Bid b &quot; + &quot; where b = :bid and a.stock.symbol = b.stock.symbol &quot; + &quot; and a.price <= b.price and a.volume >= b.volume &quot; + &quot; and NOT (a.seller = b.buyer) “ + “ and a.trade is NULL and b.trade is NULL &quot;; public class SectorBasedQueryTargetPolicy implements QueryTargetPolicy { public String [] getTargets( String query, Map < Object , Object > params, String language, List < String > slices, Object context) { Stock stock = null ; if (TradingService. MATCH_BID .equals(query)) { stock = ((Tradable)params.get(&quot; bid &quot;)).getStock(); return new String[]{slices.get(stock.getSector().ordinal())}; } return null ; } }
  59. 59. Future Work <ul><li>Support a wider notion of heterogeneity </li></ul><ul><ul><li>Different mappings to different databases </li></ul></ul><ul><ul><li>Mixing data storage technologies </li></ul></ul>
  60. 60. Future Work <ul><li>Dynamic reconfiguration </li></ul><ul><ul><li>Adding slices </li></ul></ul><ul><ul><li>Removing slices </li></ul></ul><ul><ul><li>Availability/Consistency debate </li></ul></ul><ul><li>Detection of unsupported queries </li></ul><ul><li>Stronger transaction warranty </li></ul>
  61. 61. References <ul><li>Slice Documentation </li></ul><ul><li>http://openjpa.apache.org/builds/latest/docs/manual/manual.html#ref_guide_slice </li></ul><ul><li>Article on Slice </li></ul><ul><li>http://www.ibm.com/developerworks/java/library/os-openjpa/index.html?ca=drs- </li></ul><ul><li>OpenTrader: a case-study on Slice + GWT </li></ul><ul><li>http://openjpa.apache.org/samples/opentrader </li></ul><ul><li>svn co https://svn.apache.org/repos/asf/openjpa/trunk/openjpa-examples/opentrader </li></ul>
  62. 62. Thank You!

×