Slice: OpenJPA for Distributed Persistence

WebSphere Services Technical virtual Conferences world class skill building and technical enablement Session Number: D05 Scale Your JPA Applications with Distributed Database Partitions Dr. Pinaki Poddar ppoddar@us.ibm.com © 2009 IBM Corporation Conference materials may not be reproduced in whole or in part without the prior written permission of IBM.

Application Integration & Middleware Overview  Brief tour of JPA – Design-time Features – Runtime Behavior  Role of JPA in JEE – Scalability  Horizontal Distributed Data Partition as a scaling strategy  Slice: JPA for Distributed, Partitioned Databases – Using Slice – Under the hood – Future work  Q&A 2 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware JPA uses POJO for Domain Model  Annotate as @Entity @Entity public class Customer {  Define persistent identity @Id private long id;  Annotate relational mapping private String name;  Use full power of Java @OneToMany(mappedBy=\"customer\") private List<Order> orders; – No interface to implement } – No class to inherit from – Use Collection, List, Set, Map @Entity public class Order { – Use generics @Id  Convention over configuration private long id; @ManyToOne – Implied database naming private Customer customer; – Implied persistence property } 3 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware Persistence Unit *.class Persistence Unit • Set of persistent classes META-INF/persistence.xml • Mapping metadata • Database & other configurations orm.xml 4 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware How to obtain a Persistence Unit? Inject as a resource @PersistenceUnit(unitName=“test”) private EntityManagerFactory emf; Look up in JNDI InitialiContext ctx = new InitialContext(); EntityManagerFactory emf = ctx.lookup(“myEMF”)); Instantiate via bootstrap EntityManagerFactory emf = Persistence.createEntityManagerFactory(“test”); WARNING EntityManagerFactory construction is costly 5 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware Persistence Context Persistence Context • Session/Transaction • Cache of managed instances • Persistent operations •find() •persist() •merge() •remove() •refresh() •createQuery() •… 6 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware How to obtain a Persistence Context? Inject as a resource @PersistenceContext private EntityManager em; Construct from Persistence Unit EntityManager em = emf.createEntityManager() 7 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware Persistence Context manages instances in a group Persistence Unit em1 = emf.createEntityManager(); em2 = emf.createEntityManager(); Persistence Context Persistence Context Account pc1 = Account pc2 = em1.find(Account.class,1245); em2.find(Account.class,1245); SELECT ID,NAME FROM ACCOUNT t Account WHERE t.ID=1245 ID NAME AMOUNT 2347 John $ 12000.57 1245 Mary $ 34568.89 8 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware A Question about Identity Important questions pc1 == pc2 ? pc1.equals(pc2) ? Java supports two identities JPA adds another identity • Reference-based identity • Persistence-based identity • Value-based identity •Persistence Identity defines uniqueness within a Persistence Context •An instance is managed by one and only one Persistence Context at a time 9 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware Life of Persistence Context em = emf.createEntityManager(); close(); begin(); commit(); begin(); flush(); commit(); clear(); Time Extended Persistence Context Transactional Persistence Context 10 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware Query executes in a Persistence Context  Persistence Context is factory for Query  Query is expressed in JPQL (Java Persistence Query Language)  Selected instances are added to the persistence context  Selected instances are returned in a List EntityManager em = …; String jpql = “SELECT p FROM Person c WHERE p.name=:name”; List result = em.createQuery() .setParameter(“name”, “John”) .getResultList(); 11 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware Transaction Scaling in JPA em = emf.createEntityManager() close() begin() commit() begin() flush() commit() clear() Time Extended Persistence Context Transaction-scoped Persistence Context L2 Data Cache Database Transaction 12 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware Optimistic Versioning scales transaction Optimistic Exception 1 7 begin(); 3 commit(); pc1 = find(Item.class, 1234); 10 Item:1234 5 Item:1234 UPDATE ITEM qty:5 SET QTY=87, VERSION=57 pc1.setQty(30); qty:30 version: 56 version: 56 WHERE ID=1234 AND VERSION=56 8 UPDATE ITEM SET QTY=30, VERSION=57 WHERE ID=1234 AND VERSION=56 ID 1234 QTY 5 VERSION 56 ID 1234 QTY 30 VERSION 57 2 begin(); 4 pc2 = find(Item.class, 1234); 9 commit(); Item:1234 6 Item:1234 qty:5 qty:87 version: 56 pc2.setQty(87); version: 56 13 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware Multi-level caches favor read-mostly sessions em1 = emf.createEntityManager(); em2 = emf.createEntityManager(); em2.commit() em1.begin() em1.commit() em2.begin() em1.query() em2.find() em2.remove() em2.find() Time L2 Data Cache Database Access & Transaction 14 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware Scaling against Data Volume  Data is growing rapidly – compounded annual growth rate of worldwide capacity of compliant records from 2003 to 2006 64% – Unbounded nature of the Web • From a web site, a company can generate several gigabytes of data each day 15 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware Distributed Horizonal Partition  Horizontal Partition – put different rows into different tables  Distributed Horizontal Partition – put different rows into different databases 16 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware divide et impera  Natural partition exists in many domains – Geographical (Customers by State) – Temporal (PurchaseOrders by Month) – Personal (Blog Posts by User)  Partition is natural in some scenarios – Hosted Platforms – Software-As-Service 17 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware What is Slice?  Slice is an OpenJPA module to transact with distributed, horizontally partitioned databases  Incubated as Apache Lab project in Jan 2008  Included as OpenJPA module since June 2008  Slice is bundled with OpenJPA within WAS v7 Slice is not the best thing since sliced bread 19 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware What is OpenJPA?  An implementation of JPA Specification  Default persistence provider for WebSphere EJB3 Feature Pack v 6.1 and WAS v 7.x  Apache Project since May 2007 – http://openjpa.apache.org  Operational codebase since 2002  Rich, extended, ahead-of-the-curve feature set  Powerful configurability 20 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware Architectural Tiers of Slice-based service Slice-based User Application Standard JPA API Slice OpenJPA JDBC API OpenJPA is a plugabble platform 22 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware Features of Slice No change to User-defined Application code or Distribution & Domain Model Replication Policy Slice-based User Application Flexible per-Slice Configuration Standard JPA API Slice OpenJPA Targeted Query JDBC API Parallel Query Execution Heterogeneous Databases Master-based Sequence 23 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware Persistence Unit Configuration <?xml version=\"1.0\" encoding=\"UTF-8\"?> Governed by <persistence xmlns=\"http://java.sun.com/xml/ns/persistence\" XML Schema JPA Provider xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" version=\"1.0\" is pluggable xsi:schemaLocation=\"http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_1_0.xsd\"> <persistence-unit name=\"test“ transaction=“RESOURCE_LOCAL”> <provider>org.apache.openjpa.persistence.PersistenceProviderImpl</provider> <class>domain.EntityA</class> List of known <class>domain.EntityB</class> Persistent types <properties> <property name=\"openjpa.ConnectionDriverName\" value=\"com.mysql.jdbc.Driver\"/> <property name=\"openjpa.ConnectionURL\" value=\"jdbc:mysql://localhost/test\"/> <property name=\"openjpa.jdbc.SynchronizeMappings\" value=\"buildSchema\"/> <property name=\"openjpa.Log\" value=\"SQL=TRACE\"/> </properties> </persistence-unit> Vendor-specific configuration Identified by META-INF/persistence.xml Unit Name 26 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware Per-Slice Configuration <properties> Activate Slice <property name=\"openjpa.BrokerFactory\" value=“slice\"/> Declare slices <property name=“openjpa.slice.Names” value=“One,Two,Three”/> <property name=“openjpa.slice.Master” value=“One”/> Define Data Distribution Policy <property name=\"openjpa.slice.DistributionPolicy\" value=“acme.org.MyDistroPolicy\"/> Configure each slice <property name=\"openjpa.ConnectionDriverName\" value=\"com.mysql.jdbc.Driver\"/> <property name=\"openjpa.slice.One.ConnectionURL\" value=\"jdbc:mysql://localhost/slice1\"/> <property name=“openjpa.slice.Two.ConnectionURL” value=“jdbc:mysql://localhost/slice2”/> <property name=“openjpa.slice.Three.ConnectionDriverName” value=“com.ibm.db2.jcc.DB2Driver”/> <property name=“openjpa.slice.Three.ConnectionURL” value=“jdbc:db2://mac3:50000/slice3”/> Configure common behavior <property name=\"openjpa.jdbc.SynchronizeMappings\" value=\"buildSchema\"/> </properties> </persistence-unit> META-INF/persistence.xml 27 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware Rules of Configuring slices  Each slice is identified by a logical name  All slice names can be specified by openjpa.slice.Names  Or determined implicitly – openjpa.slice.XYZ.abc declares a slice with logical name XYZ  Each slice can be configured independently  Each slice property defaults to common configuration – If openjpa.slice.XYZ.abc is not specified then it defaults to value of openjpa.abc property  A master slice is either configured by openjpa.slice.Master property  Or automatically detected by convention/heuristic as the first slice  Unreachable slices are ignored at startup if openjpa.slice.Lenient 28 propertySWG/IBM to true. Pinaki Poddar, is set © 2009 IBM Corporation

Application Integration & Middleware How to distribute data across slices? 01: EntityManager em = …; Domain Classes 02: em.getTransaction().begin(); @Entity 03: Person person = new Person(); public class Person { User Application 04: person.setName(“John”); private String name; private int age; 05: person.setAge(42); @OneToOne (cascade=ALL) 06: Address addr = new Address(); private Address address; } 07: addr.setCity(“New York”); 08: person.setAddress(addr); @Entity public class Address { 09: em.persist(person); private String city; 10: em.getTransation().commit(); } Data Distribution Policy 01: public class MyDistributionPolicy implements DistributionPolicy { 02: public String distribute(Object pc, List<String> slices, Object ctx) { 03: return ((Person)pc).getAge() > 40) 04: ? slices.get(0) : slices.get(1); 05: } 06: } 29 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware Distribution Policy public interface DistributionPolicy { /** * Gets the name of the slice where the given newly persistent * instance will be stored. * * @param pc The newly persistent or to-be-merged object. * @param slices name of the configured slices. * @param context persistence context managing the given instance. * * @return identifier of the slice. This name must match one of the * configured slice names. * @see DistributedConfiguration#getSliceNames() */ String distribute(Object pc, List<String> slices, Object context); } Slice will call this method while persisting or merging a root instance. The instance and its closure will be stored in the returned slice. 30 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware Collocation Constraint  All instances reachable from a root instance, at the time of persist(), are stored in the same slice – Because Slice can not join across databases  Compliant Domain Models are referred as Constrained Tree Schema – Customer has Orders has LineItems – http://www.devwebsphere.com/devwebsphere/2008/01/constrained-tre.html Customer 0+ Order 1+ LineItem 1 1 31 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware What if schema is not a Constrained Tree Schema? Company Department Employee Address • Partition into databases per Department • Tree Schema Constraint is violated • In which database should Company and Country reside? Country 32 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware Replicate Master Data across slices  Annotate Company and Country as @Replicated  By default, @Replicated entities are stored in all slices – or implement ReplicationPolicy @Entity @org.apache.openjpa.persistence.Replicated public class Company {..} 33 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware Replication Policy public interface ReplicationPolicy { /** * Gets the name of the slices where the given newly persistent * instance will be replicated. * * @param pc The newly persistent or to-be-merged object. * @param slices name of the configured slices. * @param context persistence context managing the given instance. * * @return identifier(s) of the slice. Each name must match one of the * configured slice names. * @see DistributedConfiguration#getSliceNames() */ String[] replicate(Object pc, List<String> slices, Object context); } Slice will call this method while persisting any @Replicated instance. 34 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware Distributed Query  Each query is executed across all slices in parallel  Performance upper bound is the size of the largest partition not the size of the entire dataset. 35 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware Distributed Query  Results from individual slices are appended List result = em.createQuery(“SELECT e FROM Employee e WHERE e.age < 30”) .getResultList(); MARY 24 2007 BILL 29 2001 ROB 22 2008 BILL 29 2001 MARY 24 2007 ROB 22 2008 slice1 slice2 slice3 NAME AGE JOIN_YEAR NAME AGE JOIN_YEAR NAME AGE JOIN_YEAR JOHN 35 2001 ROB 22 2008 HARI 31 2002 MARY 24 2007 LEUNG 37 2005 SHIVA 35 1999 SANDRA 43 1975 BILL 29 2001 JOSE 41 1987 36 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware Distributed Query (Sorting)  Results from individual slices are sorted across all slices for ORDER BY queries List result = em.createQuery(“SELECT e FROM Employee e WHERE e.age < 30 ORDER BY e.name”).getResultList(); BILL 29 2001 MARY 24 2007 ROB 22 2007 BILL 29 2001 MARY 24 2007 ROB 22 2008 slice1 slice2 slice3 NAME AGE JOIN_YEAR NAME AGE JOIN_YEAR NAME AGE JOIN_YEAR JOHN 35 2001 ROB 22 2008 HARI 31 2002 MARY 24 2007 LEUNG 37 2005 SHIVA 35 1999 SANDRA 43 1975 BILL 29 2001 JOSE 41 1987 37 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware Distributed Top-N Query  Top-N Result from each slice is merged (with ordering, if any) for LIMIT BY queries List result = em.createQuery(“SELECT e FROM Employee e ORDER BY e.age”) .setMaxResult(2).getResultList(); ROB 22 2008 MARY 24 2007 MARY 24 2007 ROB 22 2008 HARI 31 2002 JOHN 35 2001 BILL 29 2001 SHIVA 35 1999 slice1 slice2 slice3 NAME AGE JOIN_YEAR NAME AGE JOIN_YEAR NAME AGE JOIN_YEAR JOHN 35 2001 ROB 22 2008 HARI 31 2002 MARY 24 2007 LEUNG 37 2005 SHIVA 35 1999 SANDRA 43 1975 BILL 29 2001 JOSE 41 1987 38 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware Distributed Top-N Query  Top-N Results from individual slices are appended for LIMIT BY queries without an ORDER BY clause. List result = em.createQuery(“SELECT e FROM Employee e”) .setMaxResult(2).getResultList(); ROB 22 2008 MARY 24 2007 MARY 24 2007 ROB 22 2008 HARI 31 2002 JOHN 35 2001 BILL 29 2001 SHIVA 35 1999 slice1 slice2 slice3 NAME AGE JOIN_YEAR NAME AGE JOIN_YEAR NAME AGE JOIN_YEAR JOHN 35 2001 ROB 22 2008 HARI 31 2002 MARY 24 2007 LEUNG 37 2005 SHIVA 35 1999 SANDRA 43 1975 BILL 29 2001 JOSE 41 1987 39 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware Targeted Query  Query and find() can be targeted to a subset of slices by hints List result = em.createQuery(“SELECT e FROM Employee e WHERE e.age > 34”) .setHint(“openjpa.slice.Targets”, “slice1,slice3”) .getResultList(); SANDRA 43 1975 JOHN 35 2001 JOSE 41 1987 SHIVA 35 1999 SANDRA 43 1975 JOSE 41 1987 JOHN 35 2001 SHIVA 35 1999 slice1 slice2 slice3 NAME AGE JOIN_YEAR NAME AGE JOIN_YEAR NAME AGE JOIN_YEAR JOHN 35 2001 ROB 22 2008 HARI 31 2002 MARY 24 2007 LEUNG 37 2005 SHIVA 35 1999 SANDRA 43 1975 BILL 29 2001 JOSE 41 1987 40 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware Aggregate Query  Aggregate results are supported when aggregate operation is commutative to partition Number sum = (Number)em.createQuery(“SELECT SUM(e.age) FROM Employee e WHERE e.age > 30”).getSingleResult(); 78 37 107 222 78 37 107 slice1 slice2 slice3 NAME AGE JOIN_YEAR NAME AGE JOIN_YEAR NAME AGE JOIN_YEAR JOHN 35 2001 ROB 22 2008 HARI 31 2002 MARY 24 2007 LEUNG 37 2005 SHIVA 35 1999 SANDRA 43 1975 BILL 29 2001 JOSE 41 1987 41 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware Aggregate Query  Aggregate results are not supported when aggregate operation is not commutative to partition Number sum = (Number)em.createQuery(“SELECT AVG(e.age) FROM Employee e WHERE e.age > 30”).getSingleResult(); 37.0 37.0 35.6 36.5 WRONG! 3 37.0 37.0 35.6 slice1 slice2 slice3 NAME AGE JOIN_YEAR NAME AGE JOIN_YEAR NAME AGE JOIN_YEAR JOHN 35 2001 ROB 22 2008 HARI 31 2002 MARY 24 2007 LEUNG 37 2005 SHIVA 35 1999 SANDRA 43 1975 BILL 29 2001 JOSE 41 1987 42 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware Distributed Aggregate Query Limitations  Commutativity – ability to change the order of operations without changing the end result.  SUM() or MAX() is commutative to partition – SUM(D) = SUM(SUM(D1), SUM(D2), SUM(D3)) where Partition(D) = {D1,D2,D3}  But AVG() is not – AVG(D) != AVG(AVG(D1), AVG(D2), AVG(D3)) 43 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware Query for Replicated Entities  Replicated instances are detected and queried in a single slice Number sum = (Number)em.createQuery(“SELECT COUNT(c) FROM Coutry c”) .getSingleResult(); 3 @Entity @Replicated public class Country {..} 3 slice1 slice2 slice3 CODE POPULATION CODE POPULATION CODE POPULATION US 300M US 300M US 300M GERMANY 82M GERMANY 82M GERMANY 82M INDIA 1200M INDIA 1200M INDIA 1200M 44 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware Updates  Slice remembers original slice of each instance. – SlicePersistence.getSlice(Object pc) returns the logical slice name for the given argument.  If an instance is modified then the update occurs in the original slice.  Replicated instances are updated to many slices – SlicePersistence.isReplicated(Object pc)  Commit will not be invoked for a slice if no update exists for that slice 45 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware Database and Transaction  Slices can be in heterogeneous database platforms – Each slice can use its own JDBC driver  A Master slice is identified for sequence generation  Commits are executed in parallel without any warranty  If all JDBC drivers are XA-compliant then a 2-phase commit provision is available – Each slice transaction is not seen by the Application Server’s Transaction Manager. 46 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware Core Architectural constructs of OpenJPA EntityManager creates EntityManager Factory delegates delegates creates BrokerFactory Broker configured by OpenJPA StoreManager Configuration JDBCStore Manager JDBC API 48 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware Distributed Template Design Pattern • Distributed Template Design Pattern as main metaphor • on JDBC artifacts (Statement, ResultSet) • major OpenJPA artifacts such as StoreManager, Query. public class DistributedTemplate<T> implements T, Iterable<T> { protected List<T> _delegates = new ArrayList<T>(); public boolean execute(String arg0) { boolean ret = true; for (T t:this) ret = t.execute(arg0) & ret; return ret; } public Iterator<T> iterator() { return _delegates.iterator(); } } 49 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware Slice extends OpenJPA by Distributed Template EntityManager creates EntityManager Factory delegates delegates creates BrokerFactory Broker Not aware of partitioned Databases configures DistributedStoreManager applies Distributed Template Pattern Distributed Configuration JDBCStore JDBCStore Manager JDBCStore Manager Manager JDBC API 50 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware Future Work: Evolving data distribution  Gradual Redistribution – Complete migration from one slice to another is currently supported – Gradual migration of data from one slice to another • Read from one slice, write to another 52 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware Future Work: Courage under Fire  Graceful degradation – can ignore unreachable slices at bootstrap – can cope with unreachable slices at runtime – can not reconnect dynamically 53 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Application Integration & Middleware Future Work: Heterogeneity  Heterogeneous Schema – assumes each slice has identical schema – relax this assumption  Join relation across slices – this one is hard problem 54 Pinaki Poddar, SWG/IBM © 2009 IBM Corporation

Slice: OpenJPA for Distributed Persistence

0 comments

Notes on slide 1

1 Favorite