Java Web Programming on Google Cloud Platform [2/3] : Datastore


Published on

Java Web Programming on Google App Engine, July 2012

Published in: Technology
1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Java Web Programming on Google Cloud Platform [2/3] : Datastore

  1. 1. Introduction to DatastoreAssoc.Prof. Dr.Thanachart Numnonda Asst.Prof. Thanisa Kruawaisayawan July 2012
  2. 2. AgendaWhat is DataStore?Using DataStoreJPA in DataStore
  3. 3. What is DataStore?
  4. 4. What is Datastore?Google App Engine Datastore is a schema-less persistence system, whose fundamental persistence unit is called Entity, c omposed by an immutable Key and a collection of mutable pr operties.Entities can be created, updated, deleted, loaded by key and queried for properties values.DataStore is consistent and transactional, with support to current transaction.
  5. 5. The DataStoreThe Datastore is not a relational database nor a façade.Relational database technology doesn’t scale horizontally – Connection pools, shared caching are a problemThe Datastore is one of many public APIs used for accessing Google’s
  6. 6. The DataStore
  7. 7. The DataStore
  8. 8. The DataStore : OperationsTransactions and Index are based on MegaTable.File persistence its done with Google File System (GFS).Its distributed by Chubby, a lock service for loosely- coupled distributed systems.
  9. 9. BigTableBigTable is a compressed, high performance, and proprietary database system built on Google File System (GFS), Chubby Lock Service, and a few other Google programsCurrently not distributed or used outside of Google.BigTable development began in 2004. and is now used by a number of Google application Google Earth, Google Map, Gmail, Youtube, etc..
  10. 10. BigTable : DesignBigTable is a fast and extremely large-scale DBMS.It is a sparse, distributed multi-dimensional sorted map, sharing characteristics of both row-oriented and column- oriented databases. sparse because only "not null" values are persisted distributed in Google cloud persistent on Google File System multidimensional in columns values ordered lexicographically by key
  11. 11. BigTable : DesignTables are optimized for GFS by being split into multiple tablets - segments of the table.BigTable is designed to scale into the petabyte.Each table has multiple dimensions (one of which is a feld for time, allowing for versioning and garbage collection).It allows an infnite number of rows and columns.
  12. 12. Google File SystemGFS is a proprietary distributed fle system developed by Google.It is designed to provide effcient, reliable access to data using large clusters of commodity hardware.GFS grew out of an earlier Google effort, BigFiles, developed by Larry Page and Sergey Brin in the early days of Google, while it was still located in Stanford.
  13. 13. Using DataStore
  14. 14. DataStore OperationsDatastore operations are defned around entities (data models) which are objects with one or more properties Types: string, user, Boolean, and so on Entities may be recursive or self-referentialEntity relationships are one-to-many or many-to-many.Entities may be fxed or grow as needed.
  15. 15. DataStore Storage ModelEvery entity is of a particular kindEntities in a kind need not have the same properties One entity may have different “columns” from another in the same kind!Unique IDs are automatically assigned unless the user defnes a key_name
  16. 16. Compare DataStore with Others
  17. 17. DataStore Storage ModelBasic unit of storage is an Entity consisting of Kind (table) Key (primary key) Entity Group (partition) 0..N typed Properties (columns)
  18. 18. Datastore QuotasEach call to Datastore counts towards the quotaThe amount of data cannot exceed the billable  Includes properties and keys but not the indicesCPU and Datastore CPU time quotas apply
  19. 19. Using the DatastoreApplications may access the Datastore using the JDO or the JPA classes.The JDO and JPA classes are abstracted using the DataNucleus API Open source  Not very popular  Support for Java standards  Poor documentation
  20. 20. JPA in DataStore
  21. 21. Setting Up JPAThe JPA and datastore JARs must be in the apps war/WEB-INF/lib/ directory.A confguration fle named persistence.xml must be in the apps war/WEB-INF/classes/META-INF/ directory,A confguration fle tells JPA to use the App Engine datastore.The appengine-api.jar must also be in the war/WEB- INF/lib/ directory.
  22. 22. persistence.xml: Example<?xml version="1.0" encoding="UTF-8"?> <?xml version="1.0" encoding="UTF-8"?><persistence version="1.0" xmlns="" <persistence version="1.0" xmlns="" xmlns:xsi="" xmlns:xsi="" xsi:schemaLocation=" xsi:schemaLocation="">"> <persistence-unit name="thaijavaappPU" transaction-type="RESOURCE_LOCAL"> <persistence-unit name="thaijavaappPU" transaction-type="RESOURCE_LOCAL"> <provider> <provider> </provider> </provider> <non-jta-data-source/> <non-jta-data-source/> <properties> <properties> <property name="datanucleus.ConnectionURL" value="appengine"/> <property name="datanucleus.ConnectionURL" value="appengine"/> <property name="datanucleus.NontransactionalRead" value="true"/> <property name="datanucleus.NontransactionalRead" value="true"/> <property name="datanucleus.NontransactionalWrite" value="true"/> <property name="datanucleus.NontransactionalWrite" value="true"/> </properties> </properties> </persistence-unit> </persistence-unit></persistence> </persistence>
  23. 23. Getting an EntityManager InstanceAn app interacts with JPA using an instance of the EntityManager.import javax.persistence.EntityManagerFactory; import javax.persistence.EntityManagerFactory;import javax.persistence.Persistence; import javax.persistence.Persistence;public class EMF {{ public class EMF private static final EntityManagerFactory emfInstance == private static final EntityManagerFactory emfInstance Persistence.createEntityManagerFactory("transactions-optional"); Persistence.createEntityManagerFactory("transactions-optional"); public static EntityManagerFactory get() {{ public static EntityManagerFactory get() return emfInstance; return emfInstance; }}}}
  24. 24. Entity Class : Example@Entity @Entitypublic class GuestList implements Serializable {{ public class GuestList implements Serializable …… @Id @Id private String id; private String id; @Basic @Basic private User author; private User author; private String content; private String content; @Temporal(javax.persistence.TemporalType.DATE) @Temporal(javax.persistence.TemporalType.DATE) private Date visitDate; private Date visitDate; …… // Getter and Setter methods // Getter and Setter methods}}
  25. 25. Queries and IndicesA query operates on every entity of a given kind. Specify zero or more sort orders Specify zero or more flters on property valuesIndices are defned in the App Engine confguration fles Results are fetched directly from these indices; no indices are created on the fly WEB-INF/datastore-indexes.xml - non-standard flesNormalization is not recommended Optimization techniques for RDBMSs may result in poor Datastore performance!
  26. 26. Query : ExampleEntityManager em == EMF.get().createEntityManager(); EntityManager em EMF.get().createEntityManager();try {{ try Query query == em.createQuery("SELECT oo FROM GuestList AS o"); Query query em.createQuery("SELECT FROM GuestList AS o"); @SuppressWarnings("unchecked") @SuppressWarnings("unchecked") List<GuestList> results == (List<GuestList>) query.getResultList(); List<GuestList> results (List<GuestList>) query.getResultList(); for (Object obj :: results) {{ for (Object obj results) GuestList guest == (GuestList) obj; GuestList guest (GuestList) obj; String nickname == guest.getAuthor().getNickname(); String nickname guest.getAuthor().getNickname(); out.println(nickname ++ "" "" ++ guest.getId()); out.println(nickname guest.getId()); }}}} catch(Exception ex) {{ catch(Exception ex) out.println(ex); out.println(ex);}}
  27. 27. Entity RelationshipsModels association between entities.There are four types of relationship multiplicities: @OneToOne @OneToMany @ManyToOneSupports unidirectional as well as bidirectional relationships Unidirectional relationship: Entity A references B, but B doesnt reference A.
  28. 28. Example : ManyToOne Mapping
  29. 29. Example : OneToMany Mapping
  30. 30. Transactions and Entity GroupsTransaction = Group of Datastore operations that either succeed or failEntity groups are required because all grouped entities are stored in the same Datastore nodeAn entity may be either created or modifed once per transactionTransactions may fail if a different user or process tries an update in the same group at the same timeUsers decide whether to retry or roll the transaction back
  31. 31. Transaction in JPA : ExampleBook book == em.find(Book.class, "9780596156732"); Book book em.find(Book.class, "9780596156732");BookReview bookReview == new BookReview(); BookReview bookReview new BookReview();bookReview.rating == 5; bookReview.rating 5;book.getBookReviews().add(bookReview); book.getBookReviews().add(bookReview);Transaction txn == em.getTransaction(); Transaction txn em.getTransaction();txn.begin(); txn.begin();try {{ try book == em.merge(book); book em.merge(book); txn.commit(); txn.commit();}} finally {{ finally if (txn.isActive()) {{ if (txn.isActive()) txn.rollback(); txn.rollback(); }}}}
  32. 32. Unsupported Features of JPAOwned many-to-many relationships, and unowned relationships."Join" queries.Aggregation queries (group by, having, sum, avg, max, min)Polymorphic queries.
  33. 33. ResourcesGoogle App Engine for Java HOWTO, Andrew Lombardi, Mar 2010The Softer Side Of Schemas, Max Ross, May 2009Official Google App Engine Tutorial,Programming Google App Engine, Don Sanderson, OReilly, 2010
  34. 34. Thank you