This document provides information on using the Google App Engine Datastore and ORM tools to manage data in a cloud environment. It describes the key concepts of entities, properties, queries and transactions using both the low-level Datastore API and higher-level ORM such as DataNucleus. Code examples are given for common operations like entity creation, retrieval, updating and deletion in both APIs.

3. CAP and ACID CAP 정합성(Consistency). The client perceives that a set of operations has occurred all at once. 가용성(Availability). Every operation must terminate in an intended response. 단절내성(Partition tolerance). Operations will complete, even if individual components are unavailable. ACID 원자성(Atomicity). All of the operations in the transaction will complete, or none will. 일관성(Consistency). The database will be in a consistent state when the transaction begins and ends. 고립성(Isolation). The transaction will behave as if it is the only operation being performed upon the database. 지속성(Durability). Upon completion of the transaction, the operation will not be reversed. http://queue.acm.org/detail.cfm?id=1394128

4. Google BigTable System BigTable Master Server Chubby Server Tablet Server #2 Tablet Server #1 MapReduce Worker #2 MapReduce Worker #1 GFS #2 GFS #1

5. Google BigTable Architecture GAE Application DataNucleus ORM (JDO or JPA Implementation) Low Level API Datastore Meta 0 tablet Big Table Meta 1 tablet Meta 1 tablet Meta 1 tablet Tab 1 Tab 2 Tab 3 Tab 4 Tab n GFS(Google File System)

6. Google BigTable Dynamic control over Data Layout and Format Not support a full Relational Model Dynamic control over serving data from memory or disk Column oriented The map is indexed by A row key A column name A timestamp BigTable Key TimeStamp / Version Columns Index

7. LDAP(Lightweight Directory Access Protocol) The Lightweight Directory Access Protocol (LDAP) is an application protocol for accessing and maintaining distributed directory information services over an Internet Protocol (IP) network. Directory services may provide any organized set of records, often with a hierarchical structure, such as a corporate electronic mail directory. Similarly, a telephone directory is a list of subscribers with an address and a phone number.

8. Google App Engine Product Roadmap Features on Deck App Engine out of Preview SSL access on custom domains Full-text search over Datastore Support for Python 2.7 Support for running MapReduce jobs across App Engine datasets Bulk Datastore Import and Export tool Improved monitoring and alerting of application serving Logging system improvements to remove limits on size and storage Integration with Google Storage for Developer

9. DataStore의 제약사항 Write가 느림 검색 가능한 Text 사이즈: 500 자 Entity 최대 사이즈: 1Mb 최대 Batch (save/delete) : 500 Entity SQL을 사용할 수 없음 데이터베이스의 제약조건을 적용할 수 없음 Aggregation 함수를 지원하지 않음 (count, avg) 쿼리가 최대 조회 레코드: 2000 쿼리에서 Equal 연산 이외의 조건은 1개만 추가 가능 트랜잭션의 단위는 Entity Group에 한정 됨

10. Datastore API and ORM API

11. Datastore API(Low Level API) 모든 Entity는 Kind가 지정 Entity는 하나이상의 Property를 포함 특정 Kind에포함된 Entity는 동일한 Property를 갖지 않음 Entity의 Property는 이름은 동일하지만 데이터 유형이 다를수 있음 Schemaless의 특성 ID는 지정하지 않을 경우 자동 할당 됨

12. Entity App Engine Datastore에서 관리하는 객체는 Entity Entity는 하나의 Key를 포함 모든 Entity 중 유일한 구분자 Key 의 구성 Path Parent Entity의 key Entity의 Kind Entity에 할당된 이름 App이 할당한 값 Datastore가 지정한 numeric ID 지원 데이터 타임 integers, floating point values, strings, dates, binary data 등 각 엔티티는 하나이상의 Property를 포함 하나의 Property에 복수의 데이터 타입 저장 가능 하나의 Property에는 복수의 값이 저장 가능 복수의 값들의 데이터 유형은 다를 수 있음

13. Entity Model Settings Kind : Table Key : Primary Key Entity Group : Partitioning Property : column

14. Entity Example(Create) import java.util.Date;import com.google.appengine.api.datastore.DatastoreService;import com.google.appengine.api.datastore.DatastoreServiceFactory;import com.google.appengine.api.datastore.Entity;// ...DatastoreServicedatastore = DatastoreServiceFactory.getDatastoreService();Entity employee = new Entity("Employee");employee.setProperty("firstName", "Antonio");employee.setProperty("lastName", "Salieri");Date hireDate = new Date();employee.setProperty("hireDate", hireDate);employee.setProperty("attendedHrTraining", true);datastore.put(employee);

15. Entity Example(Scheme Free) import java.util.Date;import com.google.appengine.api.datastore.DatastoreService;import com.google.appengine.api.datastore.DatastoreServiceFactory;import com.google.appengine.api.datastore.Entity;// ...DatastoreServicedatastore = DatastoreServiceFactory.getDatastoreService();Entity employee = new Entity("Employee");employee.setProperty("firstName", “Sun-Jin");datastore.put(employee);

16. Entity Group Entity Group는 부모 Entity를 갖는 Entity의 Group 하나의 트랜잭션에서 여러 Entity를 조회, 생성, 수정, 삭제 할 수 가능하지만 그 범위는 단일 Entity Group에 국한 부모 Entity 를 가지 않은 Entity 를 Root Entity라고 함 Entity Group은 하나의 Root Entity와 Child Entity로 구성 하나의 EntityGroup은 동일한 분산 네트웍 서버에 저장됨 (Tablet 서버)

17. Entity Group Entity Group Root Entity (xxxx) Child Entity (xxxxyyyy) Child Entity (xxxxyyyyzzzz)

18. Entity Example(Read) import java.util.Date;import com.google.appengine.api.datastore.DatastoreService;import com.google.appengine.api.datastore.DatastoreServiceFactory;import com.google.appengine.api.datastore.Entity;// ...DatastoreServicedatastore = DatastoreServiceFactory.getDatastoreService(); Key rootKey = KeyFactory.createKey("Employee", “CEO"); Entity employee = datastore.get(rootKey);

19. Entity Example(Update) import java.util.Date;import com.google.appengine.api.datastore.DatastoreService;import com.google.appengine.api.datastore.DatastoreServiceFactory;import com.google.appengine.api.datastore.Entity;// ...DatastoreServicedatastore = DatastoreServiceFactory.getDatastoreService(); Key rootKey = KeyFactory.createKey("Employee", “CEO"); Entity employee = datastore.get(rootKey); employee.setProperty("firstName", “Sun-Jin");employee.setProperty("lastName", “Jang");datastore.put(employee);

20. Entity Example(Delete) import java.util.Date;import com.google.appengine.api.datastore.DatastoreService;import com.google.appengine.api.datastore.DatastoreServiceFactory;import com.google.appengine.api.datastore.Entity;// ...DatastoreServicedatastore = DatastoreServiceFactory.getDatastoreService(); Key rootKey = KeyFactory.createKey("Employee", “CEO"); Entity employee = datastore.get(rootKey);datastore.delete(employee);

21. Entity Example(Query) // ...DatastoreServicedatastore = DatastoreServiceFactory.getDatastoreService(); Query q = new Query(”employee") q.addFilter("lastName", Query.FilterOperator.EQUAL, lastNameParam); q.addFilter("height", Query.FilterOperator.LESS_THAN, maxHeightParam); PreparedQuerypq = datastore.prepare(q); for (Entity result : pq.asIterable()) { String firstName = (String) result.getProperty("firstName"); String lastName = (String) result.getProperty("lastName"); Long height = (Long) result.getProperty("height"); }

22. Java ORM API

23. DataNucleus DataNucleus attempts to make the whole process of persisting data a transparent process. The idea revolves around the developer having a series of Java classes that need persisting. http://www.datanucleus.org/products/accessplatform/development_process.html

24. JDO Model Define(Annotation) import javax.jdo.annotations.PersistenceCapable;@PersistenceCapablepublic class Employee { @PrimaryKey @Persistent(valueStrategy= IdGeneratorStrategy.IDENTITY) private Key key; @Persistent private String firstName; @Persistent private String lastName; @Persistent private Date hireDate; }

25. JDO Model Define(Annotation) import javax.jdo.annotations.PersistenceCapable; // …@PersistenceCapablepublic class Employee { @PrimaryKey @Persistent(valueStrategy= IdGeneratorStrategy.IDENTITY) private Key key; @Persistent private String firstName; @Persistent private String lastName; @Persistent private Date hireDate; }

26. JDO Model(Create) PersistenceManager pm = PMF.get().getPersistenceManager(); Employee e = new Employee("Alfred", "Smith", new Date()); try { pm.makePersistent(e); } finally { pm.close(); }

27. JDO Model(Update) public void updateEmployeeTitle(User user, String newTitle) { PersistenceManager pm = PMF.get().getPersistenceManager(); try { Employee e = pm.getObjectById(Employee.class, “Sun-Jin”); if (titleChangeIsAuthorized(e, newTitle) { e.setTitle(newTitle); } else { throw new UnauthorizedTitleChangeException(e, newTitle); } } finally { pm.close(); }}

28. JDO Model(Delete) public void updateEmployeeTitle(User user, String newTitle) { PersistenceManager pm = PMF.get().getPersistenceManager(); try { Employee e = pm.getObjectById(Employee.class, “Sun-Jin”); pm.deletePersistent(e); } finally { pm.close(); }}

29. JDO Model(Query) import java.util.List;import javax.jdo.Query;// ... Query query = pm.newQuery(Employee.class); query.setFilter("lastName == lastNameParam"); query.setOrdering("hireDatedesc"); query.declareParameters("String lastNameParam"); try { List<Employee> results = (List<Employee>) query.execute("Smith"); if (!results.isEmpty()) { for (Employee e : results) { // ... } } else { // ... no results ... } } finally { query.closeAll(); }

30. Demo

31. 감사합니다. http://code.google.com/p/devmento-gae-seminar