Successfully reported this slideshow.
Hybrid Appand GAE DataStore<br />대표이사 장선진<br />@jangsunjin<br /><ul><li>Mobile Phone: (+82) 010-4585-1770
 E-Mail: jangsunjin@softwareinlife.com</li></li></ul><li>MapReduce<br />MapReduce is a patented software framework introdu...
CAP and ACID<br />CAP <br />정합성(Consistency). The client perceives that a set of operations has occurred all at once.<br /...
Google BigTable System<br />BigTable Master Server<br />Chubby Server<br />Tablet Server #2<br />Tablet Server #1<br />Map...
Google BigTable Architecture<br />GAE Application<br />DataNucleus ORM<br />(JDO or JPA Implementation)<br />Low Level API...
Google BigTable<br />Dynamic control over Data Layout and Format<br />Not support a full Relational Model<br />Dynamic con...
LDAP(Lightweight Directory Access Protocol)<br />The Lightweight Directory Access Protocol (LDAP) is an application protoc...
Google App Engine Product Roadmap<br />Features on Deck<br />App Engine out of Preview<br />SSL access on custom domains<b...
DataStore의 제약사항<br />Write가 느림<br />검색 가능한 Text 사이즈: 500 자<br />Entity 최대 사이즈: 1Mb<br />최대 Batch (save/delete) : 500 Entit...
Datastore API and ORM API<br />
Datastore API(Low Level API)<br />모든 Entity는 Kind가 지정<br />Entity는 하나이상의 Property를 포함<br />특정 Kind에포함된 Entity는 동일한 Propert...
Entity<br />App Engine Datastore에서 관리하는 객체는 Entity<br />Entity는 하나의 Key를 포함<br />모든 Entity 중 유일한 구분자<br />Key 의 구성<br />Pa...
Entity Model<br />Settings<br />Kind : Table<br />Key : Primary Key<br />Entity Group : Partitioning<br />Property : colum...
Entity Example(Create)<br />import java.util.Date;import com.google.appengine.api.datastore.DatastoreService;import com.go...
Entity Example(Scheme Free)<br />import java.util.Date;import com.google.appengine.api.datastore.DatastoreService;import c...
Entity Group<br />Entity Group는 부모 Entity를 갖는 Entity의 Group<br />하나의 트랜잭션에서 여러 Entity를 조회, 생성, 수정, 삭제 할 수 가능하지만 그 범위는 단일 E...
Entity Group<br />Entity Group<br />Root Entity<br />(xxxx)<br />Child Entity<br />(xxxxyyyy)<br />Child Entity<br />(xxxx...
Entity Example(Read)<br />import java.util.Date;import com.google.appengine.api.datastore.DatastoreService;import com.goog...
Entity Example(Update)<br />import java.util.Date;import com.google.appengine.api.datastore.DatastoreService;import com.go...
Entity Example(Delete)<br />import java.util.Date;import com.google.appengine.api.datastore.DatastoreService;import com.go...
Entity Example(Query)<br />// ...DatastoreServicedatastore = DatastoreServiceFactory.getDatastoreService();<br />Query q =...
Java ORM API<br />
DataNucleus<br />DataNucleus attempts to make the whole process of persisting data a transparent process. <br />The idea r...
JDO Model Define(Annotation)<br />import javax.jdo.annotations.PersistenceCapable;@PersistenceCapablepublic class Employee...
JDO Model Define(Annotation)<br />import javax.jdo.annotations.PersistenceCapable;<br />// …@PersistenceCapablepublic clas...
JDO Model(Create)<br />        PersistenceManager pm = PMF.get().getPersistenceManager();        Employee e = new Employee...
JDO Model(Update)<br />public void updateEmployeeTitle(User user, String newTitle) {    PersistenceManager pm = PMF.get()....
JDO Model(Delete)<br />public void updateEmployeeTitle(User user, String newTitle) {    PersistenceManager pm = PMF.get()....
JDO Model(Query)<br />import java.util.List;import javax.jdo.Query;// ...    Query query = pm.newQuery(Employee.class);   ...
Demo<br />
감사합니다.<br />http://code.google.com/p/devmento-gae-seminar<br />
Upcoming SlideShare
Loading in …5
×

S03 hybrid app_and_gae_datastore_v1.0

6,359 views

Published on

Google BigTable and GAE Datastore

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

S03 hybrid app_and_gae_datastore_v1.0

  1. 1. Hybrid Appand GAE DataStore<br />대표이사 장선진<br />@jangsunjin<br /><ul><li>Mobile Phone: (+82) 010-4585-1770
  2. 2. E-Mail: jangsunjin@softwareinlife.com</li></li></ul><li>MapReduce<br />MapReduce is a patented software framework introduced by Google in 2004 to support distributed computing on large data sets on clusters of computers.<br />http://www.cs.berkeley.edu/~ballard/cs267.sp11/hw0/results/htmls/Muzaffar.html<br />
  3. 3. CAP and ACID<br />CAP <br />정합성(Consistency). The client perceives that a set of operations has occurred all at once.<br />가용성(Availability). Every operation must terminate in an intended response.<br />단절내성(Partition tolerance). Operations will complete, even if individual components are unavailable.<br />ACID<br />원자성(Atomicity). All of the operations in the transaction will complete, or none will.<br />일관성(Consistency). The database will be in a consistent state when the transaction begins and ends.<br />고립성(Isolation). The transaction will behave as if it is the only operation being performed upon the database.<br />지속성(Durability). Upon completion of the transaction, the operation will not be reversed.<br />http://queue.acm.org/detail.cfm?id=1394128<br />
  4. 4. Google BigTable System<br />BigTable Master Server<br />Chubby Server<br />Tablet Server #2<br />Tablet Server #1<br />MapReduce Worker #2<br />MapReduce Worker #1<br />GFS #2<br />GFS #1<br />
  5. 5. Google BigTable Architecture<br />GAE Application<br />DataNucleus ORM<br />(JDO or JPA Implementation)<br />Low Level API<br />Datastore<br />Meta 0 tablet <br />Big Table<br />Meta 1 tablet <br />Meta 1 tablet <br />Meta 1 tablet <br />Tab 1<br />Tab 2<br />Tab 3<br />Tab 4<br />Tab n<br />GFS(Google File System)<br />
  6. 6. Google BigTable<br />Dynamic control over Data Layout and Format<br />Not support a full Relational Model<br />Dynamic control over serving data from memory or disk<br />Column oriented<br />The map is indexed by<br />A row key<br />A column name<br />A timestamp<br />BigTable<br />Key<br />TimeStamp / Version<br />Columns<br />Index<br />
  7. 7. LDAP(Lightweight Directory Access Protocol)<br />The Lightweight Directory Access Protocol (LDAP) is an application protocol for accessing and maintaining distributed directory information services over an Internet Protocol (IP) network.<br />Directory services may provide any organized set of records, often with a hierarchical structure, such as a corporate electronic mail directory. Similarly, a telephone directory is a list of subscribers with an address and a phone number.<br />
  8. 8. Google App Engine Product Roadmap<br />Features on Deck<br />App Engine out of Preview<br />SSL access on custom domains<br />Full-text search over Datastore<br />Support for Python 2.7<br />Support for running MapReduce jobs across App Engine datasets<br />Bulk Datastore Import and Export tool<br />Improved monitoring and alerting of application serving<br />Logging system improvements to remove limits on size and storage<br />Integration with Google Storage for Developer<br />
  9. 9. DataStore의 제약사항<br />Write가 느림<br />검색 가능한 Text 사이즈: 500 자<br />Entity 최대 사이즈: 1Mb<br />최대 Batch (save/delete) : 500 Entity<br />SQL을 사용할 수 없음<br />데이터베이스의 제약조건을 적용할 수 없음<br />Aggregation 함수를 지원하지 않음 (count, avg)<br />쿼리가 최대 조회 레코드: 2000<br />쿼리에서 Equal 연산 이외의 조건은 1개만 추가 가능<br />트랜잭션의 단위는 Entity Group에 한정 됨<br />
  10. 10. Datastore API and ORM API<br />
  11. 11. Datastore API(Low Level API)<br />모든 Entity는 Kind가 지정<br />Entity는 하나이상의 Property를 포함<br />특정 Kind에포함된 Entity는 동일한 Property를 갖지 않음<br />Entity의 Property는 이름은 동일하지만 데이터 유형이 다를수 있음<br />Schemaless의 특성<br />ID는 지정하지 않을 경우 자동 할당 됨<br />
  12. 12. Entity<br />App Engine Datastore에서 관리하는 객체는 Entity<br />Entity는 하나의 Key를 포함<br />모든 Entity 중 유일한 구분자<br />Key 의 구성<br />Path<br />Parent Entity의 key<br />Entity의 Kind<br />Entity에 할당된 이름<br />App이 할당한 값<br />Datastore가 지정한 numeric ID<br />지원 데이터 타임<br />integers, floating point values, strings, dates, binary data 등<br />각 엔티티는 하나이상의 Property를 포함<br />하나의 Property에 복수의 데이터 타입 저장 가능<br />하나의 Property에는 복수의 값이 저장 가능<br />복수의 값들의 데이터 유형은 다를 수 있음<br />
  13. 13. Entity Model<br />Settings<br />Kind : Table<br />Key : Primary Key<br />Entity Group : Partitioning<br />Property : column<br />
  14. 14. Entity Example(Create)<br />import java.util.Date;import com.google.appengine.api.datastore.DatastoreService;import com.google.appengine.api.datastore.DatastoreServiceFactory;import com.google.appengine.api.datastore.Entity;// ...DatastoreServicedatastore = DatastoreServiceFactory.getDatastoreService();Entity employee = new Entity("Employee");employee.setProperty("firstName", "Antonio");employee.setProperty("lastName", "Salieri");Date hireDate = new Date();employee.setProperty("hireDate", hireDate);employee.setProperty("attendedHrTraining", true);datastore.put(employee);<br />
  15. 15. Entity Example(Scheme Free)<br />import java.util.Date;import com.google.appengine.api.datastore.DatastoreService;import com.google.appengine.api.datastore.DatastoreServiceFactory;import com.google.appengine.api.datastore.Entity;// ...DatastoreServicedatastore = DatastoreServiceFactory.getDatastoreService();Entity employee = new Entity("Employee");employee.setProperty("firstName", “Sun-Jin");datastore.put(employee);<br />
  16. 16. Entity Group<br />Entity Group는 부모 Entity를 갖는 Entity의 Group<br />하나의 트랜잭션에서 여러 Entity를 조회, 생성, 수정, 삭제 할 수 가능하지만 그 범위는 단일 Entity Group에 국한<br />부모 Entity 를 가지 않은 Entity 를 Root Entity라고 함<br />Entity Group은 하나의 Root Entity와 Child Entity로 구성<br />하나의 EntityGroup은 동일한 분산 네트웍 서버에 저장됨 (Tablet 서버)<br />
  17. 17. Entity Group<br />Entity Group<br />Root Entity<br />(xxxx)<br />Child Entity<br />(xxxxyyyy)<br />Child Entity<br />(xxxxyyyyzzzz)<br />
  18. 18. Entity Example(Read)<br />import java.util.Date;import com.google.appengine.api.datastore.DatastoreService;import com.google.appengine.api.datastore.DatastoreServiceFactory;import com.google.appengine.api.datastore.Entity;// ...DatastoreServicedatastore = DatastoreServiceFactory.getDatastoreService();<br />Key rootKey = KeyFactory.createKey("Employee", “CEO");<br />Entity employee = datastore.get(rootKey);<br />
  19. 19. Entity Example(Update)<br />import java.util.Date;import com.google.appengine.api.datastore.DatastoreService;import com.google.appengine.api.datastore.DatastoreServiceFactory;import com.google.appengine.api.datastore.Entity;// ...DatastoreServicedatastore = DatastoreServiceFactory.getDatastoreService();<br />Key rootKey = KeyFactory.createKey("Employee", “CEO");<br />Entity employee = datastore.get(rootKey);<br />employee.setProperty("firstName", “Sun-Jin");employee.setProperty("lastName", “Jang");datastore.put(employee);<br />
  20. 20. Entity Example(Delete)<br />import java.util.Date;import com.google.appengine.api.datastore.DatastoreService;import com.google.appengine.api.datastore.DatastoreServiceFactory;import com.google.appengine.api.datastore.Entity;// ...DatastoreServicedatastore = DatastoreServiceFactory.getDatastoreService();<br />Key rootKey = KeyFactory.createKey("Employee", “CEO");<br />Entity employee = datastore.get(rootKey);datastore.delete(employee);<br />
  21. 21. Entity Example(Query)<br />// ...DatastoreServicedatastore = DatastoreServiceFactory.getDatastoreService();<br />Query q = new Query(”employee") <br />q.addFilter("lastName", Query.FilterOperator.EQUAL, lastNameParam); <br />q.addFilter("height", Query.FilterOperator.LESS_THAN, maxHeightParam); <br />PreparedQuerypq = datastore.prepare(q); <br />for (Entity result : pq.asIterable()) { <br /> String firstName = (String) result.getProperty("firstName"); <br /> String lastName = (String) result.getProperty("lastName"); <br /> Long height = (Long) result.getProperty("height"); <br />}<br />
  22. 22. Java ORM API<br />
  23. 23. DataNucleus<br />DataNucleus attempts to make the whole process of persisting data a transparent process. <br />The idea revolves around the developer having a series of Java classes that need persisting. <br />http://www.datanucleus.org/products/accessplatform/development_process.html<br />
  24. 24. JDO Model Define(Annotation)<br />import javax.jdo.annotations.PersistenceCapable;@PersistenceCapablepublic class Employee {    @PrimaryKey    @Persistent(valueStrategy= IdGeneratorStrategy.IDENTITY)    private Key key; <br />    @Persistent    private String firstName;    @Persistent    private String lastName;    @Persistent    private Date hireDate; }<br />
  25. 25. JDO Model Define(Annotation)<br />import javax.jdo.annotations.PersistenceCapable;<br />// …@PersistenceCapablepublic class Employee {    @PrimaryKey    @Persistent(valueStrategy= IdGeneratorStrategy.IDENTITY)    private Key key; <br />    @Persistent    private String firstName;    @Persistent    private String lastName;    @Persistent    private Date hireDate; }<br />
  26. 26. JDO Model(Create)<br />        PersistenceManager pm = PMF.get().getPersistenceManager();        Employee e = new Employee("Alfred", "Smith", new Date());        try {            pm.makePersistent(e);        } finally {            pm.close();        }<br />
  27. 27. JDO Model(Update)<br />public void updateEmployeeTitle(User user, String newTitle) {    PersistenceManager pm = PMF.get().getPersistenceManager();    try {        Employee e = pm.getObjectById(Employee.class, “Sun-Jin”);        if (titleChangeIsAuthorized(e, newTitle) {            e.setTitle(newTitle);        } else {            throw new UnauthorizedTitleChangeException(e, newTitle);        }    } finally {        pm.close();    }}<br />
  28. 28. JDO Model(Delete)<br />public void updateEmployeeTitle(User user, String newTitle) {    PersistenceManager pm = PMF.get().getPersistenceManager();    try {        Employee e = pm.getObjectById(Employee.class, “Sun-Jin”);<br />        pm.deletePersistent(e);     } finally {        pm.close();    }}<br />
  29. 29. JDO Model(Query)<br />import java.util.List;import javax.jdo.Query;// ...    Query query = pm.newQuery(Employee.class);    query.setFilter("lastName == lastNameParam");    query.setOrdering("hireDatedesc");    query.declareParameters("String lastNameParam");    try {        List<Employee> results = (List<Employee>) query.execute("Smith");        if (!results.isEmpty()) {            for (Employee e : results) {                // ...            }        } else {            // ... no results ...        }    } finally {        query.closeAll();    }<br />
  30. 30. Demo<br />
  31. 31. 감사합니다.<br />http://code.google.com/p/devmento-gae-seminar<br />

×