S03 hybrid app_and_gae_datastore_v1.0

Hybrid Appand GAE DataStore
대표이사 장선진
@jangsunjin
Mobile Phone: (+82) 010-4585-1770
E-Mail: jangsunjin@softwareinlife.com

MapReduce
MapReduce is a patented software framework introduced by Google in 2004 to support distributed computing on large data sets on clusters of computers.
http://www.cs.berkeley.edu/~ballard/cs267.sp11/hw0/results/htmls/Muzaffar.html

CAP and ACID
CAP
정합성(Consistency). The client perceives that a set of operations has occurred all at once.
가용성(Availability). Every operation must terminate in an intended response.
단절내성(Partition tolerance). Operations will complete, even if individual components are unavailable.
ACID
원자성(Atomicity). All of the operations in the transaction will complete, or none will.
일관성(Consistency). The database will be in a consistent state when the transaction begins and ends.
고립성(Isolation). The transaction will behave as if it is the only operation being performed upon the database.
지속성(Durability). Upon completion of the transaction, the operation will not be reversed.
http://queue.acm.org/detail.cfm?id=1394128

Google BigTable System
BigTable Master Server
Chubby Server
Tablet Server #2
Tablet Server #1
MapReduce Worker #2
MapReduce Worker #1
GFS #2
GFS #1

Google BigTable Architecture
GAE Application
DataNucleus ORM
(JDO or JPA Implementation)
Low Level API
Datastore
Meta 0 tablet
Big Table
Meta 1 tablet
Meta 1 tablet
Meta 1 tablet
Tab 1
Tab 2
Tab 3
Tab 4
Tab n
GFS(Google File System)

Google BigTable
Dynamic control over Data Layout and Format
Not support a full Relational Model
Dynamic control over serving data from memory or disk
Column oriented
The map is indexed by
A row key
A column name
A timestamp
BigTable
Key
TimeStamp / Version
Columns
Index

LDAP(Lightweight Directory Access Protocol)
The Lightweight Directory Access Protocol (LDAP) is an application protocol for accessing and maintaining distributed directory information services over an Internet Protocol (IP) network.
Directory services may provide any organized set of records, often with a hierarchical structure, such as a corporate electronic mail directory. Similarly, a telephone directory is a list of subscribers with an address and a phone number.

Google App Engine Product Roadmap
Features on Deck
App Engine out of Preview
SSL access on custom domains
Full-text search over Datastore
Support for Python 2.7
Support for running MapReduce jobs across App Engine datasets
Bulk Datastore Import and Export tool
Improved monitoring and alerting of application serving
Logging system improvements to remove limits on size and storage
Integration with Google Storage for Developer

DataStore의 제약사항
Write가 느림
검색 가능한 Text 사이즈: 500 자
Entity 최대 사이즈: 1Mb
최대 Batch (save/delete) : 500 Entity
SQL을 사용할 수 없음
데이터베이스의 제약조건을 적용할 수 없음
Aggregation 함수를 지원하지 않음 (count, avg)
쿼리가 최대 조회 레코드: 2000
쿼리에서 Equal 연산 이외의 조건은 1개만 추가 가능
트랜잭션의 단위는 Entity Group에 한정 됨

Datastore API and ORM API

Datastore API(Low Level API)
모든 Entity는 Kind가 지정
Entity는 하나이상의 Property를 포함
특정 Kind에포함된 Entity는 동일한 Property를 갖지 않음
Entity의 Property는 이름은 동일하지만 데이터 유형이 다를수 있음
Schemaless의 특성
ID는 지정하지 않을 경우 자동 할당 됨

Entity
App Engine Datastore에서 관리하는 객체는 Entity
Entity는 하나의 Key를 포함
모든 Entity 중 유일한 구분자
Key 의 구성
Path
Parent Entity의 key
Entity의 Kind
Entity에 할당된 이름
App이 할당한 값
Datastore가 지정한 numeric ID
지원 데이터 타임
integers, floating point values, strings, dates, binary data 등
각 엔티티는 하나이상의 Property를 포함
하나의 Property에 복수의 데이터 타입 저장 가능
하나의 Property에는 복수의 값이 저장 가능
복수의 값들의 데이터 유형은 다를 수 있음

Entity Model
Settings
Kind : Table
Key : Primary Key
Entity Group : Partitioning
Property : column

Entity Example(Create)
import java.util.Date;import com.google.appengine.api.datastore.DatastoreService;import com.google.appengine.api.datastore.DatastoreServiceFactory;import com.google.appengine.api.datastore.Entity;// ...DatastoreServicedatastore = DatastoreServiceFactory.getDatastoreService();Entity employee = new Entity("Employee");employee.setProperty("firstName", "Antonio");employee.setProperty("lastName", "Salieri");Date hireDate = new Date();employee.setProperty("hireDate", hireDate);employee.setProperty("attendedHrTraining", true);datastore.put(employee);

Entity Example(Scheme Free)
import java.util.Date;import com.google.appengine.api.datastore.DatastoreService;import com.google.appengine.api.datastore.DatastoreServiceFactory;import com.google.appengine.api.datastore.Entity;// ...DatastoreServicedatastore = DatastoreServiceFactory.getDatastoreService();Entity employee = new Entity("Employee");employee.setProperty("firstName", “Sun-Jin");datastore.put(employee);

Entity Group
Entity Group는 부모 Entity를 갖는 Entity의 Group
하나의 트랜잭션에서 여러 Entity를 조회, 생성, 수정, 삭제 할 수 가능하지만 그 범위는 단일 Entity Group에 국한
부모 Entity 를 가지 않은 Entity 를 Root Entity라고 함
Entity Group은 하나의 Root Entity와 Child Entity로 구성
하나의 EntityGroup은 동일한 분산 네트웍 서버에 저장됨 (Tablet 서버)

Entity Group
Entity Group
Root Entity
(xxxx)
Child Entity
(xxxxyyyy)
Child Entity
(xxxxyyyyzzzz)

Entity Example(Read)
import java.util.Date;import com.google.appengine.api.datastore.DatastoreService;import com.google.appengine.api.datastore.DatastoreServiceFactory;import com.google.appengine.api.datastore.Entity;// ...DatastoreServicedatastore = DatastoreServiceFactory.getDatastoreService();
Key rootKey = KeyFactory.createKey("Employee", “CEO");
Entity employee = datastore.get(rootKey);

Entity Example(Update)
Entity employee = datastore.get(rootKey);
employee.setProperty("firstName", “Sun-Jin");employee.setProperty("lastName", “Jang");datastore.put(employee);

Entity Example(Delete)
Entity employee = datastore.get(rootKey);datastore.delete(employee);

Entity Example(Query)
// ...DatastoreServicedatastore = DatastoreServiceFactory.getDatastoreService();
Query q = new Query(”employee")
q.addFilter("lastName", Query.FilterOperator.EQUAL, lastNameParam);
q.addFilter("height", Query.FilterOperator.LESS_THAN, maxHeightParam);
PreparedQuerypq = datastore.prepare(q);
for (Entity result : pq.asIterable()) {
String firstName = (String) result.getProperty("firstName");
String lastName = (String) result.getProperty("lastName");
Long height = (Long) result.getProperty("height");
}

Java ORM API

DataNucleus
DataNucleus attempts to make the whole process of persisting data a transparent process.
The idea revolves around the developer having a series of Java classes that need persisting.
http://www.datanucleus.org/products/accessplatform/development_process.html

JDO Model Define(Annotation)
import javax.jdo.annotations.PersistenceCapable;@PersistenceCapablepublic class Employee { @PrimaryKey @Persistent(valueStrategy= IdGeneratorStrategy.IDENTITY) private Key key;
@Persistent private String firstName; @Persistent private String lastName; @Persistent private Date hireDate; }

JDO Model Define(Annotation)
import javax.jdo.annotations.PersistenceCapable;
// …@PersistenceCapablepublic class Employee { @PrimaryKey @Persistent(valueStrategy= IdGeneratorStrategy.IDENTITY) private Key key;
@Persistent private String firstName; @Persistent private String lastName; @Persistent private Date hireDate; }

JDO Model(Create)
PersistenceManager pm = PMF.get().getPersistenceManager(); Employee e = new Employee("Alfred", "Smith", new Date()); try { pm.makePersistent(e); } finally { pm.close(); }

JDO Model(Update)
public void updateEmployeeTitle(User user, String newTitle) { PersistenceManager pm = PMF.get().getPersistenceManager(); try { Employee e = pm.getObjectById(Employee.class, “Sun-Jin”); if (titleChangeIsAuthorized(e, newTitle) { e.setTitle(newTitle); } else { throw new UnauthorizedTitleChangeException(e, newTitle); } } finally { pm.close(); }}

JDO Model(Delete)
public void updateEmployeeTitle(User user, String newTitle) { PersistenceManager pm = PMF.get().getPersistenceManager(); try { Employee e = pm.getObjectById(Employee.class, “Sun-Jin”);
pm.deletePersistent(e); } finally { pm.close(); }}

JDO Model(Query)
import java.util.List;import javax.jdo.Query;// ... Query query = pm.newQuery(Employee.class); query.setFilter("lastName == lastNameParam"); query.setOrdering("hireDatedesc"); query.declareParameters("String lastNameParam"); try { List<Employee> results = (List<Employee>) query.execute("Smith"); if (!results.isEmpty()) { for (Employee e : results) { // ... } } else { // ... no results ... } } finally { query.closeAll(); }

감사합니다.
http://code.google.com/p/devmento-gae-seminar

http://blog.java2game.com	1866
http://jangsunjin.tistory.com	51
http://www.hanrss.com	13
http://www.techgig.com	9
http://translate.googleusercontent.com	1
http://blog.java2game.com&_=1329407256062 HTTP	1

S03 hybrid app_and_gae_datastore_v1.0

by Sun-Jin Jang on Jul 24, 2011

Accessibility

Categories

Upload Details

Usage Rights

6 Embeds 1,941

Statistics

S03 hybrid app_and_gae_datastore_v1.0 — Presentation Transcript