Google App
Engine for Java
Java
shin1ogawa@

...shin1ogawa
Google, Java, Eclipse, AppEngine,
Wicket, Maven, Hudson, ...
Java
• OSS
• [SF.jp]Jiemamy...
• [SF.jp]gae-j-samples...GAE/J GAE/J/Maven
• [SF.jp]asclipse...Amateras AIR GEAR AS3
• java-ja, wicket-ja, BOTTUKU...
(http://www.topgate.co.jp)
GoogleAppEngine GoogleApps

• Platform
•
•
•
•
• Datastore
•
•
•

Platform
•
• Disk
•
• AppEngine
• Google Apps Apps

Platform
• Google
• Google
•

Platform
•
•
JVM …
• Platform
• AWS-EC2 ”Amazon Cloud Watch( )” (
) ”Auto Scaling” ”Elastic Load
Balancing”
•
Relational Database
• KVS SimpleDB( )

Platform
• WebContainer
• static Container
•
• DatastoreService
• MailService
• URLFetchService
• ImageService...
• cron

Platform
•
/ , CPU
, DISK
, API ,
...

Platform
“ ”
• 1
1
• ”release”
”snapshot” …
• http://myappid.appspot.com/ ...
• http://snapthot.latest.myappid.appspot.com/ ...
•
• datastore
• python java

Platform
Web
• JVM JDK1.6 Hotspot Client VM
• Jetty
• Jetty JEE App Engine
• Jetty Comet (30 stream )
• = JVM
• static
• https off

Platform
Web /
• appengine-web.xml sessions enabled=true
• Datastore
• _ah_session Kind
• Application
SessionCleanupServlet
• Servlet web.xml

Platform
•
• Disk
•
• API
• Java
: http://code.google.com/intl/ja/appengine/docs/quotas.html

Platform
• Disk
• 3000
• Datastore 1GB
• 250
• ...
appengine

Platform
•
30
• / 10MB
• 2000
• API 7000
• ... , /

Platform
API
• Datastore 1MB
• MemcacheService Key
1MB
• URLFetchService 10
• Datasotre 1000
• JDO 1000 ...
• ...

Platform
Java
•
• Thread
• Socket
• JDK class
• white-list class -> http://
code.google.com/intl/ja/appengine/docs/java/jrewhitelist.html
• GC
• ...

1
• : Google
•
• Best practices for writing scalable applications
• Google Group
• Google App Engine for Java
• Google App Engine
• Google-App-Engine-Japan
• Google Code
• googleappengine
• datanucleus-appengine

2
• 1, 3 IRC Office Hour
• irc://irc.freenode.net/#appengine
• Google Calendar: delevepoer-calendar@google.com
• Google App Engine Blog
• AppEngine Cookbook
• Under the covers of the Google App Engine
Datastore
• tips
• shin1ogawa
• ( maven plugin )
• Google App Engine

3
• https://sites.google.com/a/topgate.co.jp/
: AppEngineSDK
Hackathon Googler

• Google Plugins for Eclipse(SDK )
• Eclipse
• AppEngine
• Ant(SDK )
• dev_appserver
• app_cfg AppEngine
• Python SDK(JavaSDK )
• index vacuum(Index Error )
• bulkloader(Datastore Export/Import)
• NetBeans IntelliJ IDEA plugin …

• war/WEB-INF/
• appengine-web.xml
...appengine
• datastore-indexes.xml
... index
• cron.xml ...cron
• appegnine-generated/
• datastore-indexes-auto.xml
... index
• local_db.bin ...Datastore

•
• SDK
local_db.bin(
) SDK
• local_db.bin AppID, versionId
local_db.bin k
• Maven
• mvnsearch appengine
• shin1ogawa Maven SDK plugin

•
• java.util.logging
• war/WEB-INF/appengine-web.xml logging.properties
• RemoteApiServlet SDK

appengine-web.xml
• on/off
• https on/off
• static
•
• logging.properties
• web.xml

Datastore
• Kind...RDB
• Entity...RDB
• Property...RDB
• Index...RDB
• Key...RDB Entity
• Filter...SQL Where
• equality filter... ”=” Filter
• inequality filter...”>”, “>=”, “<=”, “<“
• Entity Group... Entity

Datastore
• BigTable
• ...
• RDB
• ( = key )
• [0]id=1, name=”Hoge”
• [1]id=2, height=170.5, weight=65.5
• [2]id=3, tag=[“Apple”, “Java”]
•

Datastore
•
• Entity 80ms/1
• RDB Index
• Entity
•
• Query
( )
•

Datastore
•
• Entity 80ms/1
• RDB Index
• Entity
•
• Query
( )
•

Datastore
Entity Group
• Entity
• RDB Relation
• Java
• )
• Kind:Entity:id=1
• Kind:Entity:id=2
• Kind:Entity:id=3
• Kind:Entity:id=4
• Kind:Entity:id=5

Datastore
Key
• Entity Property
• Long (id) String (name) Key Id
• EntityGroup Entity Key
Entity
• Entity: Kind=Parent, id=1
• Entity: Kind=Child, name=“hoge@fuga.com”
• Entity Key
• Parent(1)/Child(“hoge@fuga.com”)

Datastore
Property
• Entity
• java.lang
• String 500
• Text
• 500 String
• Blob
•
• List Property
• List Set

Datastore
Index
• Property
• Kind Index
• Kind Key Index
• Single Property Index
• Property Index
• Composite Index
• Property Index
• Text Blob Index
(index explosion) Composite
Property Index

Datastore
Transaction
• ACID
•
• JDO
•
• Transaction EntityGroup
Entity
•

Datastore
Query
• ’<‘, ‘<=’, ‘==’, ‘>=’, ‘>’
• Like (startsWith())
• And
• Java ‘IN’, ‘!=’
• Property
inequality filter Property
• max() min()
• Join

Datastore
Query
• Index
→ Index
• Kind Index, Single Property Index
Datastore
• Composite Index datastore-indexes.xml

Datastore
NG
•
•
ID
• 1,2,3,1001,1002...
•
• 1000
• JDO 1000
Google tips
” ”

Datastore
•
•
•
• Kind
• EntityGroup (
Transaction )
• Transaction
•

Datastore
/
=
•
• ID, ShardingNo
• “ID, ShardingNo)”
•

Datastore
/
1 2 ...
• offset/limit
• +1
Filter
• Filter
k
•
• property property

Datastore
• JDO, JPA, Low-level API 3
• JPA
• JDO ORM
• Datastore
• Low-level API
• JDO Low-level API
JEE ...
Low-level API

Datastore
JDO/Entity
@PersistenceCapable(identityType = IdentityType.APPLICATION
, detachable = "true")
@Version(strategy = VersionStrategy.VERSION_NUMBER)
public class Entity {
@Persistent
@PrimaryKey(valueStrategy = IdGeneratorStrategy.IDENTITY)
Key id;
@Persistent String value;
@Persistent(defaultFetchGroup = “true”) Text text;
}
Fetch Fetch
@Version JDO (OPT_VERSION property
)

Datastore
JDO/Owned Relationship
public class Parent {
...
@Persistent(defaultFetchGroup = “true”)
List<Child> children;
}
public class Child {
@Persistent
@PrimaryKey(valueStrategy = IdGeneratorStrategy.IDENTITY)
Key id;
...
}
Child Key ( encoded-pk String )

Datastore
JDO/Owned Relationship(Key )
public class Parent {
...
}
public class Child {
...
@Persistent
@Extension(vendorName="datanucleus",
key="gae.parent-pk", value="true")
Key parentKey;
...
}
Entity Child
parentKey Entity Key EntityGroup
Ancestor Key

Datastore
JDO/Owned Relationship( )
public class Parent {
...
@Persistent(defaultFetchGroup = “true”)
List<Child> children;
}
public class Child {
...
@Persistent(mappedBy = “children”)
Parent parent;
...
}
mappedBy property

Datastore
JDO/Unowned Relationship
@PersistenceCapable(identityType = IdentityType.APPLICATION)
public class Entity {
@Persistent
@PrimaryKey(valueStrategy = IdGeneratorStrategy.IDENTITY)
Key id;
@Persistent
String value;
@Persistent
Sey<Key> otherEntityRefs;
}

Datastore
JDO/
PersistentManager pm = ... // PersistentManager ;
Parent parent = new Parent();
parent.set......
List<Child> children = new ArrayList<Child>();
children.add(child1);
parent.setChildren(children);
try {
Transaction transaction = pm.currentTransaction();
transaction.begin();
pm.makePersistent(parent);
transaction.commit();
} finally {
if (transaction.isActive())
transaction.rollback();
}

Datastore
JDO/
PersistentManager pm = ... // PersistentManager ;
Query query = pm.newQuery(Entity.class);
query.setFilter(“name == param”);
query.declareParameters(“java.lang.String param”);
query.setOrdering(“name asc, key desc”);
@SuppressWarnings("unchecked")
List<Entity> list = (List<Entity>) query.execute(“hoge”);
...
Key key = KeyFactory.createKey(“Entity”, “keyName”);
Entity entity = pm.getObjectById(Entity.class, key);
Key key = entity.getOtherEntityKey();
OtherEntity otherEntity = pm.getObjectById(OtherEntity.class, key);

Datastore
JDO/
PersistentManager pm = ... // PersistentManager ;
Query query = pm.newQuery(Entity.class);
query.setFilter(“name == param”);
query.declareParameters(“java.lang.String param”);
query.setResult(“count(this)”);
@SuppressWarnings("unchecked")
Integer count = (Integer) query.execute(“hoge”);
...
query.setResult(“key”);
List<Entity> list = (List<Entity>) query.execute(“hoge”);
”count(this)”
”key” Key

Datastore
JDO/
•
• transient, hollow, persistent/detached
• JDOHelper#getObjectState()
• PersistenceManager
InstanceLifecycleListener
• PersistenceManager#execute() List
serialize
• PersistenceManager#retrieve[All]() Property
fetch

Datastore
Low-level API/
Entity entity = new Entity(“Kind”); // Low-level API Entity
entity.setProperty(“name”, “hoge”);
entity.setProperty(“height”, 170.5);
DatastoreService service =
DatastoreServiceFactory.getDatastoreService();
service.put(entity);
Entity[] entities = ....
service.put(entities);
Entity

Datastore
Low-level API/
Query query = new Query(“Entity”);
query
.addFilter(“name”, FilterOperator.GREATER_THAN_OR_EQUAL, “hoge”)
.addSort("name", SortDirection.ASCENDING)
.addSort("__key__", SortDirection.ASCENDING);
DatastoreService service =
DatastoreServiceFactory.getDatastoreService();
List<Entity> entities = service.prepare(query).asList(
FetchOptions.Builder.withOffset(0));
Key key = KeyFactory.createKey(“Entity”, “keyName”);
Entity entity = service.get(key);
property ”__key__”

Datastore
Low-level API/
Query query = new Query(“Entity”);
query
.addFilter(“name”, FilterOperator.GREATER_THAN_OR_EQUAL, “hoge”)
.addSort("name", SortDirection.ASCENDING)
.addSort("__key__", SortDirection.ASCENDING).setKeysOnly();
DatastoreService service =
DatastoreServiceFactory.getDatastoreService();
int count = service.prepare(query).countEntities();
Entity entity = service. prepare(query).asSingleResult();
setKeysOnly()
asSingleResult() 0 null 1 Entity 2 Exception

MemcacheService
•
• ( )
•
• cache
• JCache(javax.cache), Low-level API 2
MemcacheService service =
MemcacheServiceFactory.getMemcacheService();
service.put("key", list);
Object cachedObject = service.get("key");
service.delete("key");

URLFetchService
• URL
• API java.net.URLConnection Low-level
API 2
• commons-httpclient
URLFetchService service =
URLFetchServiceFactory.getURLFetchService();
HTTPResponse response =
service.fetch(new HTTPRequest(new URL(“”)));
if (response.getResponseCode() != 200)
throw new RuntimeException();
byte[] bytes = response.getContent();

cron
•
• URL Filter/Servlet
•
•
<?xml version="1.0" encoding="UTF-8"?>
<cronentries>
<cron>
<url>/cron/fetch?name=jiemamy-sf</url>
<description>fetch the jiemamy SF.jp timeline.</description>
<schedule>every 1 minutes</schedule>
</cron>
</cronentries>

• MailService
• JavaMail(javax.mail), Low-level API 2
•
• ImageService
• low-level API
• ...
•
• UserService
• Google Account
• Google
appengine-java ML

ApiProxy
• com.google.apphosting.api.ApiProxy
• Proxy
• com.google.apphosting.api.ApiProxy.Delegate
•
• com.google.appengine.tools.development.ApiProxyL
ocal
• Delegate
ApiProxyLocal

Delegate#makeSyncCall()
• public byte[] makeSyncCall(
Environment environment,
String packageName, // service
String methodName, // service
byte[] requestBytes) throws ApiProxyException
byte[] ProtocolBuffer byte
SDK RemoteApiServlet
http makeSyncCall() Servlet
[ | YO]

UI
• UI …
…
•
Google App Engine ” ” ”
”

shin1ogawa@