This document provides an overview and summary of Google App Engine and the Datastore: - It introduces the Google developer toolkit, creating a cloud project, enabling billing, and creating a first project. - It covers the basics of the Datastore including entities, kinds, properties, keys, queries, filters, sorting, and indexes. - It compares the Datastore to traditional SQL databases, highlighting the Datastore's scalability, consistency model, and limitations around inequality filters and sorting. - Code examples are provided for creating, getting, and querying entities using the low-level and query APIs, including composite filters, ancestors, and transactions.

1. GAE Developer - Day2
Simon @ MiCloud
2014Q1

2. Preview
● Google developer toolkit installed
● Google cloud project created
● Billing enabled
● Your first project

3. Today’s tool (Windows) - GitBash
http://git-scm.com/downloads
or
http://git-scm.com/download/win

4. Testing your code with curl...
curl http://localhost:8888/[servlet_path]
ex: curl http://localhost:8888/gaedemo

5. Today
● Datastore basic
● Datastore operation
● Restrictions
● Transaction

6. Compares - Googler’s develop way
Googler’s way
AppEngine
Traditional Web
applications
Web application
framework
AppEngine
(Java, Python, Go, PHP)
Java, Perl/CGI, PHP, Ruby,
Python...
Persistent storage
NoSQL
● Datastore
● Cloud SQL
RDBMS
● MySQL
● PostgreSQL
● SQL Server
● Oracle

7. Compares - Datastore vs RDBMS
Datastore RDBMS
Query language
flexibility
SQL-like query language
● Limited to simple
filter and sort
Full support of SQL
● Table JOIN
● Flexible filtering
● Subquery
Reliability and
Scalability
Highly scalable and
reliable
Hard to scale

8. Consistency
● Strong Consistency
○ Data is always consistent among all database
instances
○ Just after write operation
○ Even if crash in the middle of write operation
● Eventual Consistency
○ Takes time until all data becomes consistent after
write
○ ex: DNS

9. Datastore mapping to RDBMS
Datastore RDBMS
Category of object Kind Table
One entry/object Entity Row
Unique identifier of data entry Key Primary Key
Individual data Property Field

10. Property
Property
Property
Datastore data model
PostEntry User
Kinds
Key: blog-1234
user: simonsu@xxx.com
message: xxxxxx
date: 3/1/2014
Key: simonsu@xxx.com
email: simonsu@xxx.com
followees: [user2@xxx.com,
user3@xxx.com]
followers:
Key: user2@xxx.com
email: user2@xxx.com
followees:
followers: [simonsu@xxx.com]
Entities
Keys

11. Low level API - Create Entity
DatastoreService datastore =
DatastoreServiceFactory.getDatastoreService();
Entity employee = new Entity("Employee");
employee.setProperty("name", "Simon Su");
employee.setProperty("hireDate", new Date());
Key empKey = datastore.put(employee);

12. Low level API - Get Entity
// Use email as key when creating entity
Entity employee = new Entity("Employee", "work-id-D001");
datastore.put(employee);
// Later, use the key to retrieve the entity
Key userKey = KeyFactory.createKey("Employee", "work-id-D001");
Entity user = datastore.get(userKey);

13. Query API
Query query = new Query("Person");
Query.Filter nameFilter = new FilterPredicate(
"name", FilterOperator.EQUAL, "John");
query.setFilter(nameFilter);
PreparedQuery results = datastore.prepare(query);

14. Query API - Filter & Sort
Query q = new Query("Person");
Query.Filter filter1 = new FilterPredicate(...);
Query.Filter filter2 = new FilterPredicate(...);
Query.Filter comboFilter =
CompositeFilterOperator.and(filter1, filter2);
q.setFilter(comboFilter);
q.addSort("name");

15. Query query = new Query("Kind");
query.setAncestor(parentKey);
Query API - Ancestor

16. Config files - Index
● Manual configure:
WEB-INF/datastore-indexes.xml
● System generated:
WEB-INF/appengine-generated/datastore-indexes-auto.xml

17. Index Example
<?xml version="1.0" encoding="utf-8"?>
<datastore-indexes>
<datastore-index kind="Widget">
<property name="x" direction="asc" />
<property name="date" direction="asc" />
</datastore-index>
<datastore-index kind="Widget">
<property name="y" direction="asc" />
<property name="date" direction="asc" />
</datastore-index>
</datastore-indexes>
<?xml version="1.0" encoding="utf-8"?>
<datastore-indexes autoGenerate="true">
</datastore-indexes>

18. Datastore statics

19. Important difference to RDBMS
● Single kind(table) query only
● Missing property is not equal to Null/None

20. Restrictions - inequality
Query for:
first_name = Cathy
last_name > Able
last_name < Mooney
Query for:
first_name > Cathy
last_name > Able

21. Restrictions - sorting
Query for:
first_name = Cathy
last_name > Able
sort by last_name Query for:
last_name > Able
sort by first_name

22. Using Transaction
DatastoreService datastore = DatastoreServiceFactory.getDatastoreService()
Transaction txn = datastore.beginTransaction();
try {
Key ekey = KeyFactory.createKey("Employee", "Joe");
Entity employee = datastore.get(eKey);
/*... reading and writing on employee ...*
datastore.put(employee);
txn.commit();
} finally {
if (txn.isActive()) {
txn.rollback();
}
}

23. Q&A
END