Google App Engine With Java And Groovy

Java and Groovy on Google App Engine ---> Ken Kousen ken.kousen@kousenit.com

Google App Engine Google's "cloud computing" solution Run applications on Google's scalable infrastructure Pay based on resources used storage, bandwidth measured by gigabyte For free, up to 500 MB of storage and up to 5 million page views/month

Running Java on GAE Original GAE version: Python interpreter and standard libraries New, "Early Look" version: JVM available So Java works, as well as other languages that compile to the JVM i.e., Groovy, Scala, JRuby, ...

Java "Early Look" Need to register at http://appengine.google.com Originally expected 10K developers Demand was so high, increased to 25K Now open to ALL Download SDK Download Eclipse plugin (optional) Can run locally or to myapp.appspot.com If Google Apps user, can map local http address (example later)

Sandbox Environment Limited access to underlying OS Manage app through application console No writing to file system Can read files uploaded by app Datastore available (discussed below) Services available (described next)

Welcome to 1999 Let's go back to those thrilling days of yesteryear... Ricky Martin, Livin' La Vida Loca Jar-Jar Binks Bill and Monica

Also in 1999 On Dec. 17th, Sun released Servlet 2.2 specification Established structure of a WAR file http://www.youtube.com/watch?v=EVESMxs4rbA Ever since then, you could deploy a WAR file to an application server Unfortunately, GAE doesn't know from WAR files

WAR, huh, ... what is it good for? Actually, that's not quite true GAE does know about war structure, just not war files gae_app src java code war normal war stuff, including classes and lib dirs WEB-INF web.xml, etc

GAE Limitations GAE running on Java 6, but with some limitations Once a request is sent to the client, no further processing can be done Request will be terminated if longer than 30 sec to complete, throwing an exception Also, No sockets No threads or timers No JNI System.gc(), System.exit(...), etc, do nothing Security issues, esp. with class loaders

GAE War Quirks Sample web.xml file Has DTD based on Web App 2.3 But also includes xmlns and version attributes (version = 2.5, no less) It's confused

Scalability About 30 active dynamic requests simultaneously Average server-side req processing time is 75 ms --> about 400 requests/sec without additional latency (note static files not affected by this limit)

GAE Services URL Fetch URL wrapper that uses Google infrastructure to retrieve resources Mail Send email Memcache "high performance in-memory key-value cache" Image Manipulation Resize, copy, rotate, flip JPEG and PNG images User Detect if user is logged in and is administrator

Welcome to the 2300s Maybe by then we'll have moved beyond relational DBs (but I doubt it...)

Datastore Distributed data storage transactional Filtering and sorting by property value type NOT a traditional relational database Google uses the term "schemaless" Uses optimistic concurrency control with retries

GAE Persistence GAE object datastore based on BigTable BigTable is a massively scalable, distributed storage system used by Google for lots of things For Java, API uses DataNucleus bytecode enhancer (No, I'd never heard of it either...)

Software Development Kit App Engine SDK Includes web server (jetty) Emulates all the GAE services SDK includes an upload tool to deploy app to GAE Command line tools included

Project Development Supports Java 5 and Java 6 (Standard Edition) GAE deployment is Java 6 Google Plugin for Eclipse version for 3.3 (Europa) http://dl.google.com/eclipse/plugin/3.3 version for 3.4 (Ganymede) http://dl.google.com/eclipse/plugin/3.4

Project Development SDK comes as a zip file Includes sample applications Some use GWT: Google Web Toolkit How to do Ajax without doing Ajax by doing it in Java GWT also comes with the Eclipse plugin

Demo: Recommended Books Warning: User Interface Poisoning Hazard Ahead

Entity Class @PersistenceCapable(identityType = IdentityType.APPLICATION) public class Book { @PrimaryKey @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) private Long id; @Persistent private String asin; @Persistent private String recommendation;

Entity Class // Other attributes populated by XML response (not persistent!) private String title; private String author; // multiple are separated by commas private String formattedPrice; private String mediumImageURL; private String detailPageURL; // contructors, getter and setter methods // equals, hashCode, toString overrides as desired }

Entity Class Book represents book at Amazon.com Amazon provides Product Advertising API formerly Amazon Associates Service http://docs.amazonwebservices.com/AWSECommerceService RESTful (sort of) web service Append parameters to request URL = base?Service=AWSECommerceService &Operation=ItemLookup &ASIN=...isbn... // ... etc ...

Amazon Web Service Input URL --> Output XML

Amazon Web Service As of May, 2009 Renamed Product Advertising API (ugh) As of August, 2009 All REST requests must be digitally signed Sample code given in Java Uses javax.crypto classes and Apache Commons Codec

DAO Interface public interface BookDAO { Book findById(Long id); Book findByAsin(String asin); Set<Book> findAllBooks(); Long addBook(Book b); boolean removeBook(Long id); }

DAO Implementation public class JdoBookDAO implements BookDAO { private PersistenceManagerFactory pmf = PMF.get(); @Override public Long addBook(Book b) { Book book = findByAsin(b.getAsin()); if (book != null) { return book.getId(); } PersistenceManager pm = pmf.getPersistenceManager();

DAO Implementation try { pm.currentTransaction().begin(); pm.makePersistent(b); pm.currentTransaction().commit(); } finally { if (pm.currentTransaction().isActive()) { pm.currentTransaction().commit(); } pm.close(); } return b.getId(); }

DAO Implementation public Set<Book> findAllBooks() { Set<Book> results = new HashSet<Book>(); PersistenceManager pm = pmf.getPersistenceManager(); Query q = pm.newQuery(Book.class); q.setOrdering("asin desc"); try { List<Book> books = (List<Book>) q.execute(); for (Book b : books) { results.add(b); } } finally { pm.close(); } return results; }

Use Cases Servlet implementations of use cases List all books Add a new book Remove a book Each is mapped in web.xml file Each goes through a service class to fill in book details Each forwards to JSP

List All Books public void doGet( HttpServletRequest req, HttpServletResponse resp) throws IOException, ServletException { AmazonBookService service = new AmazonBookService(); Set<Book> books = (Set<Book>) service.getBooks(); req.setAttribute("books", books); req.getRequestDispatcher("books.jsp").forward(req, resp); } What is that AmazonBookService? It converts XML to Book instances Enter Groovy...

Amazon Book Service Class implemented in Groovy: class AmazonBookService { def baseUrl = 'http://ecs.amazonaws.com/onca/xml' def params = ['Service':'AWSECommerceService', 'Operation':'ItemLookup', 'AWSAccessKeyId':'... long ugly key ...', 'AssociateTag':'kouitinc-20', 'ResponseGroup':'Medium'] BookDAO dao = DAOFactory.instance.bookDAO

Amazon Book Service def getBooks() { def books = dao.findAllBooks() books.each { book -> book = fillInBookDetails(book) } return books }

Amazon Book Service def fillInBookDetails(Book book) { def queryString = params.collect { k,v -> "$k=$v" }.join('&') def url = "${baseUrl}?${queryString}&ItemId=${book.asin}" def response = new XmlSlurper().parse(url) def item = response.Items.Item book.title = item.ItemAttributes.Title book.author = item.ItemAttributes.Author.collect { it }.join(', ') book.formattedPrice = item.ItemAttributes.ListPrice.FormattedPrice book.mediumImageURL = item.MediumImage.URL book.detailPageURL = item.DetailPageURL return book }

One annoying thing For Groovy classes, GAE needs all compiled classes in war/WEB-INF/classes/ ... (normal for WAR files) But couldn't get that and Groovy Eclipse plugin to keep them in classpath at same time Had to manually copy compiled classes to proper dir Grrr... but at least it worked (probably Eclipse issue)

JSP books.jsp displays all the books in the request <%@ taglib prefix="c" uri="http://java.sun.com/jsp/jstl/core" %> <%@ page isELIgnored="false" %> ... <c:forEach items="${books}" var="book"> <tr> <td><a href="${book.detailPageURL}"> <img alt="${book.title} picture" src="${book.mediumImageURL}"> </a></td>

JSP <td> <ul> <li>ISBN: ${book.asin}</li> <li style="font-weight: bold;"> <a href="${book.detailPageURL}">${book.title}</a></li> <li style="font-style: italic;">${book.author}</li> <li>${book.formattedPrice}</li> </ul> </td> <td>${book.recommendation}</td>

Resulting Display

Admin Capabilities Use GAE UserService class UserService us = UserServiceFactory.getUserService(); if (us.isUserLoggedIn()) { <p><a href="<%= us.createLogoutURL("/listbooks") %>">Logout</a></p> } else { <p><a href="<%= us.createLoginURL("/listbooks") %>">Login</a></p> } Yeah, ugly scriptlet code, but so be it

Admin Capabilities If logged in user and user is registered admin, Can add new books Can delete existing books Otherwise links aren't available Probably best to make a custom JSP tag out of this Or use a real security solution JSecurity, Spring Security, etc. Easy enough for a demo, though

Admin Capabilities Can set billing limit, too Mine is $2/day, so don't bother writing those scripts ... :)

Other Services URL fetch service Just opening a URL means we're using it already Has more capabilities for alternate request types Mail service Can only send, not receive Limits on size, frequency, but seem reasonable Memcache API implements JCache interface

Memcache Caching capability based on javax.cache package JCache: JSR 107 (under development) Cache cache = CacheManager.instance. cacheFactory.createCache(props) ... if (cache.get(book.asin)) { book = cache.get(book.asin) } else { book = fillInBookDetails(book) cache.put(book.asin,book) }

Scheduling CRON jobs supported in cron.xml <?xml version="1.0" encoding="UTF-8"?> <cronentries> <cron> <url>/listbooks</url> <description>Repopulate the cache every day at 5am</description> <schedule>every day 05:00</schedule> </cron> </cronentries>

Admin Console Administrative console located at http://appengine.google.com for non-Google Apps or http://appengine.google.com/a/domain.com if the domain is using Google Apps Current limit of 10 apps per user Can check activity, quotas, etc.

GAE vs. Amazon EC2 Amazon EC2 installs images You need to configure server, JVM, etc. GAE is a sandbox with provided services Detailed admin console available Java persistence framework built in

The Quest for the Holy Grails Grails framework Spring + Hibernate + Groovy Modular -- built on plugins Now there's a GAE plugin Uninstall hibernate, can then use JDO Some quirks, but it's early yet Will eventually be the easiest way to build a GAE app

Grails GAE Plugin 1. Create Grails app with matching name (Workaround available if not possible) 2. Uninstall Hibernate plugin 3. Install app-engine plugin (set APPENGINE_HOME) 4. Set version to 1 (or some other int) GAE doesn't like decimal version numbers 5. Deploy using package command

Opinionated Conclusions: Bad Stuff Must be able to deploy a WAR file Hard to take GAE seriously otherwise Expect this to be easy in release version Must support the full Java EE standard Sun (Oracle?) is very upset about this... Also hard to port existing apps otherwise Should support other persistence options Especially Hibernate Spring would be nice, too (might be asking a lot) But non-relational issues will persist (sorry)

Opinionated Conclusions: Good Stuff Easy to use cloud framework Great entry-level tool Scales like Google Free! You get a fully-functioning system for free Did I mention that it's free? (okay, within limits, but it's free for a while)

Links Google App Engine home http://code.google.com/appengine Source code git://github.com/kousen/recommended-books.git Deployed App URL http://recommended-books.kousenit.com GAE Plugin for Grails http://grails.org/plugin/app-engine

Google App Engine With Java And Groovy

0 comments

Notes on slide 1

2 Favorites