Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Castles in the Cloud: Developing with Google App Engine


Published on

App Engine offers developers the opportunity to deploy systems on Google's robust and scalable server-farms. App Engine provides a higher-level platform than Amazon Web Services,with automated scaling and true pay-per-use billing.

The poster-child of App Engine, "BuddyPoke", has gained over thirty million users.

With App Engine, Google has released the first public API to BigTable, its planetary datastore, which performs successfully at petabyte scale across diverse applications from search to finance to Google Earth.

This presentation will cover App Engine's features and limitations, and how to exploit this new and evolving platform.

Published in: Technology, News & Politics
  • Thanks - nice introduction to GAE (even though this may be a bit dated as of today)
    Are you sure you want to  Yes  No
    Your message goes here

Castles in the Cloud: Developing with Google App Engine

  1. 1. Castles in the Cloud: Developing with Google App Engine Kevin Noonan Calbane Ltd. IT@Cork, 20th April, 2009
  2. 2. Castles in the Cloud Introduction
  3. 3. A working definition of Cloudware • Outsourced Virtualization o Virtual server running on outsourced server farm
  4. 4. The Cloud Computing Pyramid •Michael Sheehan of GoGrid devised this classification
  5. 5. Main Players
  6. 6. Other Players • Microsoft Windows Azure o Including .NET Live Services • from Salesforce • SUN (recently announced) • Mosso • GoGrid • FlexiScale • Hosting365 • ...
  7. 7. What's Cloudware good for? (I)‫‏‬ • Guerilla development o Pay-per-use model is ideal for startups o Also good for guerillas inside the enterprise...
  8. 8. What's Cloudware good for? (II)‫‏‬ • Scaling up o Clone the virtual server o pay for extra resources (bandwidth, storage, RAM, etc.)‫‏‬
  9. 9. What's Cloudware good for? (III)‫‏‬ • In general, outsourcing IT infrastructure o pay someone else to run your servers, network, database...
  10. 10. Castles in the Cloud Who's using Cloudware & why not?
  11. 11. Startups & Large Firms on AWS • Startups o Twitter o SmugMug o Pixenate (Walter Higgin's site, from here in Ireland)‫‏‬ o Many Facebook apps, e.g. iLIke • Large firms o NASDAQ o Activision o Business Objects o Hasbro o New York Times • See:
  12. 12. Startups on Google App Engine • Startups o BuddyPoke! (30M daily pageviews on App Engine)‫‏‬ o Pixverse (PixChat)‫‏‬ o Hyperic (Cloudstatus)‫‏‬ • One large company's experiment: o GiftTag from BestBuy •
  13. 13. Barriers to Enterprise Adoption • Immaturity of the technology o Although AWS is over three years old • Concerns about reliability • Concerns about portability o no two clouds are the same • Bureaucracy & inertia • Lock-in to legacy systems • Fear of loss-of-control & of job-losses • ...
  14. 14. Black days in the Cloud • Amazon S3 went down for two consecutive hours in Feb 2008. • Amazon S3 (& Amazon SQS) went down for eight consecutive US daytime hours in July 2008. • App Engine had a five-hour partial-outage in June '08 • App Engine had a problem of increased latency (for web-serving) over a few days in early Mar '09. • Group for notification of App Engine downtime o
  15. 15. Castles in the Cloud Google App Engine
  16. 16. Python: The Original of the Species
  17. 17. Java: the All New Flavour
  18. 18. Languages on App Engine's JVM • Java • JRuby • Jython • BeanShell • JavaScript • Scala • Groovy • … • Will it Play in App Engine?
  19. 19. What is Google App Engine? • Complete web stack o Soon to be more general computing platform • Runs on Google's infrastructure • Bigtable Datastore o Like a sorted hashtable o Not a relational database o Use GQL (with Python) instead of SQL o Use JDO or JPA (with Java) for datastore work • Memcache • Image API • Mail API • Python library available for web services • SDK for development on Windows/Mac/Linux o Run & test locally before uploading to Cloud
  20. 20. A fabulous app: 'Gaebar' Alternative: use AppRocket to replicate an App Engine datastore to an SQL database
  21. 21. Limitations of Google App Engine • Applications run in “The Sandbox”: • Native Python modules (written in C) do not work. • Many Java libraries will not work (but many have been patched or will be). See the JRE whitelist: • No “long-lived” processing (requests must return in 30 seconds). • Limited “offline processing” (this is coming) o no forks, no spawned processes, no threads o Just scheduled tasks, which work like “cron jobs” on UNIX systems • HTTPS only on o and only with Google's certificate. o Due to limitation of SSL protocol, as applied to Google's Cloud.
  22. 22. Roadmap to June for App Engine • Service for storing and serving large files • Task queues for performing background processing • Incoming email support • XMPP (Jabber) API
  23. 23. 'Batch processing' on App Engine today • Use App Engine's task scheduling for “cron jobs” spaced a minute apart (or more seldom) • A hack for more frequent jobs: run a cron-job on your PC or server to periodically hit a URL on App Engine and kick off a task • An outsourced hack for more frequent jobs: use o do the same as above (perhaps more reliably!) o Cost: one hundredth of one Euro cent per 'hit' • More sophisticated: hitch the clouds together o 'App Engine Patch' has support for 'Boto' o 'Boto' is a Python library for Amazon o Use Boto (on App Engine) to queue work-packets on Amazon's Simple Queue Service (SQS) o Kick off tasks on App Engine with a method above. o Cost: $0.01 per 10,000 requests o (+ $0.10 per GB transferred in...)
  24. 24. Web tools for App Engine • “Xmas gifts” - announced in late December 2008 • Zoho Creator o generates code and web-pages for deployment o • App Engine Site Creator o “Lightweight CMS” (with GUI interface)‫‏‬ o o Apache 2.0 license
  25. 25. Billing on Google App Engine • True pay-per-use o free quotas & no bill to pay if quotas not exceeded. o quite different model to AWS • Developer can assign budget per resource. • Prices o $0.10 per CPU core-hour o $0.15 per GB-month of storage o $0.12 per GB outgoing bandwidth o $0.10 per GB incoming bandwidth o $0.0001 per email recipient for emails sent by the application
  26. 26. Free Quotas on App Engine •Daily outgoing bandwidth: 10 GB •Daily incoming bandwidth 10 GB •Daily requests 1.3 million • Daily CPU-hours: 46 • More quotas (mail, datastore, images, memcache)
  27. 27. Other limitations on App Engine • 1 GB of free storage per app. •100 MB Memcache storage limit • App Engine apps can now receive requests and send responses of up to 10MB. • Caveat: 1 MB limit on datastore/memcache API call • You can deploy up to 1,000 code and 1,000 static files with an app. Workaround: use ZipImport (with Python) to store files in a Zip-archive: • 150MB cumulative limit on an app's code and static files (as opposed to storage in the datastore).
  28. 28. Atypical Uses of App Engine • Run Open Source software (e.g. blog / CMS): http:/ • Store your static files on App Engine o You can use App Engine as rudimentary 'Content Delivery Network' • Expose a web-service on App Engine (while doing your main development elsewhere) REST library for Django & App Engine: • Expose an interface to BigTable and exploit its scalable storage: REST interface to datastore: • Use RESTlets for Java equivalent of the two above:
  29. 29. Brief Case Study: LongURLPlease • LongURLPlease expands “tiny URLs” to the original long versions. • Original version written in a day by Darragh Curran, with barely any previous Python experience. • Launched publicly on December 23rd, 2008. • Rapidly became a platform in itself: at least nine other applications built on the API within 8 weeks • Over four thousand downloads of the Firefox plugin • 100K – 200K requests per day (each request can batch up to 4 URLs) • Running costs: zero (since its usage is well below the free quotas)
  30. 30. Where to go from here (Python) • App Engine Helper tutorial o • App Engine Patch article (I recommend AEP over the “helper” above) o • App Engine cheatsheet: • Code samples o
  31. 31. Where to go from here (Java) • Sign up for the Java “early look” • Install the Eclipse plugin (& SDK) (The Java samples are in the SDK.) • Read Google's docs and follow the tutorial: • Or pick your own language and get up-and-running on the JVM:
  32. 32. Castles in the Cloud Reference 2/27/2009
  33. 33. Articles & Papers • 10 Reasons Enterprises Aren't Ready to Trust the Cloud o • The Information Factories, by George Gilder o o quot;When the network becomes as fast as the processor, the computer hollows out and spreads across the network.quot; - in 1993, from Sun Microsystems' CTO, Eric Schmidt (now CEO of Google) • Above the Clouds: A Berkeley View of Cloud Computing o
  34. 34. Resources for Google App Engine • Google App Engine: • Videos from conference Google held in 2008 o • Chat with developers from the App Engine team o first Wednesday of the month: 3am next day (Dublin/London time). o third Wednesday of the month: 5pm (Dublin/London time). o IRC channel: #appengine on o • Google Group o • Paper on Bigtable: o
  35. 35. Book (I)‫‏‬ Developing with Google App Engine   by Eugene Ciurana Published: Feb 2009
  36. 36. Book (II)‫‏‬ Google App Engine in Action   by Noah Gift & Michael Orr To be published: May 2009
  37. 37. Book (III)‫‏‬ Google App Engine   by Dan Sanderson To be published: Sept 2009
  38. 38. Castles in the Cloud Wrapping up 2/27/2009
  39. 39. Credits • Photos used under Creative Commons license: • Photo Credits o o o o o o o o o o o o o o
  40. 40. Cloudware is the arsenal of freedom • Go guerilla o Experiment with cloudware o Consider using cloudware for your new projects o Test-drive your favourite language on the App Engine SDK o Await availability for access to the production Java runtime
  41. 41. Contact Information Kevin Noonan kevin_noonan on Twitter