Castles in the Cloud:
Developing with Google App Engine
Kevin Noonan
Calbane Ltd.
IT@Cork, 20th April, 2009
Castles in the Cloud




Introduction
A working definition of Cloudware




 •   Outsourced Virtualization
      o   Virtual server running on outsourced server...
The Cloud Computing Pyramid




•Michael Sheehan of GoGrid devised this classification
Main Players
Other Players
•   Microsoft Windows Azure
     o    Including .NET Live Services

•   Force.com from Salesforce
•   SUN (r...
What's Cloudware good for? (I)‫‏‬




•   Guerilla development
     o   Pay-per-use model is ideal for startups
     o   A...
What's Cloudware good for? (II)‫‏‬




•   Scaling up
     o   Clone the virtual server
     o   pay for extra resources (...
What's Cloudware good for? (III)‫‏‬




•   In general, outsourcing IT infrastructure
     o   pay someone else to run you...
Castles in the Cloud

Who's using Cloudware & why not?
Startups & Large Firms on AWS
•   Startups
     o   Twitter
     o   SmugMug
     o   Pixenate (Walter Higgin's site, from...
Startups on Google App Engine

•   Startups
     o   BuddyPoke! (30M daily pageviews on App Engine)‫‏‬
     o   Pixverse (...
Barriers to Enterprise Adoption

•   Immaturity of the technology
     o    Although AWS is over three years old

•   Conc...
Black days in the Cloud

•   Amazon S3 went down for two consecutive hours in Feb 2008.


•   Amazon S3 (& Amazon SQS) wen...
Castles in the Cloud

Google App Engine
Python: The Original of the Species
Java: the All New Flavour
Languages on App Engine's JVM
•   Java
•   JRuby
•   Jython
•   BeanShell
•   JavaScript
•   Scala
•   Groovy
•   …
•   Wi...
What is Google App Engine?
•   Complete web stack
     o   Soon to be more general computing platform

•   Runs on Google'...
A fabulous app: 'Gaebar'




Alternative: use AppRocket to replicate an App Engine datastore to an SQL database
http://cod...
Limitations of Google App Engine
•   Applications run in “The Sandbox”:
    http://code.google.com/appengine/docs/java/run...
Roadmap to June for App Engine
•   Service for storing and serving large files


•   Task queues for performing background...
'Batch processing' on App Engine today
•   Use App Engine's task scheduling for “cron jobs” spaced a minute apart (or more...
Web tools for App Engine
•   “Xmas gifts” - announced in late December 2008


•   Zoho Creator
     o   generates code and...
Billing on Google App Engine
• True pay-per-use
 o   free quotas & no bill to pay if quotas not exceeded.
 o   quite diffe...
Free Quotas on App Engine
•Daily outgoing bandwidth: 10 GB

•Daily incoming bandwidth 10 GB

•Daily requests 1.3 million

...
Other limitations on App Engine
• 1 GB of free storage per app.

•100 MB Memcache storage limit

• App Engine apps can now...
Atypical Uses of App Engine

•   Run Open Source software (e.g. blog / CMS):
http:/groups.google.com/group/google-appengin...
Brief Case Study: LongURLPlease
• LongURLPlease expands “tiny URLs” to the original long versions.

• Original version wri...
Where to go from here (Python)
• App Engine Helper tutorial
 o   http://blog.pas.net.au/2009/02/17/django-and-google-app-e...
Where to go from here (Java)
•   Sign up for the Java “early look”
    http://appengine.google.com/promo/java_runtime


• ...
Castles in the Cloud

 Reference




2/27/2009
Articles & Papers
•   10 Reasons Enterprises Aren't Ready to Trust the Cloud
     o   http://gigaom.com/2008/07/01/10-reas...
Resources for Google App Engine
•   Google App Engine: http://appengine.google.com


•   Videos from conference Google hel...
Book (I)‫‏‬
       Developing with Google App Engine
                        
              by Eugene Ciurana


          ...
Book (II)‫‏‬

             Google App Engine in Action
                           
             by Noah Gift & Michael Orr...
Book (III)‫‏‬

          Google App Engine
                        
          by Dan Sanderson


          To be published...
Castles in the Cloud

 Wrapping up




2/27/2009
Credits
•   Photos used under Creative Commons license:
•   Photo Credits
     o   http://www.flickr.com/photos/notsogoodp...
Cloudware is the arsenal of freedom




•   Go guerilla
     o   Experiment with cloudware
     o   Consider using cloudwa...
Contact Information
     Kevin Noonan




 kevin_noonan on Twitter

 http://www.calbane.com
Upcoming SlideShare
Loading in...5
×

Castles in the Cloud: Developing with Google App Engine

4,079

Published on

App Engine offers developers the opportunity to deploy systems on Google's robust and scalable server-farms. App Engine provides a higher-level platform than Amazon Web Services,with automated scaling and true pay-per-use billing.

The poster-child of App Engine, "BuddyPoke", has gained over thirty million users.

With App Engine, Google has released the first public API to BigTable, its planetary datastore, which performs successfully at petabyte scale across diverse applications from search to finance to Google Earth.

This presentation will cover App Engine's features and limitations, and how to exploit this new and evolving platform.

Published in: Technology, News & Politics
1 Comment
7 Likes
Statistics
Notes
  • Thanks - nice introduction to GAE (even though this may be a bit dated as of today)
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
4,079
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
204
Comments
1
Likes
7
Embeds 0
No embeds

No notes for slide

Transcript of "Castles in the Cloud: Developing with Google App Engine"

  1. 1. Castles in the Cloud: Developing with Google App Engine Kevin Noonan Calbane Ltd. IT@Cork, 20th April, 2009
  2. 2. Castles in the Cloud Introduction
  3. 3. A working definition of Cloudware • Outsourced Virtualization o Virtual server running on outsourced server farm
  4. 4. The Cloud Computing Pyramid •Michael Sheehan of GoGrid devised this classification
  5. 5. Main Players
  6. 6. Other Players • Microsoft Windows Azure o Including .NET Live Services • Force.com from Salesforce • SUN (recently announced) • Mosso • GoGrid • FlexiScale • Hosting365 • ...
  7. 7. What's Cloudware good for? (I)‫‏‬ • Guerilla development o Pay-per-use model is ideal for startups o Also good for guerillas inside the enterprise...
  8. 8. What's Cloudware good for? (II)‫‏‬ • Scaling up o Clone the virtual server o pay for extra resources (bandwidth, storage, RAM, etc.)‫‏‬
  9. 9. What's Cloudware good for? (III)‫‏‬ • In general, outsourcing IT infrastructure o pay someone else to run your servers, network, database...
  10. 10. Castles in the Cloud Who's using Cloudware & why not?
  11. 11. Startups & Large Firms on AWS • Startups o Twitter o SmugMug o Pixenate (Walter Higgin's site, from here in Ireland)‫‏‬ o Many Facebook apps, e.g. iLIke • Large firms o NASDAQ o Activision o Business Objects o Hasbro o New York Times • See: http://aws.amazon.com/solutions/case-studies/
  12. 12. Startups on Google App Engine • Startups o BuddyPoke! (30M daily pageviews on App Engine)‫‏‬ o Pixverse (PixChat)‫‏‬ o Hyperic (Cloudstatus)‫‏‬ • One large company's experiment: o GiftTag from BestBuy • http://code.google.com/appengine/casestudies.html
  13. 13. Barriers to Enterprise Adoption • Immaturity of the technology o Although AWS is over three years old • Concerns about reliability • Concerns about portability o no two clouds are the same • Bureaucracy & inertia • Lock-in to legacy systems • Fear of loss-of-control & of job-losses • ...
  14. 14. Black days in the Cloud • Amazon S3 went down for two consecutive hours in Feb 2008. • Amazon S3 (& Amazon SQS) went down for eight consecutive US daytime hours in July 2008. • App Engine had a five-hour partial-outage in June '08 • App Engine had a problem of increased latency (for web-serving) over a few days in early Mar '09. • Group for notification of App Engine downtime o http://groups.google.com/group/google-appengine-downtime-notify
  15. 15. Castles in the Cloud Google App Engine
  16. 16. Python: The Original of the Species
  17. 17. Java: the All New Flavour
  18. 18. Languages on App Engine's JVM • Java • JRuby • Jython • BeanShell • JavaScript • Scala • Groovy • … • Will it Play in App Engine? http://groups.google.com/group/google-appengine-java/web/will-it-play-in-app-engine
  19. 19. What is Google App Engine? • Complete web stack o Soon to be more general computing platform • Runs on Google's infrastructure • Bigtable Datastore o Like a sorted hashtable o Not a relational database o Use GQL (with Python) instead of SQL o Use JDO or JPA (with Java) for datastore work • Memcache • Image API • Mail API • Python library available for Force.com web services • SDK for development on Windows/Mac/Linux o Run & test locally before uploading to Cloud
  20. 20. A fabulous app: 'Gaebar' Alternative: use AppRocket to replicate an App Engine datastore to an SQL database http://code.google.com/p/approcket/
  21. 21. Limitations of Google App Engine • Applications run in “The Sandbox”: http://code.google.com/appengine/docs/java/runtime.html#The_Sandbox • Native Python modules (written in C) do not work. • Many Java libraries will not work (but many have been patched or will be). See the JRE whitelist: http://code.google.com/appengine/docs/java/jrewhitelist.html • No “long-lived” processing (requests must return in 30 seconds). • Limited “offline processing” (this is coming) o no forks, no spawned processes, no threads o Just scheduled tasks, which work like “cron jobs” on UNIX systems • HTTPS only on XYZ.appspot.com o and only with Google's certificate. o Due to limitation of SSL protocol, as applied to Google's Cloud.
  22. 22. Roadmap to June for App Engine • Service for storing and serving large files • Task queues for performing background processing • Incoming email support • XMPP (Jabber) API
  23. 23. 'Batch processing' on App Engine today • Use App Engine's task scheduling for “cron jobs” spaced a minute apart (or more seldom) • A hack for more frequent jobs: run a cron-job on your PC or server to periodically hit a URL on App Engine and kick off a task • An outsourced hack for more frequent jobs: use webcron.org o do the same as above (perhaps more reliably!) o Cost: one hundredth of one Euro cent per 'hit' • More sophisticated: hitch the clouds together o 'App Engine Patch' has support for 'Boto' o 'Boto' is a Python library for Amazon o Use Boto (on App Engine) to queue work-packets on Amazon's Simple Queue Service (SQS) o Kick off tasks on App Engine with a method above. o Cost: $0.01 per 10,000 requests o (+ $0.10 per GB transferred in...)
  24. 24. Web tools for App Engine • “Xmas gifts” - announced in late December 2008 • Zoho Creator o generates code and web-pages for deployment o http://creator.zoho.com/ • App Engine Site Creator o “Lightweight CMS” (with GUI interface)‫‏‬ o http://code.google.com/p/app-engine-site-creator/ o Apache 2.0 license
  25. 25. Billing on Google App Engine • True pay-per-use o free quotas & no bill to pay if quotas not exceeded. o quite different model to AWS • Developer can assign budget per resource. • Prices o $0.10 per CPU core-hour o $0.15 per GB-month of storage o $0.12 per GB outgoing bandwidth o $0.10 per GB incoming bandwidth o $0.0001 per email recipient for emails sent by the application
  26. 26. Free Quotas on App Engine •Daily outgoing bandwidth: 10 GB •Daily incoming bandwidth 10 GB •Daily requests 1.3 million • Daily CPU-hours: 46 • More quotas (mail, datastore, images, memcache) http://code.google.com/appengine/docs/quotas.html
  27. 27. Other limitations on App Engine • 1 GB of free storage per app. •100 MB Memcache storage limit • App Engine apps can now receive requests and send responses of up to 10MB. • Caveat: 1 MB limit on datastore/memcache API call • You can deploy up to 1,000 code and 1,000 static files with an app. Workaround: use ZipImport (with Python) to store files in a Zip-archive: http://code.google.com/appengine/articles/django10_zipimport.html • 150MB cumulative limit on an app's code and static files (as opposed to storage in the datastore).
  28. 28. Atypical Uses of App Engine • Run Open Source software (e.g. blog / CMS): http:/groups.google.com/group/google-appengine/web/google-app-engine-open-source-projects • Store your static files on App Engine o You can use App Engine as rudimentary 'Content Delivery Network' • Expose a web-service on App Engine (while doing your main development elsewhere) REST library for Django & App Engine: http://code.google.com/p/python-rest/ • Expose an interface to BigTable and exploit its scalable storage: REST interface to datastore: http://code.google.com/p/app3/ • Use RESTlets for Java equivalent of the two above: http://wiki.restlet.org/docs_1.2/13-restlet/252-restlet.html
  29. 29. Brief Case Study: LongURLPlease • LongURLPlease expands “tiny URLs” to the original long versions. • Original version written in a day by Darragh Curran, with barely any previous Python experience. • Launched publicly on December 23rd, 2008. • Rapidly became a platform in itself: at least nine other applications built on the API within 8 weeks • Over four thousand downloads of the Firefox plugin • 100K – 200K requests per day (each request can batch up to 4 URLs) • Running costs: zero (since its usage is well below the free quotas)
  30. 30. Where to go from here (Python) • App Engine Helper tutorial o http://blog.pas.net.au/2009/02/17/django-and-google-app-engine-tutorial/ • App Engine Patch article (I recommend AEP over the “helper” above) o http://code.google.com/appengine/articles/app-engine-patch.html • App Engine cheatsheet: http://short.ie/cheatongae • Code samples o http://code.google.com/p/google-app-engine-samples/downloads/list
  31. 31. Where to go from here (Java) • Sign up for the Java “early look” http://appengine.google.com/promo/java_runtime • Install the Eclipse plugin (& SDK) http://code.google.com/appengine/docs/java/tools/eclipse.html (The Java samples are in the SDK.) • Read Google's docs and follow the tutorial: http://code.google.com/appengine/docs/java/overview.html • Or pick your own language and get up-and-running on the JVM: http://groups.google.com/group/google-appengine-java/web/will-it-play-in-app-engine
  32. 32. Castles in the Cloud Reference 2/27/2009
  33. 33. Articles & Papers • 10 Reasons Enterprises Aren't Ready to Trust the Cloud o http://gigaom.com/2008/07/01/10-reasons-enterprises-arent-ready-to-trust-the-cloud/ • The Information Factories, by George Gilder o http://www.wired.com/wired/archive/14.10/cloudware_pr.html o quot;When the network becomes as fast as the processor, the computer hollows out and spreads across the network.quot; - in 1993, from Sun Microsystems' CTO, Eric Schmidt (now CEO of Google) • Above the Clouds: A Berkeley View of Cloud Computing o http://www.eecs.berkeley.edu/Pubs/TechRpts/2009/EECS-2009-28.html
  34. 34. Resources for Google App Engine • Google App Engine: http://appengine.google.com • Videos from conference Google held in 2008 o http://sites.google.com/site/io/ • Chat with developers from the App Engine team o first Wednesday of the month: 3am next day (Dublin/London time). o third Wednesday of the month: 5pm (Dublin/London time). o IRC channel: #appengine on irc.freenode.net o http://en.wikipedia.org/wiki/List_of_IRC_clients • Google Group o http://groups.google.com/group/google-appengine/ • Paper on Bigtable: o http://labs.google.com/papers/bigtable.html
  35. 35. Book (I)‫‏‬ Developing with Google App Engine   by Eugene Ciurana Published: Feb 2009   http://apress.com/
  36. 36. Book (II)‫‏‬ Google App Engine in Action   by Noah Gift & Michael Orr To be published: May 2009   http://www.manning.com/gift/
  37. 37. Book (III)‫‏‬ Google App Engine   by Dan Sanderson To be published: Sept 2009   http://oreilly.com/
  38. 38. Castles in the Cloud Wrapping up 2/27/2009
  39. 39. Credits • Photos used under Creative Commons license: • Photo Credits o http://www.flickr.com/photos/notsogoodphotography/ o http://www.flickr.com/photos/barto/ o http://www.flickr.com/photos/kitoy/ o http://www.flickr.com/photos/wolfgangstaudt/ o http://www.flickr.com/photos/motumboe/ o http://www.flickr.com/photos/ragnar1984/ o http://www.flickr.com/photos/janekm/ o http://www.flickr.com/photos/raylopez/ o http://www.flickr.com/photos/pagedooley/ o http://www.flickr.com/photos/gustty/ o http://www.flickr.com/photos/mike9alive/ o http://www.flickr.com/photos/tambako/ o http://www.flickr.com/photos/nate o http://www.flickr.com/photos/freewine/
  40. 40. Cloudware is the arsenal of freedom • Go guerilla o Experiment with cloudware o Consider using cloudware for your new projects o Test-drive your favourite language on the App Engine SDK o Await availability for access to the production Java runtime
  41. 41. Contact Information Kevin Noonan kevin_noonan on Twitter http://www.calbane.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×