A Gentle Introduction to Google App Engine

1. A Gentle Introduction to
Google App Engine
10/2009
Robin Bhattacharyya
twitter.com/rcb

2. Quick Survey

3. Quick Survey
Heard of App Engine before today?

4. Quick Survey
Heard of App Engine before today?
Installed App Engine SDK?

5. Quick Survey
Heard of App Engine before today?
Installed App Engine SDK?
Started a sample app on localhost?

6. Quick Survey
Heard of App Engine before today?
Installed App Engine SDK?
Started a sample app on localhost?
Deployed an app to App Engine?

7. Quick Survey
Heard of App Engine before today?
Installed App Engine SDK?
Started a sample app on localhost?
Deployed an app to App Engine?
Plan to use App Engine for production?

8. Quick Survey
Heard of App Engine before today?
Installed App Engine SDK?
Started a sample app on localhost?
Deployed an app to App Engine?
Plan to use App Engine for production?
Already using App Engine in production?

9. What exactly is Google App Engine?

10. What exactly is Google App Engine?
Host web apps on Google’s infrastructure

11. What exactly is Google App Engine?
Host web apps on Google’s infrastructure
No server maintenance, auto scaling

12. What exactly is Google App Engine?
Host web apps on Google’s infrastructure
No server maintenance, auto scaling
No database maintenance, uses BigTable

13. What exactly is Google App Engine?
Host web apps on Google’s infrastructure
No server maintenance, auto scaling
No database maintenance, uses BigTable
Free quota, 1G storage, 5M hits/month

14. What exactly is Google App Engine?
Host web apps on Google’s infrastructure
No server maintenance, auto scaling
No database maintenance, uses BigTable
Free quota, 1G storage, 5M hits/month
SDK for development, easy deployment

15. What exactly is Google App Engine?
Host web apps on Google’s infrastructure
No server maintenance, auto scaling
No database maintenance, uses BigTable
Free quota, 1G storage, 5M hits/month
SDK for development, easy deployment
Easy to get started, create up to 10 apps

16. Getting Started with App Engine

17. Getting Started with App Engine
Signup and add your-app at appspot.com

18. Getting Started with App Engine
Signup and add your-app at appspot.com
Get SDK at code.google.com/appengine

19. Getting Started with App Engine
Signup and add your-app at appspot.com
Get SDK at code.google.com/appengine
Write a simple helloworld WSGI app

20. Getting Started with App Engine
Signup and add your-app at appspot.com
Get SDK at code.google.com/appengine
Write a simple helloworld WSGI app
Run local server with dev_appserver.py .

21. Getting Started with App Engine
Signup and add your-app at appspot.com
Get SDK at code.google.com/appengine
Write a simple helloworld WSGI app
Run local server with dev_appserver.py .
Deploy to Google with appcfg.py update .

22. Getting Started with App Engine
Signup and add your-app at appspot.com
Get SDK at code.google.com/appengine
Write a simple helloworld WSGI app
Run local server with dev_appserver.py .
Deploy to Google with appcfg.py update .
View your app at your-app.appspot.com

23. How to Run with Python on App Engine

24. How to Run with Python on App Engine
from wsgiref.handlers import CGIHandler
def helloworld( environ, start_response):
status = '200 OK'
headers = [('Content-Type', 'text/plain')]
start_response(status, headers)
return ['Hello, World!']
def main():
CGIHandler().run(helloworld)
if __name__ == '__main__':
main()

25. How to Run with Python on App Engine
from wsgiref.handlers import CGIHandler
def helloworld( environ, start_response):
status = '200 OK'
headers = [('Content-Type', 'text/plain')]
start_response(status, headers)
return ['Hello, World!']
def main():
CGIHandler().run(helloworld)
if __name__ == '__main__':
main()

26. How to Run with Python on App Engine
from wsgiref.handlers import CGIHandler
def helloworld( environ, start_response):
status = '200 OK'
headers = [('Content-Type', 'text/plain')]
start_response(status, headers)
return ['Hello, World!']
def main():
CGIHandler().run(helloworld)
if __name__ == '__main__':
main()

27. How to Run with Python on App Engine
from wsgiref.handlers import CGIHandler
def helloworld( environ, start_response):
status = '200 OK'
headers = [('Content-Type', 'text/plain')]
start_response(status, headers)
return ['Hello, World!']
def main():
CGIHandler().run(helloworld)
if __name__ == '__main__':
main()

28. How to Run with Python on App Engine
from wsgiref.handlers import CGIHandler
def helloworld( environ, start_response):
status = '200 OK'
headers = [('Content-Type', 'text/plain')]
start_response(status, headers)
return ['Hello, World!']
def main():
CGIHandler().run(helloworld)
if __name__ == '__main__':
main()

29. A Little App Engine History

30. A Little App Engine History
April 7th 2008, Released, Python only,
10,000 signups, limited to 3 apps per user.

31. A Little App Engine History
April 7th 2008, Released, Python only,
10,000 signups, limited to 3 apps per user.
Monthly updates, open signups, projected
pricing, image, memcache APIs, 10 apps
per user, admin dashboard, download logs.

32. A Little App Engine History
April 7th 2008, Released, Python only,
10,000 signups, limited to 3 apps per user.
Monthly updates, open signups, projected
pricing, image, memcache APIs, 10 apps
per user, admin dashboard, download logs.
April 7th 2009, added Java Support, cron,
datastore import/export, firewall support.

33. A Little App Engine History
April 7th 2008, Released, Python only,
10,000 signups, limited to 3 apps per user.
Monthly updates, open signups, projected
pricing, image, memcache APIs, 10 apps
per user, admin dashboard, download logs.
April 7th 2009, added Java Support, cron,
datastore import/export, firewall support.
Monthly updates, Task Queue, XMPP,
incoming mail soon to be released.

34. App Engine Services

35. App Engine Services
Datastore - BigTable - Scalable Read/Write

36. App Engine Services
Datastore - BigTable - Scalable Read/Write
Memcache - shared among your instances

37. App Engine Services
Datastore - BigTable - Scalable Read/Write
Memcache - shared among your instances
Google Accounts - basic Login and Logout

38. App Engine Services
Datastore - BigTable - Scalable Read/Write
Memcache - shared among your instances
Google Accounts - basic Login and Logout
Mail - send email and soon receive email

39. App Engine Services
Datastore - BigTable - Scalable Read/Write
Memcache - shared among your instances
Google Accounts - basic Login and Logout
Mail - send email and soon receive email
Images - scale, compose, adjust colors

40. App Engine Services
Datastore - BigTable - Scalable Read/Write
Memcache - shared among your instances
Google Accounts - basic Login and Logout
Mail - send email and soon receive email
Images - scale, compose, adjust colors
Cron - up to 20 periodic webhooks

41. App Engine Services
Datastore - BigTable - Scalable Read/Write
Memcache - shared among your instances
Google Accounts - basic Login and Logout
Mail - send email and soon receive email
Images - scale, compose, adjust colors
Cron - up to 20 periodic webhooks
TaskQueue - dynamic concurrent webhooks

42. App Engine Services
Datastore - BigTable - Scalable Read/Write
Memcache - shared among your instances
Google Accounts - basic Login and Logout
Mail - send email and soon receive email
Images - scale, compose, adjust colors
Cron - up to 20 periodic webhooks
TaskQueue - dynamic concurrent webhooks
XMPP - send/recv your-app@appspot.com

43. A Look at Big Table

44. A Look at BigTable
Atomic - writes either commit or fail

45. A Look at BigTable
Atomic - writes either commit or fail
Consistent - writes are synchronous

46. A Look at BigTable
Atomic - writes either commit or fail
Consistent - writes are synchronous
MVCC - writers never block readers

47. A Look at BigTable
Atomic - writes either commit or fail
Consistent - writes are synchronous
MVCC - writers never block readers
Scalable Write - entities sharded to groups

48. A Look at BigTable
Atomic - writes either commit or fail
Consistent - writes are synchronous
MVCC - writers never block readers
Scalable Write - entities sharded to groups
Transactions - inter-table txns possible

49. A Look at BigTable
Atomic - writes either commit or fail
Consistent - writes are synchronous
MVCC - writers never block readers
Scalable Write - entities sharded to groups
Transactions - inter-table txns possible
Schema-less - add properties on the fly

50. A Look at BigTable
Atomic - writes either commit or fail
Consistent - writes are synchronous
MVCC - writers never block readers
Scalable Write - entities sharded to groups
Transactions - inter-table txns possible
Schema-less - add properties on the fly
Indexed - adds indexes at write time

51. A Look at BigTable
Atomic - writes either commit or fail
Consistent - writes are synchronous
MVCC - writers never block readers
Scalable Write - entities sharded to groups
Transactions - inter-table txns possible
Schema-less - add properties on the fly
Indexed - adds indexes at write time
Queries - seek 1st index, then scan down

52. App Engine Restrictions

53. App Engine Restrictions
Read-only access to file system.

54. App Engine Restrictions
Read-only access to file system.
No C-extensions, pure python only.

55. App Engine Restrictions
Read-only access to file system.
No C-extensions, pure python only.
Applications cannot create new threads.

56. App Engine Restrictions
Read-only access to file system.
No C-extensions, pure python only.
Applications cannot create new threads.
10 MB request and response size limit.

57. App Engine Restrictions
Read-only access to file system.
No C-extensions, pure python only.
Applications cannot create new threads.
10 MB request and response size limit.
1 MB datastore entity size limit.

58. App Engine Restrictions
Read-only access to file system.
No C-extensions, pure python only.
Applications cannot create new threads.
10 MB request and response size limit.
1 MB datastore entity size limit.
1000 results max per datastore query.

59. App Engine Restrictions
Read-only access to file system.
No C-extensions, pure python only.
Applications cannot create new threads.
10 MB request and response size limit.
1 MB datastore entity size limit.
1000 results max per datastore query.
30 second deadline per request/response.

60. App Engine Advantages

61. App Engine Advantages
Auto Scaling - No need to over provision.

62. App Engine Advantages
Auto Scaling - No need to over provision.
Static Files - Static files use Google's CDN

63. App Engine Advantages
Auto Scaling - No need to over provision.
Static Files - Static files use Google's CDN
Easy Logs - View logs in web console.

64. App Engine Advantages
Auto Scaling - No need to over provision.
Static Files - Static files use Google's CDN
Easy Logs - View logs in web console.
Easy Deployment - Literally 1-click deploy.

65. App Engine Advantages
Auto Scaling - No need to over provision.
Static Files - Static files use Google's CDN
Easy Logs - View logs in web console.
Easy Deployment - Literally 1-click deploy.
Free Quota - 99% of apps will pay nothing.

66. App Engine Advantages
Auto Scaling - No need to over provision.
Static Files - Static files use Google's CDN
Easy Logs - View logs in web console.
Easy Deployment - Literally 1-click deploy.
Free Quota - 99% of apps will pay nothing.
Affordable Scaling - Prices better than AWS.

67. App Engine Advantages
Auto Scaling - No need to over provision.
Static Files - Static files use Google's CDN
Easy Logs - View logs in web console.
Easy Deployment - Literally 1-click deploy.
Free Quota - 99% of apps will pay nothing.
Affordable Scaling - Prices better than AWS.
No config - No need to config OS or servers.

68. App Engine Advantages
Auto Scaling - No need to over provision.
Static Files - Static files use Google's CDN
Easy Logs - View logs in web console.
Easy Deployment - Literally 1-click deploy.
Free Quota - 99% of apps will pay nothing.
Affordable Scaling - Prices better than AWS.
No config - No need to config OS or servers.
Easy Security - Google patches OS/servers.

69. App Engine Case Study: Ruly
Link shortener called Ruly at http://ru.ly
Built with Web2py, AppEngine, BigTable.

70. App Engine Case Study: Ruly
Uses a 'WideCounter' to generate short IDs in a
sharded and scalable way on BigTable.
class WideCounter(object):
class WideCounter(db.Model):
c = db.IntegerProperty('Count', default=0, required=True)
def next(self, namespace='', shard=None):
if shard is None:
shard = random.randint(0, self.max_shard)
key_name = '/%s/%s/%s' % (self.namespace, namespace, shard)
def txn():
counter = self.WideCounter.get_by_key_name(key_name)
if counter:
counter.c += 1
else:
counter = self.WideCounter(key_name=key_name)
counter.put()
return counter.c
count = db.run_in_transaction(txn)
return self.to_code(shard, self.padleft) + self.to_code(count, self.padright)

71. Browse Logs in Web Console (Screenshot)

72. Manage App in Web Console (Screenshot)

73. Study Best Practices Early

74. Study Best Practices Early
Stop thinking in terms of SQL joins at
request time, because joins will not scale.

75. Study Best Practices Early
Stop thinking in terms of SQL joins at
request time, because joins will not scale.
Do not compute results at request time,
instead pre-compute results and simply
return answers at request time.

76. Study Best Practices Early
Stop thinking in terms of SQL joins at
request time, because joins will not scale.
Do not compute results at request time,
instead pre-compute results and simply
return answers at request time.
Break up large tasks into smaller tasks and
use the task queue to execute them.

77. Study Best Practices Early
Stop thinking in terms of SQL joins at
request time, because joins will not scale.
Do not compute results at request time,
instead pre-compute results and simply
return answers at request time.
Break up large tasks into smaller tasks and
use the task queue to execute them.
Design your app using these guidelines, so
you do not have to rewrite it if it becomes
popular, because it will automatically scale.

78. Coming Soon to AppEngine

79. Coming Soon to AppEngine
Service for storing and serving large files

80. Coming Soon to AppEngine
Service for storing and serving large files
Incoming email support

81. Coming Soon to AppEngine
Service for storing and serving large files
Incoming email support
More support for map-reduce

82. Coming Soon to AppEngine
Service for storing and serving large files
Incoming email support
More support for map-reduce
Cursors for continuing results of Datastore
queries past the 1000 entity limit

83. Coming Soon to AppEngine
Service for storing and serving large files
Incoming email support
More support for map-reduce
Cursors for continuing results of Datastore
queries past the 1000 entity limit
Alerting system for exceptions in your app

84. Coming Soon to AppEngine
Service for storing and serving large files
Incoming email support
More support for map-reduce
Cursors for continuing results of Datastore
queries past the 1000 entity limit
Alerting system for exceptions in your app
Datastore dump and restore facility

85. Photo Credits
http://www.flickr.com/photos/13476480@N07/3711908745/sizes/o/
http://www.flickr.com/photos/niallkennedy/3422255193/sizes/l/
http://www.flickr.com/photos/30050915@N05/3046571642/sizes/o/
http://www.flickr.com/photos/robbie1/442214088/
http://www.flickr.com/photos/zavo/205961178/
http://www.flickr.com/photos/annapickard/20988642/
http://www.flickr.com/photos/billsphotos/2536260790/

86. Questions
Robin Bhattacharyya
twitter.com/rcb