0
Building TweetEngine


                                  Ikai Lan
Developer Relations, Google App Engine
                 ...
Goals of this talk

                     Using TweetEngine to
                     explain key concepts of
               ...
What is App Engine?




     App Engine is a platform

     You build & test your app

     Then upload your app to Google...
The king of ease-of-use




                   Extremely rapid development
                   Very low barrier of entry
  ...
Components
Getting TweetEngine




Application: http://tweetengine.net/

Code: http://github.com/Arachnid/tweetengine

Prerequisites:...
OAuth
                                        Secure method for granting
                                        permissio...
Standard OAuth flow (User)
What does the User see?

1. Link to login with Twitter
2. User is redirected to an http:
   //t...
Standard OAuth flow (Server)
What does the server do?

1. Acquire request token
2. Generate login URL
3. Redirect or link ...
TweetEngine + OAuth

 OAuth request token saved in TwitterAccount model
 Saved per Twitter Account
 oauth.py handles signi...
Logging in to Twitter using OAuth
   # src/tweetengine/add.py
   auth_token =
     self.request.get("oauth_token")
   auth...
Initializing an OAuth client

# src/oauth.py

OAuthClient.__init__(self, "twitter", consumer_key,
 consumer_secret,
 "http...
Twitter API calls
  - Web handlers user AJAX
  - Backend calls go to TwitterApiHandler
  # src/tweetengine/handlers/apis.p...
OAuth

 The "de facto" standard for authentication on the web
 against a third party API
 TweetEngine's support can be eas...
i18n - internationalization
  Problem: maintaining
  localized versions
  Localization is more than
  translations
  Tweet...
Workflow


1.   Developer marks strings as requiring translation
2.   Translators work on .PO files
3.   These are compile...
What does a .po file look like?

#. Default: "Collaborative tweeting"
#: tweetengine/templates/base.pt:36
msgid "subline"
...
Sample template code

tweetengine/templates/index.pt:

<metal:title fill-slot="title" i18n:translate="title-welcome">Welco...
src/tweetengine/handlers/base.py
 # Instantiate Chameleon template loader
from chameleon.zpt.loader import TemplateLoader
...
Remember: i18n > translations

  Resizing UI elements for languages with long/short words
  Right-to-left languages (Hebre...
App Engine is also about tooling




   Image source: http://www.flickr.com/photos/sparr0/4584964212/
Two App Engine tools
 AppStats                                                       Task Queues
    Application profile t...
AppStats
           Track expensive requests
           Expose bottlenecks in your
           code
           Show you wha...
AppStats example
AppStats example (con't)
How does it work?
 Uses Call Hooks
    pre‐call hook records start .me, request, stack
    post‐call hook records end .me,...
AppStats for TweetEngine
  Add AppStats as WSGI Middleware:
app = recording.appstats_wsgi_middleware(app)
  Add AppStats G...
Task Queues

 Execute background tasks
 Throttle control
 ETA
 Can be unique!
 Invoked using a web handler
Standard async worker architecture
Task Queues in App Engine
 # Creating a task # src/tweetengine/model.py taskqueue.Task
 (eta=send_time, name=task_name, ur...
Task Queues in TweetEngine

 Used to schedule Tweets
 ETA parameter used to set an approximate execution time
 Tasks are n...
Task Queues

 Allow developers to build asynchronous workers without
 hassle of building out queueing/polling system
 Hand...
Summary: TweetEngine technologies

1. OAuth - "Valet key" for internet applications
2. i18n - Using templates and message ...
Questions?




Application: http://tweetengine.net/

Code: http://github.com/Arachnid/tweetengine

ikai.l@google.com
Twitt...
Upcoming SlideShare
Loading in...5
×

Building TweetEngine

1,005

Published on

0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,005
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
17
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Building TweetEngine"

  1. 1. Building TweetEngine Ikai Lan Developer Relations, Google App Engine ikai.l@google.com Twitter: @ikai
  2. 2. Goals of this talk Using TweetEngine to explain key concepts of building cloud applications Explaining App Engine as a development environment Whet your appetite!
  3. 3. What is App Engine? App Engine is a platform You build & test your app Then upload your app to Google App Engine runs everything No need to worry about machines, network, storage, scalability, etc.
  4. 4. The king of ease-of-use Extremely rapid development Very low barrier of entry Simple yet robust syntax Rich library of packages/modules App Engine's first language API
  5. 5. Components
  6. 6. Getting TweetEngine Application: http://tweetengine.net/ Code: http://github.com/Arachnid/tweetengine Prerequisites: Python 2.5 buildout lxml (for i18n features)
  7. 7. OAuth Secure method for granting permissions to third-party integrations Allows sites to use a trusted identity provider (Google, Twitter) "Valet key" for the internet Image and "valet key" source: (http://hueniverse.com/oauth/guide/intro/)
  8. 8. Standard OAuth flow (User) What does the User see? 1. Link to login with Twitter 2. User is redirected to an http: //twitter.com URL 3. User logs in, grants access 4. User is redirected back to site
  9. 9. Standard OAuth flow (Server) What does the server do? 1. Acquire request token 2. Generate login URL 3. Redirect or link user to this URL 4. After user has logged in, Twitter will redirect to a callback URL on server 5. Server saves token passed to callback URL
  10. 10. TweetEngine + OAuth OAuth request token saved in TwitterAccount model Saved per Twitter Account oauth.py handles signing apis.py handles proxied Twitter API calls
  11. 11. Logging in to Twitter using OAuth # src/tweetengine/add.py auth_token = self.request.get("oauth_token") auth_verifier = self.request.get("oauth_verifier") user_info = client.get_user_info(auth_token, auth_verifier=auth_verifier) # Create the twitter account account = model.TwitterAccount. get_or_insert( user_info["username"], oauth_token=user_info["token"], oauth_secret=user_info["secret"], name=user_info["name"], picture=user_info["picture"])
  12. 12. Initializing an OAuth client # src/oauth.py OAuthClient.__init__(self, "twitter", consumer_key, consumer_secret, "http://twitter.com/oauth/request_token", "http://twitter.com/oauth/access_token", callback_url)
  13. 13. Twitter API calls - Web handlers user AJAX - Backend calls go to TwitterApiHandler # src/tweetengine/handlers/apis.py for k,v in params.items(): if isinstance(v, unicode): params[k] = v.encode('utf8') # Join all of the params together. params_str = "&".join(["%s=%s" % (encode(k), encode(params[k])) for k in sorted(params)]) # Join the entire message together per the OAuth specification. message = "&".join(["GET" if method == urlfetch.GET else "POST", encode(url), encode(params_str)]) # Create a HMAC-SHA1 signature of the message. key = "%s&%s" % (self.consumer_secret, secret) # Note compulsory "&". signature = hmac(key, message, sha1) digest_base64 = signature.digest().encode("base64").strip() params["oauth_signature"] = digest_base64 return urlencode(params)
  14. 14. OAuth The "de facto" standard for authentication on the web against a third party API TweetEngine's support can be easily generalized to be used on any OAuth supporting site Highly recommend: New sites, do not build identity system, use OAuth or OpenID
  15. 15. i18n - internationalization Problem: maintaining localized versions Localization is more than translations TweetEngine already supports English, German and Italian! Image source: http://www.flickr. com/photos/natematias/22132919
  16. 16. Workflow 1. Developer marks strings as requiring translation 2. Translators work on .PO files 3. These are compiled into efficient .POT files 4. At runtime, translations are looked up when pages rendered using "Accepts-Header"
  17. 17. What does a .po file look like? #. Default: "Collaborative tweeting" #: tweetengine/templates/base.pt:36 msgid "subline" msgstr "Tweeting collaborativo" #. Default: "Welcome!" #: tweetengine/templates/index.pt:3 msgid "title-welcome" msgstr "Benvenuto!"
  18. 18. Sample template code tweetengine/templates/index.pt: <metal:title fill-slot="title" i18n:translate="title-welcome">Welcome!</metal:title> tweetengine/templates/base.pt: <h2><a href="/" i18n:translate="subline">Collaborative tweeting</a></h2> Run from TweetEngine root: bin/i18nize
  19. 19. src/tweetengine/handlers/base.py # Instantiate Chameleon template loader from chameleon.zpt.loader import TemplateLoader from tweetengine import model from tweetengine.menu import mainmenu tpl_path = os.path.join(os.path.dirname(__file__), "..", "templates") tpl_loader = TemplateLoader(tpl_path) # Later on in the file template_vars['target_language'] = self.request.headers.get('Accept-Language', None) tpl = tpl_loader.load('base.pt') template_vars['master'] = tpl.macros['master'] tpl = tpl_loader.load('macros.pt') template_vars['macros'] = tpl.macros tpl = tpl_loader.load(template_file) self.response.out.write(tpl(**template_vars))
  20. 20. Remember: i18n > translations Resizing UI elements for languages with long/short words Right-to-left languages (Hebrew, Arabic) and layouts Time and date formats Metric/English system Much more ... this just a place to start!
  21. 21. App Engine is also about tooling Image source: http://www.flickr.com/photos/sparr0/4584964212/
  22. 22. Two App Engine tools AppStats Task Queues Application profile that is Background queueing App Engine aware and worker mechanism Image source: http://www.flickr.com/photos/spbatt/3928384579/
  23. 23. AppStats Track expensive requests Expose bottlenecks in your code Show you what code is being executed the most for "hotspot optimization" Run in production with minimal overhead
  24. 24. AppStats example
  25. 25. AppStats example (con't)
  26. 26. How does it work? Uses Call Hooks pre‐call hook records start .me, request, stack post‐call hook records end .me, mcycles, response Measuring times: real time using Python’s wall clock API API megacycles using rpc.cpu_usage_mcycles CPU megacycles using quota.get_request_cpu_usage() Get stack contents from Python’s sys._getframe() Recording accumulates in memory object Written to memcache at end of each request
  27. 27. AppStats for TweetEngine Add AppStats as WSGI Middleware: app = recording.appstats_wsgi_middleware(app) Add AppStats GUI to URL mapping (app.yaml): - url: /stats.* script: appstats/ui.py Dev: http://localhost:8080/stats Prod (as admin): http://APPID.appspot.com/stats That's it!
  28. 28. Task Queues Execute background tasks Throttle control ETA Can be unique! Invoked using a web handler
  29. 29. Standard async worker architecture
  30. 30. Task Queues in App Engine # Creating a task # src/tweetengine/model.py taskqueue.Task (eta=send_time, name=task_name, url=twitter. ScheduledTweetHandler.URL_PATH).add() # Task Handler # src/tweetengine/handlers/twitter.py class ScheduledTweetHandler(webapp.RequestHandler): def post(self): publishApprovedTweets() def publishApprovedTweets(): # Iterate through unsent OutgoingTweets and send them
  31. 31. Task Queues in TweetEngine Used to schedule Tweets ETA parameter used to set an approximate execution time Tasks are named - this allows us to set times to check for OutgoingTweets that should be sent at this time and not duplicate work (we don't have to use cron)
  32. 32. Task Queues Allow developers to build asynchronous workers without hassle of building out queueing/polling system Handles task uniqueness and transactional task creation Can be scheduled, throttled
  33. 33. Summary: TweetEngine technologies 1. OAuth - "Valet key" for internet applications 2. i18n - Using templates and message bundles in web applications 3. AppStats - App Engine request profiling toolkit 4. Task Queues - Background work for App Engine applications
  34. 34. Questions? Application: http://tweetengine.net/ Code: http://github.com/Arachnid/tweetengine ikai.l@google.com Twitter: @ikai
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×