• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Google AppEngine as a perfect mashup platform

Google AppEngine as a perfect mashup platform



My talk at budapest.py meetup, July 2009. See http://www.meetup.com/budapest-py/ for the details.

My talk at budapest.py meetup, July 2009. See http://www.meetup.com/budapest-py/ for the details.



Total Views
Views on SlideShare
Embed Views



2 Embeds 8

http://www.slideshare.net 6
http://www.linkedin.com 2



Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

    Google AppEngine as a perfect mashup platform Google AppEngine as a perfect mashup platform Presentation Transcript

    • budapest.py Google Appengine as a perfect mashup platform Janos Hardi janos@liligo.com http://github.com/yaanno
    • What is a mashup? Some original idea implemented on the top of various 3rd party web services by mixing them.
    • What do you need for a good mashup? A smashing idea Technical background 3rd party services
    • An example A good idea (me thinks): rd A travel search bot based on a 3 party engine plus Twitter (+ Jabber) messaging. Search types: flight, hotel, inspiration, etc.
    • Sideslide Twitter bots: http://twitter.pbworks.com/Bots Twitter mashups: http://twitter.pbworks.com/Mashups
    • Google AppEngine Technical background: Database (datastore) Cache (memcache) 3rd party apis via HTTP (urlfetch) ... and many many more ...
    • 3rd party web services Good starting points: programmableweb.com (apis) mashable.com (news)
    • Mashup ingredients Twitter API Jabber API Yahoo Trip Planner API Travel Search APIs
    • Travel Search APIk Kayak Liligo Cleartrip
    • Kayak Travel Search API - Kayak infrastructure - Slightly complicated data exchange - XML based - Poor documentation - Discontinued! Now you have “RSS APIs” ...
    • Liligo Travel Search API - Liligo infrastructure - Complicated setup procedure - XML based - Good documentation
    • Cleartrip Travel Search API - Cleartrip infrastructure - Dozens of services - Simple data exchange - XML based - Excellent documentation - From India with love
    • Yahoo! API - Stone simple - XML based - Good documentation
    • Twitter API - Simple as stone - XML/JSON based - Good documentation
    • Google (Data) APIs Loads of stuff, amazing. http://code.google.com/p/gdata-python-client/
    • Now what? What we have is ether nothing or almost useless.
    • Demo http://tripideas-bot.appspot.com Dumped (partially) :(
    • Web based bot? Cron service calls /cron resource /cron sets up default tasks Task manager processes these tasks (queue api)
    • Default tasks - Do we have new messages? - Do we have new searches stored? - Do we have unreplied messages? ... and a lot more ...
    • How? Apparently we don't have parallel / async events, processes and so on but we have a nice Task Queue api.
    • Task Queue class MessageHandler(webapp.RequestHandler): def get(self): task_type = self.request.get('type') queue = Queue(name='messaging') if task_type == 'collect': task = Task( url='/worker/collect_messages', method='GET', ) queue.add(task)
    • Task Queue class SearchHandler(webapp.RequestHandler): def get(self): task = Task( url = '/worker/search', params = { 'engine' : 'yahoo', 'searchtype' : 'inspire', 'query' : query['query'], 'maxresults' : '30', 'request_id' : query['request_id'], }, method = 'GET', ) queue.add(task)
    • Message Worker import logging from worker import Worker from .modules.messaging import Messaging class MessageWorker(Worker): def __init__(self): logging.debug('new messaging worker spawned') def get(self): messaging = Messaging() messaging.collect() messaging.process() messaging.save()
    • Search Worker import logging from worker import Worker from .modules.search import Search class SearchWorker(Worker): def __init__(self): logging.debug('new search worker spawned') def get(self): request = self.request.query_string query = self.normalize(request) search = Search(query) raw_result = search.search()
    • Processing #1 def collect(self): self.messages = t.status_replies() def process(self): self.messages = self._serialize(self.messages) def _serialize(self,serializable): return json.loads(serializable)
    • Processing #2 import xml.etree.ElementTree as ET xml = ET.fromstring(self.result) trips = xml.findall("trips/trip") for trip in trips: id = trip.get("id") priceelem = trip.find("price") price = priceelem.text url = priceelem.get("url")
    • Page render & Cache class UserPage(Page): def get(self,*args): userpageid = str('userpage_' + args[0]) page = memcache.get(userpageid) if page is None: userid = long(args[0]) author = Author.all().filter('author_id', userid).get() messages = Message.all().filter('author_id', author.key().id()).fetch(20) results = { 'author': author, 'results':[] } page = self.prerender('user_page', {'results':results}) memcache.put(page) self.response.out.write(page)
    • Jabber on Appengine Natively http://googleappengine.blogspot.com/2009/02/roadm External service http://morethanseven.net/2009/02/21/example-using PyGtalkRobot based http://code.google.com/p/aiassistant/
    • Thanks for your time!