Google AppEngine as a perfect mashup platform

1,526 views
1,390 views

Published on

My talk at budapest.py meetup, July 2009. See http://www.meetup.com/budapest-py/ for the details.

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,526
On SlideShare
0
From Embeds
0
Number of Embeds
24
Actions
Shares
0
Downloads
0
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Google AppEngine as a perfect mashup platform

  1. 1. budapest.py Google Appengine as a perfect mashup platform Janos Hardi janos@liligo.com http://github.com/yaanno
  2. 2. What is a mashup? Some original idea implemented on the top of various 3rd party web services by mixing them.
  3. 3. What do you need for a good mashup? A smashing idea Technical background 3rd party services
  4. 4. An example A good idea (me thinks): rd A travel search bot based on a 3 party engine plus Twitter (+ Jabber) messaging. Search types: flight, hotel, inspiration, etc.
  5. 5. Sideslide Twitter bots: http://twitter.pbworks.com/Bots Twitter mashups: http://twitter.pbworks.com/Mashups
  6. 6. Google AppEngine Technical background: Database (datastore) Cache (memcache) 3rd party apis via HTTP (urlfetch) ... and many many more ...
  7. 7. 3rd party web services Good starting points: programmableweb.com (apis) mashable.com (news)
  8. 8. Mashup ingredients Twitter API Jabber API Yahoo Trip Planner API Travel Search APIs
  9. 9. Travel Search APIk Kayak Liligo Cleartrip
  10. 10. Kayak Travel Search API - Kayak infrastructure - Slightly complicated data exchange - XML based - Poor documentation - Discontinued! Now you have “RSS APIs” ...
  11. 11. Liligo Travel Search API - Liligo infrastructure - Complicated setup procedure - XML based - Good documentation
  12. 12. Cleartrip Travel Search API - Cleartrip infrastructure - Dozens of services - Simple data exchange - XML based - Excellent documentation - From India with love
  13. 13. Yahoo! API - Stone simple - XML based - Good documentation
  14. 14. Twitter API - Simple as stone - XML/JSON based - Good documentation
  15. 15. Google (Data) APIs Loads of stuff, amazing. http://code.google.com/p/gdata-python-client/
  16. 16. Now what? What we have is ether nothing or almost useless.
  17. 17. Demo http://tripideas-bot.appspot.com Dumped (partially) :(
  18. 18. Web based bot? Cron service calls /cron resource /cron sets up default tasks Task manager processes these tasks (queue api)
  19. 19. Default tasks - Do we have new messages? - Do we have new searches stored? - Do we have unreplied messages? ... and a lot more ...
  20. 20. How? Apparently we don't have parallel / async events, processes and so on but we have a nice Task Queue api.
  21. 21. Task Queue class MessageHandler(webapp.RequestHandler): def get(self): task_type = self.request.get('type') queue = Queue(name='messaging') if task_type == 'collect': task = Task( url='/worker/collect_messages', method='GET', ) queue.add(task)
  22. 22. Task Queue class SearchHandler(webapp.RequestHandler): def get(self): task = Task( url = '/worker/search', params = { 'engine' : 'yahoo', 'searchtype' : 'inspire', 'query' : query['query'], 'maxresults' : '30', 'request_id' : query['request_id'], }, method = 'GET', ) queue.add(task)
  23. 23. Message Worker import logging from worker import Worker from .modules.messaging import Messaging class MessageWorker(Worker): def __init__(self): logging.debug('new messaging worker spawned') def get(self): messaging = Messaging() messaging.collect() messaging.process() messaging.save()
  24. 24. Search Worker import logging from worker import Worker from .modules.search import Search class SearchWorker(Worker): def __init__(self): logging.debug('new search worker spawned') def get(self): request = self.request.query_string query = self.normalize(request) search = Search(query) raw_result = search.search()
  25. 25. Processing #1 def collect(self): self.messages = t.status_replies() def process(self): self.messages = self._serialize(self.messages) def _serialize(self,serializable): return json.loads(serializable)
  26. 26. Processing #2 import xml.etree.ElementTree as ET xml = ET.fromstring(self.result) trips = xml.findall("trips/trip") for trip in trips: id = trip.get("id") priceelem = trip.find("price") price = priceelem.text url = priceelem.get("url")
  27. 27. Page render & Cache class UserPage(Page): def get(self,*args): userpageid = str('userpage_' + args[0]) page = memcache.get(userpageid) if page is None: userid = long(args[0]) author = Author.all().filter('author_id', userid).get() messages = Message.all().filter('author_id', author.key().id()).fetch(20) results = { 'author': author, 'results':[] } page = self.prerender('user_page', {'results':results}) memcache.put(page) self.response.out.write(page)
  28. 28. Jabber on Appengine Natively http://googleappengine.blogspot.com/2009/02/roadm External service http://morethanseven.net/2009/02/21/example-using PyGtalkRobot based http://code.google.com/p/aiassistant/
  29. 29. Thanks for your time!

×