Rapid web development using tornado web and mongodb

Rapid Web Development with
Tornado Web and MongoDB
Ikai Lan
Twitter: @ikai

Friday, June 10, 2011

About the speaker

• Developer Relations at Google
• Software Engineering background
• Lives in San Francisco, CA
• Writes Java, Python on a day to day basis
• Twitter: @ikai


This talk

• Why Tornado Web and MongoDB?
• Whirlwind tour of Tornado (get it?!)
• Intro to MongoDB
• Brief look at a demo app


What I won’t be talking
about

• Scalability - loaded topic
• Reliability - another loaded topic


So why tornado/
mongo?


Where does this joy
come from?


Freedom.

So really, my talk should
have been called “The
Joy of Tornado Web and
Mongo DB”


Intangibles

• Tornado - fast development, very few rules
• MongoDB - freedom from a predeﬁned
schema
• Python, of course. Dynamically typed
variables, duck typing plus everything else
that is cool about the language


Tornado Web history
• Open Sourced by Facebook
• ... formerly of FriendFeed
• ... former Googlers (that’s why the
framework looks like App Engine’s webapp
- they wrote it)
• Tornado Web powers FriendFeed


A whirlwind tour

Tornado features
• Out of the box auth with Twitter,
Facebook, Google and FriendFeed
• Out of box support for localized strings
• Simple templating that allows arbitrary
Python code
• Asynchronous requests
• Small codebase

Simple handlers
import tornado.ioloop
import tornado.web

class HelloHandler(tornado.web.RequestHandler):
def get(self):
self.write("Hello, world :)")

class GoodbyeHandler(tornado.web.RequestHandler):
def get(self):
self.render("goodbye.html",
message=”Goodbye, world”)

application = tornado.web.Application([
(r"/hello", HelloHandler),
(r"/goodbye", GoodbyeHandler),
])

if __name__ == "__main__":
application.listen(8888)
tornado.ioloop.IOLoop.instance().start()


Asynchronous nature
class MainHandler(tornado.web.RequestHandler):

@tornado.web.asynchronous
def get(self):
http = tornado.httpclient.AsyncHTTPClient()
http.fetch("http://friendfeed-api.com/v2/feed/bret",
callback=self.on_response)

def on_response(self, response):
if response.error: raise tornado.web.HTTPError(500)
json = tornado.escape.json_decode(response.body)
self.write("Fetched " + str(len(json["entries"])) + " entries "
"from the FriendFeed API")
self.finish()


Unrestrictive templating
{% for student in [p for p in people if p.student and p.age > 23] %}
<li>{{ escape(student.name) }}</li>
{% end %}

# Sample of arbitrary functions in templates
def add(x, y):
return x + y
template.execute(add=add)

### The template
{{ add(1, 2) }}


Lies, damn lies and benchmarks


Saving data

MongoDB is an object
database

• No schema - anything that can be a JSON
object can be stored
• You can deﬁne new collections on the ﬂy


Getting started with
Mongo in 4 steps
• Download a binary for your system
• Create a data directory (mkdir -p /data/db)
• Run mongod
• Install pymongo
• ... start writing code!


Pymongo code sample
connection = pymongo.Connection()
database = connection["your_database"]

def get_or_create_location_by_id(location_id):
"""
Attempts to fetch location data from database. If it doesn't exist,
create it. Note that this is NOT concurrency safe.
"""
location_data = database["locations"].find_one({ "_id" : location_id })
if location_data is None:
location_data = { "_id" : location_id,
"guards": [],
"owner" : None,
"history" : [],
"last_extort_time" : None
}
database.location.save(location_data, safe=True)
return location_data


Pymongo queries
# Add Tyson as a guard to every location owned by “Joe Smith”
locations = database["locations"].find({ "owner" : "Joe Smith" })
for location in locations:
location["guards"].append("Tyson")
database["locations"].save(location)

# Find everyone who has Tyson as a guard
locations = database["locations"].find({"guards" : "Tyson"})

# Find everyone who has *ONLY* Tyson as a guard
locations = database["locations"].find({"guards" : ["Tyson"]}) # Note []s

# Find everyone who has Tyson as a guard whose owner is “Ikai Lan”
locations = database["locations"].find({"guards" : "Tyson",
"owner" : "Ikai Lan" })


Mold as you go along

Freedom from

• ... waiting to get started
• ... predeﬁning a schema
• ... worrying about complex data structures
that don’t ﬁt well into the SQL box.
Anything that is JSON can be stored!


More MongoDB
features
• Simple Geospatial indexing
• GridFS - distributed ﬁlesystem running on
MongoDB
• Out of the box mapreduce
• Out of the box replication, data partitioning


Putting it all together
• MobSquare, a location based game that
uses the Facebook API.
• Mashup: Maﬁa Wars + FourSquare
• https://github.com/ikai/mobsquare-demo
• “Check in” to locations, take them over,
extort money and ﬁght other gangs


Why are Tornado/
Mongo a ﬁt?
• If I haven’t said it enough yet, they’re fun
• Facebook API performance varies -
asynchronous so we can serve requests
while waiting
• Highly structured player data


OAuth 2 upgrade ﬂow
class OnLoginHandler(tornado.web.RequestHandler):

@tornado.web.asynchronous
def get(self):
# Store this somewhere
code = self.get_argument("code")
access_token_url = ACCESS_TOKEN_URL_TPL + code
client = httpclient.AsyncHTTPClient()
client.fetch(access_token_url, self.on_fetched_token)

def on_fetched_token(self, response):
""" Callback inokved when the auth_token is fetched """
matches = ACCESS_TOKEN_REGEX.search(response.body)
if matches:
access_token = matches.group(1)
client = httpclient.AsyncHTTPClient()
# lambda is effectively a function factory for us
client.fetch(API["profile"] % access_token,
lambda response: self.on_profile_fetch(response, access_token))

def on_profile_fetch(self, response, access_token):
""" Callback invoked when we have fetched the user's profile """
profile = json.loads(response.body)
profile["access_token"] = access_token
profile_id = db.save_profile(profile)
self.set_secure_cookie("user_id", str(profile_id))
self.redirect("/") # implictly calls self.finish()


Known gotchas

• Immaturity of frameworks, tools
• Tornado templating errors result in
somewhat useless stack traces
• MongoDB nuances
• Tornado community fairly small


Questions?

• Ikai Lan - @ikai on Twitter
• Github: https://github.com/ikai
• Google Proﬁle: https://proﬁles.google.com/
ikai.lan/about
• Thank you for having me at Pycon!


Attributions

• Slide 6: Kirstin Jennings - “big smile!” http://www.flickr.com/photos/methyl_lives/2973265796/

• Slide 8: Tony Blay - “I am a bird now!” http://www.flickr.com/photos/toniblay/59415205/

• Slide 12: Antonio Sofi - “whirlwind” http://www.flickr.com/photos/webgol/36546734

• Slide 17: Tornado Web documentation - http://www.tornadoweb.org/documentation

• Slide 18: Christine of Robot Brainz - “Cassette” http://www.flickr.com/photos/robotbrainz/
2786347158/

• Slide 23: Samuel Stroube - “Play-Doh on a Picnic Table” http://www.flickr.com/photos/samoube/
203586664


Rapid web development using tornado web and mongodb

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (7)

Similar to Rapid web development using tornado web and mongodb

Similar to Rapid web development using tornado web and mongodb (20)

More from ikailan

More from ikailan (14)

Recently uploaded

Recently uploaded (20)

Rapid web development using tornado web and mongodb