2. What is The Bachelor?
1. The show
Reality dating game show
2. Competition
Contestants compete to be selected through elimination rounds.
3. Limited Participants
Only 25 participants in the TV show; a Facebook game lets the
audience play too!
3. The Development Team
Nicholas Asch
Alice Bevan-McGregor
Nicolas Cadou
Blaise Laflamme
Egor Miadzvedeu
Zachary Allatt
11. Architecture
Core API and Data Access
We keep them separate
Instead of littering the API with stuff like this
u = UserIO.one(user_id)
if not u:
topic = []
else:
topic = u['answers'].get(event_id, {}).get(str(topic_id), [])
We use object model methods
topic = UserEvent.get_topic_answers(user_id, event_id, topic_id)
The code then become easier to understand
def matches(self, user_id, event_id, topic_id):
""" Return a list of users whose answer is similar. """
topic = UserEvent.get_topic_answers(user_id, event_id, topic_id)
if topic is None:
return None
return UserEvent.matches(user_id, event_id, topic_id)
12. MongoDB
schema-less
non-relational
full of awesomeness!
Well-suited to building clusters.
Harder to break.
The State of MongoDB in Python
Choices for MongoDB in Python are limited to MongoEngine and MongoKit.
So we rolled our own on top of raw pymongo.
But then we were even more limited.
Switching to MongoEngine.
Really nice!
13. Migrations
NoSQL == no migrations, right?
WRONG!!
DB is schema-less
But for the sake of sanity, app shouldn't be
Data and content can change
Migration modes
In Python, on-the-fly
Migration scripts
Auto-migrate on deployment
14. Lettuce
Is lettuce a vegetable or a testing platform?!
Feature: Main show
In order ward off boredom and hopefully get laid
As a facebook user and bachelor wannabe
I want to take part in the best of the best
And that means playing the Bachelor game
# Actors:
# - B: Bachelor
# - C: Contestant
# - S: System
# - F: Friend
Scenario: B starts a show
Given B has enough points to start a show
When B initiates a new show
And B selects four of his pictures
And B starts the audition
Then B should loose the expected points to start a show
And B should not yet have access to the audition results
And B should not be able to start another show
Scenario: S selects the audition winners
Given the allotted audition time has elapsed
And 24 C or more have entered the audition
When S selects the 24 best matches
Then there should be 24 C in the show
And the lives of countless others shall be ruined
15. Lettuce
Lettuce steps map to Python code:
Scenario: C looses episode 1
Given C has lost episode 1
When C asks for episode 1 results
Then C should be informed that he failed the profound cuteness test
And C should gain the points earned by episode 1 losers
And C should should not have access to episode 2
@step(u'Then C should be informed that he failed the profound cuteness
test')
def
then_c_should_be_informed_that_he_failed_the_profound_cuteness_test(step):
w.golem.as_bachelor()
losers = w.golem.get('12_roses_losers', True)
for l in losers:
user = w.golem.user_api.get_user(l)
w.golem.as_contestant(user)
show = w.golem.show_api.get_by_id(w.golem.show_id)
is_winner = w.golem.show_api.collect_12roses(show=show)
assert not is_winner
16.
17. Facebook Integration
Facebook lets you take advantage of an existing account, their friends, and
social sharing. Great for marketting, no fun for developers.
# What's the purpose of this... there's no id when you ask to add the
# app so it raises the exception before a user can register
# ncadou: the purpose of this check is to prevent the creation of blank
# users when the facebook API actually fails. See commit 7bfef9df70f7.
#if not 'id' in user:
# # TODO log something
# raise Exception('Facebook API error')
if user.get('error', None):
return dict()
18. Facebook API Issues
Only the most critical ones...
Downtime
Speed (Latency in every request)
Sudden Changes
Documentation - Comparable to Early-Stage FOSS
19. How Your Public API Can Be
Better
K.I.S.S.
Namespaces are one honking great idea -- let's do
more of those!
Seriously, hierarchical organization is easy to understand and use.
Document every API call
Explain what it does and any nuances.
Document every variable
Type, possible options (and what each option means), and if its
required.
20. Mapping Game Logic to Objects
Multiplayer social games are step and role based.
21. System Tasks and Timers
Certain tasks have time limits or system requirements
def make_steps():
g = GameSteps()
# Episode 0 - Audition
g.add(bachelor=g.step('show_create', episode=0))
g.add(bachelor=g.step('audition_wait', has_url=False, episode=0,
can_skip=True),
contestant=g.step('audition_enter', episode=0))
g.add(contestant=g.step('audition_wait', can_skip=True, has_url=False,
episode=0))
g.add(system=g.step('select_contestants', has_url=False, is_stop=True,
timer=Tuneable().get_durations(0),
callback=callback.select_contestants))
g.add(bachelor=g.step('audition_result', label='Reveal Audition
Results',
episode=0),
contestant=g.step('audition_result', label='Reveal Audition
Results',
episode=0))
22. Worker Processes
MongoDB as RPC.
Needed for timed game events.
Needed because Facebook's API is slow.
Background task execution.
Immediate or scheduled.
Captures responses and exceptions.
Acts as a state machine with locking.
23. Worker Processes
MongoDB as RPC.
Stores job data in a collection.
Notifications via capped collection.
Uses Futures for parallel execution of tasks.
Uses APScheduler for timed execution of tasks.
Atomic locking using "update if current" mechanic.
24. Capped Collections
MongoDB as a low-latency queue.
Limited size, optionally limited document count.
Ring buffer design.
Insert order.
Updates allowed… mostly.
Used by MongoDB for replication.
Tailable cursors.
Long-poll push, like IMAP IDLE.
Live demonstration time!
25. Example Job Record
Stored in a permanant collection.
{
"_id" : ObjectId("4ea3717f9bfbb601d2000002"),
"state" : "new", // pending, dead, cancelled, running, finished
"callable" : "c__builtin__nprintnp1n.",
"args" : [ "Task", 0 ],
"kwargs" : { },
"created" : ISODate("2011-10-23T01:44:31.446Z"),
"creator" : [ "Lucifer", 298, 466 ],
"owner" : null, // [ "Lucifer", 324, 456 ]
// If scheduled, not immediate:
"when": ISODate("...")
// If in progress or completed...
"acquired" : ISODate("..."),
// If completed...
"result" : null,
"exception" : null,
"completed" : ISODate("..."),
}
27. Example Queue Runner
Python generators are teh win.
def queue(collection, query=None):
if not collection.find():
# This is to prevent a terrible infinite busy loop while empty.
collection.insert(dict(nop=True))
last = None
query = query or {}
cursor = collection.find(query, slave_ok=True, tailable=True,
await_data=True)
while True: # Primary retry loop.
try:
while cursor.alive: # Inner record loop; may time
out.
for record in cursor:
last = record['_id']
yield record
except OperationFailure:
pass
retry_query = {"_id": {"$gte": last}}
retry_query.update(query)
cursor = collection.find(retry_query, slave_ok=True,
tailable=True, await_data=True)
28. Example Job Handler
Job locking to prevent accidental execution.
def handler(self, job_id):
# Build the dictionary update.
update = dict(acquired=datetime.utcnow(), state="running",
owner=self.identity)
try:
result = self.jobs.update(dict(_id=job_id, state="pending",
owner=None),
{"$set": update}, safe=True)
except:
raise AcquireFailed()
if not result['updatedExisting']: raise AcquireFailed()
try:
job = self.jobs.find(dict(_id=job_id), limit=1,
fields=['callable', 'args', 'kwargs'])[0]
except: # This should, in theory, never happen unless MongoDB goes
away.
raise AcquireFailed()
obj = pickle.loads(job['callable'].encode('ascii'))
args = job.get('args', [])
kwargs = job.get('kwargs', {})
return obj(*args, **kwargs)
