Rapidly Developing FarmVille<br />How we built and scaled a #1 Facebook game in 5 weeks<br />AmittMahajan<br />Lead Develo...
FarmVille<br />
5 week development cycle<br />FarmVille<br />6 developers (PHP/Flash)<br />2 artists<br />3 producers/designers<br />
At Launch<br />FarmVille<br />18K users/day after 24 hrs<br />1M users/day after 4 days<br />
Today<br />FarmVille<br />110M+ Installs <br />31M Players/Day<br />
My Goals<br />Reduce the amount of time it takes you to develop your games<br />Give you a blueprint for how to reliably s...
What slows down developers?<br />Other developers<br />Design / Art<br />Production (Copy)<br />Lack of knowledge<br />
Increasing Developer Efficiency<br />Flash (Client)<br />PHP (Server)<br />All developers know both<br />Design DOESN’T ru...
Data Driven Design<br />Shoot for a content pipeline that doesn’t need a developer<br />Data driven can be as easy as a de...
String Tables<br />A string table is an external file that holds strings for the app<br />Best practice to prepare for loc...
Abstracted Network Layer<br />Feature developers get client/server communication and serialization for free<br />Goal <br />
Abstracted Network Layer<br />PHP<br />Flash<br />Feature A<br />Feature B<br />Feature A Service<br />Feature B Service<b...
Network Layer Batching<br />No need to do the work twice<br />Reduced server load in FarmVille by 50%!<br />Server<br />Cl...
Network Layer Validation<br />Solves problem of unfiltered input<br />Ensures data is received by server in-order it was s...
Social Network Wrapper<br />Single place to perform Facebook API calls<br />Working with a highly dynamic API can be diffi...
Continuous Deployment<br />Build latest version of source-repo<br />Deploy to test “auto app” on Facebook to surface produ...
Continuous Deployment<br />FarmVille QA Process<br />Auto app<br />Staging<br />Production<br />Quick<br />Smoke<br />Full...
FarmVille Traffic Growth<br />1 Million DAUsevery week for 20 weeks<br />source: developeranalytics.com<br />
Scalable Server Architecture<br />We had no choice but to scale on the cloud<br />Every part of the server architecture sc...
Round Robin DNS<br />…<br />Load Balancer #1<br />Load Balancer #N<br />     Auto-scaling web      array (PHP)<br />…<br /...
All you need to implement is…<br />…<br />Web Server #M<br />Web Server #1<br />…everything else is off-the-shelf componen...
Reducing Load-times<br />Users want responsive pages<br />Show something immediately<br />Player’s won’t wait for your app...
Avoid Remote Calls<br />Remote server calls are slow and unreliable<br />Aim for no remote calls during load<br />Embed da...
Caching Slow Calls<br />Build in FacebookAPI caching within the social network wrapper<br />Write a DB wrapper to generate...
FarmVille Page Profiler<br />Catch and eliminate all un-cached Facebook and DB calls<br />Goal <br />
Fault Tolerance<br />Servers do go down randomly<br />Build redundancy on all levels of server architecture<br />Facebook ...
Runtime Config<br />How do we progressively keep the game running?<br />Create all features with kill-switches<br />Create...
Is it still running?<br />Notify your team when things break using server monitoring (Nagios/Munin)<br />What to watch<br ...
Stats Tracking<br />How do we know what users are seeing?<br />Have the client send back statistics<br />Number of errors<...
Before Launching<br />Get confidence that your stuff actually works<br />Perform load testing<br />Social games are a mara...
Questions?<br />
Thank You!<br />
Upcoming SlideShare
Loading in...5
×

Rapidly Building FarmVille: How we built and scaled a #1 Facebook game in 5 weeks (GDC 2010)

12,211

Published on

Amitt Mahajan discusses the techniques used in the development of the hit social game FarmVille. This presentation covers the processes used by the FarmVille development team to quickly release the game and then scale it to support millions of daily players.

(Originally presented at GDC 2010)

Published in: Technology
0 Comments
9 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
12,211
On Slideshare
0
From Embeds
0
Number of Embeds
8
Actions
Shares
0
Downloads
196
Comments
0
Likes
9
Embeds 0
No embeds

No notes for slide

Rapidly Building FarmVille: How we built and scaled a #1 Facebook game in 5 weeks (GDC 2010)

  1. 1. Rapidly Developing FarmVille<br />How we built and scaled a #1 Facebook game in 5 weeks<br />AmittMahajan<br />Lead Developer – FarmVille<br />March 9, 2010<br />
  2. 2. FarmVille<br />
  3. 3. 5 week development cycle<br />FarmVille<br />6 developers (PHP/Flash)<br />2 artists<br />3 producers/designers<br />
  4. 4. At Launch<br />FarmVille<br />18K users/day after 24 hrs<br />1M users/day after 4 days<br />
  5. 5. Today<br />FarmVille<br />110M+ Installs <br />31M Players/Day<br />
  6. 6. My Goals<br />Reduce the amount of time it takes you to develop your games<br />Give you a blueprint for how to reliably scale your games to millions without your servers falling over<br />
  7. 7. What slows down developers?<br />Other developers<br />Design / Art<br />Production (Copy)<br />Lack of knowledge<br />
  8. 8. Increasing Developer Efficiency<br />Flash (Client)<br />PHP (Server)<br />All developers know both<br />Design DOESN’T rule all<br />Developers co-own features with designers<br />
  9. 9. Data Driven Design<br />Shoot for a content pipeline that doesn’t need a developer<br />Data driven can be as easy as a designer-editable XML file<br />No devneeded!<br />
  10. 10. String Tables<br />A string table is an external file that holds strings for the app<br />Best practice to prepare for localization<br />Developers are not blocked by production<br />Allows quick response to Facebook TOS changes<br />
  11. 11. Abstracted Network Layer<br />Feature developers get client/server communication and serialization for free<br />Goal <br />
  12. 12. Abstracted Network Layer<br />PHP<br />Flash<br />Feature A<br />Feature B<br />Feature A Service<br />Feature B Service<br />Networked Actions<br />Action Dispatcher<br />Network Layer<br />Network Layer<br />Validation Checks<br />Validation Checks<br />AMF Protocol (RPC)<br />Action Batching<br />Result Batching<br />
  13. 13. Network Layer Batching<br />No need to do the work twice<br />Reduced server load in FarmVille by 50%!<br />Server<br />Client<br />Only One Request<br />onPlow()<br />Plow<br />Plow<br />onPlow()<br />Plant<br />Plant<br />onPlant()<br />Plow<br />…<br />
  14. 14. Network Layer Validation<br />Solves problem of unfiltered input<br />Ensures data is received by server in-order it was sent<br />
  15. 15. Social Network Wrapper<br />Single place to perform Facebook API calls<br />Working with a highly dynamic API can be difficult<br />Abstracting FB calls makes them easier to integrate<br />Allows for cross-platform games<br />
  16. 16. Continuous Deployment<br />Build latest version of source-repo<br />Deploy to test “auto app” on Facebook to surface production issues early<br />
  17. 17. Continuous Deployment<br />FarmVille QA Process<br />Auto app<br />Staging<br />Production<br />Quick<br />Smoke<br />Full Test<br />Pass<br />
  18. 18. FarmVille Traffic Growth<br />1 Million DAUsevery week for 20 weeks<br />source: developeranalytics.com<br />
  19. 19. Scalable Server Architecture<br />We had no choice but to scale on the cloud<br />Every part of the server architecture scales horizontally<br />No single points of failure<br />Take the DB out of the equation<br />
  20. 20. Round Robin DNS<br />…<br />Load Balancer #1<br />Load Balancer #N<br /> Auto-scaling web array (PHP)<br />…<br />Web Server #M<br />Web Server #1<br />User Data <br />& Updates<br />Memcache Pool<br />Lazy Writes<br />DB Layer<br />Data<br />Reads<br />…<br />DB-#K<br />Master<br />DB-#1<br />Master<br />…<br />DB-#K<br />Slave<br />DB-#1<br />Slave<br />
  21. 21. All you need to implement is…<br />…<br />Web Server #M<br />Web Server #1<br />…everything else is off-the-shelf components!<br />…<br />
  22. 22. Reducing Load-times<br />Users want responsive pages<br />Show something immediately<br />Player’s won’t wait for your app to load<br />Stream non-critical content<br />
  23. 23. Avoid Remote Calls<br />Remote server calls are slow and unreliable<br />Aim for no remote calls during load<br />Embed data into Javascript<br />Facebook<br />Iframe<br />Inline-JS here<br />Flash ClientEmbed<br />
  24. 24. Caching Slow Calls<br />Build in FacebookAPI caching within the social network wrapper<br />Write a DB wrapper to generate and cache SQL<br />
  25. 25. FarmVille Page Profiler<br />Catch and eliminate all un-cached Facebook and DB calls<br />Goal <br />
  26. 26. Fault Tolerance<br />Servers do go down randomly<br />Build redundancy on all levels of server architecture<br />Facebook is a dependency also<br />Aim for DB-less & Facebook-less modes<br />“Defcon”-style error management<br />
  27. 27. Runtime Config<br />How do we progressively keep the game running?<br />Create all features with kill-switches<br />Create a web dashboard to allow non-technical folks to help out<br />
  28. 28. Is it still running?<br />Notify your team when things break using server monitoring (Nagios/Munin)<br />What to watch<br />Server load/traffic graphs<br />Memcacheevictions<br />
  29. 29. Stats Tracking<br />How do we know what users are seeing?<br />Have the client send back statistics<br />Number of errors<br />Load-times<br />Take a metrics driven approach to error handling<br />
  30. 30. Before Launching<br />Get confidence that your stuff actually works<br />Perform load testing<br />Social games are a marathon not a sprint<br />Sleep before launching!<br />
  31. 31. Questions?<br />
  32. 32. Thank You!<br />
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×