Your SlideShare is downloading. ×
At Scale With Style
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

×

Introducing the official SlideShare app

Stunning, full-screen experience for iPhone and Android

Text the download link to your phone

Standard text messaging rates apply

At Scale With Style

1,677
views

Published on

In the world of social gaming, the classic 2-tier of web application does not cut it anymore. We need new and better solutions. …

In the world of social gaming, the classic 2-tier of web application does not cut it anymore. We need new and better solutions.

Follow along the evolution of game servers at Wooga and get an in-depth look into the next-generation backend putting the combined forces of Erlang and Ruby to work. Learn how scalability, reliability, concurrency control and beautiful code do not need to be mutually exclusive.


0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,677
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
24
Comments
0
Likes
3
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
No notes for slide

Transcript

  • 1. AT  SCALE  WITH  STYLE Erlang  <3  <3  <3  RubyMar$n  Rehfeld,  @klickmich
  • 2. At  Scale  With  Style How  we  roll Server  architectures How  to  innovate Mind  the  limits
  • 3. Typical  game  architecture Client HTTP Backend API
  • 4. Typical  game  architecture Backend State  Changes ValidaHon Persistence
  • 5. The  scale  is  interesHng 14  billion  requests  /  month >100,000  DB  operaHons  /  second >50,000  DB  updates  /  second
  • 6. Wooga’s  approach  to  developmentSmall  independent  teams  for  each  gameTeam  gets  to  choose  tools  and  technologies Same  team  also  does  ops  a,er  going  liveCulture  of  sharing Look  around  what  is  there,  pick/adapt  exis;ng   solu;ons,  but  take  ownership  for  what  you   include
  • 7. ExisHng  backends  –  Technology  landscape
  • 8. At  Scale  With  Style How  we  roll Server  architectures How  to  innovate Mind  the  limits
  • 9. Most  games  use  stateless  applicaHon  servers Server Database
  • 10. And  then  there’s  sharding lb app app app app app app app app app My My SQL SQL slave slave
  • 11. More  app  servers,  more  sharding lb app app app app app app app app app app app app app app app app app app My My My My My My My My SQL SQL SQL SQL SQL SQL SQL SQL slave slave slave slave slave slave slave slave
  • 12. Wait,  seriously?! lb lbapp app app app app app app app app app app app appapp app app app app app app app app app app app appapp app app app app app app app app app app app appMy My My My My redis redis redis redis redisSQL SQL SQL SQL SQLslave slave slave slave slave slave slave slave slave slave
  • 13. Find  the  flaw “Stateless  applicaHon  servers   guarantee  one  thing: The  data  is  never where  you  need  it!” Paolo  Negri,  Developer  @  Wooga
  • 14. Strong  session  pa`ern User  starts  playing many  transformaHons of  the  same  set  of  data User  stops  playing
  • 15. Stateful  servers  and  DBs Server Database One  Game  Session
  • 16. Stateful  game  serverOne  process  per  acHve  user  gaming  session ...  holds  the  current  state  and  is  the  only  one  that  can   modify  it  (strong  encapsulaHon) ...  handles  all  API  calls  for  the  given  user  one  a,er  the  other   (concurrency  control  through  actor  model) ...  loads  the  game  state  from  storage  and  writes  it  back   periodically  and  on  process  termina;on (;meout  =  user  stopped  playing)
  • 17. The  DB  is  no  longer  the  bo`leneck Stateless Stateful 30.000 22.500 15.000 700 7.500 0 DB  operations  /  second
  • 18. Magic  Land  uses  ErlangDetails:Awesome  presentaHon  onthe  Magic  Land  game  serverby  @knuHn  &  @hungryblankh`p://www.slideshare.net/wooga/from-­‐0-­‐to-­‐1000000-­‐daily-­‐users-­‐with-­‐erlang
  • 19. At  Scale  With  Style How  we  roll Server  architectures How  to  innovate Mind  the  limits
  • 20. Erlang  &  RubyErlang  is  great Concurrent,  robust Great  for  opera;onRuby  is  great Concise,  expressive,  flexible Great  for  development
  • 21. Bringing  two  worlds  together Server Worker session sender Worker session Worker ... Worker receiver session Worker
  • 22. Do  you  know  Mongrel2? Web  Server  that  hooks up  applicaHons  via➡ We  chose  the  same  queue  setup  &   message  format Server Worker session sender Worker session Worker ... Worker receiver session Worker
  • 23. ConnecHng  the  dotsMongrel2  h`p://mongrel2.org/  protocol  &  ZeroMQ  setupErlang:  h`ps://github.com/hungryblank/emongrel2Ruby rack-­‐mongrel2  fork  hKps://github.com/khiltd/khi-­‐rack-­‐mongrel2 rack  protocol  hKp://rack.rubyforge.org Sinatra  hKp://www.sinatrarb.com/➡ essenHally  we  are  speaking  HTTP  over  ZeroMQ and  can  hook  up  any  Rack-­‐based  Ruby  web   framework
  • 24. Example  controller  in  Ruby app.game_action /:actor/fruit_tree/self/shake, :observable => true, :affects => [:fruit_trees, :user], :params => [:x, :y] do x, y = params[:x], params[:y] fruit_trees[x, y].shake endDSL-­‐like  definiHon  of  game  acHonSkinny  as  controllers  should  be
  • 25. Example  model  in  Ruby class FruitTree < Tree property :last_shake_time, :type => Integer, :default => 0 property :collectable_fruit_count, :type => Integer, :default => 0 def shake raise G8::Error::Validation, "no fruit!" unless carries_fruit? session.user.xp += 1 session.user.energy -= 1 self.last_shake_time = game_time self.collectable_fruit_count = config.fruit_count end endEasily  unit  testableMinimal  amount  of  code
  • 26. Game  stateGame  state  is  split  in  mulHple  parts user,  map,  fruit_trees  etc.Erlang  does  not  care  about  content Serialized  Ruby  objectsErlang  does  know  mapping  of  state  parts  to  URLs
  • 27. Looking  back  at  the  game  acHonapp.game_action /:actor/fruit_tree/self/shake, :observable => true, :affects => [:fruit_trees, :user], :params => [:x, :y] do x, y = params[:x], params[:y] fruit_trees[x, y].shakeendMapping  of  state  parts  to  game  acHons Worker  knows  mapping Worker  pushes  mapping  to  Erlang  on  startup Erlang  can  query  mapping  if  needed
  • 28. NICE! http://www.flickr.com/photos/aigle_dore/
  • 29. At  Scale  With  Style How  we  roll Server  architectures How  to  innovate Mind  the  limits
  • 30. Performance  impact  for  large  payloads 200  kB  payload 6000 5000Requests  /  s 4000 Pure 3000 Erlang 2000 Ruby 1000 ZMQ 400  rps 0 1 #  of  workers 8
  • 31. NOT  CPU  bound25% Fixed  bandwith  limit  @  300  MB/s
  • 32. This  has  happened  before2,000,000"1,500,000"1,000,000" 500,000" 0" Apr*10" Jul*10" Oct*10" Jan*11" Apr*11" Jul*11" Oct*11"
  • 33. Example  controller  in  Erlang shake(Args, State) -> Coords = g8_map:coords(Args), Map = g8_game:get_map(State), Tree = g8_map:find(Coords, Map), NewTree = g8_fruit_tree:shake(Tree), UserEffects = [incr_xp, decr_energy], NewMap = g8_map:place(Coords, NewTree, Map), [observable, {state, g8_game:set_map(NewMap, g8_user:apply(UserEffects, State))}].
  • 34. Example  model  in  Erlang shake(Tree) -> validate_carries_fruit(Tree), FruitCount = g8_fruit_tree_config:fruit_count(Tree), Tree#fruit_tree{last_shake_time = g8_game:gametime(), collectable_fruit_count = FruitCount}.
  • 35. ✓ Y ES
  • 36. QuesHons? MarHn  Rehfeld @klickmichslideshare.net/wooga wooga.com/jobs