Deploying the Graph

513 views

Published on

A talk given in November 2013 to the infrastructure coders group in Melbourne on how PlayUp deploys the backend for it's latest generation of applications, using packer and amazon web services.

  • Be the first to comment

  • Be the first to like this

Deploying the Graph

  1. 1. Deploying the Graph November 2013 Kevin O’Neill CTO PlayUp @kevinoneill Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  2. 2. Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  3. 3. Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  4. 4. Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  5. 5. Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  6. 6. A little History Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  7. 7. Engage & Entertain Around Live Sport Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  8. 8. Mobile Games Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  9. 9. SMS Based Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  10. 10. Number of Runs Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  11. 11. Complex Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  12. 12. Carrier Integration Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  13. 13. Smart Phone Revolution Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  14. 14. Move from a Content Creator to a Content Enabler Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  15. 15. Top 10 in Engagement Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  16. 16. Over 3 Million Likes Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  17. 17. 30% - 40% Active Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  18. 18. Manually Curated Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  19. 19. We Know there's an Opportunity Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  20. 20. The Graph Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  21. 21. Scala Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  22. 22. Scala • • • • • A Scalable language Object-Oriented Functional Seamless Java Interoperability Functions are Objects Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  23. 23. AKKA Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  24. 24. AKKA • • • • Actors Fault Tolerance Location Transparency Transactors Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  25. 25. Neo4j Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  26. 26. Neo4j • • • • Graph Database Vertices and Edges Properties Full Text Index Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  27. 27. Whiteboard Outlining the bits Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  28. 28. Runtime Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  29. 29. AWS Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  30. 30. AWS • • • • us-west m2.2xlarge ubuntu 13.04 jre 7 Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  31. 31. JVM Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  32. 32. JVM • • • Graph - 8G of Heap Others 128M - 1G Parallel GC Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  33. 33. The remaining memory is reserved for file system cache and memory mapped buffers Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  34. 34. Building the Components Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  35. 35. SBT Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  36. 36. Builds the jars Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  37. 37. Debian Packages Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  38. 38. Debian Packages • • • Places binaries in opt Adds an upstart file for the Permissions on data and log directories Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  39. 39. graph-server.upstart setuid  ubuntu   setgid  ubuntu   ! console  log   ! chdir  /var/lib/playup/spot/graph-­‐server   ! exec  java  -­‐server  -­‐Xms4096M  -­‐Xmx8192M  -­‐Xss1M        -­‐XX:MaxPermSize=256M  -­‐XX:+UseParallelGC        -­‐jar  /opt/playup/spot/graph-­‐server/run.jar        com.playup.spot.graph.service.NeoKernel Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  40. 40. Base configuration is for production Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  41. 41. Baking Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  42. 42. Packer Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  43. 43. Create machine images for multiple platforms from a single config Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  44. 44. Images have lots of benefits Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  45. 45. Fast Deployment Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  46. 46. Stability Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  47. 47. Testability Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  48. 48. Packer makes creating images a breeze Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  49. 49. Example Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  50. 50. Builders "builders":  [          {              "type":  "amazon-­‐ebs",              "instance_type":  "t1.micro",              "region":  "ap-­‐southeast-­‐2",              "ami_name":  "spot-­‐graph-­‐{{user  `version`}}-­‐{{timestamp}}",              "source_ami":  "ami-­‐4bf26f71",              "ssh_username":  "ubuntu",              "tags":  {                  "Name":  "spot-­‐graph-­‐{{user  `version`}}  ({{isotime}})",                  "OS":  "Ubuntu",                  "Version":  "{{user  `version`}}-­‐{{timestamp}}"              },              "ami_regions":["us-­‐west-­‐2"]          }  … Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  51. 51. Builders "builders":  [          {              "type":  "amazon-­‐ebs",              "instance_type":  "t1.micro",              "region":  "ap-­‐southeast-­‐2",              "ami_name":  "spot-­‐graph-­‐{{user  `version`}}-­‐{{timestamp}}",              "source_ami":  "ami-­‐4bf26f71",              "ssh_username":  "ubuntu",              "tags":  {                  "Name":  "spot-­‐graph-­‐{{user  `version`}}  ({{isotime}})",                  "OS":  "Ubuntu",                  "Version":  "{{user  `version`}}-­‐{{timestamp}}"              },              "ami_regions":["us-­‐west-­‐2"]          }  … Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  52. 52. Builders "builders":  [          {              "type":  "amazon-­‐ebs",              "instance_type":  "t1.micro",              "region":  "ap-­‐southeast-­‐2",              "ami_name":  "spot-­‐graph-­‐{{user  `version`}}-­‐{{timestamp}}",              "source_ami":  "ami-­‐4bf26f71",              "ssh_username":  "ubuntu",              "tags":  {                  "Name":  "spot-­‐graph-­‐{{user  `version`}}  ({{isotime}})",                  "OS":  "Ubuntu",                  "Version":  "{{user  `version`}}-­‐{{timestamp}}"              },              "ami_regions":["us-­‐west-­‐2"]          }  … Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  53. 53. Builders "builders":  [          {              "type":  "amazon-­‐ebs",              "instance_type":  "t1.micro",              "region":  "ap-­‐southeast-­‐2",              "ami_name":  "spot-­‐graph-­‐{{user  `version`}}-­‐{{timestamp}}",              "source_ami":  "ami-­‐4bf26f71",              "ssh_username":  "ubuntu",              "tags":  {                  "Name":  "spot-­‐graph-­‐{{user  `version`}}  ({{isotime}})",                  "OS":  "Ubuntu",                  "Version":  "{{user  `version`}}-­‐{{timestamp}}"              },              "ami_regions":["us-­‐west-­‐2"]          }  … Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  54. 54. Builders "builders":  [          {              "type":  "amazon-­‐ebs",              "instance_type":  "t1.micro",              "region":  "ap-­‐southeast-­‐2",              "ami_name":  "spot-­‐graph-­‐{{user  `version`}}-­‐{{timestamp}}",              "source_ami":  "ami-­‐4bf26f71",              "ssh_username":  "ubuntu",              "tags":  {                  "Name":  "spot-­‐graph-­‐{{user  `version`}}  ({{isotime}})",                  "OS":  "Ubuntu",                  "Version":  "{{user  `version`}}-­‐{{timestamp}}"              },              "ami_regions":["us-­‐west-­‐2"]          }  … Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  55. 55. Variables    "variables":  {          "version":  "development"      }   Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  56. 56. Provisioners    "provisioners":  [        {              "script":  "scripts/update-­‐packages.sh",              "type":  "shell"          },          {              "script":  "scripts/install-­‐puppet.sh",              "type":  "shell"          },          {              "type":  "file",              "source":  "packages",              "destination":  "/tmp"          },          {              "script":  "scripts/install-­‐spot-­‐packages.sh",              "type":  "shell"          },          {              "type":  "puppet-­‐masterless",              "manifest_file":  "puppet/spot-­‐graph.pp",              "module_paths":  ["puppet/local",  "puppet/common"]          },          {              "script":  "scripts/build-­‐date.sh",              "type":  "shell"          }     Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  57. 57. Provisioners    "provisioners":  [        {              "script":  "scripts/update-­‐packages.sh",              "type":  "shell"          },          {              "script":  "scripts/install-­‐puppet.sh",              "type":  "shell"          },          {              "type":  "file",              "source":  "packages",              "destination":  "/tmp"          },          {              "script":  "scripts/install-­‐spot-­‐packages.sh",              "type":  "shell"          },          {              "type":  "puppet-­‐masterless",              "manifest_file":  "puppet/spot-­‐graph.pp",              "module_paths":  ["puppet/local",  "puppet/common"]          },          {              "script":  "scripts/build-­‐date.sh",              "type":  "shell"          }     Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  58. 58. Provisioners    "provisioners":  [        {              "script":  "scripts/update-­‐packages.sh",              "type":  "shell"          },          {              "script":  "scripts/install-­‐puppet.sh",              "type":  "shell"          },          {              "type":  "file",              "source":  "packages",              "destination":  "/tmp"          },          {              "script":  "scripts/install-­‐spot-­‐packages.sh",              "type":  "shell"          },          {              "type":  "puppet-­‐masterless",              "manifest_file":  "puppet/spot-­‐graph.pp",              "module_paths":  ["puppet/local",  "puppet/common"]          },          {              "script":  "scripts/build-­‐date.sh",              "type":  "shell"          }     Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  59. 59. Provisioners    "provisioners":  [        {              "script":  "scripts/update-­‐packages.sh",              "type":  "shell"          },          {              "script":  "scripts/install-­‐puppet.sh",              "type":  "shell"          },          {              "type":  "file",              "source":  "packages",              "destination":  "/tmp"          },          {              "script":  "scripts/install-­‐spot-­‐packages.sh",              "type":  "shell"          },          {              "type":  "puppet-­‐masterless",              "manifest_file":  "puppet/spot-­‐graph.pp",              "module_paths":  ["puppet/local",  "puppet/common"]          },          {              "script":  "scripts/build-­‐date.sh",              "type":  "shell"          }     Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  60. 60. Builder Provisioners    "provisioners":  [   !   …   !        {              "type":  "shell",              "script":  "scripts/vagrant.sh",              "only":  ["virtualbox"]          },          {              "type":  "shell",              "script":  "scripts/guest-­‐additions.sh",              "only":  ["virtualbox"]          },   !   … Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  61. 61. Build $  packer  build  -­‐-­‐only  "amazon-­‐ebs"  template.json   Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  62. 62. In a few minutes you have a shiny new image Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  63. 63. CI Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  64. 64. CI • • • • Variables source ami region default build is for local development Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  65. 65. Starting an instance Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  66. 66. Remember Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  67. 67. Backed config is for Production* Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  68. 68. We have staging plus multiple developer environments Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  69. 69. CloudInit Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  70. 70. Configure values in /etc/playup/… Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  71. 71. Configure the puppet master* Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  72. 72. Cloud Formation Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  73. 73. Which images Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  74. 74. What CloudInit values Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  75. 75. Challenges Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  76. 76. Too many to list … Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  77. 77. Persistant Data Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  78. 78. Service Registration Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  79. 79. Cluster Membership Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  80. 80. Investigating Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  81. 81. Docker Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  82. 82. Apache Mesos Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  83. 83. Come Join Us Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  84. 84. Disrupting Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  85. 85. Hiring Selectively Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  86. 86. Talk to Me or Andrea Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  87. 87. You miss 100 percent of the the shots you don't take – Wayne Gretzky Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013
  88. 88. Questions? Kevin O’Neill CTO - PlayUp @kevinoneill kevin@playup.com Kevin O’Neill - CTO PlayUp - @kevinoneill Infrastructure Coders - Melbourne - November 2013

×