Scaling an API

611 views

Published on

Scaling an API. A masterclass given at Seedcamp London product day 2012

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
611
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
0
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide
  • Good Afternoon big lunch, not put you sleep share my experiences So from reboots, To Redis, still works my journey Not how make Not why should But What happens when BIG
  • So who I am? Ciaran, CTO Skimlinks. former seedcamp Content monetization company service ordinated architecture Internal and external APIs why me? Numbers!
  • 14 Billion API requests 4 Billion Page views 145 Million Unique users monetize 1million Websites 26 thousand customers didn’t start like that
  • Started Skimbit.com Rebranded skimit.com social decision making and bookmarking tool Launched 2007 topic create a page
  • bookmark items brides maids shoes friends and family comments / rating bit like another site
  • But we’ll come back to that site in a bit
  • going ok steady base of users ticking along nicely something BIG happened
  • We got Techcrunched!! Website didn’t go down So …
  • Partied! cut short then something else happened
  • hosted at FlexiScale anyone? No? Small, local, cloud CEO’s number on speed dial Everything we need
  • App very straight forward It looked like this Few still like this? node.js on solar powered raspberry pi’s Ours & we were happy with it one day this happened ->
  • three day outage to the hour wasn’t our fault Well really it was
  • the cloud floated away human error deleted main storage restore via remount Not enough hardware ship from Germany Didn’t have a backup plan
  • We panicked! CEO’s number no answer
  • Our app is simple Just deploy Not that easy We thought But…
  • DNS servers were OUR server Email records on OUR Server Even customer uploaded images on it This is how NOT to do it
  • DNS was the hardest 24 Hour TTL on NS records
  • Email right after Already using Google Apps 30 min TTL on MX Once DNS back ..
  • Finally Back up images to S3
  • Shortly after this We started October 2008
  • Well more like this Fist customer live Nov 10 2008
  • We business means new website Nov 2008
  • New hosting company The Planet Real dedicated servers No cloud float Backward? Sensible
  • New App New architecture Half dozen servers
  • 2 API boxes Mostly read Writes local
  • Master slave DB MySQL replication Batch writes to master
  • Finally client app Reports Website Etc Last mention
  • Simple Unbreakable
  • Growth over the next few years Interesting events
  • Signed Wordpress
  • Pinterest took off!
  • Zoom in Monetizable events More insight
  • BF 2009
  • BF 2010
  • BF 2011
  • Remember our site
  • Hosting = planet Real severs Never float
  • Archictecure = this Never could break
  • Stopped recording Important lesson Only record what you need Later we did record
  • International traffic growing 40% = USA 20% = UK 10% = France 30% =others like germany canada
  • Started to improve New Website easly 2010
  • New product Skimwords Analyses content Creates links
  • Great thing New products go on new servers Eventually all on AWS This where start talking about the right way
  • Created API boxen Apache PHP Memcached
  • Scaled out API Added ELB Backed by MYSQL Skimwords = MongoDB
  • Amazon regions Availabily zones Mysql Master slave Mongo Master Slave Mongo 1.6 Ultra DNS = GLB Failover Short TTL
  • It worked Latency dropped huge
  • World Graphs Blue = europe Green = us east Red = us west
  • We were where BF ‘10
  • And this was coming Pinterest over 1 year Growing fast
  • First step New website Early 2011
  • We bought Atma
  • And did some research Bake off Apache Worker = Not thread safe PHP Drivers Mongo at the time
  • Picked Python Why not scala? Many reasons Team skills
  • New Jay Box Jay = Json Ngix PHP/Apache Tornado / Python Flume for logs+events
  • Make a Cluster Jays ELB Mysql Memcached Flume C Tree Filter + Redis
  • Again International Clusters Balance with Ultra DNS Mysql Master Slave Flume -> S3 Hadoop Hbase
  • Other tools Puppet Jenkins-ci / RMP’s / Hubot Fastly Api Axle
  • Monitoring Local Cacti Icinga aka nagios Pindom & Padgerduty
  • NOC Always watching
  • Remember numbers
  • Good Afternoon big lunch, not put you sleep share my experiences So from reboots, To Redis, still works my journey Not how make Not why should But What happens when BIG
  • My Details Hope you got something from this We learned a lot Hope you can apply to your business here for the rest of day Hope you all get a lot out of the rest of the week
  • Scaling an API

    1. 1. Scaling an APIFrom Reboots to Redis, how (not) to do it
    2. 2. { "fullName": "Ciarán Rooney", "workLife": { "company": "Skimlinks", "position": "CTO", "website": "http://skimlinks.com" }, "homeLife": { "email": "ciaran.rooney@gmail.com", "twitter": "@CiaranR", "website": "http://ciaran.ie" "hobby": "http://phplondon.org" }}
    3. 3. { "dataYear": "2012", "dataMonth": "august", "dataPeriod": "31 days", "dataRequests": { "apiRequests": "14,256,654,541", "impressions": "4,073,142,857", "uniqueUsers": "145,137,679" }, "customers": { "websitesPowered": "987,348", "customers": "26,379", }}
    4. 4. BIND / DNS eMailimageStorage
    5. 5. eMailimageStorage
    6. 6. imageStorage
    7. 7. APIs
    8. 8. DatabaseMaster/Slave
    9. 9. Client App
    10. 10. Client AppDatabaseMaster/SlaveAPIs
    11. 11. WordPress
    12. 12. WordPress Pinterest
    13. 13. Black Friday 09
    14. 14. Black Friday 09 Black Friday 10
    15. 15. Black Friday 09 Black Friday 11 Black Friday 10
    16. 16. Client AppDatabaseMaster/SlaveAPIs
    17. 17. WordPress
    18. 18. SkimWords ™
    19. 19. APIxx
    20. 20. CLUSTERxxAPIxx APIxx APIxx APIxx APIxx APIxx APIxx APIxx ELB ELB
    21. 21. US-WEST US-EAST EU-WESTCLUSTERxx CLUSTERxx CLUSTERxx CLUSTERxx CLUSTERxx
    22. 22. Black Friday 10
    23. 23. Pinterest
    24. 24. JAYxxApache Flume
    25. 25. CLUSTERxx Apache Flume Collector C Prefix Tree C NLP FilteringJAYxx JAYxx JAYxx JAYxx JAYxx JAYxx JAYxx JAYxx ELB ELB
    26. 26. US-WEST US-EAST EU-WEST Apache Flume MasterCLUSTERxx CLUSTERxx CLUSTERxx CLUSTERxx CLUSTERxx
    27. 27. Cacti
    28. 28. { "dataYear": "2012", "dataMonth": "august", "dataPeriod": "31 days", "dataRequests": { "apiRequests": "14,256,654,541", "impressions": "4,073,142,857", "uniqueUsers": "145,137,679" }, "customers": { "websitesPowered": "987,348", "customers": "26,379", }}
    29. 29. Scaling an APIFrom Reboots to Redis, how (not) to do it Questions?
    30. 30. { "fullName": "Ciarán Rooney", "workLife": { "company": "Skimlinks", "position": "CTO", "website": "http://skimlinks.com" }, "homeLife": { "email": "ciaran.rooney@gmail.com", "twitter": "@CiaranR", "website": "http://ciaran.ie" "hobby": "http://phplondon.org" }}

    ×