SlideShare a Scribd company logo
1 of 59
Download to read offline
The problems with
eating your own
dog food

Phil Calçado
SoundCloud
Thursday, October 17, 13
Thursday, October 17, 13
>11 hours of
audio uploaded
every minute
Thursday, October 17, 13
~200
million users
every month
Thursday, October 17, 13
Thursday, October 17, 13
Thursday, October 17, 13
http://bit ly/tech-behind-new-twittercom
Thursday, October 17, 13
Thursday, October 17, 13
Just before
the beta

release
Thursday, October 17, 13
http://bit ly/improving-performance-twittercom
Thursday, October 17, 13
Thursday, October 17, 13
Ship it?

Thursday, October 17, 13
Ship it!
(just overprovision the api)

Thursday, October 17, 13
~3 back-end requests

Thursday, October 17, 13
~159 back-end requests

Thursday, October 17, 13
First to break
were the

haproxy
Thursday, October 17, 13
Then

memcached

Thursday, October 17, 13
Then

Rails/MySQL

Thursday, October 17, 13
Rails
won’t go away
need faster api

Thursday, October 17, 13
SoundCloud.com

Thursday, October 17, 13
Sounds Social Activity
& Sets Graph Stream
Premium
Search API
Features

Thursday, October 17, 13
Social
Graph

Sounds
& Sets

API

Activity
Stream

Premium
Features
Thursday, October 17, 13

Search
100%

75%

50%

25%

0%

Original

Database
Thursday, October 17, 13

Services Extracted

Other

HTTP
100%

75%

50%

25%

0%

Original

Database
Thursday, October 17, 13

Services Extracted

Other

HTTP
about the same, tbh
100%

75%

50%

25%

0%

Original

Database
Thursday, October 17, 13

Services Extracted

Other

HTTP
Rails
won’t go away

need faster Rails

Thursday, October 17, 13
Thursday, October 17, 13
lots of I/O
concurrency friendly

Thursday, October 17, 13
Rails
doesn’t like

parallelism
Thursday, October 17, 13
time
Thursday, October 17, 13
Can we
make it like

concurrency?
Thursday, October 17, 13
time
Thursday, October 17, 13
time
Thursday, October 17, 13
Thursday, October 17, 13
Thursday, October 17, 13
What
else

is there?
Thursday, October 17, 13
Thursday, October 17, 13
Thursday, October 17, 13
Thursday, October 17, 13
Thursday, October 17, 13
Thursday, October 17, 13
time
Thursday, October 17, 13
time
Thursday, October 17, 13
Thursday, October 17, 13
What
about

the network?
Thursday, October 17, 13
GET /tracks/trending/music

Thursday, October 17, 13
GET /tracks/trending/music
GET /tracks/genres

Thursday, October 17, 13
GET /tracks/trending/music
GET /tracks/genres
GET /tracks/genres/jazz
Thursday, October 17, 13
GET /tracks/trending/music
GET /tracks/genres
GET /tracks/genres/jazz
GET /tracks/123
Thursday, October 17, 13
What
about

a custom API?
Thursday, October 17, 13
GET /tracks/suggested

Thursday, October 17, 13
GET /tracks/suggested
GET /tracks/123

Thursday, October 17, 13
api.soundcloud.com

iOS
Thursday, October 17, 13

Android Desktop Widget 3rd Party
api-mobile

iOS
Thursday, October 17, 13

api web

api-partn

Android Desktop Widget 3rd Party
What
comes

next?
Thursday, October 17, 13
Stop
babysitting

Rails
Thursday, October 17, 13
Enable &
empower

app devs

Thursday, October 17, 13
“Experience
based”

api?

Thursday, October 17, 13
phil calçado
http://philcalcado.com
@pcalcado

www.soundcloud.com

Thursday, October 17, 13

More Related Content

What's hot (11)

gurltalkad
gurltalkadgurltalkad
gurltalkad
 
Elvis Presley The Fool 1973
Elvis Presley   The Fool 1973Elvis Presley   The Fool 1973
Elvis Presley The Fool 1973
 
Callister stephen ppp 4.4
Callister stephen ppp 4.4Callister stephen ppp 4.4
Callister stephen ppp 4.4
 
manejo de repositorios
manejo de repositoriosmanejo de repositorios
manejo de repositorios
 
Elvis Presley His Hand In Mine 1960
Elvis Presley   His Hand In Mine 1960Elvis Presley   His Hand In Mine 1960
Elvis Presley His Hand In Mine 1960
 
How to book gigs
How to book gigsHow to book gigs
How to book gigs
 
New botkier womens ines wedge sandal peach snake39.5 eu9.5 m us
New botkier womens ines wedge sandal peach snake39.5 eu9.5 m usNew botkier womens ines wedge sandal peach snake39.5 eu9.5 m us
New botkier womens ines wedge sandal peach snake39.5 eu9.5 m us
 
Audio on the web
Audio on the webAudio on the web
Audio on the web
 
Elvis Presley Hits Collection(Ktv)
Elvis Presley Hits Collection(Ktv)Elvis Presley Hits Collection(Ktv)
Elvis Presley Hits Collection(Ktv)
 
Pronunciation of final -s and final -ed
Pronunciation of final -s and final -edPronunciation of final -s and final -ed
Pronunciation of final -s and final -ed
 
Baer_Travis_4.4
Baer_Travis_4.4Baer_Travis_4.4
Baer_Travis_4.4
 

Viewers also liked

Structuring apps in Scala
Structuring apps in ScalaStructuring apps in Scala
Structuring apps in Scala
Phil Calçado
 
Evolutionary Architecture at Work
Evolutionary  Architecture at WorkEvolutionary  Architecture at Work
Evolutionary Architecture at Work
Phil Calçado
 

Viewers also liked (20)

Os cuidados e os limites do Responsive Web Design
Os cuidados e os limites do Responsive Web DesignOs cuidados e os limites do Responsive Web Design
Os cuidados e os limites do Responsive Web Design
 
CSS 4 - What's coming up
CSS 4 - What's coming upCSS 4 - What's coming up
CSS 4 - What's coming up
 
Structuring apps in Scala
Structuring apps in ScalaStructuring apps in Scala
Structuring apps in Scala
 
Cloud Reliability Patterns
Cloud Reliability PatternsCloud Reliability Patterns
Cloud Reliability Patterns
 
Desafio dos testes em uma arquitetura de micro serviços
Desafio dos testes em uma arquitetura de micro serviçosDesafio dos testes em uma arquitetura de micro serviços
Desafio dos testes em uma arquitetura de micro serviços
 
Three Years of Microservices at SoundCloud - Distributed Matters Berlin 2015
Three Years of Microservices at SoundCloud - Distributed Matters Berlin 2015Three Years of Microservices at SoundCloud - Distributed Matters Berlin 2015
Three Years of Microservices at SoundCloud - Distributed Matters Berlin 2015
 
Evolutionary Architecture at Work
Evolutionary  Architecture at WorkEvolutionary  Architecture at Work
Evolutionary Architecture at Work
 
Rhein-Main Scala Enthusiasts — Your microservice as a Function
Rhein-Main Scala Enthusiasts — Your microservice as a FunctionRhein-Main Scala Enthusiasts — Your microservice as a Function
Rhein-Main Scala Enthusiasts — Your microservice as a Function
 
ScalaItaly 2015 - Your Microservice as a Function
ScalaItaly 2015 - Your Microservice as a FunctionScalaItaly 2015 - Your Microservice as a Function
ScalaItaly 2015 - Your Microservice as a Function
 
Microservices - Quebrando gigantes em pequenos
Microservices - Quebrando gigantes em pequenosMicroservices - Quebrando gigantes em pequenos
Microservices - Quebrando gigantes em pequenos
 
Vida longa para SOA com microservices
Vida longa para SOA com microservicesVida longa para SOA com microservices
Vida longa para SOA com microservices
 
Amadurecendo Equipes com Microservices
Amadurecendo Equipes com MicroservicesAmadurecendo Equipes com Microservices
Amadurecendo Equipes com Microservices
 
Um pequeno estudo sobre a microsoft
Um pequeno estudo sobre a microsoftUm pequeno estudo sobre a microsoft
Um pequeno estudo sobre a microsoft
 
Integração utilizando REST API e Microservices
Integração utilizando REST API e MicroservicesIntegração utilizando REST API e Microservices
Integração utilizando REST API e Microservices
 
Flexbox and Grid Layout: How you will structure layouts tomorrow.
Flexbox and Grid Layout: How you will structure layouts tomorrow.Flexbox and Grid Layout: How you will structure layouts tomorrow.
Flexbox and Grid Layout: How you will structure layouts tomorrow.
 
DOM, CSSOM e RenderThree - Introdução ao Browser Render Path
DOM, CSSOM e RenderThree - Introdução ao Browser Render PathDOM, CSSOM e RenderThree - Introdução ao Browser Render Path
DOM, CSSOM e RenderThree - Introdução ao Browser Render Path
 
Microservices
MicroservicesMicroservices
Microservices
 
Introdução à Microservices
Introdução à MicroservicesIntrodução à Microservices
Introdução à Microservices
 
Workshop soa, microservices e devops
Workshop soa, microservices e devopsWorkshop soa, microservices e devops
Workshop soa, microservices e devops
 
Microservices: uma abordagem para arquitetura de aplicações (Devcamp 2015)
Microservices: uma abordagem para arquitetura de aplicações (Devcamp 2015)Microservices: uma abordagem para arquitetura de aplicações (Devcamp 2015)
Microservices: uma abordagem para arquitetura de aplicações (Devcamp 2015)
 

More from Phil Calçado

the afterparty: refactoring after 100x hypergrowth
the afterparty: refactoring after 100x hypergrowththe afterparty: refactoring after 100x hypergrowth
the afterparty: refactoring after 100x hypergrowth
Phil Calçado
 
don't try this at home: self-improvement as a senior leader
don't try this at home: self-improvement as a senior leaderdon't try this at home: self-improvement as a senior leader
don't try this at home: self-improvement as a senior leader
Phil Calçado
 
From microservices to serverless - Chicago CTO Summit 2019
From microservices to serverless - Chicago CTO Summit 2019From microservices to serverless - Chicago CTO Summit 2019
From microservices to serverless - Chicago CTO Summit 2019
Phil Calçado
 
The Not-So-Straightforward Road From Microservices to Serverless
The Not-So-Straightforward Road From Microservices to ServerlessThe Not-So-Straightforward Road From Microservices to Serverless
The Not-So-Straightforward Road From Microservices to Serverless
Phil Calçado
 
Microservices vs. The First Law of Distributed Objects - GOTO Nights Chicago ...
Microservices vs. The First Law of Distributed Objects - GOTO Nights Chicago ...Microservices vs. The First Law of Distributed Objects - GOTO Nights Chicago ...
Microservices vs. The First Law of Distributed Objects - GOTO Nights Chicago ...
Phil Calçado
 
An example of Future composition in a real app
An example of Future composition in a real appAn example of Future composition in a real app
An example of Future composition in a real app
Phil Calçado
 
From a monolithic Ruby on Rails app to the JVM
From a monolithic  Ruby on Rails app  to the JVMFrom a monolithic  Ruby on Rails app  to the JVM
From a monolithic Ruby on Rails app to the JVM
Phil Calçado
 
Applying Evolutionary Architecture on a Popular API
Applying Evolutionary Architecture on a  Popular APIApplying Evolutionary Architecture on a  Popular API
Applying Evolutionary Architecture on a Popular API
Phil Calçado
 
SoundCloud Masterclass on Brazil
SoundCloud Masterclass on BrazilSoundCloud Masterclass on Brazil
SoundCloud Masterclass on Brazil
Phil Calçado
 
SpeakerConf: my findings in trying to use this functional programming busines...
SpeakerConf: my findings in trying to use this functional programming busines...SpeakerConf: my findings in trying to use this functional programming busines...
SpeakerConf: my findings in trying to use this functional programming busines...
Phil Calçado
 
[GOTO Copenhagen 2012] The Startup Hangover: Supporting 15 mil Users
[GOTO Copenhagen 2012] The Startup Hangover: Supporting 15 mil Users[GOTO Copenhagen 2012] The Startup Hangover: Supporting 15 mil Users
[GOTO Copenhagen 2012] The Startup Hangover: Supporting 15 mil Users
Phil Calçado
 

More from Phil Calçado (20)

the afterparty: refactoring after 100x hypergrowth
the afterparty: refactoring after 100x hypergrowththe afterparty: refactoring after 100x hypergrowth
the afterparty: refactoring after 100x hypergrowth
 
don't try this at home: self-improvement as a senior leader
don't try this at home: self-improvement as a senior leaderdon't try this at home: self-improvement as a senior leader
don't try this at home: self-improvement as a senior leader
 
The Economics of Microservices (redux)
The Economics of Microservices (redux)The Economics of Microservices (redux)
The Economics of Microservices (redux)
 
From microservices to serverless - Chicago CTO Summit 2019
From microservices to serverless - Chicago CTO Summit 2019From microservices to serverless - Chicago CTO Summit 2019
From microservices to serverless - Chicago CTO Summit 2019
 
The Not-So-Straightforward Road From Microservices to Serverless
The Not-So-Straightforward Road From Microservices to ServerlessThe Not-So-Straightforward Road From Microservices to Serverless
The Not-So-Straightforward Road From Microservices to Serverless
 
Ten Years of Failing Microservices
Ten Years of Failing MicroservicesTen Years of Failing Microservices
Ten Years of Failing Microservices
 
The Next Generation of Microservices
The Next Generation of MicroservicesThe Next Generation of Microservices
The Next Generation of Microservices
 
The Next Generation of Microservices — YOW 2017 Brisbane
The Next Generation of Microservices — YOW 2017 BrisbaneThe Next Generation of Microservices — YOW 2017 Brisbane
The Next Generation of Microservices — YOW 2017 Brisbane
 
The Economics of Microservices (2017 CraftConf)
The Economics of Microservices  (2017 CraftConf)The Economics of Microservices  (2017 CraftConf)
The Economics of Microservices (2017 CraftConf)
 
Microservices vs. The First Law of Distributed Objects - GOTO Nights Chicago ...
Microservices vs. The First Law of Distributed Objects - GOTO Nights Chicago ...Microservices vs. The First Law of Distributed Objects - GOTO Nights Chicago ...
Microservices vs. The First Law of Distributed Objects - GOTO Nights Chicago ...
 
Finagle @ SoundCloud
Finagle @ SoundCloudFinagle @ SoundCloud
Finagle @ SoundCloud
 
A Brief Talk On High-Performing Organisations
A Brief Talk On High-Performing OrganisationsA Brief Talk On High-Performing Organisations
A Brief Talk On High-Performing Organisations
 
Finagle-Based Microservices at SoundCloud
Finagle-Based Microservices at SoundCloudFinagle-Based Microservices at SoundCloud
Finagle-Based Microservices at SoundCloud
 
An example of Future composition in a real app
An example of Future composition in a real appAn example of Future composition in a real app
An example of Future composition in a real app
 
From a monolithic Ruby on Rails app to the JVM
From a monolithic  Ruby on Rails app  to the JVMFrom a monolithic  Ruby on Rails app  to the JVM
From a monolithic Ruby on Rails app to the JVM
 
Applying Evolutionary Architecture on a Popular API
Applying Evolutionary Architecture on a  Popular APIApplying Evolutionary Architecture on a  Popular API
Applying Evolutionary Architecture on a Popular API
 
SoundCloud Masterclass on Brazil
SoundCloud Masterclass on BrazilSoundCloud Masterclass on Brazil
SoundCloud Masterclass on Brazil
 
SpeakerConf: my findings in trying to use this functional programming busines...
SpeakerConf: my findings in trying to use this functional programming busines...SpeakerConf: my findings in trying to use this functional programming busines...
SpeakerConf: my findings in trying to use this functional programming busines...
 
[GOTO Copenhagen 2012] The Startup Hangover: Supporting 15 mil Users
[GOTO Copenhagen 2012] The Startup Hangover: Supporting 15 mil Users[GOTO Copenhagen 2012] The Startup Hangover: Supporting 15 mil Users
[GOTO Copenhagen 2012] The Startup Hangover: Supporting 15 mil Users
 
(v2.0) Better Functional Design Through Test-Driven Development
(v2.0) Better Functional Design Through Test-Driven Development(v2.0) Better Functional Design Through Test-Driven Development
(v2.0) Better Functional Design Through Test-Driven Development
 

Recently uploaded

Recently uploaded (20)

The Metaverse: Are We There Yet?
The  Metaverse:    Are   We  There  Yet?The  Metaverse:    Are   We  There  Yet?
The Metaverse: Are We There Yet?
 
Portal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russePortal Kombat : extension du réseau de propagande russe
Portal Kombat : extension du réseau de propagande russe
 
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
Choosing the Right FDO Deployment Model for Your Application _ Geoffrey at In...
 
A Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System StrategyA Business-Centric Approach to Design System Strategy
A Business-Centric Approach to Design System Strategy
 
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdfIntroduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
Introduction to FDO and How It works Applications _ Richard at FIDO Alliance.pdf
 
What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024What's New in Teams Calling, Meetings and Devices April 2024
What's New in Teams Calling, Meetings and Devices April 2024
 
Where to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdfWhere to Learn More About FDO _ Richard at FIDO Alliance.pdf
Where to Learn More About FDO _ Richard at FIDO Alliance.pdf
 
Designing for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at ComcastDesigning for Hardware Accessibility at Comcast
Designing for Hardware Accessibility at Comcast
 
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
ASRock Industrial FDO Solutions in Action for Industrial Edge AI _ Kenny at A...
 
1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPT
1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPT1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPT
1111 ChatGPT Prompts PDF Free Download - Prompts for ChatGPT
 
Oauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoftOauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoft
 
Powerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara LaskowskaPowerful Start- the Key to Project Success, Barbara Laskowska
Powerful Start- the Key to Project Success, Barbara Laskowska
 
Your enemies use GenAI too - staying ahead of fraud with Neo4j
Your enemies use GenAI too - staying ahead of fraud with Neo4jYour enemies use GenAI too - staying ahead of fraud with Neo4j
Your enemies use GenAI too - staying ahead of fraud with Neo4j
 
ERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage IntacctERP Contender Series: Acumatica vs. Sage Intacct
ERP Contender Series: Acumatica vs. Sage Intacct
 
WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024
 
PLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. StartupsPLAI - Acceleration Program for Generative A.I. Startups
PLAI - Acceleration Program for Generative A.I. Startups
 
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
 
TopCryptoSupers 12thReport OrionX May2024
TopCryptoSupers 12thReport OrionX May2024TopCryptoSupers 12thReport OrionX May2024
TopCryptoSupers 12thReport OrionX May2024
 
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdfThe Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
The Value of Certifying Products for FDO _ Paul at FIDO Alliance.pdf
 
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...
FDO for Camera, Sensor and Networking Device – Commercial Solutions from VinC...
 

APIs: The Problems with Eating your Own Dog Food