0
HOW TO
SCALE YOUR
APP AND WIN
THE CLOUD
CHALLENGE
QUENTIN ADAM
@WAXZCE
2013
Quentin ADAM
Clever Cloud CEO
@waxzce on twitter
http://www.waxzce.org
WHO I AM ?
Java, scala, python, nodejs, php… apps scaling automatically in the cloud.
We cover your ass, you can focus on your own st...
WHEN YOU
NEED TO
SCALE
THERE ARE 2 WAYS
GROWING AND GROWING
UNTIL YOU EXPLODE OR
BECOME WEIRD
OR SPLIT THE WORK AND
MAKE YOUR SOFTWARE
WORK AS A TEAM
Build an army of fat app
YOU CAN DO BOTH
SO WE NEED TO BE ABLE
TO DISPATCH THE WORK
SCALE OUT
• Many workers
doing the same
thing
• No SPOF
• Growing is more
easy
...
SO, HOW TO
SCALE OUT
?
JUST SOME FACTS
SPLIT PROCESS AND
STORAGE
Storage
• Databases
• Files
• Sessions
• Events
• …
Code
• Can be replicated
• Stateless
• Proce...
Picking one instance or another doesn’t matter
STATELESSNESS IS THE KEY
CONSIDER MORE
THINGS AS DATA
• User account
• Users data
• Files
• Sessions
• Events
CHOOSE
DATASTORE
WISELY
YOU CAN SHOULD USE MANY DATASTORES
DATASTORE CHOICES
ARE DRIVEN BY USAGE
Make
decisions
based on
needs
Do I need
atomicity of
requests ?
Do I need
concurrent...
• Not a big volume
• DB have to manage
data TTL
• Data model : K/V
• Multiple writes at the
same time
• High availability
...
• Not a big volume
It’s OK, PG can handle
small quantity of data
• DB have to manage
data TTL
No, I have to do it
manually...
• Not a big volume
It’s OK, redis can
handle small quantity
of data
• DB have to manage
data TTL
Yes Redis can do it
• Dat...
• Not a big volume
It’s OK, CB can handle
small quantity of data
• DB have to manage
data TTL
Yes CB can do it
• Data mode...
USE ONLINE
DATABASE / BE
READY TO TEST
IN JUST A FEW
MINUTES
NO NEED TO TRASH YOUR COMPUTER
DON’T BE THAT GUY
DO NOT USE A
TECHNOLOGY BECAUSE
YOU <3 IT OR BECAUSE
IT’S HYPE : USE IT
BECAUSE IT FITS YOUR
NEEDS
BALANCE YOUR LEARNING C...
DO NOT CREATE MONSTERS
COMMON MISTAKES
DO NOT USE
MEMORY AS
DATABASE
LIKE : SHARED / GLOBAL VARIABLE,
CACHE “IN THE CODE”, INTENSIVE
SESSION USAGE…
DO NOT USE A VARIABLE
FOR MORE THAN ONE
REQUEST
2 + 2 = 4
FOR SAME INPUT, SAME OUTPUT
GET do not change data on server
BE HTTP CONSISTENT
And data will be lost
CODE WILL FAIL
DO NOT USE FILE
SYSTEM AS DATASTORE
File system are POSIX compliant
• POSIX is ACID
• POSIX is powerful but is bottleneck
...
CAREFUL USE OF
DARK MAGIC
SPLIT THE CODE :
MODULES
• Smallest code
base
• Deploy as
service for
each other
• Focus on best
technology for
a problem
SCALE YOUR TEAM
MODULARIZE YOUR TEAM
USE EVENT BROKER TO
MODULARIZE YOUR APP
• AMQP
• Celery
• 0MQ
• Redis
• JMS
• Some case : hadoop, akka…
• …
CRON is not an...
MAKE HARD
COMPUTATION ASYNC
ALWAYS USE A
REVERSE PROXY
Y U NOT USE ONE ?
DO NOT BUILD “THE
SERVER” WITH NO DOC
USE PROCESS
DEPLOYMENT
EASY MOVING OR
INCIDENT MANAGEMENT
KEEP CALM UNDER FIRE
TRACK BUG & GET METRICS
Quentin ADAM
Twitter : @waxzce
THX FOR LISTENING
& QUESTIONS TIME
How to scale your app and win the cloud challenge
Upcoming SlideShare
Loading in...5
×

How to scale your app and win the cloud challenge

2,096

Published on

while42.org #1 paris event

Published in: Technology
0 Comments
3 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,096
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
20
Comments
0
Likes
3
Embeds 0
No embeds

No notes for slide

Transcript of "How to scale your app and win the cloud challenge "

  1. 1. HOW TO SCALE YOUR APP AND WIN THE CLOUD CHALLENGE QUENTIN ADAM @WAXZCE 2013
  2. 2. Quentin ADAM Clever Cloud CEO @waxzce on twitter http://www.waxzce.org WHO I AM ?
  3. 3. Java, scala, python, nodejs, php… apps scaling automatically in the cloud. We cover your ass, you can focus on your own stuff http://www.clever-cloud.com PAAS PROVIDER
  4. 4. WHEN YOU NEED TO SCALE THERE ARE 2 WAYS
  5. 5. GROWING AND GROWING UNTIL YOU EXPLODE OR BECOME WEIRD
  6. 6. OR SPLIT THE WORK AND MAKE YOUR SOFTWARE WORK AS A TEAM
  7. 7. Build an army of fat app YOU CAN DO BOTH
  8. 8. SO WE NEED TO BE ABLE TO DISPATCH THE WORK SCALE OUT • Many workers doing the same thing • No SPOF • Growing is more easy • Introduce best practice SCALE UP • 1 Fat instance • 1 Fat application • SPOF (single point of failure) • Hard to maintain • Always has a limit • Short term meaning
  9. 9. SO, HOW TO SCALE OUT ? JUST SOME FACTS
  10. 10. SPLIT PROCESS AND STORAGE Storage • Databases • Files • Sessions • Events • … Code • Can be replicated • Stateless • Process
  11. 11. Picking one instance or another doesn’t matter STATELESSNESS IS THE KEY
  12. 12. CONSIDER MORE THINGS AS DATA • User account • Users data • Files • Sessions • Events
  13. 13. CHOOSE DATASTORE WISELY YOU CAN SHOULD USE MANY DATASTORES
  14. 14. DATASTORE CHOICES ARE DRIVEN BY USAGE Make decisions based on needs Do I need atomicity of requests ? Do I need concurrent access ? Do I mostly read or write ? Do I need relational ? Do I need big storage capacity ? Do I need high availability ?
  15. 15. • Not a big volume • DB have to manage data TTL • Data model : K/V • Multiple writes at the same time • High availability I need to store sessions QUICK EXAMPLE
  16. 16. • Not a big volume It’s OK, PG can handle small quantity of data • DB have to manage data TTL No, I have to do it manually • Data model : K/V No, PG is relational (mainly) • Multiple writes at the same time No, PG is Atomic • High availability PG is awesome ;-) Use of PG bouncer or similar allow good clustering I need to store sessions QUICK EXAMPLE
  17. 17. • Not a big volume It’s OK, redis can handle small quantity of data • DB have to manage data TTL Yes Redis can do it • Data model : K/V Yes • Multiple writes at the same time No, redis is pseudo Atomic (master/slave) • High availability Redis is great, but cauterization is rude… I need to store sessions QUICK EXAMPLE
  18. 18. • Not a big volume It’s OK, CB can handle small quantity of data • DB have to manage data TTL Yes CB can do it • Data model : K/V Yes • Multiple writes at the same time OK, this is possible with memcached protocol • High availability Clustering is built in, no downtime  I need to store sessions QUICK EXAMPLE
  19. 19. USE ONLINE DATABASE / BE READY TO TEST IN JUST A FEW MINUTES NO NEED TO TRASH YOUR COMPUTER
  20. 20. DON’T BE THAT GUY
  21. 21. DO NOT USE A TECHNOLOGY BECAUSE YOU <3 IT OR BECAUSE IT’S HYPE : USE IT BECAUSE IT FITS YOUR NEEDS BALANCE YOUR LEARNING CURVE WITH THE TIME SAVED
  22. 22. DO NOT CREATE MONSTERS
  23. 23. COMMON MISTAKES
  24. 24. DO NOT USE MEMORY AS DATABASE LIKE : SHARED / GLOBAL VARIABLE, CACHE “IN THE CODE”, INTENSIVE SESSION USAGE…
  25. 25. DO NOT USE A VARIABLE FOR MORE THAN ONE REQUEST
  26. 26. 2 + 2 = 4 FOR SAME INPUT, SAME OUTPUT
  27. 27. GET do not change data on server BE HTTP CONSISTENT
  28. 28. And data will be lost CODE WILL FAIL
  29. 29. DO NOT USE FILE SYSTEM AS DATASTORE File system are POSIX compliant • POSIX is ACID • POSIX is powerful but is bottleneck • File System is nightmare of ops • File System is create coupling (host provider/OS/language) • Free SPOF multi tenant File System is a unicorn STORE IN DATABASE, OR DATASTORE LIKE S3 (AWS) DEDICATED TO FILE MANAGEMENT
  30. 30. CAREFUL USE OF DARK MAGIC
  31. 31. SPLIT THE CODE : MODULES • Smallest code base • Deploy as service for each other • Focus on best technology for a problem
  32. 32. SCALE YOUR TEAM MODULARIZE YOUR TEAM
  33. 33. USE EVENT BROKER TO MODULARIZE YOUR APP • AMQP • Celery • 0MQ • Redis • JMS • Some case : hadoop, akka… • … CRON is not an event queue
  34. 34. MAKE HARD COMPUTATION ASYNC
  35. 35. ALWAYS USE A REVERSE PROXY Y U NOT USE ONE ?
  36. 36. DO NOT BUILD “THE SERVER” WITH NO DOC
  37. 37. USE PROCESS DEPLOYMENT
  38. 38. EASY MOVING OR INCIDENT MANAGEMENT
  39. 39. KEEP CALM UNDER FIRE
  40. 40. TRACK BUG & GET METRICS
  41. 41. Quentin ADAM Twitter : @waxzce THX FOR LISTENING & QUESTIONS TIME
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×