Drupal Cloud to the rescue? Servers, Files, CDNs and Fun!

Cloud to the rescue?
Servers, Files, CDNs and Fun
Questions!

Alex McFadyen, CTO

drupal.org : a_c_m || drupal.org/user/195063
email : alex@demotix.com
twitter : @a_c_m

Introduction : Demotix.com

Demotix.com
A two minute (or less) introduction

http://prezi.com/_ugoyxm4qprv/demotixcom-2-minutes/

Drupal Developer Days – Barcelona June 16th 2012
ALEX McFADYEN | alex@demotix.com | @a_c_m

Introduction : Demotix.com
Some Demotix numbers
●
1.5million+ page views a month
●
Very very long tale of content
●
Spikes where we get 100k's views in a single day
– Oh how I love Varnish
●
14GB+ database
●
750,000+ nodes, even more terms
●
3-7k new nodes per day

All running from 3 “real” servers. 1 Web, 1 DB, 1 Util.
(Having moved “out” of the cloud 3 years ago)


Introduction : Why care?

Why not just pay for it to be someone else’s
problem?

●
Control (you NEED it)
●
Complexity (doesn't fit generic solutions)
●
Policy

●
If you can. Do.


Introduction : “The Cloud”
To the world
“Cloud computing refers to the delivery of computing and storage
capacity as a service to a heterogeneous community of end-
recipients”
- http://en.wikipedia.org/wiki/Cloud_computing

To me (a Drupal dev)
“The cloud is just a buzz word meaning quick to set up VPSs,
perhaps 'unlimited' storage space and even a CDN if your lucky”
- Me, just now.


Introduction : “The Cloud”
3 parts we were interested in

Power = Virtual Private Servers
Storage = Disks. Big ones.
Delivery = CDNs and other tricks


Power : Why VPSs rock

@ New servers anytime
@ Scale up and down anytime
@ Short/no contracts
@ Total control (I Y Drush, root access, etc)
@ Cheap !


Power : Why VPSs don't

@ Total control (I Y Drush, root access, etc)

“With great power comes great responsibility”
- Uncle Ben

●
You always secretly wanted be a sysadmin right?
●
You want to be woken up at 3am if it goes wrong?
●
root@server:~# rm -rf /



@ Location (Where did I just put that data?)

●
EU has laws about where you can send and store users
data.
●
Your business may not allow for you to store data in the
cloud.
●
Cloud may not be where your users are.



@ Slices (How big is your share?)

●
Most (not all) cloud providers slice servers up into equal
cpu/ram/disk space chunks and sell them off.
●
Some let you add on resources you need, others don't.

@ Poor disk IO
●
Are your disks local or over the network?


@ More servers, more problems
●
Once your past dedicated use servers (web, db etc) you
have to do work on the app to support more servers.
See last session “How to sleep without the server-crash-fear
(High Availability and High Performance Hosting for Drupal)”

@ Support
●
Do you have any?
●
How about at 3am?
●
On Christmas day



@ Bad neighbours & overselling
●
Starts out great, then reddit moves in next door.

@ Cloud A != Cloud B
●
Disks, CPU/Ram/Disk ratio, network speed, location,
management systems etc


Power : What to do?
@ Benchmarks
●
Drupal benchmark? Omega8.cc Room today at 4pm.
●
http://cloudharmony.com cpu/ram/disk/web/db


Power : What to do?
@ Try before you buy
●
Some companies will give you a free week.
●
Put as close as you can 1:1 copy of your current site on
it and...

@ Test, test and test some more
●
AB / Siege test your caching system.
●
Instead replay your log files with HTTPERF or Jmeter.
●
Monitor the results with Munin/Cacti/Newrelic.


Power : Review
Why VPS rock Why they don't
@ New servers anytime
@ Total control
@ Scale up and down anytime @ Location
@ Short/no contracts @ Slices
@ Poor disk IO
@ Total control @ More servers, more problems
@ Cheap ! @ Support
@ Overselling
@ Cloud A != Cloud B

What to do
@ Benchmark @ Try before you buy

@ Test, test and test some more


Storage : The problem
Storage = /sites/*/files

@ Multi web head environment
@ Web heads need more space than
their slice allows
@* LOTS of files in a single dir (10k+)


Storage : The solutions

https://www.getpantheon.com/news/inside-pantheon-
valhalla-filesystem

@ Server with
big disk(s) / SAN / EBS + NFS
●
Tried, tested, works
●
Not very exciting
●
Single point of failure
●
Depending on your cloud could need a 64GB
ram server !!


Storage : The solutions

@ API storage + Fuse (Cloudfuse*, s3fs)
●
Theoretically unlimited storage
●
CDN with little or no extra effort
●
A bit bleeding edge / untested

@ Gluster FS / Similar
●
Impressive solution
●
Makes trade-offs drupal doesn't need
●
Complicated


Delivery : Do CDNs matter?
Dev server in Paris

YES

http://www.dotcom-monitor.com/WebTools/technical-tools-browser-test.aspx


Delivery : Types of CDN
http://wimleers.com/article/key-properties-of-a-cdn

@ Push (CloudFiles, S3 etc)
●
No redundant traffic
●
Complex set up

@ Pull (lots)
●
Very easy to set up
●
Redundant traffic

@ DNS (Cloudflare/incapsula)
●
Takes over your DNS
●
Does more than “just” CDN
●
We have not tested


Conclusions

@ The Cloud isn't a magic bullet
@ The best benchmark is your own app,
with replayed traffic.
@ Cloud providers differ a LOT
@ Storage can be complicated
@ CDNs can give you an easy big win


What did we do?
@ Power : Cloud, Yes … but still in testing
●
Using Dediserve.com
https://billing.dediserve.com/?affid=148
●
Still re-tuning web heads PHP-FPM vs mod_php.
●
Hopefully live next week!

@ Storage : NFS
●
Simple, tried and tested.
●
Using a NFS mount used by all web heads.

@ CDN : Yes! CDN pull.
●
Easy win ! Very easy to set up.
●
Dediserve's CDN.
●
Also using CDN module to spread assets over several
domains (improves front end performance)

* Lots of files / * Cloud fuse
@ Lots of files ...
Close to 1M files in a single directory
●
Wrote custom drupal 6 module to fix
●
As part of the file save hooks, it moved the file to a folder
structure based on the filename.
●
/sites/default/files/<new_folder>/images/123.jpg
●
On drupal.org, but alpha

@ Cloudfuse + CDN + Lots of files
●
Working proof of concept
●
Uses Rackspace cloud files for both storage and CDN
●
Requires pre-generation of imagecache
●
Not published … yet.


Thank you
Questions?

Alex McFadyen, CTO

drupal.org : a_c_m || drupal.org/user/195063
email : alex@demotix.com
twitter : @a_c_m

Drupal Cloud to the rescue? Servers, Files, CDNs and Fun!

More Related Content

What's hot

Viewers also liked

Similar to Drupal Cloud to the rescue? Servers, Files, CDNs and Fun!

Recently uploaded

Drupal Cloud to the rescue? Servers, Files, CDNs and Fun!