Łukasz Klimek : www.softinn.eu
PLAN
1. Introduction
2. Cloudflare basics
3. Performance
4. Security
5. Show me the results!
6. Cloudflare and Drupal
7. Questions / discussion
2
3
DRUPAL HOSTING NEEDS
Shared
hosting
Cloud /
dedicated
server
Complex
infrastructure
4
THE PROBLEM
• Spam bots
• Comments
• User registrations
• Worms, viruses, trojans
• Traffic peaks
• Event websites
5
FIGHTING SPAM
• Captcha-style (Captcha / reCAPTCHA)
• Already cracked. By Google themselves ;-)
• Mollom
• captcha
• text analysis
• user reputation
• …
6
7
PERFORMANCE ISSUES
We still process our PHP scripts!
• Huge CPU utilization
• Memory consumption
• DoS in case of multiple concurrent connections
8
INCREASING PERFORMANCE
• APC
• memcache
• boost
• …
• Minimize number of requests
• Combine & minify CSS / JS
• Website code refactoring
9
NOT ENOUGH?
• Separate DB server
• Separate host for static content
• Reverse proxy (Varnish)
10
SO WE GET…
11
12
ADDING REDUNDANCY 
13
LOOKS COMPLEX?
And that’s just the beginning 
• No development/staging servers
• No shared storage between servers
• No backups
• No monitoring
• No Internet connection redundancy
• Issues with bandwidth consumption
• …
14
15
• 99.9% uptime
• Defend against bots
& spam
• Handle traffic peaks
• Decrease server load
• Minimize bandwidth
usage
• Minify CSS and JS
LET’S SUMMARIZE THE NEEDS
16
17
18
WHAT IS CLOUDFLARE?
• Content Delivery Network (CDN)
• Web Application Firewall
• Code optimizer
• Traffic statistics
• Application platform
19
WHAT IS CLOUDFLARE? (2)
20
CLOUDFLARE NETWORK
21
22
CLOUDFLARE AS A CDN
• Works like „reverse proxy”
• Caching of static files
• Caching of dynamic (generated) pages for
anonymous users
• No bandwidth limits / fees
23
PERFORMANCE SETTINGS
• Caching level:
• Aggressive: http://softinn.eu/pic.jpg?with=query
• Simplified: http://softinn.eu/pic.jpg?ignore=this-query-string
• Basic: http://softinn.eu/pic.jpg
24
RULES
• Ability to customize performance & security
settings based on URLs
• Up to 3 rules in Free plan, 20 in Pro plan
• IMO the most important tool in Cloudflare
25
CODE OPTIMIZATIONS
Auto Minify - remove unnecessary characters
• JS
• CSS
• HTML
Rocket Loader
• Loads JS asynchronously (after window.onload)
• Can have some side-effects
Website Preloader
• Detects most often used static resources
• Fetches these resources to browser’s cache
26
ROCKET LOADER
27
IMAGES
Mirage 2
• Asynchronous image loading
• All images in a single request
Polish - image otimization
• Lossless
• Remove metadata
• Average reduction of size: about 21%
• Lossy
• Additional lossy compression
• Average reduction of size: 48%
28
MIRAGE 2.0
29
30
SECURITY OPTIONS
E-mail address obfuscation
Server side exclude (SSE)
Browser integrity check – HTTP headers inspection (incl.
User-agent)
Visitor reputation
Hotlink protection
• HTTP Referers that are not in-zone and not blank will be
denied access
• Hotlink-ok mechanism (eg. http://softinn.eu/hotlink-
ok/img.gif
SSL support 
31
THREAT CONTROL
32
SUSPICIOUS VISITORS
Captcha
Ability to blacklist / whitelist IPs
• Drupal module: Cloudflare
33
WEB APPLICATION FIREWALL
Set of security rules to address most common
threats
• OWASP TOP 10
• Cloudflare-designed:
PHP, WHCMS, Joomla, Wordpress, …
• No Drupal-specific rules 
34
ALWAYS ONLINE
• Limited version of your site is always online
• Only the most popular pages
• No POST and SSL support
• Crawler-based - crawling every 7, 3 or 1 day
• Triggers:
• HTTP status 502 or 504
• Connection timeout, SSL errors etc.
35
36
EXAMPLE STATISTICS
37
NOT A SILVER BULLET
• Logged-in users
• Cache invalidation
• Performance of non-cached pages
38
CACHE INVALIDATION
There are only two hard things in Computer Science:
cache invalidation and naming things.
-- Phil Karlton (after
http://martinfowler.com/bliki/TwoHardThings.html)
1. Cloudflare stores copy of a page in the cache
2. User changes this page
3. How can Cloudflare know that the page has
changed?
39
• 99.9% uptime
• Defend against bots
& spam
• Handle traffic peaks
• Decrease server load
• Minimize bandwidth
usage
• Minify CSS and JS
DOES IT SOLVE OUR NEEDS?
40
41
PREPARING TO DEPLOY CLOUDFLARE
1. Cache expiration policy
2. Plan your URLs / pathauto config
http://www.site.com/can-cache/...
3. Views expiration settings (Views Content
Cache?)
4. Apache configuration (proper expiration of
static content)
42
Expire monitors content
updates
Expire invokes
hook_expire_cache()
(cfpurge_expire_cache())
Cloudflare API:
zone_file_purge
• https://drupal.org/project/expire
• https://drupal.org/project/cfpurge
• Define „Cache everything” rule on Cloudflare
• CFPurge still needs some work; only 16 installs 
• Lack of Views integration
43
CACHE INVALIDATION: EXPIRE + CFPURGE
CLOUDFLARE + DRUPAL: QUICK START
Review Cloudflare performance settings (Auto Minify, Caching Level, Mirage, Polish, …)
Review Cloudflare security settings (obfuscation, hotlink protection, …)
Whitelist important IP addresses (monitoring, APIs, …)
Create Cloudflare Rules (/admin/*, /user/*, …)
Handle remote (client) IP address correctly
Install & configure modules (cloudflare, CFPurge, expire)
Change DNS delegation
Create Cloudflare account
44
DNS CONFIGURATION
45
TO DO – TASKS FOR COMMUNITY
• 502 / 504 on errors (compatibility with
Cloudflare Always Online)
https://drupal.org/node/2268487
• Views expiration
• Expire all views that use CT https://drupal.org/node/2146797 (won’t
fix )
• Integrate Expire with Views Content Cache
https://drupal.org/node/1786436 (won’t fix )
• Integrate blacklists with antispam modules
(Mollom etc.)
46
THANK YOU!
47
Łukasz Klimek
E-mail: Lukasz@softinn.eu
Mobile: +48 66 999 2096
Skype: casatm | Twitter @lklimek
http://tinyurl.com/lklimek
http://goo.gl/2dEgs7
Software Inn
www.softinn.eu

Cloudflare and Drupal - fighting bots and traffic peaks

  • 1.
    Łukasz Klimek :www.softinn.eu
  • 2.
    PLAN 1. Introduction 2. Cloudflarebasics 3. Performance 4. Security 5. Show me the results! 6. Cloudflare and Drupal 7. Questions / discussion 2
  • 3.
  • 4.
    DRUPAL HOSTING NEEDS Shared hosting Cloud/ dedicated server Complex infrastructure 4
  • 5.
    THE PROBLEM • Spambots • Comments • User registrations • Worms, viruses, trojans • Traffic peaks • Event websites 5
  • 6.
    FIGHTING SPAM • Captcha-style(Captcha / reCAPTCHA) • Already cracked. By Google themselves ;-) • Mollom • captcha • text analysis • user reputation • … 6
  • 7.
  • 8.
    PERFORMANCE ISSUES We stillprocess our PHP scripts! • Huge CPU utilization • Memory consumption • DoS in case of multiple concurrent connections 8
  • 9.
    INCREASING PERFORMANCE • APC •memcache • boost • … • Minimize number of requests • Combine & minify CSS / JS • Website code refactoring 9
  • 10.
    NOT ENOUGH? • SeparateDB server • Separate host for static content • Reverse proxy (Varnish) 10
  • 11.
  • 12.
  • 13.
  • 14.
    LOOKS COMPLEX? And that’sjust the beginning  • No development/staging servers • No shared storage between servers • No backups • No monitoring • No Internet connection redundancy • Issues with bandwidth consumption • … 14
  • 15.
  • 16.
    • 99.9% uptime •Defend against bots & spam • Handle traffic peaks • Decrease server load • Minimize bandwidth usage • Minify CSS and JS LET’S SUMMARIZE THE NEEDS 16
  • 17.
  • 18.
  • 19.
    WHAT IS CLOUDFLARE? •Content Delivery Network (CDN) • Web Application Firewall • Code optimizer • Traffic statistics • Application platform 19
  • 20.
  • 21.
  • 22.
  • 23.
    CLOUDFLARE AS ACDN • Works like „reverse proxy” • Caching of static files • Caching of dynamic (generated) pages for anonymous users • No bandwidth limits / fees 23
  • 24.
    PERFORMANCE SETTINGS • Cachinglevel: • Aggressive: http://softinn.eu/pic.jpg?with=query • Simplified: http://softinn.eu/pic.jpg?ignore=this-query-string • Basic: http://softinn.eu/pic.jpg 24
  • 25.
    RULES • Ability tocustomize performance & security settings based on URLs • Up to 3 rules in Free plan, 20 in Pro plan • IMO the most important tool in Cloudflare 25
  • 26.
    CODE OPTIMIZATIONS Auto Minify- remove unnecessary characters • JS • CSS • HTML Rocket Loader • Loads JS asynchronously (after window.onload) • Can have some side-effects Website Preloader • Detects most often used static resources • Fetches these resources to browser’s cache 26
  • 27.
  • 28.
    IMAGES Mirage 2 • Asynchronousimage loading • All images in a single request Polish - image otimization • Lossless • Remove metadata • Average reduction of size: about 21% • Lossy • Additional lossy compression • Average reduction of size: 48% 28
  • 29.
  • 30.
  • 31.
    SECURITY OPTIONS E-mail addressobfuscation Server side exclude (SSE) Browser integrity check – HTTP headers inspection (incl. User-agent) Visitor reputation Hotlink protection • HTTP Referers that are not in-zone and not blank will be denied access • Hotlink-ok mechanism (eg. http://softinn.eu/hotlink- ok/img.gif SSL support  31
  • 32.
  • 33.
    SUSPICIOUS VISITORS Captcha Ability toblacklist / whitelist IPs • Drupal module: Cloudflare 33
  • 34.
    WEB APPLICATION FIREWALL Setof security rules to address most common threats • OWASP TOP 10 • Cloudflare-designed: PHP, WHCMS, Joomla, Wordpress, … • No Drupal-specific rules  34
  • 35.
    ALWAYS ONLINE • Limitedversion of your site is always online • Only the most popular pages • No POST and SSL support • Crawler-based - crawling every 7, 3 or 1 day • Triggers: • HTTP status 502 or 504 • Connection timeout, SSL errors etc. 35
  • 36.
  • 37.
  • 38.
    NOT A SILVERBULLET • Logged-in users • Cache invalidation • Performance of non-cached pages 38
  • 39.
    CACHE INVALIDATION There areonly two hard things in Computer Science: cache invalidation and naming things. -- Phil Karlton (after http://martinfowler.com/bliki/TwoHardThings.html) 1. Cloudflare stores copy of a page in the cache 2. User changes this page 3. How can Cloudflare know that the page has changed? 39
  • 40.
    • 99.9% uptime •Defend against bots & spam • Handle traffic peaks • Decrease server load • Minimize bandwidth usage • Minify CSS and JS DOES IT SOLVE OUR NEEDS? 40
  • 41.
  • 42.
    PREPARING TO DEPLOYCLOUDFLARE 1. Cache expiration policy 2. Plan your URLs / pathauto config http://www.site.com/can-cache/... 3. Views expiration settings (Views Content Cache?) 4. Apache configuration (proper expiration of static content) 42
  • 43.
    Expire monitors content updates Expireinvokes hook_expire_cache() (cfpurge_expire_cache()) Cloudflare API: zone_file_purge • https://drupal.org/project/expire • https://drupal.org/project/cfpurge • Define „Cache everything” rule on Cloudflare • CFPurge still needs some work; only 16 installs  • Lack of Views integration 43 CACHE INVALIDATION: EXPIRE + CFPURGE
  • 44.
    CLOUDFLARE + DRUPAL:QUICK START Review Cloudflare performance settings (Auto Minify, Caching Level, Mirage, Polish, …) Review Cloudflare security settings (obfuscation, hotlink protection, …) Whitelist important IP addresses (monitoring, APIs, …) Create Cloudflare Rules (/admin/*, /user/*, …) Handle remote (client) IP address correctly Install & configure modules (cloudflare, CFPurge, expire) Change DNS delegation Create Cloudflare account 44
  • 45.
  • 46.
    TO DO –TASKS FOR COMMUNITY • 502 / 504 on errors (compatibility with Cloudflare Always Online) https://drupal.org/node/2268487 • Views expiration • Expire all views that use CT https://drupal.org/node/2146797 (won’t fix ) • Integrate Expire with Views Content Cache https://drupal.org/node/1786436 (won’t fix ) • Integrate blacklists with antispam modules (Mollom etc.) 46
  • 47.
    THANK YOU! 47 Łukasz Klimek E-mail:Lukasz@softinn.eu Mobile: +48 66 999 2096 Skype: casatm | Twitter @lklimek http://tinyurl.com/lklimek http://goo.gl/2dEgs7 Software Inn www.softinn.eu