Akshay Mathur gives a presentation on techniques for scaling applications with security and visibility in the cloud. He discusses 8 growth phases applications typically go through including load balancing, gaining insights, content optimization, offloading services, content switching, preventing bot traffic and DDoS attacks, continuous delivery, and the need for a unified cloud application front end solution to manage these phases. He introduces Appcito CAFE as a service that provides capabilities across availability, performance, security and DevOps to simplify application scaling in the cloud.

1. Techniques for Scaling Application
with Security and Visibility
in Cloud
Akshay Mathur
@akshaymathu of @appcito

2. Let’s Know Each Other
• Do you Manage applications?
• Hosting providers?
• Priorities?
• Tools?
• Why are you attending?
• What are your Goal?
• Happy Users, Happy DevOps, Happy Servers
2@akshaymathu

3. Akshay Mathur
• 15+ years in IT industry
• Currently Product Manager at Appcito
• Mostly worked with Startups
• From Conceptualization to Stabilization
• At different functions i.e. development, testing, release, marketing, devops
• With multiple technologies
• Founding Team Member of
• ShopSocially (Enabling “social” for retailers)
• AirTight Neworks (Global leader of WIPS)
@akshaymathu 3

4. Ground Rules
• Tweet now: #TechNext @akshaymathu @appcito
• Disturb Everyone later
• Not by phone rings
• Not by local talks
• By more information and questions
@akshaymathu 4

5. How Applications are Changing

6. Traditional Application
• Monolithic components
• All application layers in a box
• Complex objects
• Box specific sessions
• Designed for vertical scale
• Self managed deployment
@akshaymathu 6

7. New Age Scalability
@akshaymathu 7

8. Cloud Computing Landscape
@akshaymathu 8

9. Architectural Mind-shift
@akshaymathu 9

10. Modern Application
• Light weight services
• Application layers designed for
network communication
• Cloud deployment
• Designed for horizontal scale
@akshaymathu 10

11. @akshaymathu 11

12. Growing Applications

13. Growth Phase 1: Load Balancing
• Replicate the box
• Have a load balancer
@akshaymathu 13

14. Questions before Growing Further
• About Insights:
• Are all server instances healthy?
• When should I add more servers?
• What is the traffic volume and its
pattern?
• What areas of application are used
most?
• What are problematic areas?
• Who access my application?
• What devices, browsers, apps are in
use?
• About Optimization:
• How can I serve more traffic using
existing servers?
• Does all the serves must be of same
type, running same code?
• Can the content be compressed,
cached?
• What to do for optimizing content for
various devices?
• Do I really need to redirect traffic to a
different URLs for specific servings?
• Does managing so many URLs for same
functionality makes sense?
• Can someone take care of SSL
termination?
@akshaymathu 14

15. Growth Phase 2: Insights
• Google Analytics, Statcounter etc. only provide
information after page load
• Information about programmatic access is missing
• Access logs provide true information about traffic
• Logs are typically in each box rather than a central place
• Difficult to read; log parsers also provide minimal
information
• Need to push logs to some analytics engine and
configure analytics engine for getting meaningful
information out
@akshaymathu 15

16. Growth Phase 3: Content Optimization
• Compressing the response
• Optimizing images
• In-lining the external resources
• JS
• CSS
• Images as base64
• Caching (as needed)
• Prefetching (if possible)
• Google’s PageSpeed does it well for HTML pages
@akshaymathu 16

17. Growth Phase 4: Offloading
• SSL Handshake
• Encryption and Decryption
• Connection handling
• Content optimization
• Anything that can be done asynchronously
e.g. sending email, tweets etc.
• Point solutions are available for each of
these
@akshaymathu 17
App Servers
Apache + Pylons
Message Queue
RabbitMQ
Background
Worker Nodes
Celery
SSL Terminator
Content Optimizer

18. Growth Phase 5: Content Switching
• Serve different content from different servers (reverse proxy)
• Static files (JS, CSS, Images) may be served from a web server; App server is not
needed
• High frequency requests may be served from different server
• Different app servers may be used for the use case they are optimized for
• Have different set of servers for different geographies
• Dedicate a few servers for specific customer
• Dedicate servers for specific functions e.g. authentication, API serving etc.
• HA Proxy is most popular tool here
• NginX is also used as reverse proxy
@akshaymathu 18

19. Web Servers
NginX
App Servers
Mongral + Brubeck
App Servers
Apache + Pylons
Web Servers
Apache + Wordpress
NoSql Datastore
Redis
NoSql Datastore
MongoDB
Sql Datastore
MySql
Corporate
website
Main dynamic
content
High frequency
requests
High speed storage Main Storage
Content Switching
Reverse Proxy

20. Growth Phase 6: Denying BOT Traffic
• Traffic from bad BOTs is about 30%
• Amounts to 30% wastage of server
resources
• Various fingerprinting techniques
are there for identifying the BOTs
• IP reputation
• UA analysis
• Pattern analysis
• JS insertion
• Advance algorithms
@akshaymathu 20

21. Growth Phase 7: Preventing Data Theft
• Typical ways are:
• SQL/object injection
• Cross Site Scripting (XSS)
• File include
• Malware inclusion
• Exploiting vulnerabilities of coding, framework,
language, platform
• Scan the deployment regularly
• Fix any vulnerability by applying patches
• Use Web Application Firewall (WAF)
@akshaymathu 21

22. Growth Phase 8: Preventing from DDoS Attack
• Volumetric attack
• Many clients make connections with
server
• Clients send huge traffic to the server
• Traffic is typically bogus
• Prevention
• Rapidly increase scale to consume
connections/traffic
• Rate limit connections/requests
• Delay/Deny bogus traffic
• Blacklist BAD clients
• Protocol exploits
• Attacker crafts traffic knowing the
timeouts and limits of protocol
• Slow moving bogus traffic hogs
resources of server
• Prevention
• Setup policy to apply aggressive limits
and timeouts in case of heavy load
• Terminate connection when unusual
behavior is observed
• Blacklist BAD client
@akshaymathu 22

23. Growth Phase 7: Continuous Delivery
• Upgrade the system without disturbing availability
• Why Continuous Delivery?
@akshaymathu 23

24. Continuous Delivery
• Considerations:
• Zero down time
• Even a little downtime means a lot for
high volume applications
• Seamless re-orientation of live traffic
from old to new deployment
• User experience has to be smooth
• Easy roll back
• Minimize the impact in case something
goes wrong
• Technique: Blue Green deployments
• Deploy old and new version in parallel
and switch the traffic
• Switch using DNS
• Switch using fixed NATed IP addresses
• Switch using external tools like load
balancer or reverse proxy

25. 25@akshaymathu

26. App & Traffic
Metrics
What is Needed Overall?
26
Availability Performance Security DevOps
Advanced Load
Balancing
Content Switching
Application Fluency
Elastic & Self-Scaling
Continuous
Deployment
Request Mirroring
Request Replay
Programmable
Policies
Per Application
Control
Front-End
Optimization
Mobile and Web
Client App
optimization
Caching &
compression
Predictive API
caching
Application & Server
offloading
Application Firewall
Elastic SSL
Anomaly Detection
DDoS Prevention
BOT Protection
Trends &
Correlations Anomalies
Policy
Recommendations
Analytics & Insights

27. CDN
Custom Scripts, Rules, Alert Management Aggregation across instances
Application Front-End Architecture
• Spaghetti of point solutions
• Multiple points of failure, redundancy difficult to setup
• Not elastic and cloud native
@akshaymathu 27

28. CDN
Application Front-End Architecture with CAFE
• All services for application under one consolidated product
• Easy Activation of capabilities closer to application
• Application policy is coordinated across services and policy enforced
@akshaymathu 28
Availability Security Performance Continuous
Deployment
Appcito Cloud Application Front-End (CAFE)

29. Cloud Application Front End
(CAFE)
Taking Cloud Applications from Good to Great

30. Appcito CAFE Service
Insights &
Analytics
Content
Optimization
Application
Security & DDoS
Prevention
Unified Functionality Available As
SaaS Delivery
Simple Activation
No Code Change
For
Dev /Ops
Cloud-agnostic
App Owner
Elastic
Continuous
Delivery
Availability &
Elasticity

31. Typical Deployment
Customer’s Cloud
Customer’s
End Users
app
server
app
server
Load
Balancer
app
server
DNS
Network Subnet
Availability Zone

32. Deployment with CAFE
Customer’s Cloud
Customer’s
End Users
app
server
app
server
Load
Balancer
app
server
Appcito Cloud
CAFE Barista
Management, Control, Analytics
DNS
CAFE
PEP
Network Subnet
Availability Zone

33. CAFE Configuration Model
• Think Out of the box (literally)
• Think in terms of
• Applications
• Traffic flow
• Request patterns
• Forget about
• Box provisioning
• Box configuration
• Networking flow
• L2/L3 access control
@akshaymathu 33

34. Production A (Blue)
Production B (Green)
Launch
Upgrade
Traffic Splitting
80% 20%
Appcito CAFE
80%
20%
CAFE Blue/Green Technique
• Steer traffic NOT switch
• Test with production traffic
• Move with confidence
• Compare performance and take informed
decisions

35. App & Traffic
Metrics
Appcito CAFE Service Capabilities
35
Availability Performance Security DevOps
Advanced Load
Balancing
Content
Switching
Application
Fluency
Elastic & Self-
Scaling
Continuous
Deployment
Request
Mirroring
Request Replay
Programmable
Policies
Per Application
Control
Front-End
Optimization
Optimization for
client
Caching &
compression
Predictive caching
Application &
Server offloading
Application
Firewall
Elastic SSL
Anomaly
Detection
DDoS
BOT Protection
Trends &
Correlations
Anomalies
Detection
Policy
Recommendation
Analytics & Insights

36. Thanks
@akshaymathu 36
@akshaymathu
akshay@appcito.com