This document discusses techniques for scaling out Apache web servers to improve performance and reliability. It covers adding redundancy with hardware components like RAID disk mirroring and redundant power supplies. It also discusses scaling out the application tier vertically by moving services to separate hosts, and horizontally by load balancing traffic across multiple servers. Load balancing can be done with techniques like DNS round robin, network load balancers, and load balancing appliances. The document also addresses session state management across servers and caching static content to improve performance.
My talk at ScaleConf 2017 in Cape Town on some tips and tactics for scaling WordPress, with reference to WordPress.com and the container-based VIP Go platform.
Video of my talk is here: https://www.youtube.com/watch?v=cs0DcY80spw
My talk at ScaleConf 2017 in Cape Town on some tips and tactics for scaling WordPress, with reference to WordPress.com and the container-based VIP Go platform.
Video of my talk is here: https://www.youtube.com/watch?v=cs0DcY80spw
XenServer Virtualization In Cloud EnvironmentsTim Mackey
= As presented at the CloudStack Silicon Valley Meetup in September 2015. =
XenServer is a virtualization platform which has been deployed in a variety of industries and to support a multitude of workloads. In this session we discuss some of the components which make it valuable not just for traditional server and desktop virtualization, but also within "the cloud". This includes discussion of VM density, network scalability, containers (such as Docker) and GPU virtualization. We end with coverage of how XenServer templates are represented within Apache CloudStack.
The speaker Jaskey(RocketMQ Committer) shared core features of RocketMQ, including domain model, order message, transactional message, batch message, message filter by SQL92 and so on.
Selecting the correct hypervisor for CloudStack 4.5Tim Mackey
Apache CloudStack supports multiple hypervisors out of the box, and the obvious question is which hypervisor is best for CloudStack. In this session we cover core CloudStack components such as networking, storage and virtualization functions to present which hypervisor is able to meet a given requirement. The core take-away is that with an understanding of the services to be delivered the correct hypervisor, or hypervisors, can be selected with relative ease. This deck is as delivered at CloudStack Days 2015 in Seattle.
Getting started with MariaDB? Whether it is on your laptop or server, containers are great ephemeral vessels for your applications. But what about the data that drives your business? It must survive containers coming and going, maintain its availability and reliability, and grow when you need it.
Kafka Tutorial: Streaming Data ArchitectureJean-Paul Azar
Kafka tutorial covers Java examples for Producers and Consumers. Also covers why Kafka is important and what Kafka is. Takes a look at the whole ecosystem around Kafka. Discusses low-level details about Kafka needed for successful deploys and performance tuning like batching, compression, partitioning, and replication.
XenServer Virtualization In Cloud EnvironmentsTim Mackey
= As presented at the CloudStack Silicon Valley Meetup in September 2015. =
XenServer is a virtualization platform which has been deployed in a variety of industries and to support a multitude of workloads. In this session we discuss some of the components which make it valuable not just for traditional server and desktop virtualization, but also within "the cloud". This includes discussion of VM density, network scalability, containers (such as Docker) and GPU virtualization. We end with coverage of how XenServer templates are represented within Apache CloudStack.
The speaker Jaskey(RocketMQ Committer) shared core features of RocketMQ, including domain model, order message, transactional message, batch message, message filter by SQL92 and so on.
Selecting the correct hypervisor for CloudStack 4.5Tim Mackey
Apache CloudStack supports multiple hypervisors out of the box, and the obvious question is which hypervisor is best for CloudStack. In this session we cover core CloudStack components such as networking, storage and virtualization functions to present which hypervisor is able to meet a given requirement. The core take-away is that with an understanding of the services to be delivered the correct hypervisor, or hypervisors, can be selected with relative ease. This deck is as delivered at CloudStack Days 2015 in Seattle.
Getting started with MariaDB? Whether it is on your laptop or server, containers are great ephemeral vessels for your applications. But what about the data that drives your business? It must survive containers coming and going, maintain its availability and reliability, and grow when you need it.
Kafka Tutorial: Streaming Data ArchitectureJean-Paul Azar
Kafka tutorial covers Java examples for Producers and Consumers. Also covers why Kafka is important and what Kafka is. Takes a look at the whole ecosystem around Kafka. Discusses low-level details about Kafka needed for successful deploys and performance tuning like batching, compression, partitioning, and replication.
In order to obtain the best performance possible out of your AEP server, the core architecture provides methods to reuse job processes multiple times. This talk will cover how the mechanism functions, what performance improvements you might expect as well as what potential problems you might encounter, how to use pooling in protocols and applications, and how the administrator or package developers can configure and debug specialized job pools for their particular applications
Where to start? - the first 2 hours of performance troubleshooting
• The performance cheat sheet: cover all the basics before you start
• Data collections and mining the logs
• Common techniques to improve performance
Like all frameworks, Drupal comes with a performance cost, but there are many ways to minimise that cost.
This session explores different and complementary ways to improve performance, covering topics such as caching techniques, performance tuning, and Drupal configuration.
We'll touch on benchmarking before presenting the results from applying each of the performance techniques against copies of a number of real-world Drupal sites.
Whats new in Microsoft Windows Server 2016 Clustering and StorageJohn Moran
In this webinar we will learn what the High Availability & Storage team in Microsoft has cooked up for us in Windows Server 2016, which is being launched at Microsoft Ignite at the end of September.
There’s lots of new stuff in this release, including better high availability for Hyper-V, greater control over resource utilization, improved fault tolerance of transient events, newer design options for stretch or multi-site clusters, a whole new way of doing software defined storage with SATA and NVMe drives, built-in block-level storage replication, and hyper-convergence without having to break the bank.
A recap of some of the most interesting things learned from the AWS re:Invent 2013 Conference. Easily the most intense and educational conference I've ever attended.
Jay Kreps on Project Voldemort Scaling Simple Storage At LinkedInLinkedIn
Jay Kreps on Project Voldemort Scaling Simple Storage At LinkedIn. This was a presentation made at QCon 2009 and is embedded on LinkedIn's blog - http://blog.linkedin.com/
TechTarget Event - Storage Architectures for the Modern Data Center - Howard ...NetApp
Keynote Presentation: How Storage Function Follows Architecture
Presented by Howard Marks, Founder and Chief Scientist, Deep Storage, LLC
Storage buyers today are faced with a broader variety of choices than ever before. Unfortunately, the architecture of the storage system they select will forever determine how well that system adapts to changes in their data center. While flash does make almost every storage system faster, the system's scalability, flexibility and manageability are determined not by the media but by the system's architecture.
This session will examine how storage system architectures predetermine how systems behave in the real world. We'll see how common storage architectures affect performance, scalability, quality of service, snapshots and vVol support.
A study and practice of OpenStack release Kilo HA deployment. The Kilo document has some errors, and it's hardly find a detailed document to describe how to deploy a HA cloud based on Kilo release. Hope this slides can provide some clues.
2. Agenda
• Introduction
• Redundancy in Hardware
• Building Out: Separate Tiers
• Building Out: Load Balancing
• Caching Content
• Conclusion
3. Introduction
• Why do This?
– Scalability (Oh my gosh, I’m so popular!)
– Reliability (We need five nines!)
• Why NOT do This?
– It costs money
4. Redundancy in Hardware
• Moving Parts Break
– Hard Disks
– Power Supplies
– Hard Disks
– Fans
– Hard Disks
• Buy High Quality Disks
– Refurbished, OEM, Brand Name
– Which has longer warranty?
– Which is more reliable?
5. Server Configuration
• Mirror those Disks
– Install the RAID utility
– Have it warn you
– RAID is no good if you don’t learn of
failures!
• Redundant Power Supplies
– On different circuits
10. Load Balancing Schemes
• DNS Tricks
• Peer Server Pools
– Network Load Balancing (Win2k3)
– Wackamole
• Load Balancing Appliance
– Box from F5, Juniper, Cisco, Foundry, …
– Linux Virtual Server
11. DNS Round-Robin
• Easy!
• Multiple A Records in DNS Zone File
• Not Smart:
– DNS Lookups are cached
– Load on Server
– Server Outage
12. Example Zone File
scalingout.org. 86400 IN SOA ns.scalingout.org. sctemme.scalingout.org. (
2006051401 ; Serial
86400 ; refresh (1 day)
7200 ; retry (2 hours)
8640000 ; expire (10 days)
86400 ) ; minimum (1 day)
scalingout.org. IN NS bagheera.scalingout.org.
gw IN A 10.11.0.1
bagheera IN A 10.11.0.2
; ...
mail IN CNAME bagheera
ns IN CNAME bagheera
www IN A 10.11.0.113
IN A 10.11.0.114
IN A 10.11.0.115
13. Peer-based: NLB
• Windows 2000 Server Enterprise Ed.,
Windows Server 2003
• Up to 32 hosts in cluster
• All hosts assume cluster IP, MAC
• NLB makes LB decision
– Only one host gets to answer TCP
handshake
• Should be application independent
14. Peer-based: Wackamole
• High Availability Solution
• When Host Fails
– Other hosts take over its IP addresses
– Distribute IP addresses among cluster
– Every IP address reliably available
• No Load Balancing!
– Use with RR DNS (or something)
http://www.backhand.org/wackamole/
16. Load Balancing
• One Load Balancer
• Many Web Servers
• Choice of Balancing Schemes
– Round-robin, Least Used, …
• Reliability
– Heartbeats, unavailable servers don’t
receive requests
• Feature War
17. Linux Virtual Server
• Free, Open Source, etc.
• IP Virtual Server module in kernel
• Lots of auxiliary modules
– Like a box of Legos
– May come with Your Distribution
• Do It Yourself
http://www.linuxvirtualserver.org/
18. Example: mod_proxy_balancer
• New in Apache HTTP Server 2.2
• Part of mod_proxy
• Two Load Balancing Methods
– By number of requests
– By number of bytes
• Detects failed backends
22. Tomcat Configuration
• Put same content on all Tomcats
• Edit conf/server.xml:
• jvmRoute must match jk worker
name!
<Engine name="Catalina" defaultHost="localhost" jvmRoute="tc1">
23. Problem: Session State
• HTTP is Stateless
• Apps use Sessions
– Cookies
– URL Encoding
• Session created on single server
– Broken by Load Balancing
– PHP: sessions stored on disk
24. Solutions: Session State
• “Sticky” routing on Load Balancer
• Store State in DB
• Put benign State in Cookie
– But don’t trust the client too much
• Replicate Sessions on Back-end
25. Tomcat Session Replication
• Share HttpSession objects across
instances
• One instance dies, session lives on
• Apache will route requests to other
instance
• Uses IP Multicast
26. Session Replication Config
• Uncomment <Cluster> element in
server.xml
• Put empty <distributable /> element in
<web-app> element in web.xml
27. Caching Content
• Dynamic Content is Expensive
• Static Content Relatively Cheap
• Several Approaches:
– Dynamic caching
– Pre-rendering popular pages
(index.rss…)
27
28. mod_cache Configuration
28
<IfModule mod_cache.c>
<IfModule mod_disk_cache.c>
CacheRoot /raid1/cacheroot
CacheEnable disk /
# A page modified 100 min. ago will expire in 10 min.
CacheLastModifiedFactor .1
# Always check again after 6 hours
CacheMaxExpire 21600
</IfModule>
</IfModule>
29. Make Popular Pages Static
• RSS Feeds
• Popular catalog queries
• … (Check your access log)
29
30. Static Page Substitution
30
<Directory "/home/sctemme/inst/blog/httpd/htdocs">
Options +Indexes
Order allow,deny
Allow from all
RewriteEngine on
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ /cgi-bin/blosxom.cgi/$1 [L,QSA]
</Directory>
31. Tuning the Database Tier
• Not my area (sorry)
• Give Money to Oracle Consultants
– (or MySQL) (or …)
• Tip: Separate Read and Write
Operations
– Replicate from Write db to Read db
– Read db data slightly stale
• Does it matter?
32. Putting it All Together
Client
Internet
Read-only
Write-only
33. Monitoring the Farm
• Monitor for outages
– More boxes, more failure
– HA can mask failures
• Monitor for performance
– Utilization
– Trends
34. Monitoring Solutions
• Nagios
– Check services, hosts for outage
– Highly configurable, extendable
– Worth your time investment
• Ganglia
– Monitor for performance
– See Brad Nicholes’s session
35. Monitoring Caveats
• Takes Time, Effort
– Highly flexible products
• You can’t fix it
– If you don’t know it’s broken
• You can’t tune it
– If you don’t know the bottlenecks
36. Conference Roadmap
• Monitoring 2.0 - Zenoss, the next level of IT
management (Training)
• Apache Performance Tuning Part 1:
Scaling Up
• Load-balancing with Apache HTTPD 2.2
and later
• Scaling the download infrastructure with
your success
• Break My Site