MySQL Web Reference Architecture

MySQL Web Reference Architecture
Best Practices for Innovating on the Web
Ricky Setyawan
MySQL Principal Sales Consultant
Ricky.setyawan@oracle.com
Copyright © 2013, Oracle and/or its affiliates. 1 All rights reserved.

Safe Harbour Statement
THE FOLLOWING IS INTENDED TO OUTLINE OUR GENERAL PRODUCT
DIRECTION. IT IS INTENDED FOR INFORMATION PURPOSES ONLY,
AND MAY NOT BE INCORPORATED INTO ANY CONTRACT. IT IS NOT A
COMMITMENT TO DELIVER ANY MATERIAL, CODE, OR
FUNCTIONALITY, AND SHOULD NOT BE RELIED UPON IN MAKING
PURCHASING DECISIONS. THE DEVELOPMENT, RELEASE, AND
TIMING OF ANY FEATURES O FUNCTIONALITY DESCRIBED FOR
ORACLE’S PRODUCTS REMAINS AT THE SOLE DISCRETION OF
ORACLE.

2x DATA GROWTH EVERY 14
MONTHS
6.7BN MOBILE SUBS IN 2012
1.2 BILLION iOS & ANDROID APPS DOWNLOADED
PER WEEK
$1TR BY 2014
$700BN IN 2011
85% HANDSETS SHIPPED WITH A
BROWSER
2.4BN USERS
$1.7TR GLOBAL CONTRIBUTION
70+ NEW DOMAINS EVERY 60 SECONDS
350m TWEETS PER DAY
875k TPM DURING US
PRESIDENTIAL ELECTION
1BN+ USERS
20M APPS PER DAY
72 HOURS UPLOADED
EVERY MINUTE

MySQL: Powering the Web & the Cloud

MySQL Web Reference Architectures
Best practices for innovating in
web & mobile services
– Fast Time to Market
– Open, Agile and Highly Adaptable
– Scalable, secure and highly available
– Reduced Cost, Risk & Complexity
On-Premise or in the Cloud

Reference Architectures: Design Patterns
 On-Line Retail
– Small, Medium & Large: Database load & size
– User Authentication & Session Management
– Content Management
– eCommerce
– Analytics & Big Data Integration
 Social Networking
 Operational Best Practices
 Documented in Reference Architecture Guide

Reference Architecture Sizing

Small: Web Reference Architecture

Small: Web Reference Architecture
 Single server supporting:
– Users & sessions
– eCommerce
– Content management
MySQL replication
– Analytics and Backups
 If traffic volumes increase, scale session
management first on a separate server
Limited Scalability
Recommended to start with the Medium Architecture

MySQL Default Storage Engine
InnoDB
 Data integrity: Fully ACID-compliant, crash-safe, Foreign Keys
 High concurrency: Row level locking, MVCC
 Scale up: 48+ threads, Scale-Out: MySQL replication
 Flexible: On-Line DDL, Full-text search, SQL & NoSQL APIs
 Instrumented & Monitored: Performance_Schema,
MySQL Enterprise Monitor

3 x Core Services
Security
Auditing
Backup

MySQL Enterprise Security
External Authentication
 MySQL Authentication API – enables addition of user authentication
- MySQL Enterprise Security provides ready-made authentication modules
 PAM (Pluggable Authentication Modules)
- Access external authentication methods
- Standard interface (Unix, LDAP, Kerberos, others)
- proxied and non-proxied users
 Windows
- Access native Windows services
- Authenticate users already logged into Windows (Windows Active Directory)
Integrates MySQL with existing security infrastructures and SOPs.

MySQL Enterprise Audit
Policy-based Auditing for MySQL Applications
 MySQL Audit API– enables users to add auditing
 MySQL Enterprise Audit provides ready-made policy auditing
 Out-of-the-box logging of connections, logins, query activity across
all or specific MySQL servers
 User defined policies, filtering and log rotation
 Dynamically enabled, disabled: no server restart
 XML-based audit stream
Adds regulatory compliance to MySQL applications.
HIPAA, Sarbanes-Oxley, PCI, etc.

MySQL Enterprise Audit Flow
1. DBA Enables on Server1
2. User Joe Connects and Queries Server1
Server1
3. Joe’s connection, query logged

MySQL Enterprise Backup
 Online, high performance backup for InnoDB (scriptable interface)
 Full, Incremental, Partial Backups (with compression)
 Point in Time, Full, Partial Recovery options
 Enterprise Advisor Monitoring and Alerts on Backup Operations
 Metadata on status, progress, history
 Unlimited Database Size
 Cloud backup to Amazon S3
 Cross-Platform
- Windows, Linux, Unix
MEB Backup
MEB Backup
Files
Files
MySQL
MySQL
Database Files
Database Files
mmyyssqqlblbaacckkuupp
Ensures quick, online backup and recovery of your on premise and Cloud
based MySQL applications.

Medium: Web Reference Architecture

Best Practices - Overview
Medium Web Reference Architecture
 Server ratio: 10 application servers to each MySQL Server
– More for PHP applications, fewer for Java
– Add more slaves as the application tier scales
 Caching layer deployed in session & content management
components
– Memcached or Redis are common
– Reads fulfilled from cache, relieving load on the source database
servers
– NoSQL Memcached APIs can enable compression of caching and
data layers

Best Practices – Content Management
 Each slave can handle around 5,000 concurrent
users
 Each master handles 20 slaves
– More slaves are often used in larger environments
MySQL Replication for high availability & scale-out
Meta data of content assets managed by MySQL
 Distributed File System / CDN / Cloud (i.e. S3) for
physical storage of assets on
– High quality SAN (redundancy for HA)
– Across commodity nodes
XOR

Best Practices – Sessions & eCommerce
 Session Management & eCommerce
– Session data maintained for up to 1 hour in a dedicated partition, rolling
partitions used to delete aged data
 NoSQL Memcached API for InnoDB can be used to scale session
management
– eCommerce HA
 Semi-Synchronous replication or clustering (DRBD, shared storage, etc.)
– If web traffic grows, move Session Management to MySQL Cluster
 Persist session data for real-time personalization of user experience
 99.999% availability and in-memory data management can reduce need
for DRBD & caching
 Data is captured in Analytics Database, increasingly replicated to Big
Data system for analysis

MySQL HA
Solutions

MySQL Replication
 Duplicates database from a “master” to a “slave”
– Redundant copies of the data provide foundation for High
Availability
– Scale out by distributing queries across the replication cluster
Master
Slaves
Web / App Servers
Writes & Reads Reads

High Availability & Scalability
MySQL Replication
 Scale-out within and across data centers
 Self-healing and crash-safe
 Multiple topologies supported
– Master/Slave, Cascading, Circular
 Asynchronous as default with Semi-
Synchronous as an option
 Replication utilities for fast provisioning
 Monitoring & best practices with MySQL
Enterprise Monitor
Relay Log

MySQL 5.6: Evolving Replication

Shared-Disk Clustering for HA
• Stricter data durability, integrity constraints
– Shared storage persists commits across
instances
– Clustering software manages data access
– Auto-failover of applications and database
– Deploy with MySQL replication for slave
scale-out
• MySQL Certified & Supported
Solutions
• Oracle VM Template
• Windows Failover Clustering
• Oracle Solaris Cluster
VIP
Clients

Shared Nothing Clustering for HA
 DRBD + Clustering
– Based on distributed storage, not a SAN
– Synchronous replication eliminates risk of
data loss
 Open source, mature & proven
 Certified and fully supported by Oracle
– DRBD integrated into Oracle Linux
Unbreakable Enterprise Kernel R2
– Pacemaker and Corosync for clustering /
failover
– Updates to stack via ULN channel

Large: Web Reference Architecture

MySQL Cluster for Sessions & eCommerce
MySQL Cluster Data Nodes
Clients
Application Layer
Management
Data Layer

MySQL & Hadoop Use-Case in Web Architecture
Users
Browsing
Recommendations
Profile,
Purchase
History
Web Logs:
Pages Viewed
Comments Posted
Social media updates
Preferences
Brands “Liked”
Recommendations
Telephony Stream

MySQL in the Big Data Lifecycle
DECIDE
BI Solutions
ACQUIRE
Applier
ANALYZE ORGANIZE

New: MySQL Applier for Hadoop
 Real-time streaming of events from
MySQL to Hadoop
– Supports move towards “Speed of
Thought” analytics
 Connects to the binary log, writes
events to HDFS via libhdfs library
 Each database table mapped to a Hive
data warehouse directory
 Enables eco-system of Hadoop tools to
integrate with MySQL data
 See dev.mysql.com for articles
 Available for download now
– labs.mysql.com

 10x faster read/write access to
InnoDB or MySQL Cluster
 Bypasses SQL parsing
 Fast look-ups and data
ingestion
 Maintain ACID guarantees
 Maintain SQL for complex
queries
 Combine with On-Line DDL for
schema evolution
NoSQL APIs for MySQL

MySQL Enterprise Scalability
MySQL Thread Pool
 MySQL default thread-handling – excellent performance, can limit
scalability as user connections grow
 Connections assigned to 1 thread for the life of the connection, same thread used
for all statements
 Thread Pool API, enables users to build their own thread pool
 MySQL Thread Pool improves sustained performance/scale as user
connections grow
 Thread Pool contains configurable number of thread groups, each manages up to
4096 re-usable threads
 Threads are prioritized, statements queued to limit concurrent executions, load on
server, improve scalability as connections grow

With Thread Pool Enabled
More than 20x Better Scalability with Thread Pool

Extra Large: Social Network Reference Architecture

Extra Large: Social Network

Best Practices
Social Networking Reference Architecture
 Introduces Sharding
– Implemented at the application layer for scaling very high volume of
writes
– Data divided into smaller sets, distributed across low-cost hardware
– Shards based on Hash of a single column – ie. User ID
 “Functional” sharding also an option
 Sharding only needed in a small percentage of workloads
– MySQL scale-up, coupled with replication will get users a long way!
– Most Web and mobile workloads are still read-intensive, ie record is
read before updates applied
– MySQL Cluster supports Auto-Sharding + Cross-Shard JOINs

MySQL Enterprise Edition
Highest Levels of Security, Performance and Availability
MySQL Enterprise
Oracle Premier
Lifetime Support
Oracle Product
Certifications/Integrations
MySQL Enterprise
High Availability
Security
MySQL Enterprise
Scalability
MySQL Enterprise
Monitor/Query Analyzer
MySQL Enterprise
Backup
MySQL Workbench
MySQL Enterprise
Audit

MySQL Enterprise Monitor
 Real-time MySQL performance and availability monitoring
 Visually find & fix problem queries
 Disk monitoring for capacity planning
 Cloud friendly architecture (no agents)
 Start monitoring MySQL in 10 minutes
 Remote agent option provides OS monitoring
New!

Automated Advisors and Alerts
MySQL Cluster
Monitors and Advises on
status/ performance of
MySQL Cluster Data Nodes
14 Advisors, 250+ Rules, 80+ MySQL & OS charts
Custom
Saves time writing, deploying, versioning, maintaining custom scripts.
Administration
Optimal Start up and
Run time Configuration
Security
Unplanned Security
Changes/Loopholes
Upgrade
Bugs/Upgrades that
affect current installation
Replication
Master/Slave Latency
Memory Usage
Monitors and advises on
optimal memory/cache
settings
Schema
Unplanned Schema
Change
Performance
Optimal Performance
Variable Settings
Built by DBA to Enforce
Organization specific
best practices
Backup
Backup/Recovery
processes
Query Analysis
queries and queries
analysis
Availability
MySQL process and
connection availability
Agent
Checks status of each
MySQL Enterprise
monitor agent
Operating System
Host operating system
performance

MySQL Query Analyzer
 Real-time query performance
 Visual correlation graphs
 Find & fix expensive queries
 Detailed query statistics
 Query Response Time index (QRTi) “With the MySQL Query Analyzer, we were able to
Improved
identify and analyze problematic SQL code, and triple
our database performance. More importantly, we were
able to accomplish this in three days, rather than
taking weeks.”
Keith Souhrada
Software Development Engineer
Big Fish Games

Oracle Premier Lifetime Support for MySQL
Rely on the Experts
 Largest Team of MySQL Experts
 Backed by MySQL Developers
 Forward Compatible Hot Fixes
 MySQL Maintenance Releases
 Direct Access to MySQL Support Engineers
 Support in 29 Languages
 24/7/365
 Unlimited Incidents
 Knowledge Base
 MySQL Consultative Support
Only From
Oracle
"The rep that assisted me was simply
outstanding. He immediately recognized
the cause of my problem and provided the
resolution.”
mysql.com/support/quotes

Next Steps
MySQL Web Reference Architectures Guide
http://www.mysql.com/why-mysql/white-papers/mysql_wp_high-availability_webrefarchs.php
MySQL 5.6: Developer and DBA Guide
http://www.mysql.com/why-mysql/white-papers/whats-new-mysql-5-6/
 Engage MySQL Consulting
http://www.mysql.com/consulting/

Reference Architecture - Summary
Designed as a springboard to
innovating on the web
Based on insight from most
successful web properties
Best practices & repeatable
technologies for Scale & HA

MySQL Web Reference Architecture

More Related Content

What's hot

Viewers also liked

Similar to MySQL Web Reference Architecture

Recently uploaded

MySQL Web Reference Architecture