Oracle MySQL Tutorial -- MySQL NoSQL Cloud Buenos Aires Nov, 13 2014

ORACLE MySQL Enterprise
Tutorial
Manuel Contreras
Rico
manuel.contreras@oracle.com
Principal SW Consultant
ORACLE MySQL
Noviembre, 2014
Copyright © 2014 Oracle and/or its affiliates. All rights reservOerda.c l|e Confidential – Internal/Restricted/Highly Restricted

5 años de innovación Oracle
Mayor inversión e innovación al ecosistema
Jan 2010 Sept 2014
Copyright © 2014, Oracle and/or its affiliates. All rights reserved. |
2
2x Engineering Staff
3x QA Staff
2x Support Staff

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |
MySQL ~ Big Data
3

Oracle Confidential – Internal/Restricted/Highly Restricted
Arquitectura MySQL
4
MySQL Server

Oracle Confidential – Internal/Restricted/Highly Restricted
MySQL Storage Engines
Motores de Almacenamiento
5
MySQL Server

MySQL Storage Engines
• Pluggable architecture
6
InnoDB MyISAM NDBCluster
ACID
Propósito General
Not ACID
Bulk operations / Catálogo
consultas
ACID in-Memory DB
real time OLTP
Archive
Data Compression
Memory
Volatile Data
High Performance
3rd Party
Community

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 7
MySQL Replication
sharding & high availability
App
Writes & Reads
MySQL Master Server
Writes
Index & Binary
Log Rotation
MySQL Slave Server
Writes
Reads
Replication
Relay Log

Oracle database & MySQL
** Oracle eBusiness Suite
* Custom Ad-Hoc ERP / CRM CRM / ERP

Caracterísiticas
Herramientas de
Avanzadas administración
Soporte Técnico
• Scalability
• High Availability
• Security
• Audit
• Monitoring
• Backup
• Development
• Administration
• Migration
• Technical Support
• Consultative Support
• Oracle Certifications
MySQL Enterprise Edition
12

MySQL Scalability Thread Pool ( Performance )
Mejor escalabilidad en MySQL Enterprise
MySQL 5.5.16
Oracle Linux 6.1, Unbreakable Kernel 2.6.32
2 sockets, 24 cores, 2 X 12-core
Intel(R) Xeon(R) X5670 2.93GHz CPUs
72GB DDR3 RAM
2 X LSI SCSI Disk (MR9261-8i) (597GB)
mysql.com/products/enterprise/scalability.html

• Visión consolidada en todo el entorno de
MySQL, en las instalaciones y en la nube
• Automatizado, monitoring emite alertas
basado en normas (SMTP, SNMP)
• Query Analyzer, seguimiento, análisis y
puesta a punto, en correlación con los
gráficos del monitor
• Identifica los servidores problemáticos
• Monitor de replicación en tiempo real con
auto-descubrimiento de topologías
maestro-esclavo
MySQL Enterprise Monitor 3.0
Ahorra tiempo. Soluciona
problemas de manera proactiva.
MySQL Enterprise Edition : Monitor

MySQL Enterprise Backup
• Online, non-locking backup & recovery
– Complete MySQL instance backup (datos &
Config )
– Partial backup & restore
• Direct Cloud storage backups (S3, etc.)
• Incremental backups
• Point-in-time recovery
• Compressed & encryption
• Backup to tape (SBT)
• Backup validation
• Online backup MySQL (Windows, Linux,
Unix)
21

MySQL Workbench EE + Backup
• Programar & administrar
backup jobs
– Create new backup jobs
– Schedule backup jobs para
ejecución automática
– Visualiza backup jobs actuales
– Visualiza actividad backup
• Restore backups
– Full, partial, incremental …
13

MySQL Enterprise Monitor + Backup
14
• Monitor backup performance
• Asegurar que los backups
programados han sido
exitosos y están al día.

▪ Diagnostica problemas de desempeño código SQL
problemático.
▪ Correlación gráfica de código SQL
▪ Encuentra y optimiza cuellos de botella
▪ Identifica la fuente de problemas en desempeño
▪ Estadísticas detallas de ejecución de queries SQL
▪ Query Response Time index (QRTi)
Con el analizador de consultas MySQL, hemos sido
capaces de identificar y analizar el código SQL problemático
y triplicar nuestro rendimiento de base de datos. Más
importante aún, hemos sido capaces de lograr esto en tres
días, en lugar de tomar semanas
Keith Souhrada
Ingeniero de Desarrollo Software
Big Fish Games
MySQL Query Analyzer
mysql.com/products/enterprise/query.html

MySQL Enterprise Security
• SSL enabled
• Access control
– Enterprise authentication (PAM, Windows, LDAP, etc.)
– Proxy users
• Auditing & monitoring
– MySQL security advisors
–Oracle Audit Vault
• Oracle Database Firewall
16

MySQL Enterprise Encryption
• Librerías MySQL encryption
– Symmetric encryption AES256
– Public-key / asymmetric cryptography
• Administración de Keys PKI
–Generate public and private keys
– Key exchange methods: RSA, DSA, DH
• Sign & verify data
– Cryptographic hashing for digital signing, verification, & validation
17

MySQL Cluster CGE
18
SQL & NoSQL Flexibility
In-Memory
Database

MySQL: Sakila & El Chavo del 8
19
playfulplay.com/lavecindaddeelchavo/index.php
Caso de estudio:
MySQL Cluster CGE habilita un Sharding
transparente para la aplicación ( NDBCluster )
Facebook Game #1 Categoría Familiar
- Cloud Elástico bajo demanda
- In-memory database
- Arquitectura Non-Shared Disk
>10 MILLONES de suscriptores
>80 MIL jugadores concurrentes

La vecindad de El Chavo del 8
20

MySQL en la Nube The #1 RDBMS in the Cloud
SaaS
21
Hosting IaaS, PaaS

MySQL Enterprise Replication Monitor
22
• Auto-discover de topología de
replicación
• Monitor de replicación Master/Slave
( Replication advisor )
• Mejores prácticas en configuración de
Grupos de Replicación ( Async, Semi-sync
)
"I use the MySQL Enterprise Monitor
every day to monitor and keep tabs
on our MySQL databases. Quick one
stop shopping for keeping tabs on
them.”
-Wes Homer,
Sr System and Network
Administrator
mysql.com/products/enterprise/monitor.html

ORACLE — MySQL Integration
Oracle Enterprise Manager 12c
Oracle Linux
Oracle VM
Oracle Clusterware
Oracle Secure Backup
Oracle Fusion Middleware
Oracle Golden Gate
Oracle Audit Vault
Oracle Database Firewall
MyOracle Support

ORACLE Enterprise Manager 12c GA
24
Performance
Security
Availability
• Availability monitoring
• Performance monitoring
• Configuration monitoring
• Permite crear custom
threshold basados en
reportes de incidencia
• MySQL auto-detection

Soporte MySQL Enterprise
Technical support & Consultative Support
• El equipo más grande de ingenieros MySQL
• Apoyado por MySQL developers
• World-class support, en 29 languages ( Español )
• Hot fixes & maintenance releases
• 24x7 x 365
• Incidentes ilimitados
• Soporte Consultivo
• Global scale and reach
Ayuda inmediata para cualquier
problema de MySQL, además de
asesoría de expertos
41

Información Adicional
26
MySQL Products
http://www.mysql.com/products/
MySQL Enterprise
http://www.mysql.com/products/enterprise/
MySQL FAQ
http://dev.mysql.com/doc/refman/5.6/en/faqs-general.html

MySQL Enterprise Server
Arq. Referencia
27

Custom OLTP App
Load balancer:
Fabric-Aware Connectors: Java, PHP, .NET, ( C Labs ); otros Load Balancers
MySQL Fabric Controller
MySQL Enterprise Monitor
VLAN1
Arquitectura MySQL Enterprise Edition
Drools (Business Rule Management System)
Red para sincronización
de datos entre nodos MySQL
MySQL Enterprise
Primario
DAS LUN (1)
VLAN2 ( Privada )
MySQL Enterprise
Secundario
DAS LUN (2)
Red de servicio
Application Server & MySQL
OLAP
BI Reporting Analítica

Arquitectura MySQL Enterprise Edition — Detalle Técnico
PRODUCCION — MySQL Enterprise (2 Servidores virtuales / físicos)
CPU: 16 Cores CPU x86 64 bits; RAM: 16G ( o según tamaño de Base de datos / volumen de transacciones )
Storage: 1 Logic Unit asociada a un punto de montaje para datos $DATADIR, para cada unos de los nodos ( Primario &
Secundario), como disco no compartido que permita un alto performance de lectura / escritura ( p.e. RAID 10 )
OS: Linux varias distribuciones, Solaris, Mac OSX, MS Windows 2012 Server
Red: 2 puertos de Gigabit ethernet configurados con VLAN1 red de servicio & VLAN2 red privada de sincronización de datos
entre nodos
MySQL Fabric Controller / MySQL Enterprise Monitor ( 1 servidor virtual / físico )
CPU: 8 Cores CPU x86 64 bits; RAM: 8G; HDD Disco Interno
PRE-PRODUCCION ( QA ) — MySQL Enterprise
1 Servidores virtual / físico con:
CPU: 16 Cores CPU x86 64 bits; RAM: 16G ( o según tamaño de Base de datos / volumen de transacciones )
Storage: 1 Logic Unit asociada a un punto de montaje para datos $DATADIR de MySQL Enterprise Monitor & Fabric Controller
OS: Linux varias distribuciones, Solaris, Mac OSX, MS Windows 2012 Server
Red: VLAN1: red de servicio
Total 4 Servidores MySQL Enterprise Edition ( ambiente QA & PROD )

MySQL & High Availability
Oracle Confidential – Internal/Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

Oracle MySQL HA & Scaling Solutions
MySQL
Replication
MySQL
Fabric
Oracle VM
Template
Oracle
Clusterware
Solaris
Cluster
Windows
Cluster DRBD MySQL
Cluster
App Auto-Failover ✖ ✔ ✔ ✔ ✔ ✔ ✔ ✔
Data Layer Auto-Failover ✖ ✔ ✔ ✔ ✔ ✔ ✔ ✔
Zero Data Loss MySQL 5.7 MySQL 5.7 ✔ ✔ ✔ ✔ ✔ ✔
Platform Support All All Linux Linux Solaris Windows Linux All
Clustering Mode Master +
Slaves
Master +
Slaves
Active/
Passive
Active/
Passive
Active/
Passive
Active/
Passive
Active/
Passive
Multi-
Master
Failover Time N/A Secs Secs + Secs + Secs + Secs + Secs + < 1 Sec
Scale-out Reads ✔ ✖ ✖ ✖ ✖ ✖ ✔
Cross-shard operations N/A ✖ N/A N/A N/A N/A N/A ✔
Transparent routing ✖ For HA ✔ ✔ ✔ ✔ ✔ ✔
Shared Nothing ✔ ✔ ✖ ✖ ✖ ✖ ✔ ✔
Storage Engine InnoDB+ InnoDB+ InnoDB+ InnoDB+ InnoDB+ InnoDB+ InnoDB+ NDB
Single Vendor Support ✔ ✔ ✔ ✔ ✔ ✖ ✔ ✔
31

MySQL Enterprise HA : Shared Storage
32
• Stricter data durability, integrity
constraints
– Shared storage persists commits across
instances
– Clustering software manages data access
– Auto-failover of applications and database
– Deploy with MySQL Fabric for scale-out
• MySQL certified & supported solutions
– Oracle Clusterware
– Windows Failover Clustering
– Oracle Solaris Cluster
Virtual IP
Clients

MySQL Enterprise HA : Shared Nothing
33
• DRBD + Clustering
– Based on distributed storage, not a SAN
– Synchronous replication eliminates risk
of data loss
• Open source, mature, & proven
• Certified and fully supported by
Oracle
– DRBD integrated into Oracle Linux
Unbreakable Enterprise Kernel R2
– Pacemaker and Corosync for clustering /
failover
– Updates to stack via ULN channel

MySQL Enterprise HA : Oracle VM Templates
• Oracle Linux
• Oracle VM
• Oracle VM Manager
• Oracle Cluster File System 2 (OCFS2)
• MySQL Database (Enterprise Edition)
34
• Pre-installed & pre-configured
• Full integration & QA testing
• Single point of support
Oracle VM Server Pool
Oracle VM Servers
ocfs2
Oracle VM
Manager
SAN / iSCSI
Oracle VM
Secure Live
Migration (SSL)
Oracle VM
Automatic Fault
Detection &
Recovery

MySQL Cluster CGE: NDBCluster
35
• Automatic Sharding
• In-memory engine
• Sync replication
• Real-time OLTP
• 99.999 %
• Requires MySQL Cluster CGE
binaries / suscription
• NoSQL / SQL supported
• Minimum 3 physical servers, 4
production environments
MySQL Cluster Data Nodes
Clients
Application Layer
Management
Data Layer

MySQL Fabric 1.5.2
36

http://www.mysql.com/products/enterprise/fabric.html
MySQL Fabric
MySQL Fabric es un framework extensible & fácil de utilizar, para administrar ambientes
de clustering con MySQL. El framework implementa 3 funcionalidades: Tolerancia a
Fallas, Distribución de carga Sharding, & aprovisionamiento de entornos de manera
automática.
MySQL Fabric, gestiona también grupos de servidores con MySQL Replication y permite
un failover automático gestionado por el nodo MySQL Fabric Controller en combinación
a los Drivers Fabric-Aware: Java, PHP, Python & .NET y próximamente Lenguaje C.
El Fabric connector, puede en rutar transacciones en base a un criterio definido por el
desarrollador, permitiendo también escalar de manera horizontal en activo - activo en
múltiples grupos de data shards.
Por otro lado, la extensión Sharding de MySQL Fabric, permite que el desarrollador pueda
distribuir bases de datos, sobre un conjunto de servidores físicos gestionados por MySQL
Fabric, esto con el fin de poder escalar elásticamente agregando o reduciendo el número
de nodos en el grupo de servidores.
37

High Availability + Sharding-Based Scale-out
MySQL Fabric 1.5
• High Availability
• Auto-promotion failover & transparente
para la application failover
• Opcionalmente scale-out a través de
data sharding
– Application provee shard key
– Range ó Hash
• Conectores Fabric-aware: Python,
Java, PHP, .NET, C (labs)
– Lower latency, bottleneck-free
• Server provisioning a través de
OpenStack
MySQL Fabric
Connector
• Application
Read-slaves
mappings
SQL
HA group
Read-slaves
HA group
Connector
• Application
38

MySQL Fabric Framework (HA)
All Data
Primary Secondary
Extra Read
Replicas
MySQL Fabric
Controller
SQL Queries
State &
Routing Info
HA Group
Coordination
and Control
39

MySQL Replication & MySQL Fabric HA
• MySQL Replication is the initial implementation used in HA Groups
– PRIMARY = Replication Master & receives all writes
– SECONDARY = Replication Slave & receives share of reads
• Failover
– MySQL Fabric detects failure of PRIMARY/Master
– Selects a SECONDARY/Slave and promotes it
– Updates State Store
– Updated state fetched by Fabric-aware connectors
How this effects failover
40

High-Availability Group Concept
• Abstract Concept
– Set of servers
– Server attributes
• Connector Attributes
– Connection information
– Mode: read-only, read-write, ...
–Weight: distribute load
• Management Attributes
– State: state/role of the server
State: Primary
Mode: Read-Write
Host: server-1.example.com
41

Create HA Groups and add Servers
• Definiendo el grupo
mysqlfabric group create my_group
• Agregando servidores al grupo
mysqlfabric group add my_group server1.example.com
mysqlfabric group add my_group server2.example.com
42

Create HA Groups and add Servers
• Promoviendo un servidor a Primario Master
mysqlfabric group promote my_group
• Activación del FailOver detector
mysqlfabric group activate my_group
43

Demostración MySQL
Fabric
44

MySQL Fabric ( Sharding )
45

Benefits of Sharding
• Write scalability
– Can handle more writes
• Large data set
– Database too large
– Does not fit on single server
• Improved performance
– Smaller index size
– Smaller working set
– Improve performance (reads and
writes)
UID 10000-20000 UID 20001-40000
46

MySQL Fabric Sharding & Provisioning Features
• Connector API Extensions
– Support Transactions
– Support full SQL
• Decision logic in connector
– Reducing network load
• Shard Multiple Tables
– Using same key
• Global Updates
–Global tables
– Schema updates
• Sharding Functions
– Range ( Integer, String, datetime )
– Hash (Consistent)
• Shard Operations
– Shard move
– Shard split
• Server Provisioning
– Integrated with OpenStack & other
frameworks
47

MySQL Fabric Node
Sharding Architecture
Shards
Application
Global
Group
Global Updates
Shard
Updates
Replication
48
Connector
Connector
Connector

MySQL Fabric (HA + Sharding)
Global Data
Primary Secondary
Shard 1
Primary Secondary
Shard 2
Primary Secondary
MySQL Fabric
Controller
SQL Queries
Server/Shard State &
Mapping
Global Group HA Group
Coordination and
Control
HA Group
Extra Read Replicas Extra Read Replicas
49

Shard #1
Routing Transactions
Shard #2
Shard #3
Executor
App Server
Connector
Cache
State Store
App Server
Connector
Cache
50

MySQL Fabric: Sharding Setup
• Set up some groups
– my_global – for global updates
– my_group.N – for the shards
– Add servers to the groups
• Create a shard mapping
– A “distributed database”
– Mapping keys to shards
– Give information on what tables are sharded
• Add shards
51

MySQL Fabric:
Moving and Splitting Shards
• Moving a shard (id=5) from existing group to another (my_group.8)
mysqlfabric sharding move 5 my_group.8
• Splitting a shard (id=5) into two parts with new half stored in group
my_group.6
mysqlfabric sharding split 5 my_group.6
52

Connector API: Shard Specific Query
• Indicate tables to be used in query
– Property: tables
– Fabric will compute map
• Indicate read-only queries
– Property: mode
• Provide sharding key
– Property: key
– Fabric will compute shard
• Joins within the shard (or with
global tables) supported
conn.set_property(tables=["test.subscribers"], key=sub_no, mode=fabric.MODE_READONLY)
cur = conn.cursor()
cur.execute(
"SELECT first_name, last_name FROM subscribers WHERE sub_no = %s", (sub_no)
)
for row in cur:
print row
53

Connector API: Global Update
• Set global scope
– Property: scope
– Query goes to global group
conn.set_property(tables=[], scope='GLOBAL')
cur = conn.cursor()
cur.execute("ALTER TABLE test.subscribers ADD nickname VARCHAR(64)")
54

> mysqlfabric provider register
my_stack
my_user my_password
http://8.21.28.222:5000/v2.0/
--tenant=my_user_role
--provider_type=OPENSTACK
> mysqlfabric machine create
my_stack
--image id=8c92f0d9-79f1-4d95-
b398-86bda7342a2d
--flavor name=m1.small
> mysqlfabric machine list my_stack
• Fabric creates new machines,
& MySQL Servers
– Initially using OpenStack Nova
– Other frameworks on the way
(OpenStack Trove, AWS,…)
• Server setup
– Clones slave
– Sets up replication
– Performs custom operations
55
Server Provisioning – OpenStack Nova Integration

MySQL Fabric executor
• Event driven
– Events will trigger execution of
procedures
– Procedures can trigger events themselves
– Each step of a procedure is called a job
• Procedures
– Written in Python
– Interacts with servers
– Write state changes into backing store
– Lock manager for conflict resolution
• Conservative two-phase locking strategy
• Avoid deadlocks
Backing
Store
Queue
Events

Example of User-Defined Executor Script:
Automatically replacing a server in a group on failure
• Register procedure for event
– @on_event decorator
– Accept event to register for
• Fetch the group the server
belonged to
• Fetch a new server from the
provider
• Add the server to the group
@on_event(SERVER_LOST)
def _add_server(group_id, server_uuid):
group = Group.fetch(group_id)
machines = PROVIDER.create_machines(
parameters
)
server = MySQLServer( server_uuid,
address
)
MySQLServer.add(server)
group.add(server)
_configure_as_slave(server)

AMQP? MySQL XML-RPC
MySQL Fabric Node
Extensible Architecture
Connector
Connector
Connector
MySQL Fabric
Framework
Executor State Store
(persister)
Shard
HA
Prov
???
MySQL
Backing
Store
SQL
58

MySQL Fabric: Goals & Features
• Connector API Extensions
– Support Transactions
– Support full SQL
• Fabric-Aware Connectors at GA:
– PHP + Doctrine, Python, Java + Hibernate,
.NET, C (pre-GA)
• Decision logic in connector
– Reducing latency &network load
• Load Balancing
– Read-Write Split
– Distribute transactions
• Global Updates
– Global data
– Schema updates
• Sharding Functions
– Range
– (Consistent) Hash
• Shard Operations
– Shard move
– Shard split
• Server Provisioning
– OpenStack Integration (& other
frameworks)
59

MySQL Fabric – Current Limitations
• Routing is dependent on Fabric-aware
connectors
– Currently Java (+ Hibernate), PHP (+
Doctrine), Python, .NET & C (labs)
• MySQL Fabric node is a single (non-redundant
process)
– HA Maintained as connectors continue to
route using local caches
• Establishes asynchronous replication
– Manual steps to switch to semisynchronous
• Sharding not completely transparent
to application (must provide shard
key – column from application
schema)
• No cross-shard joins or other queries
• Management is through CLI, MySQL
protocol or XML/RPC API
– No GUI
60

Oracle MySQL HA & Scaling Solutions
MySQL
Replicatio
n
MySQL
Fabric
Oracle VM
Template
Oracle
Clusterwar
e
Solaris
Cluster
Windows
Cluster DRBD MySQL
Cluster
App Auto-Failover ✖ ✔ ✔ ✔ ✔ ✔ ✔ ✔
Data Layer Auto-
✖ ✔ ✔ ✔ ✔ ✔ ✔ ✔
Failover
Zero Data Loss MySQL 5.7 MySQL 5.7 ✔ ✔ ✔ ✔ ✔ ✔
Platform Support All All Linux Linux Solaris Windows Linux All
Clustering Mode Master +
Slaves
Master +
Slaves
Active/
Passive
Active/
Passive
Active/
Passive
Active/
Passive
Active/
Passive
Multi-
Master
Failover Time N/A Secs Secs + Secs + Secs + Secs + Secs + < 1 Sec
Scale-out Reads ✔ ✖ ✖ ✖ ✖ ✖ ✔
Cross-shard operations N/A ✖ N/A N/A N/A N/A N/A ✔
Transparent routing ✖ For HA ✔ ✔ ✔ ✔ ✔ ✔
Shared Nothing ✔ ✔ ✖ ✖ ✖ ✖ ✔ ✔
Storage Engine InnoDB+ InnoDB+ InnoDB+ InnoDB+ InnoDB+ InnoDB+ InnoDB+ NDB
Single Vendor Support ✔ ✔ ✔ ✔ ✔ ✖ ✔ ✔
61

MySQL Fabric Resources
• Download and try
http://dev.mysql.com/downloads/fabric/
• Documentation
http://dev.mysql.com/doc/mysql-utilities/en/fabric.html
• MySQL Fabric on the web
http://www.mysql.com/products/enterprise/fabric.html
• Forum (MySQL Fabric, Sharding, HA, Utilities)
http://forums.mysql.com/list.php?144
• Tutorial: MySQL Fabric - adding High Availability and Scaling to MySQL
http://www.clusterdb.com/mysql-fabric/mysql-fabric-adding-high-availability-and-scaling-to-mysql
• White Paper: MySQL Fabric - A Guide to Managing MySQL High Availability and Scaling Out
http://www.mysql.com/why-mysql/white-papers/mysql-fabric-product-guide
• Webinar Replays
http://www.mysql.com/news-and-events/on-demand-webinars/#en-20-41
62

ORACLE MySQL Enterprise
Manuel Contreras Rico
manuel.contreras@oracle.com
Principal Software Consultant
Latinoamérica & El Caribe ORACLE MySQL
Noviembre, 2014
Copyright © 2014 Oracle and/or its affiliates. All rights reservOerda.c l|e Confidential – Internal/Restricted/Highly Restricted

Oracle MySQL Tutorial -- MySQL NoSQL Cloud Buenos Aires Nov, 13 2014

More Related Content

What's hot

Viewers also liked

Similar to Oracle MySQL Tutorial -- MySQL NoSQL Cloud Buenos Aires Nov, 13 2014

Recently uploaded

Oracle MySQL Tutorial -- MySQL NoSQL Cloud Buenos Aires Nov, 13 2014