Arquiteturas de Referência Web
Conjunto de boas práticas
Airton Lastori
fev-2015
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |2
Safe Harbour Statement
The following is intended t...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |3
s
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |4
Agenda
 Requisitos Web, níveis de escala
 Arquit...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |5
Desdobramentos na Infraestrutura
Ágil, aberta & a...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |6 Fonte: http://db-engines.com/en/ranking_trend (dez...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |7
Os
MELHORES
COMPONENTES
da pilha
MySQL: Aplicações...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |8
Alguns clientes MySQL
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |9
MySQL ISV Partners in LAD
Brasil
América
Central
A...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |11
Edward Screven & Tomas Ulin no Open World 2014
 ...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |12
@MySQLBR
meetup.com/MySQL-BR
facebook.com/MySQLBR...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |13
Arquiteturas de Referência
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |14
Referências de Escala
Social Network
Small Medium...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |15
Pequena Escala
Social Network
Small Medium Large ...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |16
Pequena Escala
 Um único servidor para:
– Perfis...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |17
Replicação MySQL
 Duplique databases de um “mast...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |18
Boas Práticas
 Use o Storage Engine correto
– In...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |19
20082008
até 4 CPU
MySQL 5.0
MySQL AB
até 16 CPU
...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |22
MySQL 5.5.28
MySQL 5.6.7
+234% Ganhos de Performa...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |23
MySQL 5.5.28
MySQL 5.6.7
+151% Ganhos de Performa...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |24
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |25
Performance
Security
Availability
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |26
MySQL Enterprise Backup
 Backup online para Inno...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |28
2. Usuário conecta-se e executa queries
MySQL Ent...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |30
Authenticate
CREATE USER win_joe
IDENTIFIED WITH ...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |32
60x Melhor Escalabilidade comThread Pool
MySQL 5....
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |33
18x Melhor Escalabilidade com Thread Pool
Thread ...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |34
MySQL Enterprise Encryption
Standards Based Crypt...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |35
Média Escala
Social Network
Small Medium Large Ex...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |36
Média Escala
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |37
Boas Práticas
 Proporção de 10 servidores de apl...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |38
Boas Práticas – Gerenciador de Contéudo
Arquitetu...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |39
Boas Práticas – Sessões e e-Commerce
 Gerenciame...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |40
Alta-Disponibilidade: soluções certificadas
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |42
 Mais performance com multi-threaded
slaves, row...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |43
 Permite self-healing
 Failover & recovery auto...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |44
• Soluções homologadas e suportadas pela Oracle M...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |45
 http://www.oracle.com/technetwork/database/data...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |46
Oracle VM Servers
Oracle VM Server Pool
ocfs2
Ora...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |47
oracle.com/technetwork/server-storage/solaris-clu...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |48
 Opção nativa do Windows certificada para MySQL ...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |49
mysql.com/why-mysql/white-papers/mysql_wp_drbd.ph...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |50
API Nodes
Arquitetura MySQL Cluster
Data Nodes
nd...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |51
MySQL Cluster
• Escalabilidade linear de Escritas...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |54
Soluções de HA homologadas pela Oracle
MySQL
Repl...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |55
Grande Escala
Social Network
Small Medium Large E...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |56
Grande Escala
Session Management eCommerce
MySQL ...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |57
Caso de uso: Loja On-Line
Usuários
Navegação
Reco...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |58
 sistema de publicidade online que
identifica o ...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |59
BI Solutions
ACQUIRE
ORGANIZEANALYZE
DECIDE
Hadoo...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |60
Novo: MySQL Hadoop Applier
 Integração em tempo ...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |61
Mais sobre como MySQL e Big Data
Whitepaper:
MySQ...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |62
Acesso chave-valor ao InnoDB
Via Memcached API
Us...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |63
MySQL 5.7: InnoDB Memcached
Contribuição do Faceb...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |64
Escala Extra Grande (Redes Sociais)
Social Networ...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |65
MySQL Central @ Oracle Open World 2014
Alguns cas...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |66
Extra Large: Social Network
Shard Catalog
Authent...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |67
Boas Práticas
 Introdução de técnicas de Shardin...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |68
Sharding Implementation
Master
Slave
Clients
Slav...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |69
MySQL Fabric 1.4 RC (MySQL Utilities 1.4.2)
 Alt...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |70
Hardware
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |71
 Up to 64 x86-64 bit CPU threads (MySQL 5.6 and ...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |72
MySQL Cluster Hardware Selection
SQL Layer
• 4 - ...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |73
MySQL Cluster Hardware Selection
Data Layer
• Up ...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |74
Entry-Level Mid-Range High-End
1 x SATA 7200 RPM
...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |75
Recommended Minimum High-End Recommendation
2 x S...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |76
Considere MySQL Enterprise para
implementar boas ...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |77
MySQL Enterprise Edition
Subscrição com renovação...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |78
MySQL Cluster Carrier-Grade Edition
Subscrição co...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |79
Conclusão
 Foco no mercado e soluções web
 Reco...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |80
Saiba Mais
 MySQL Web Reference Architectures Wh...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |81
MySQL for
Beginners Ed 3
MySQL for Database
Admin...
Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |82
@MySQLBR
meetup.com/MySQL-BR
facebook.com/MySQLBR...
Arquiteturas de Referência Web
Conjunto de boas práticas
Perguntas?
Upcoming SlideShare
Loading in...5
×

Arquiteturas de referência MySQL

4,364

Published on

Uma visão sobre como projetar arquiteturas web com bases de dados de alta escala, baseado em casos de referência como Facebook, Twitter e Youtube.

Descrição: O MySQL é utilizado na aplicação principal dos maiores websites do mundo, incluindo Facebook, Twitter, LinkedIn e YouTube, assim como em alguns dos que mais crescem, como Tumblr, Pinterest e box.com. Nesta apresentação daremos uma visão sobre como projetar arquiteturas web com bases de dados de alta escala, seja implantado no localmente ou na nuvem.
Serão apresentadas - Arquiteturas de Referência para pequeno, médio e grande porte para aplicações web e móveis, abordando tópicos como:
-Como escalar o banco de dados enquanto seu novo site, fenômeno da web, simplesmente decola
-Como proporcionar agilidade, alta-disponibilidade, segurança e gerenciamento simplificado dos dados
-Como simplificar a integração com a grandes sistemas de análise de dados e big data
-Como manter baixos níveis de custo, risco e complexidade

Published in: Technology, News & Politics

Arquiteturas de referência MySQL

  1. 1. Arquiteturas de Referência Web Conjunto de boas práticas Airton Lastori fev-2015
  2. 2. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |2 Safe Harbour Statement The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
  3. 3. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |3 s
  4. 4. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |4 Agenda  Requisitos Web, níveis de escala  Arquiteturas de Referência – Escalas & Topologias – Tecnologias envolvidas na Infraestrutura – Boas práticas  Um pouco sobre Hardware  Considerações Operacionais  Conclusão
  5. 5. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |5 Desdobramentos na Infraestrutura Ágil, aberta & adaptável Simples & replicável Baixa latência Escalável e Diponível Segurança extrema Baixo TCO
  6. 6. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |6 Fonte: http://db-engines.com/en/ranking_trend (dezembro/14)
  7. 7. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |7 Os MELHORES COMPONENTES da pilha MySQL: Aplicações Web da próxima geração Aplicações On-Premise, na Nuvem ou Distribuídas
  8. 8. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |8 Alguns clientes MySQL
  9. 9. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |9 MySQL ISV Partners in LAD Brasil América Central Argentina México
  10. 10. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |11 Edward Screven & Tomas Ulin no Open World 2014  [video] youtube.com/watch?v=fzCpd4j72jA Oracle Chief Corporate Architect & VP of MySQL Engineering
  11. 11. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |12 @MySQLBR meetup.com/MySQL-BR facebook.com/MySQLBR MySQLBR
  12. 12. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |13 Arquiteturas de Referência
  13. 13. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |14 Referências de Escala Social Network Small Medium Large Extra Large Queries/Second <500 <5,000 10,000+ 25,000+ Transactions/Second <100 <1,000 10,000+ 25,000+ Concurrent Read Users <100 <5,000 10,000+ 25,000+ Concurrent Write Users <10 <100 1,000+ 2,500+ Database Size Sessions <2 GB <10 GB 20+ GB 40+ GB eCommerce <2 GB <50 GB 50+ GB 200+ GB Analytics (Multi-Structured Data) <10 GB <1TB 10+ TB 100+ TB Content Management (Meta-Data) <10 GB <500 GB 1+ TB 2+ TB
  14. 14. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |15 Pequena Escala Social Network Small Medium Large Extra Large Queries/Second <500 <5,000 10,000+ 25,000+ Transactions/Second <100 <1,000 10,000+ 25,000+ Concurrent Read Users <100 <5,000 10,000+ 25,000+ Concurrent Write Users <10 <100 1,000+ 2,500+ Database Size Sessions <2 GB <10 GB 20+ GB 40+ GB eCommerce <2 GB <50 GB 50+ GB 200+ GB Analytics (Multi-Structured Data) <10 GB <1TB 10+ TB 100+ TB Content Management (Meta-Data) <10 GB <500 GB 1+ TB 2+ TB
  15. 15. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |16 Pequena Escala  Um único servidor para: – Perfis de usuário e sessões – eCommerce – Gerenciamento de conteúdo  Replicação MySQL – Analítico e Backups  Se o tráfego aumentar, o gerenciamento de sessões pode ser direcionado para outro servidor. Escalabilidade limitada. Se já é previsto aumento de demanda, inicie com Arquitetura para Média escala. MySQL Enterprise Monitor MySQL Enterprise Backup MySQL Master MySQLReplication Analytics Slave 2 Backup Slave 1
  16. 16. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |17 Replicação MySQL  Duplique databases de um “master” para um “slave” – Cópias redundantes do dados, peça fundamental na Alta Dispinibilidade – Escale horizontalmente distribuindo queries nos slaves Master Slaves Web / App Servers Writes & Reads Reads
  17. 17. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |18 Boas Práticas  Use o Storage Engine correto – InnoDB atende a maior parte dos cenários, default a partir do MySQL 5.5 – Atualize para MySQL 5.6  Hot Backup e Recovery rápido  Monitoramento pró-ativo  Mantenha a qualidade do tempo de reposta das queries  Auditoria  Reforçe políticas de segurança com autenticação externa  Thread Pool para escalabilidade vertical Arquiteturas de Pequena Escala
  18. 18. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |19 20082008 até 4 CPU MySQL 5.0 MySQL AB até 16 CPU MySQL 5.1 Sun até 32 CPU MySQL 5.5 Oracle até 48 CPU MySQL 5.6 Evolução da escalabilidade do MySQL 20092009 20102010 20132013 Use InnoDB para usufruir melhor de hardwares modernos.
  19. 19. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |22 MySQL 5.5.28 MySQL 5.6.7 +234% Ganhos de Performance 0 2.000 4.000 6.000 8.000 10.000 12.000 14.000 16.000 18.000 32 64 128 256 512 TransaçõesporSegundo Conexões MySQL 5.6 vs. 5.5 - Leituras (Linux) Oracle Linux 6 Intel(R) Xeon(R) E7540 x86_64 MySQL leveraging: - 48 of 96 available CPU threads - 2 GHz, 512GB RAM Performance MySQL 5.6: InnoDB SysBench Benchmarks
  20. 20. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |23 MySQL 5.5.28 MySQL 5.6.7 +151% Ganhos de Performance 0 2.000 4.000 6.000 8.000 10.000 12.000 32 64 128 256 512 TransaçõesporSegundo Conexões MySQL 5.6 vs. 5.5 – Escritas+Leituras (Linux) Oracle Linux 6 Intel(R) Xeon(R) E7540 x86_64 MySQL leveraging: - 48 of 96 available CPU threads - 2 GHz, 512GB RAM Performance MySQL 5.6: InnoDB SysBench Benchmarks
  21. 21. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |24
  22. 22. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |25 Performance Security Availability
  23. 23. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |26 MySQL Enterprise Backup  Backup online para InnoDB, tamanho ilimitado  Backups automatizados: completos, incrementais, parciais  Compressão, Criptografia  Metadados de status, progresso e histórico  Compatível SBT, Oracle Secure Backup  Suporte Cloud (Amazon S3)  Point in Time Recovery, Parcial ou Completo  Multi-Plataforma: Windows, Linux, Unix Backup Files MEB Backup Files Database MySQL Database Files mysqlbackupmysqlbackup Backup e Recovery mais rápidos e online.
  24. 24. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |28 2. Usuário conecta-se e executa queries MySQL Enterprise Audit 1. DBA habilita no Server1 Server1 3. Conexões e queries dos usuários aparecem no log
  25. 25. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |30 Authenticate CREATE USER win_joe IDENTIFIED WITH authentication_windows AS ‘joe'; LDAP/AD PAM / Windows Auth PAM / Windows Auth Usuário/senha definidos no diretório Privilégios definidos no MySQL e mapeados para usuários/grupos do diretório MySQL Enterprise Security External Authentication Plug-Ins
  26. 26. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |32 60x Melhor Escalabilidade comThread Pool MySQL 5.6.11 Oracle Linux 6.3, Unbreakable Kernel 2.6.32 4 sockets, 24 cores, 48 Threads Intel(R) Xeon(R) E7540 2GHz CPUs 512GB DDR3 RAM Thread Pool habilitado/desabilitado
  27. 27. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |33 18x Melhor Escalabilidade com Thread Pool Thread Pool habilitado/desabilitado MySQL 5.6.11 Oracle Linux 6.3, Unbreakable Kernel 2.6.32 4 sockets, 24 cores, 48 Threads Intel(R) Xeon(R) E7540 2GHz CPUs 512GB DDR3 RAM
  28. 28. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |34 MySQL Enterprise Encryption Standards Based Cryptography  MySQL encryption libraries – Symmetric encryption AES256 – Public-key / asymmetric cryptography  Key management – Generate public and private keys – Key exchange methods: RSA, DSA, DH  Sign and verify data – Cryptographic hashing for digital signing, verification, & validation
  29. 29. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |35 Média Escala Social Network Small Medium Large Extra Large Queries/Second <500 <5,000 10,000+ 25,000+ Transactions/Second <100 <1,000 10,000+ 25,000+ Concurrent Read Users <100 <5,000 10,000+ 25,000+ Concurrent Write Users <10 <100 1,000+ 2,500+ Database Size Sessions <2 GB <10 GB 20+ GB 40+ GB eCommerce <2 GB <50 GB 50+ GB 200+ GB Analytics (Multi-Structured Data) <10 GB <1TB 10+ TB 100+ TB Content Management (Meta-Data) <10 GB <500 GB 1+ TB 2+ TB
  30. 30. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |36 Média Escala
  31. 31. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |37 Boas Práticas  Proporção de 10 servidores de aplicação para cada MySQL Server – Mais para aplicações PHP, menos para Java – Adicione mais slaves quando a camada de aplicação escalar  Camada de Cache para gerenciamento de sessões & componentes do Gerenciador de Conteúdo – Memcached ou Redis são comuns – Leituras atendidas pelo cache, aliviando a carga no MySQL – API NoSQL Memcached do MySQL 5.6 ou MySQL Cluster podem implementar compressão e/ou persistência nas camadas de cache Arquiteturas de Média Escala
  32. 32. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |38 Boas Práticas – Gerenciador de Contéudo Arquiteturas de Média Escala XOR  Cada slave atende por volta de 5.000 usuários simultâneos  Cada master atende por volta de 20 slaves – Mais slaves podem ser necessários em ambientes maiores  Replicação do MySQL para alta disponibilidade e escalabilidade horizontal  Meta dados dos arquivos gerenciados pelo MySQL  Distributed File System / CDN / Cloud (e. S3) para armazenamento físico dos arquivos – SAN de alta qualidade, com redundancia para HA, ou – Hardware commodity com software especializado (ex. MogileFS)
  33. 33. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |39 Boas Práticas – Sessões e e-Commerce  Gerenciamento de Sessões & e-Commerce – Dados das Sessões mantidos por aproximadamente 1 hora em uma partição dedicada; partições antiga são excluídas – API NoSQL Memcached do MySQL 5.6 ou MySQL Cluster ser usadas – Alta diponibilidade em e-Commerce:  Semi-Synchronous Replication or clusterização via DRBD ou shared storage – Se o tráfego aumentar, mova o Gerenciamento de Sessões para o MySQL Cluster  Persistência dos dado de sessão em tempo real, dados in-memory reduzem a necessidade caching  99,999% de disponibilidade (multi-master), elimina necessidade de DRBD ou shared storage  Bases separadas para Dados Analíticos (ex. Data Mart, Relatórios) Arquiteturas de Média Escala
  34. 34. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |40 Alta-Disponibilidade: soluções certificadas
  35. 35. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |42  Mais performance com multi-threaded slaves, row image minimal, binlog group commit  Maior robustez e consistência com event checksums, crash safe slaves  Mais flexibilidade e disponibilidade com GTID, self-healing, auto-failover, Replication Utilities e MySQL Enterprise Monitor Master # Slave # Replicação melhorada no MySQL 5.6
  36. 36. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |43  Permite self-healing  Failover & recovery automatizado - mysqlfailover (Utilitário)  Switchover & administração - mysqlrpladmin (Utilitário)  Alta disponibilidade com MySQL core - Elimina necessidade de soluções de terceiros; Fácil de estender HA UtilitiesMonitoramento Master Falhou Slaves Slave Promovido para Master Disponibilidade com Replicação no MySQL 5.6 Replication HA Utilities (Python)
  37. 37. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |44 • Soluções homologadas e suportadas pela Oracle MySQL – Oracle Clusterware – Oracle VM Template – Windows Failover Clustering – Oracle Solaris Cluster VIP Clients Clusterização baseada em Shared-Storage
  38. 38. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |45  http://www.oracle.com/technetwork/database/database- technologies/clusterware/downloads/index.html MySQL Grid Control Agent for Clusterware Oracle Clusterware
  39. 39. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |46 Oracle VM Servers Oracle VM Server Pool ocfs2 Oracle VM Manager SAN / iSCSI Secure Live Migration (SSL) Oracle VM Automatic Fault Detection & Recovery Oracle VM mysql.com/why-mysql/white-papers/mysql_wp_oracle-vm-template-for-mee.php • Oracle Linux • Oracle VM • Oracle VM Template for MySQL EE • Oracle Cluster File System 2 (OCFS2) • MySQL Database 5.5 Enterprise Edition Pré-instalado & Configurado Integrado & Testado Suporte de um único fornecedor Oracle VM Template for MySQL Enterprise
  40. 40. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |47 oracle.com/technetwork/server-storage/solaris-cluster/overview/index.html • Heartbeat e monitoramento baseado no Kernel • SPARC e x86. Solaris Virtualization-aware • Agente MySQL incluíd o no Oracle Solaris Cluster Oracle Solaris Clustering
  41. 41. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |48  Opção nativa do Windows certificada para MySQL Enterprise  Para Falhas ou Manutenção Planejada  Cluster gerenciado pela interface snap-in MS Failover Cluster Management*, não requer scripts customizados  Perda de serviço = alguns segundos + tempo de recovery do InnoDB: • Aplicação vê temporariamente uma perda de conexão e reconecta no mesmo IP • Slave na Replicação recupera-se automaticamente  Quorum (3o voto), dados (InnoDB + schema) & binários (opcional) em armazenamento compartilhado • teste com iSCSI & FCAL *suporte técnico do componente MS Failover Cluster Management deve ser contratado da Microsoft App Virtual IP Data Bin Slave App App Vote Windows Server Failover Clustering mysql.com/why-mysql/white-papers/mysql_wp_windows_failover_clustering.php
  42. 42. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |49 mysql.com/why-mysql/white-papers/mysql_wp_drbd.php • Certificado e completamente suportado pela Oracle • DRBD integrado ao Oracle Linux Unbreakable Enterprise Kernel R2 • Pacemaker e Corosync para clustering & failover • Updates na pilha via ULN channel • Baseado em storage distribuído, não requer SAN • Replicação síncrona elimina o risco de perda de dados • Maduro & já utilizado em muitos casos Oracle Linux + DRBD Stack
  43. 43. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |50 API Nodes Arquitetura MySQL Cluster Data Nodes ndbd ndbd Management mgm_ndbd Clients Application Layer Data Layer ndbd ndbd
  44. 44. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |51 MySQL Cluster • Escalabilidade linear de Escritas e Leituras • Elasticidade, crescimento incremental, distrib. simétricaEscalabilidade • 99,999% de disponibilidade, 5min / ano • Self-healing, failover menor que 1s, geo-replicaçãoAlta Disponibilidade • Performance em tempo real, in-memory e em disco • Latência baixa e preditiva, acessos paralelosAlta Performance • Modelo relacional ACID, SQL e Foreign Keys • APIs NoSQL (C++, Java, Memcached, Node.js)Flexibilidade • Open Source, suporte e ferramentas comerciais opcionais • Hardware commodityBaixo TCO
  45. 45. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |54 Soluções de HA homologadas pela Oracle MySQL Replication MySQL Fabric Oracle VM Template Solaris Cluster Windows Cluster DRBD MySQL Cluster Clustering Mode Master + Slaves Master + Slaves Active/Pass ive Active/Pass ive Active/Passi ve Active/Pa ssive Multi- Master App Auto-Failover ✖ ✔ ✔ ✔ ✔ ✔ ✔ Data Layer Auto-Failover ✖ ✔ ✔ ✔ ✔ ✔ ✔ Zero Data Loss MySQL 5.7 ✔ ✔ ✔ ✔ ✔ ✔ Failover Time N/A Secs Secs + Secs + Secs + Secs + < 1 Sec Scale-out Reads ✔ ✖ ✖ ✖ ✖ ✔ Cross-shard operations N/A ✖ N/A N/A N/A N/A ✔ Transparent routing ✖ For HA ✔ ✔ ✔ ✔ ✔ Shared Nothing ✔ ✔ ✖ ✖ ✖ ✔ ✔ Storage Engine InnoDB InnoDB InnoDB InnoDB InnoDB InnoDB NDB Single Vendor Support ✔ ✔ ✔ ✔ ✖ ✔ ✔ Platform Support All All Linux Solaris Windows Linux All
  46. 46. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |55 Grande Escala Social Network Small Medium Large Extra Large Queries/Second <500 <5,000 10,000+ 25,000+ Transactions/Second <100 <1,000 10,000+ 25,000+ Concurrent Read Users <100 <5,000 10,000+ 25,000+ Concurrent Write Users <10 <100 1,000+ 2,500+ Database Size Sessions <2 GB <10 GB 20+ GB 40+ GB eCommerce <2 GB <50 GB 50+ GB 200+ GB Analytics (Multi-Structured Data) <10 GB <1TB 10+ TB 100+ TB Content Management (Meta-Data) <10 GB <500 GB 1+ TB 2+ TB
  47. 47. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |56 Grande Escala Session Management eCommerce MySQL Master Content Management Slave 1 MySQL Master Analytics MySQL Enterprise Monitor Slave 2 Slave 3 Slave 4 Slave 5 Slave 6 Slave 7 Slave 8 Slave 9 Slave 10 Slave N Slave 2 Node Group 2 F2 F4 Node3Node4 F2 F4 Node Group 1 F1 F3 Node3Node4 F1 F3 MySQL Cluster Data Nodes MySQL Servers Node Group 2 F2 F4Node3Node4 F2 F4 Node Group 1 F1 F3 Node3Node4 F1 F3 MySQL Cluster Data Nodes MySQL Servers Slave 1 Slave 3 Hadoop Cluster MySQL Enterprise Backup Caching / Application Servers Distributed Storage XOR
  48. 48. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |57 Caso de uso: Loja On-Line Usuários Navegação Recomendações Web Logs: Páginas Visitadas, Comentários, Posts Perfil, Histórico de Compras Recomendações Interações em Mídias Sociais, Preferências, Marcas com “Curtir” Conversas Telefônicas, mensagens
  49. 49. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |58  sistema de publicidade online que identifica o visitante e exibe conteúdo direcionado ao seu perfil  2 TB de web logs são capturados, com 22 bilhões de linhas por mês processadas pelo MySQL  Atualmente, considerando apenas uma instância MySQL, há 8 TB de dados armazenados com 5GB de crescimento por dia Caso de Sucesso Big Data “O MySQL é uma parte essencial da nossa estratégia de Big Data. A integração com Hadoop permite-nos melhorar e crescer o nosso negócio de publicidade digital com rapidez e agilidade.” Josafá Santos IT Manager, boo-box mysql.com/why-mysql/case-studies/1billion-advertisements-mysql-hadoop.html
  50. 50. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |59 BI Solutions ACQUIRE ORGANIZEANALYZE DECIDE Hadoop Applier Big Data Lifecycle com MySQL+Hadoop
  51. 51. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |60 Novo: MySQL Hadoop Applier  Integração em tempo real do MySQL para Hadoop  Cada tabela mapeada para um diretório Hive data warehouse  Possibilita usufruir de ferramentas de análise do eco-sistema Hadoop  Construído sobre MySQL Binlog API e libhdfs library  Artigos dev.mysql.com  Download labs.mysql.com
  52. 52. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |61 Mais sobre como MySQL e Big Data Whitepaper: MySQL and Hadoop – Big Data Integration December 2012 mysql.com/why-mysql/white-papers/mysql- and-hadoop-guide-to-big-data-integration
  53. 53. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |62 Acesso chave-valor ao InnoDB Via Memcached API Usa clients Memcached existentes Sem SQL parsing, ultra-rápido Acesso SQL e NotOnlySQL Operações chave-valor mas também queries SQL, JOINs, FKs, etc. Memcached com Persistência no InnoDB Método transparente para persistir os dados em memória volátil Memcached plug-in no mysqld , mapeado à API nativa do InnoDB Shared process para baixa latência SQL (MySQL Client) InnoDB Storage Engine MySQL Server Memcached plugin Application/Client NoSQL (Memcached Prot.) mysqld http://dev.mysql.com/doc/refman/5.6/en/innodb-memcached.html Flexibilidade MySQL 5.6: InnoDB NoSQL com Memcached
  54. 54. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |63 MySQL 5.7: InnoDB Memcached Contribuição do Facebook Intel(R) Xeon(R) CPU X7560 x86_64 8 sockets x 6 cores-HT (96 CPU threads) 2000Mhz, 256G RAM Oracle Linux 6.2 6x mais rápido que o MySQL 5.6 1,150,000 QPS 0 200000 400000 600000 800000 1000000 1200000 1400000 8 16 32 64 128 256 512 1024 QueriesperSecond Concurrent User Sessions 5.7.3 5.6
  55. 55. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |64 Escala Extra Grande (Redes Sociais) Social Network Small Medium Large Extra Large Queries/Second <500 <5,000 10,000+ 25,000+ Transactions/Second <100 <1,000 10,000+ 25,000+ Concurrent Read Users <100 <5,000 10,000+ 25,000+ Concurrent Write Users <10 <100 1,000+ 2,500+ Database Size Sessions <2 GB <10 GB 20+ GB 40+ GB eCommerce <2 GB <50 GB 50+ GB 200+ GB Analytics (Multi-Structured Data) <10 GB <1TB 10+ TB 100+ TB Content Management (Meta-Data) <10 GB <500 GB 1+ TB 2+ TB
  56. 56. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |65 MySQL Central @ Oracle Open World 2014 Alguns cases apresentados...  [slides] oracle.com/openworld/mysql
  57. 57. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |66 Extra Large: Social Network Shard Catalog Authentication MySQL Master Analytics Slave 2 Node Group 2 F2 F4 Node3Node4 F2 F4 Node Group 1 F1 F3 Node3Node4 F1 F3 MySQL Cluster Data Nodes MySQL Servers Slave 1 Slave 3 Hadoop Cluster MySQL Enterprise Backup Node Group 2 F2 F4 Node3Node4 F2 F4 Node Group 1 F1 F3 Node3Node4 F1 F3 MySQL Cluster Data Nodes MySQL Servers App /Caching ServersCentral Databases Customers 1% – 33% Customers 34% – 66% Customers 67% – 100% Slave 1 Slave 2 Slave N Shards MySQL Enterprise Monitor Slave 1 Slave 2 Slave N Slave 1 Slave 2 Slave N
  58. 58. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |67 Boas Práticas  Introdução de técnicas de Sharding – Normalmente implementado na camada de aplicação para escalar um volume muito alto de escritas (ex. Youtube Vitess, Twitter Gizzard, Tumblr JetPants etc) – Dados divididos em pequenos conjuntos, distribuído em servidores de baixo custo – Distribuição baseada em Hash de uma única coluna (ex. User ID) – Particionamento Funcional também é uma opção  Use com moderação, certifique-se de tentar as recomendações anteriores – Replicação convencional e escalabilidade vertical serão suficientes em muitos casos, com menor complexidade – Sharding é realmente necessário em uma pequena porcentagem de casos – Muitos serviços Web & Mobile ainda possuem mais leituras que escritas – MySQL Cluster implementa Auto-Sharding com JOINs entre shards Arquiteturas de Escala Extra Grande
  59. 59. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |68 Sharding Implementation Master Slave Clients Slaves Shards Reads Writes Partitioning Logic 11 22 33 44 55
  60. 60. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |69 MySQL Fabric 1.4 RC (MySQL Utilities 1.4.2)  Alta disponibilidade: – Monitoramento do Server com failover automático e transparente para aplicação  Conectores devem ser “Fabric-aware”: – Java, PHP, Python  Sem Proxy, sem latencia extra e SPoF  Escalabilidade de escrita com sharding: – Aplicação define a shard key (Range ou Hash) – Ferramentas para re-sharding – Tabelas globais & global updates Alta-disponibilidade + escalabilidade de escrita com sharding MySQL Fabric Connector Application Read-slaves SQL Master group Read-slaves Master group
  61. 61. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |70 Hardware
  62. 62. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |71  Up to 64 x86-64 bit CPU threads (MySQL 5.6 and above).  Recommended RAM at least equal to or larger than “hottest” (most regularly accessed) data set.  Linux, Oracle Solaris or Microsoft Windows operating systems.  Minimum of 4 x SSDs or HDDs. 8 – 16 drives will increase performance for I/O intensive applications. – Hardware RAID with battery-backed cache. – RAID 10 recommended. RAID 5 is suitable if the workload is read- intensive.  2 x Network Interface Cards and 2 x Power Supply Units for redundancy. The Perfect MySQL Server
  63. 63. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |72 MySQL Cluster Hardware Selection SQL Layer • 4 - 24 x86-64 bit CPU threads • Minimum 4GB of RAM • Memory not as critical at this layer • Requirements influenced by connections and buffers. • 2 x Network Interface Cards & 2 x PSUs • Linux, Solaris or Windows operating systems.
  64. 64. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |73 MySQL Cluster Hardware Selection Data Layer • Up to 64 x86-64 bit CPU threads • Use high frequency: enables faster processing of messages • Calculating RAM per Data Node (in-memory database) • Database Size * # Replicas * 1.25 / # data node • 50GB database * 2 replicas * 1.25 / 2 data nodes = 64GB of RAM • Non-indexed columns can be stored on disk, reduces RAM requirements • 2 x NICs and 2 x PSUs • Deep-dive into network best-practices in Ref Archs Guide
  65. 65. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |74 Entry-Level Mid-Range High-End 1 x SATA 7200 RPM • For read-mostly • No redundancy (but other data node is the mirror) 1 x SAS 10K RPM or SSD • Heavy duty (many MB/s) • No redundancy (but other data node is the mirror) 4 x SAS 10-15K RPM or SSDs • Heavy duty (many MB/s) • Disk redundancy (RAID1+0), hot swap • REDO, LCP, BACKUP – written sequentually in small chunks (256KB) • If possible, use Odirect = 1 LCP REDOLOG LCP REDOLOG LCP / REDOLOG MySQL Cluster Hardware Selection Disk Subsystem for Checkpointing
  66. 66. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |75 Recommended Minimum High-End Recommendation 2 x SAS 10K RPM or 2 x SSD • Use High-End Recommendation for heavy read / write workloads • (1000's of 10KB records per sec) of data (i.e. Content Delivery platforms) • Having TABLESPACE on separate disk is good for read performance • Enable WRITE_CACHE on devices TABLESPACE LCP REDOLOG UNDOLOG UNDOLOG LCP (REDO LOG / UNDO LOG) TABLESPACE 1 TABLESPACE 2 4 x SAS 10-15K RPM or SSD (REDO LOG)(REDO LOG) MySQL Cluster Hardware Selection Disk Data Storage
  67. 67. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |76 Considere MySQL Enterprise para implementar boas práticas operacionais
  68. 68. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |77 MySQL Enterprise Edition Subscrição com renovação anual: ferramentas + serviços Plug-ins Suporte Backup Monitor & Workbench Escalabilidade Segurança Auditoria
  69. 69. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |78 MySQL Cluster Carrier-Grade Edition Subscrição com renovação anual: ferramentas + serviços Cluster Plug-ins Suporte Backup Monitor & Workbench Cluster Manager Gerenciamento Consolidado
  70. 70. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |79 Conclusão  Foco no mercado e soluções web  Recomendações baseadas em casos de muito sucesso da web  Considerar os desafios operacionais de escalabilidade, performance e alta- disponibilidade
  71. 71. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |80 Saiba Mais  MySQL Web Reference Architectures Whitepaper http://www.mysql.com/why-mysql/white-papers/mysql_wp_high-availability_webrefarchs.php  MySQL 5.6: Developer and DBA Guide http://www.mysql.com/why-mysql/white-papers/whats-new-mysql-5-6/  MySQL Enterprise com Suporte Consultivo marcos.trujillo@oracle.com
  72. 72. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |81 MySQL for Beginners Ed 3 MySQL for Database Administrators Ed 3.1 MySQL Performance Tuning MySQL High Availability MySQL Cluster MySQL DBA Introduction to MySQL 5.5 MySQL Developers Techniques MySQL for Beginners Ed 3 MySQL and PHP Developing Dynamic Web Applications MySQL for Developers Ed 2 MySQL Advanced Stored Procedures MySQLDeveloper education.oracle.com/mysql Opcional Necessário Treinamentos e certificações Oracle Certified Professional, MySQL 5.6 DBA Oracle Certified Expert, MySQL Cluster DBA Certificações Oracle Certified Professional, MySQL 5.6 Developer
  73. 73. Copyright © 2013, Oracle and/or its affiliates. All rights reserved. |82 @MySQLBR meetup.com/MySQL-BR facebook.com/MySQLBR Obrigado!
  74. 74. Arquiteturas de Referência Web Conjunto de boas práticas Perguntas?
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×