Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification f...
Upcoming SlideShare
Loading in...5
×

MySQL Cluster e Big Data

1,290

Published on

O MySQL Cluster é um produto derivado do popular MySQL Server que é o "M" do LAMP stack e pode ser usado como solução para problemas de Big Data.
Trata-se de um banco de dados distribuído com arquitetura shared-nothing e que pode oferecer 99,999% de disponibilidade, performance superior a 1 bilhão de escritas por minuto, escalabilidade linear, conformidade com o modelo ACID, flexibilidade das interfaces NoSQL e geo-replicação multi-master.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
1,290
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
27
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

MySQL Cluster e Big Data

  1. 1. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 121 Airton Lastori airton.lastori@oracle.com 12-jul-2013 MySQL Cluster e Big Data – visão geral
  2. 2. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 122 @MySQLBR meetup.com/MySQL-BR facebook.com/MySQLBR MySQLBR
  3. 3. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 123 Agenda  Problemas e soluções para Big Data  O que é o MySQL Cluster?  Como começar?  Como funciona?  Perguntas?
  4. 4. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 124 Problemas e soluções para Big Data
  5. 5. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 125 O que é? Big Data = Datasets volumosos: 90% dos dados gerados nos últimos 2 anos Crescimento muito rápido: ~50x maior velocidade Fontes variadas: web, dispositivos móveis, sensores, câmeras… produção voluntária e não-volutária de qualidade varíavel Variabilidade de formatos: dados estruturados e não-estruturados Se bem aproveitados, muitas oportunidades, muito valor Não há consenso na definição, mas há fatos
  6. 6. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 126 Desafios Big Data VOLUME VELOCIDADE SOCIAL VARIEDADE BLOG SMART METER VARIABILIDADE 10110010100 10010011010 10101011100 10101010010 PROBLEMA: os sistemas convencionais não foram projetados para isso!
  7. 7. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 127 Novas demandas, novas soluções Transações Arquitetura tradicional: Decisões basedas nos dados do seu banco de dados Big Data: Decisões baseadas em todos dados capturáveis Video e Imagens Dados gerados por máquinas Dados Sociais Documentos Transações
  8. 8. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 128
  9. 9. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 129 ACQUIRE ORGANIZEANALYZE DECIDE Big Data Lifecycle
  10. 10. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1210 BI Solutions ACQUIRE ORGANIZEANALYZE DECIDE Hadoop Applier Big Data Lifecycle com MySQL+Hadoop
  11. 11. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1211 Por que Hadoop? (1/2) Projetado para resolver problemas de Big Data  Armazenamento de data sets de grandes volumes  Processamento paralelo em grande velocidade  Facilidade em tratar de fontes de dados variadas  Variabilidade nos formatos dos dados – estruturados e não estruturados (schemaless) 90% com Projetos Piloto com Hadoop no final de 2012 Fonte: gigaom.com
  12. 12. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1212  Escalabilidade horizontal para milhares de nós – Hardware commodity ou Cloud  Alta-disponibilidade com replicação e self-healing  Inicialmente modelo de processamento batch (Map/Reduce) – Extensível com queries interativas via Apache Drill, Cloudera Impala, Stinger etc. Por que Hadoop? (2/2) Projetado para resolver problemas de Big Data
  13. 13. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1213 Caso de uso: Loja On-Line Usuários Navegação Recomendações Web Logs: Páginas Visitadas, Comentários, Posts Perfil, Histórico de Compras Recomendações Interações em Mídias Sociais, Preferências, Marcas com “Curtir” Conversas Telefônicas, mensagens
  14. 14. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1214  sistema de publicidade online que identifica o visitante e exibe conteúdo direcionado ao seu perfil  2 TB de web logs são capturados, com 22 bilhões de linhas por mês processadas pelo MySQL  Atualmente, considerando apenas uma instância MySQL, há 8 TB de dados armazenados com 5GB de crescimento por dia Caso de Sucesso Big Data “O MySQL é uma parte essencial da nossa estratégia de Big Data. A integração com Hadoop permite-nos melhorar e crescer o nosso negócio de publicidade digital com rapidez e agilidade.” Josafá Santos IT Manager, boo-box mysql.com/why-mysql/case-studies/1billion-advertisements-mysql-hadoop.html
  15. 15. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1215 boo-box
  16. 16. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1216 BI Solutions ACQUIRE ORGANIZEANALYZE DECIDE Hadoop Applier Big Data Lifecycle com MySQL+Hadoop
  17. 17. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1217 Novo: MySQL Hadoop Applier  Integração em tempo real do MySQL para Hadoop  Cada tabela mapeada para um diretório Hive data warehouse  Possibilita usufruir de ferramentas de análise do eco-sistema Hadoop  Construído sobre MySQL Binlog API e libhdfs library  Artigos dev.mysql.com  Download labs.mysql.com
  18. 18. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1218 Mais sobre como MySQL e Big Data Whitepaper: MySQL and Hadoop – Big Data Integration December 2012 mysql.com/why-mysql/white-papers/mysql- and-hadoop-guide-to-big-data-integration
  19. 19. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1219 MySQL+Hadoop
  20. 20. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1220 O que é o MySQL Cluster?
  21. 21. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1221 MySQL Cluster • Escalabilidade linear de Escritas e Leituras • Elasticidade, crescimento incremental, distrib. simétricaEscalabilidade • 99,999% de disponibilidade, 5min / ano • Self-healing, failover menor que 1s, geo-replicaçãoAlta Disponibilidade • Performance em tempo real, in-memory e em disco • Latência baixa e preditiva, acessos paralelosAlta Performance • Modelo relacional ACID, SQL e Foreign Keys • APIs NoSQL (C++, Java, Memcached, Node.js)Flexibilidade • Open Source, suporte e ferramentas comerciais opcionais • Hardware commodityBaixo TCO
  22. 22. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1225 Enterprise Management Services and Utilities Backup & Recovery Monitor Workbench Utilities Connection Pool, SQL Interface, Parser, Optimizer, Caches Clients & Connectors Native C API, JDBC, ODBC, .Net, PHP, Ruby, Python, VB, Perl mysqld Clients and Apps Arquitetura MySQL Server Storage Engines InnoDB, MyISAM, Memory, Archive, Cluster (NDB API), etc… Filesystems, Files & Logs Data, Index, Logs…
  23. 23. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1226 Cluster Data Nodes NDB NDB Connection Pool, SQL Interface, Parser, Optimizer, Caches Enterprise Management Services and Utilities Backup & Recovery Monitor Workbench Utilities Clients & Connectors Native C API, JDBC, ODBC, .Net, PHP, Ruby, Python, VB, Perl Arquitetura MySQL Cluster Storage Engines InnoDB, MyISAM, Memory, Archive, Cluster (NDB API), etc… Filesystems, Files & Logs Data, Index, Logs… mysqld Clients and Apps ndbd mgm_ndbd Management
  24. 24. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1227 Storage Engines MyISAM InnoDB NDB Cluster Transações ✖ ✔ ✔ Nível de Lock Tabela Linha Linha Replicação Assíncrona ou Semi-síncrona Assíncrona ou Semi-síncrona, crash-safe Síncrona, Multi-master, crash-safe sem SPOF Foreign Keys ✖ ✔ ✔ Full-text indexes ✔ ✔ ✖ Compressão de dados somente Leitura ✔ ✖ Caches somente Índices Dados e Índices Dados e Índices Suporte Geoespacial Tipos de Dados e Índices somente Tipos de Dados somente Tipos de Dados Limite de armazenamento 256TB 64TB 384EB (~3TB em RAM) dev.mysql.com/doc/refman/5.5/en/storage-engines.html
  25. 25. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1228 InnoDB vs NDB Cluster dev.mysql.com/doc/refman/5.6/en/mysql-cluster-compared.html Preferível InnoDB • Datasets ou linhas muito grandes, muitos BLOBs, tabelas histórico, fulltext search • Memória RAM mais limitada • Transações muito longas ou com isolamento diferente de READ COMMITTED • Foreign keys sob uso intensivo • Muitos full table scans, queries analíticas Preferível NDB Cluster • Escalabilidade de escrita, multi-master, auto- sharding • 99,999% uptime com operações online: adição de nós, upgrade, manutenção do esquema • Baixa latência para transações mais curtas, performance tempo-real, paralelismo • APIs NoSQL schemaless: Java, node.js, REST, C++, memcached • Uso limitado de colunas BLOB • Foreign keys são suportadas, porém podem causar impacto na performance em condições extremas
  26. 26. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1229 NDB + InnoDB InnoDB externo InnoDB local NDB Cluster Data Nodes MySQL Cluster Replicação Síncrona Replicação Assíncrona NDB API
  27. 27. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1230 Replicação síncrona Replicação assíncrona InnoDB + NDB Cluster Opções com Geo-Replicação Replicação síncrona entre os grupos de nós para HA Geo-Replicação assíncrona ou síncrona entre nós remotos NDB para redundância geográfica Replicação assíncrona entre Storage Engines diferentes para aplicações especializadas como geração de relatórios Cluster 1 InnoDB InnoDB InnoDB Cluster 2
  28. 28. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1232 O que NÃO é o MySQL Cluster 1. SGBD de uso geral, “bala de prata” • Tabelas NDB ainda não são recomendadas para todos tipos de aplicações / workloads • NDB ainda não é 100% compatível com InnoDB, mas podem ser usados em conjunto • MySQL Cluster requer relativamente mais memória RAM 2. Cluster do tipo Shared-All que requer Shared Storage • Como, por exemplo, Oracle RAC • Cada Data Node tem sua unidade de armazenamento independente 3. Replicação tradicional do MySQL Server • MySQL Cluster possui seu próprio mecanismo interno de replicação síncrona • Também pode ser usado opcionalmente em conjunto com replicação externa tradicional do MySQL 4. “Apenas” o MySQL Server (mysqld) • MySQL Cluster possui outros componentes: ndbd, ndb_mgmd 5. Produto de código fechado • Possui edição community (GPL) e carrier-grade-edition (suporte e ferramentas comerciais)
  29. 29. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1233 Alguns Clientes MySQL Cluster
  30. 30. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1234  sistema de recomendações de games casuais que identifica o visitante e exibe conteúdo direcionado ao seu perfil  já nos testes, com 2 data nodes, capacidade de 87k INSERTs por segundo e 3k SELECTs por segundo  latência extremamente baixa (sub- milisegundos) e alta-disponibilidade  NoSQL ClusterJ API (Java) Caso de Sucesso MySQL Cluster “As a strategic project, we couldn’t afford to take any chances. MySQL Cluster provided us with a proven and trusted solution to meet the demands of both our business and our users.” Sean Chighizola Database Director, Big Fish mysql.com/why-mysql/case-studies/mysql-cs-bigfish.html
  31. 31. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1235
  32. 32. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1236  sistema global de detecção de fraudes que monitora transações financeiras, sessões dos usuários e histórico em tempo real  Trabalha com mais de 100TB e 100m de usuários, transações ACID, R/W <1 segundo em qualquer parte do mundo com geo- replicação  Escalabilidade linear com 99,999% de disponibilidade, na nuvem AWS Caso de Sucesso ACID em tempo real “Technologies such as MySQL Cluster enables users to get the best of both world’s: the agility of NoSQL systems with the trust, maturity and reliability of the SQL model.” Daniel Austin Chief Architect, Paypal mysql.com/customers/view/?id=1223
  33. 33. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1237 PayPal
  34. 34. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1238  Social game para Facebook  2 milhões usuários, com +30k novos usuários por dia  10K usuários concorrentes, ~10K TPS  99.999% uptime Caso de Sucesso Online Games “The MySQL support service has been essential in helping us for troubleshooting and giving recommendations for the production cluster.” Carlos Morales DBA, playfulplay.com blogs.oracle.com/MySQL/entry/mysql_cluster_powers_el_chavo
  35. 35. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1239 Playful Play
  36. 36. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1240 MySQL Cluster Customers mysql.com/customers/cluster
  37. 37. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1241 Como começar a usar o MySQL Cluster?
  38. 38. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1242 MySQL Cluster Auto-Install downloads.mysql.com/tutorials/cluster/mysql_wp_cluster_quickstart.pdf
  39. 39. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1258 1. Download MCM/Cluster edelivery.oracle.com: 2. Unzip e execute o comando com usuário diferente de root: MySQL Cluster Manager Para testes em uma única máquina C:MySQLmcmbin> mcmd --bootstrap MySQL Cluster Manager 1.1.2 started Connect to MySQL Cluster Manager by running “C:MySQLmcmbinmcm" -a NOVA:1862 Configuring default cluster 'mycluster'... Starting default cluster 'mycluster'... Cluster 'mycluster' started successfully ndb_mgmd NOVA:1186 ndbd NOVA ndbd NOVA mysqld NOVA:3306 mysqld NOVA:3307 ndbapi * Connect to the database by running “C:MySQLmcmclusterbinmysql" -h NOVA -P 3306 -u root
  40. 40. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1259 Teste via MySQL Workbench
  41. 41. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1261 Teste via MySQL Workbench 2/2
  42. 42. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1262 <estado:SP,Campinas> prefix key value <estado:SP,Campinas> key value Prefix Table Key-col Val-col policy estado: mapa.cidades sigla_estado cidade cluster Config tables sigla_estado ... cidade ... SP ... Campinas ... Tabela cidades do DB mapa visão da Aplicação visão MySQL Cluster Cluster & Memcached Schema configurável SELECT * FROM mapa.cidades WHERE cidade LIKE ’C%’;
  43. 43. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1263 Exemplos Memcached API Blog: Scalable, persistent, HA NoSQL Memcache storage using MySQL Cluster 15 February 2012 clusterdb.com/mysql-cluster/scalabale-persistent-ha-nosql-memcache- storage-using-mysql-cluster
  44. 44. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1264 var nosql = require('mysql-js'); var annotations = new nosql.TableMapping('cidade').apply ToClass(Cidade); var dbProperties = nosql.ConnectionProperties('ndb'); nosql.openSession(dbProperties, Cidade, annotations, onSession); Connector modular para vários back-ends: • direta com NDB • via MySQL Server Novo: Node.js connector 1/4
  45. 45. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1265 var onSession = function(err, session) { if (err) {console.log(err); process.exit(0);} else { var data = new Cidade('Guarulhos', 'SP'); session.persist(data, onInsert, data, session); } }; Novo: Node.js connector 2/4
  46. 46. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1266 var onInsert = function(err, object, session) { if (err) {console.log(err);} else { console.log('Inserido: ' + JSON.stringify(object)); session.find(Cidade,'Guarulhos', onFind); } }; Novo: Node.js connector 3/4
  47. 47. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1267 var onFind = function(err, result) { if (err) {console.log(err);} else { console.log('Encontrado: ' + JSON.stringify(result)); } process.exit(0); }; Novo: Node.js connector 4/4
  48. 48. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1268 Exemplos JavaScript / Node.js API Blog: MySQL Cluster Tutorial: NoSQL JavaScript Connector for Node.js 17 April 2013 blogs.oracle.com/MySQL/entry/mysql_cluster_tutorial_nosql_api
  49. 49. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1269 Qual API devo usar? SQL • Padrão da indústria • Joins e Queries complexas • Modelo relacional ClusterJ / OpenJPA • ORM Java • Simplicidade • Acesso nativo e rápido ao Cluster • Ex: Web e Telco memcached • chave/valor • Simples de usar • Driver para diversas linguagens • Ex: PHP Proxy node.js • Javascript • Mesma tecnologia do browser no lado do servidor • Ex: Mobile Apps mod_ndb • REST/JSON • HTML • Uso do Apache httpd C++ • Performance extrema • Tempo real • Desenvolved or experiente • Mais baixo nível
  50. 50. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1270 Como o MySQL Cluster funciona?
  51. 51. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1271 Data Layer Application Layer Management mgm_ndbd MySQL Cluster: arquitetura simplificada mysqld Clients MySQL Cluster Data Nodes ndbd ndbd ndbd ndbd
  52. 52. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1272 Como escalar leituras e escritas? • Escalabilidade linear de Escritas e Leituras • Elasticidade, crescimento incremental, distrib. simétricaEscalabilidade • 99.999% de disponibilidade, 5min / ano • Self-healing, failover menor que 1s, geo-replicaçãoAlta Disponibilidade • Performance em tempo real, in-memory e em disco • Latência baixa e preditiva, acessos paralelosAlta Performance • Modelo relacional ACID, SQL e Foreign Keys • APIs NoSQL (C++, Java, Memcached, Node.js)Flexibilidade • Open Source, suporte e ferramentas comerciais opcionais • Hardware commodityBaixo TCO
  53. 53. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1273 Data Node 1 Data Node 2 Data Node 3 Data Node 4 Table T1 P2 P3 P4 P1 Data Sharding: escalando ESCRITAS
  54. 54. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1274 Data Node 1 Data Node 2 F1 Data Node 3 Data Node 4 Table T1 P2 P3 P4 P1 Data Sharding: escalando ESCRITAS
  55. 55. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1275 Data Node 1 Data Node 2 F1 F3 Data Node 3 Data Node 4 Table T1 P2 P3 P4 P1 Data Sharding: escalando ESCRITAS
  56. 56. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1276 Data Node 1 Data Node 2 F1 F3 Data Node 3 Data Node 4 F2 Table T1 P2 P3 P4 P1 Data Sharding: escalando ESCRITAS
  57. 57. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1277 Data Node 1 Data Node 2 F1 F3 Data Node 3 Data Node 4 F2 F4 Table T1 P2 P3 P4 P1 Data Sharding: escalando ESCRITAS
  58. 58. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1278 Data Layer Application Layer Management mgm_ndbd MySQL Cluster: auto-sharding Table T1 P2 P3 P4 P1F1 F3 F3 F1 F2 F4 F4 F2 ndbd ndbd ndbd ndbd Clients mysqld Table T1 visão da Aplicação visão MySQL Cluster SQL NDB API
  59. 59. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1279 Como garantir a alta-disponibilidade? • Escalabilidade linear de Escritas e Leituras • Elasticidade, crescimento incremental, distrib. simétricaEscalabilidade • 99.999% de disponibilidade, 5min / ano • Self-healing, failover menor que 1s, geo-replicaçãoAlta Disponibilidade • Performance em tempo real, in-memory e em disco • Latência baixa e preditiva, acessos paralelosAlta Performance • Modelo relacional ACID, SQL e Foreign Keys • APIs NoSQL (C++, Java, Memcached, Node.js)Flexibilidade • Open Source, suporte e ferramentas comerciais opcionais • Hardware commodityBaixo TCO
  60. 60. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1280 Data Node 1 Data Node 2 F1 F1 Data Node 3 Data Node 4 Table T1 P2 P3 P4 P1 Data Sharding + Replicação
  61. 61. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1281 Data Node 1 Data Node 2 F1 F3 F1 Data Node 3 Data Node 4 Table T1 P2 P3 P4 P1 F3 Data Sharding + Replicação
  62. 62. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1282 Data Node 1 Data Node 2 F1 F3 F3 F1 Data Node 3 Data Node 4 F2 F2 Table T1 P2 P3 P4 P1 Alta-Disponibilidade: Replicação Síncrona
  63. 63. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1283 Data Node 1 Data Node 2 F1 F3 F3 F1 Data Node 3 Data Node 4 F2 F4 F4 F2 Table T1 P2 P3 P4 P1 Data Sharding + Replicação
  64. 64. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1284 Data Node 1 Data Node 2 F1 F3 F3 F1 Data Node 3 Data Node 4 F2 F4 F4 F2 Group1 Table T1 P2 P3 P4 P1 Data Sharding + Replicação Group2
  65. 65. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1285 Data Node 1 Data Node 2 F1 F3 F3 F1 Data Node 3 Data Node 4 F2 F4 F4 F2 Group1 Table T1 P2 P3 P4 P1 Data Sharding + Replicação Group2
  66. 66. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1286 MySQL Cluster Data Nodes Data Layer Application Layer Management mgm_ndbd MySQL Cluster: alta-disponibilidade Table T1 F1 F3 F3 F1 F2 F4 F4 F2 Management mgm_ndbd mysqldmysqld ndbd ndbd ndbd ndbd Clients
  67. 67. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1287 Como garantir a durabilidade e performance? • Escalabilidade linear de Escritas e Leituras • Elasticidade, crescimento incremental, distrib. simétricaEscalabilidade • 99.999% de disponibilidade, 5min / ano • Self-healing, failover menor que 1s, geo-replicaçãoAlta Disponibilidade • Performance em tempo real, in-memory e em disco • Latência baixa e preditiva, acessos paralelosAlta Performance • Modelo relacional ACID, SQL e Foreign Keys • APIs NoSQL (C++, Java, Memcached, Node.js)Flexibilidade • Open Source, suporte e ferramentas comerciais opcionais • Hardware commodityBaixo TCO
  68. 68. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1288 In-Memory, parallel, AQL, Pushdown Joins 1. Memória RAM mais barata e redes mais rápidas • Escrever em memória RAM remota é mais rápido que em disco local 2. Múltiplas threads trabalhando em paralelo em diversas máquinas • Uso de arquiteturas multi-core 3. Adaptative Query Localization • Queries mais complexas, envolvendo apenas Data Nodes necessários 4. Pushdown Joins • JOINs resolvidos nos Data Nodes, em paralelo
  69. 69. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1289  NoSQL C++ API, flexaSynch benchmark  30 x Intel E5-2600 Intel Servers, 2 socket, 64GB  ACID Transactions, Synchronous Replication 0 5 10 15 20 25 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 MillionsofUPDATEsper Second MySQL Cluster Data Nodes 20 Milhões de UPDATEs / seg
  70. 70. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1292 Flexibilidade na interação com os dados • Escalabilidade linear de Escritas e Leituras • Elasticidade, crescimento incremental, distrib. simétricaEscalabilidade • 99.999% de disponibilidade, 5min / ano • Self-healing, failover menor que 1s, geo-replicaçãoAlta Disponibilidade • Performance em tempo real, in-memory e em disco • Latência baixa e preditiva, acessos paralelosAlta Performance • Modelo relacional ACID, SQL e Foreign Keys • APIs NoSQL (C++, Java, Memcached, Node.js)Flexibilidade • Open Source, suporte e ferramentas comerciais opcionais • Hardware commodityBaixo TCO
  71. 71. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1293 Data Layer Clients Application Layer Management MySQL Cluster: NoSQL APIs Management ndbd ndbd ndbd ndbd
  72. 72. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1294 Os mesmos dados acessados simultaneamente através de interfaces SQL e NoSQL APIs de Acesso aos Dados
  73. 73. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1295 Como manter TCO baixo? • Escalabilidade linear de Escritas e Leituras • Elasticidade, crescimento incremental, distrib. simétricaEscalabilidade • 99.999% de disponibilidade, 5min / ano • Self-healing, failover menor que 1s, geo-replicaçãoAlta Disponibilidade • Performance em tempo real, in-memory e em disco • Latência baixa e preditiva, acessos paralelosAlta Performance • Modelo relacional ACID, SQL e Foreign Keys • APIs NoSQL (C++, Java, Memcached, Node.js)Flexibilidade • Open Source, suporte e ferramentas comerciais opcionais • Hardware commodityBaixo TCO
  74. 74. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1296 Principais componentes do TCO 1. Custo de aquisição • Licenciamento do Software (desenvolvimento e produção) • Hardware necessário • Conhecimento da equipe • Produtividade do time de desenvolvimento 2. Custo de manutenção e evolução • Hospedagem e manutenção do hardware • Conhecimento da equipe • Estabilidade e maturidade do software • Suporte do fabricante • Produtividade: Ferramentas de gerenciamento, monitoramento e backup • Produtividade do time de evolução do produto
  75. 75. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1297 Carrier Grade Edition (CGE) - comercial MySQL Cluster Suporte Monitor & Backup Plug-ins Manager mysql.com/products/cluster/features.html
  76. 76. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1298 Aprenda mais…
  77. 77. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 1299 Mais sobre como começar com MySQL Cluster Whitepaper: MySQL Cluster Evaluation Guide June 2013 mysql.com/why-mysql/white-papers/mysql- cluster-evaluation-guide
  78. 78. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12100 Próximo passo: documentação Reference Manual Cap. 17.1, MySQL Cluster Overview 1. MySQL Cluster Core Concepts 2. MySQL Cluster Nodes, Node Groups, Replicas, and Partitions 3. MySQL Cluster Hardware, Software, and Networking Requirements 4. MySQL Cluster Development History 5. MySQL Server Using InnoDB Compared with MySQL Cluster 6. Known Limitations of MySQL Cluster dev.mysql.com/doc/refman/5.6/en/mysql-cluster.html
  79. 79. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12101 Mais sobre como MySQL Cluster para Web Whitepaper: Guide to Scaling Web Databases with MySQL Cluster June 2013 mysql.com/why-mysql/white-papers/guide-to- scaling-web-databases-with-mysql-cluster
  80. 80. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12102 MySQL Boot Camp Accelerated MySQL Performance Tuning Boot Camp Accelerated MySQL for Begginers MySQL for Database Administrators MySQL Performance Tuning MySQL High Availability MySQL Cluster MySQL DBA MySQL Boot Camp Accelerated MySQL for Developers MySQL Performance Tuning Boot Camp Accelerated MySQL for Begginers MySQL and PHP Developing Dynamic Web Applicationg MySQL Advanced Stored Procedures MySQLDeveloper mysql.com/training Certificações Opcional Necessário Treinamentos e certificações
  81. 81. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12103 Sumário  O MySQL Cluster é um produto derivado do popular MySQL Server que é o "M" do LAMP stack e pode ser usado como solução para problemas de Big Data.  Trata-se de um banco de dados distribuído com arquitetura shared-nothing e que pode oferecer 99,999% de disponibilidade, performance superior a 1 bilhão de escritas por minuto, escalabilidade linear, conformidade com o modelo ACID, flexibilidade das interfaces NoSQL e geo-replicação multi- master.  Instale, migre algumas tabelas e comece a usar!
  82. 82. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12104 @MySQLBR facebook.com/MySQLBR Obrigado! Slides disponíves em…
  83. 83. Copyright © 2012, Oracle and/or its affiliates. All rights reserved. Insert Information Protection Policy Classification from Slide 12105 Perguntas? MySQL Cluster e Big Data – visão geral
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×