0
Lightning talks       Giuseppe Maxia       Continuent, Inc                          April 11, 2012Wednesday, April 11, 12
Lineup             Stewart Smith        The 5 minutes tour of Drizzle                    Jeff Mace     What can possibly g...
The 5 minutes tour of                             Drizzle                             Stewart SmithWednesday, April 11, 12
Wednesday, April 11, 12
2008...Wednesday, April 11, 12
http://www.flickr.com/photos/shaylor/53819053/Wednesday, April 11, 12
2011...Wednesday, April 11, 12
Drizzle7Wednesday, April 11, 12
2012...Wednesday, April 11, 12
Drizzle7.1Wednesday, April 11, 12
What is                          Drizzle?Wednesday, April 11, 12
Modular                 architectureWednesday, April 11, 12
PluggableWednesday, April 11, 12
Infrastructure            AwareWednesday, April 11, 12
Community                   DevelopedWednesday, April 11, 12
MulticoreWednesday, April 11, 12
ConcurrencyWednesday, April 11, 12
Web focusWednesday, April 11, 12
UTF-8Wednesday, April 11, 12
EBCDIC                            !=Wednesday, April 11, 12
Enable                          othersWednesday, April 11, 12
Modern                          codebaseWednesday, April 11, 12
C++Wednesday, April 11, 12
STLWednesday, April 11, 12
BoostWednesday, April 11, 12
No                           linked-list                          innovationWednesday, April 11, 12
CollaborationWednesday, April 11, 12
ParticipationWednesday, April 11, 12
Build                    businessesWednesday, April 11, 12
Support & ConsultingWednesday, April 11, 12
TestingWednesday, April 11, 12
TransactionalWednesday, April 11, 12
LoggingWednesday, April 11, 12
ReplicationWednesday, April 11, 12
Multi-source              ReplicationWednesday, April 11, 12
AuthWednesday, April 11, 12
LDAPWednesday, April 11, 12
PAMWednesday, April 11, 12
In database                    tablesWednesday, April 11, 12
No bad dataWednesday, April 11, 12
strict_modeWednesday, April 11, 12
No GOTCHAsWednesday, April 11, 12
No implicit                          COMMITWednesday, April 11, 12
BSD Client                       LibrariesWednesday, April 11, 12
New in 7.1Wednesday, April 11, 12
Multi-source              ReplicationWednesday, April 11, 12
XtraBackupWednesday, April 11, 12
IPv6 data                            typeWednesday, April 11, 12
ZeroMQWednesday, April 11, 12
Syslog is                           defaultWednesday, April 11, 12
query_log                            pluginWednesday, April 11, 12
Auth from                           tablesWednesday, April 11, 12
Server-side                   JavascriptWednesday, April 11, 12
HTTP JSON                APIWednesday, April 11, 12
Percona                          InnoDB                          PatchesWednesday, April 11, 12
Wednesday, April 11, 12
What could possibly go                         wrong?                           Jeff MaceWednesday, April 11, 12
What Could Possibly                                  Go Wrong?                                       Je! Mace   ©Continuen...
http://mingle2.com/blog/view/web-developer-mind   ©Continuent 2012       59Wednesday, April 11, 12
http://www.flickr.com/photos/brhefele/4026340146/   ©Continuent 2012       60Wednesday, April 11, 12
http://www.flickr.com/photos/themoly/4336260658/   ©Continuent 2012       61Wednesday, April 11, 12
©Continuent 2012       62Wednesday, April 11, 12
©Continuent 2012       62Wednesday, April 11, 12
What can I possibly do?                     •    Relax                     •    Have a plan                     •    Keep ...
http://www.flickr.com/photos/devenlaney/5767271943   ©Continuent 2012       64Wednesday, April 11, 12
MAKE.DATABASES.FAST                          Nicolai PlumWednesday, April 11, 12
MAKE.DATABASES.FAST                                                                      Nicolai Plum    Booking.com, part...
The business        • Booking.com              – Leading online hotel reservation agency              – Part of the Pricel...
The challenge    Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www...
The problem…        •   Hundreds of database servers        •   Many replication slaves        •   Few DBAs, some more sys...
If you do this manually               You are doing it wrong    Booking.com, part of priceline.com (Nasdaq: PCLN), headqua...
Among our weapons are…        •   Network OS installation        •   Puppet configuration management        •   Database c...
Server installation        • Server Database        • PXE netboot        • CentOS kickstart    Booking.com, part of pricel...
System configuration        • Now we have a server that can run Puppet        • Puppet does              – OS setup       ...
Database cloning        • Now we have a server ready to run a database        • Copying databases based on LVM snapshots  ...
Things we learned        • Waiting for actions to complete takes a lot of person          time – automate batches of work ...
Things we learned               DBAs are toolsmiths,              not database installers    Booking.com, part of pricelin...
Thank you        • … if you like the way we work, we are hiring        • http://www.booking.com/jobs    Booking.com, part ...
Schemadoc                            Bill KarwinWednesday, April 11, 12
Schemadoc                            Bill Karwin, Percona Inc.Wednesday, April 11, 12
Schema Can Be Unclear             CREATE TABLE AccountActivity (                 Id        INT PRIMARY KEY,               ...
Schema Can Be Unclear             CREATE TABLE AccountActivity (                 Id        INT PRIMARY KEY,               ...
Schema Can Be Unclear             CREATE TABLE AccountActivity (                 Id        INT PRIMARY KEY,               ...
Schema Can Be Unclear             CREATE TABLE AccountActivity (                 Id        INT PRIMARY KEY,               ...
Schema Can Be Unclear             CREATE TABLE AccountActivity (                 Id        INT PRIMARY KEY,               ...
Comments!             CREATE TABLE AccountActivity (                Id        INT PRIMARY KEY,                Name      VA...
Length       • Limits increased in MySQL 5.5.3:               • Per column: 1024 characters               • Per index: 102...
“Is there a tool for MySQL                           like javadoc for Java code?”                                         ...
development                                        dba                             documentation                          ...
schemadoc                                      www.percona.comWednesday, April 11, 12
Usage       $ schemadoc -v h=dbserver,D=EcommerceDB,u=root,p=xxxx       Loading metadata...       Running online analysis....
Wednesday, April 11, 12
hooray.                                    www.percona.comWednesday, April 11, 12
But Wait, There’s More       • Parses SQL dump file       • Reads from a live MySQL instance       • Document many schemas...
Schema Analysis       • Number of indexes and columns per data type       • Columns with same name but different type     ...
Schema Analysis       • Tables that look like Entity-Attribute-Value       • Tables that look like Polymorphic Association...
yes, we get it.                                            www.percona.comWednesday, April 11, 12
Future of Schemadoc       • Integrate with Percona Toolkit library       • Read extra metadata (foreign keys)       • Show...
Wednesday, April 11, 12
Future of Schemadoc                                            www.percona.comWednesday, April 11, 12
launchpad.net/                            schemadoc                                       www.percona.comWednesday, April ...
Its the application,                                 stupid!                                Robert HodgesWednesday, April ...
It’s the Application, Stupid!                             (Or: Multi-Master Enlightenment                                 ...
Multi-Master Replication Is This Easy           tungsten-sandbox-2.0.4 -m 5.1.54              -i $RELDIR/tungsten-replicat...
Multi-Master Apps Not So Much                        Airline        Airline        Airline                      Reservatio...
Who Is in Which Seat on Flight 3523?                    Book Bob         Bump Bob from     Book Sue                   in S...
Rules for Multi-Master Apps                     1. Isolate destructive changes to a single                        master t...
Rules for Multi-Master Apps                     3. Coordinate across masters to aggregate                        results  ...
Multi-Master Enlightenment                          Scalable multi-master applications are                          eventu...
NuoDB overview                              Robert BuckWednesday, April 11, 12
The Elastically Scalable Database™                                       Copyright © 2012 NuoDBWednesday, April 11, 12
NuoDB Summary           ‣       SQL/ACID: Emergent Architecture           ‣       Elastically Scale: Demand-driven, auto-b...
Emergent Database Architecture          “An emergent           behavior can         appear when a        number of simple ...
Instantly Elastic     ‣ Second machine more         than doubles TPS due to         caching effects     ‣ Second machine i...
Linearly Scalable                                                 30000"                          Nodes    TPS      MySQL ...
... even on Amazon EC2       ‣ NuoDB scales linearly                                        12000"         on EC2         ...
We Work with Your SQL Tools                                                                    MS Excel (and other MS tool...
NuoDB System Architecture                                                 TM1                                             ...
NuoDB Summary           ‣          100% SQL           ‣          100% ACID           ‣          100% Elastic Scaling      ...
NuoDB                          The Elastically Scalable Database™                              Copyright © 2012 NuoDBWedne...
Diagnosis and possible                             fixes                            David HoloboffWednesday, April 11, 12
DIAGNOSIS AND                          POSSIBLE FIXES:               A MULTIPLE CHOICE PRIMER                 FOR BEGINNIN...
This talk is for entertainment value only.                          Any perceived similarity between what you             ...
Wednesday, April 11, 12
Wednesday, April 11, 12
Wednesday, April 11, 12
Wednesday, April 11, 12
Are you being asked the right question?Wednesday, April 11, 12
Wednesday, April 11, 12
Wednesday, April 11, 12
Wednesday, April 11, 12
Wednesday, April 11, 12
Wednesday, April 11, 12
What do you do with a slow database server?Wednesday, April 11, 12
Wednesday, April 11, 12
What do you do if swap memory is always being used up?Wednesday, April 11, 12
Wednesday, April 11, 12
Wednesday, April 11, 12
Wednesday, April 11, 12
Wednesday, April 11, 12
Wednesday, April 11, 12
Wednesday, April 11, 12
Wednesday, April 11, 12
Wednesday, April 11, 12
Wednesday, April 11, 12
Wednesday, April 11, 12
Wednesday, April 11, 12
Wednesday, April 11, 12
Wednesday, April 11, 12
Wednesday, April 11, 12
Wednesday, April 11, 12
Wednesday, April 11, 12
Wednesday, April 11, 12
OLTP and OLAP                             Lawrence SchwartzWednesday, April 11, 12
OLTP and OLAP                          Have Your Cake and Eat it Too!                                Lawrence SchwartzWedn...
Avoiding the B-tree Valley of Death                   “The real end-game for Big Data is to have transactional a          ...
B-trees Perform Poorly on Disk      MySQL + InnoDB                        MySQL + TokuDB     Based on B-Tree Indexes      ...
B-trees Can’t Handle Big Data      MySQL + InnoDB                       MySQL + TokuDB  • Performance Drops to a Trickle a...
B-trees Aren’t Agile      MySQL + InnoDB                     MySQL + TokuDB  • Big Database Schema Changes Take     • Chan...
B-trees Don’t Compress Well      MySQL + InnoDB                      MySQL + TokuDB  • Compression ~2x to ~2.5x Typical   ...
TokuDB Eliminates Death Valley                          “While TokuDB® is effectively an operational database technology, ...
Wednesday, April 11, 12
Building a scalable                                system                               Baron SchwartzWednesday, April 11,...
Wednesday, April 11, 12
Wednesday, April 11, 12
Wednesday, April 11, 12
How to become a rock                       star DBA                          Sheeri CabralWednesday, April 11, 12
How to Become a Rock Star DBA                              Sheeri Cabral, Mozilla                               Oracle ACE...
Do what you love....Wednesday, April 11, 12
What you really loveWednesday, April 11, 12
Have a Twin BrotherWednesday, April 11, 12
CodeWednesday, April 11, 12
Write                                             Tutorials / HOW-TOs                          MariaDB/Percona Documentati...
Make MisteaksMistakesWednesday, April 11, 12
Teach othersWednesday, April 11, 12
Teach others                          You know more than you think you do!Wednesday, April 11, 12
Keep LearningWednesday, April 11, 12
ShareWednesday, April 11, 12
Organize              User Group                    Magazine                Ignite Event                                  ...
Connect PeopleWednesday, April 11, 12
Avoid burnoutWednesday, April 11, 12
Have a hobbyWednesday, April 11, 12
Have SupportWednesday, April 11, 12
Be tall...or notWednesday, April 11, 12
Find Ways Others                            Can Help YouWednesday, April 11, 12
Some Help You Can Get        MySQL Community Team        Volunteers – UG video: Richard Laskey        “What can I help you...
It Takes a Community        Permission to use pictures granted by:        Ronald Bradford        Francesca Simon-Carboni P...
Cruelty on database                                clusters                                Giuseppe MaxiaWednesday, April ...
Cruel things that you can                          inflict on your replicated                    cluster ... and get away ...
I must confess   ©Continuent 2012             185Wednesday, April 11, 12
I must confess                          I am a serial killer   ©Continuent 2012                 185Wednesday, April 11, 12
I kill   ©Continuent 2012       186Wednesday, April 11, 12
I kill                     •    database servers   ©Continuent 2012                 186Wednesday, April 11, 12
I kill                     •    database servers                     • client connections   ©Continuent 2012              ...
I kill                     •    database servers                     • client connections                     • transactio...
I kill                     •    database servers                     • client connections                     • transactio...
I am armed and dangerous                                 +                          = "class 2" weapon   ©Continuent 2012 ...
And of course I carry ...(GASP)                                    Perl!   ©Continuent 2012               188Wednesday, Ap...
I have a criminal mindset                     •    Like a bank robber                     •    or an airplane hijacker    ...
Whenever I get a new database               cluster                     •    I open "re                                   ...
when a cluster is working               just "ne ...   ©Continuent 2012         191Wednesday, April 11, 12
when a cluster is working               just "ne ...                     •    I pull the rug from under                   ...
when a cluster is working               just "ne ...                     •    I pull the rug from under                   ...
when a cluster is working               just "ne ...                     •    I pull the rug from under                   ...
I am a Harry Potter fan   ©Continuent 2012                  192Wednesday, April 11, 12
So I get carried away, and I use the               unforgivable curses                                        $ kill -9   ...
We have di#erent               standpoints   ©Continuent 2012        194Wednesday, April 11, 12
We have di#erent               standpoints                     •    My colleagues say that I                          am c...
We have di#erent               standpoints                     •    My colleagues say that I                          am c...
Have you ever reported a bug ...                     • about a failure that happens after                          9000 sw...
Have you ever reported a bug ...                     • about a failure that happens after                          9000 sw...
Things I do that make developers say               "it is not fair"   ©Continuent 2012              196Wednesday, April 11...
Things I do that make developers say               "it is not fair"                     •    using the installer with wron...
Things I do that make developers say               "it is not fair"                     •    using the installer with wron...
Things I do that make developers say               "it is not fair"                     •    using the installer with wron...
Things I do that make developers say               "it is not fair"                     •    using the installer with wron...
Things I do that make developers say               "it is not fair"                     •    using the installer with wron...
Why do I do cruel things to replication               clusters?   ©Continuent 2012               197Wednesday, April 11, 12
Why do I do cruel things to replication               clusters?                     • Because, most of the time:          ...
Why do I do cruel things to replication               clusters?                     • Because, most of the time:          ...
in the end, I get away with cruelty                I am not bad. I am                just drawn that way                  ...
Upcoming SlideShare
Loading in...5
×

Lightning talks percona live mysql_2012

2,342

Published on

Lightning talks at Percona Live MySQL Conference and Expo 2012

Published in: Technology
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
2,342
On Slideshare
0
From Embeds
0
Number of Embeds
1
Actions
Shares
0
Downloads
25
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide

Transcript of "Lightning talks percona live mysql_2012"

  1. 1. Lightning talks Giuseppe Maxia Continuent, Inc April 11, 2012Wednesday, April 11, 12
  2. 2. Lineup Stewart Smith The 5 minutes tour of Drizzle Jeff Mace What can possibly go wrong? Nicolai Plum MAKE.DATABASES.FAST Bill Karwin Schemadoc Robert Hodges Its the application, stupid! Robert Buck NuoDB overview David Holoboff Diagnosis and possible fixes Lawrence Schwartz OLTP and OLAP Baron Schwartz Building a scalable system Sheeri Cabral How to become a rock star DBA Giuseppe Maxia Cruelty on database clustersWednesday, April 11, 12
  3. 3. The 5 minutes tour of Drizzle Stewart SmithWednesday, April 11, 12
  4. 4. Wednesday, April 11, 12
  5. 5. 2008...Wednesday, April 11, 12
  6. 6. http://www.flickr.com/photos/shaylor/53819053/Wednesday, April 11, 12
  7. 7. 2011...Wednesday, April 11, 12
  8. 8. Drizzle7Wednesday, April 11, 12
  9. 9. 2012...Wednesday, April 11, 12
  10. 10. Drizzle7.1Wednesday, April 11, 12
  11. 11. What is Drizzle?Wednesday, April 11, 12
  12. 12. Modular architectureWednesday, April 11, 12
  13. 13. PluggableWednesday, April 11, 12
  14. 14. Infrastructure AwareWednesday, April 11, 12
  15. 15. Community DevelopedWednesday, April 11, 12
  16. 16. MulticoreWednesday, April 11, 12
  17. 17. ConcurrencyWednesday, April 11, 12
  18. 18. Web focusWednesday, April 11, 12
  19. 19. UTF-8Wednesday, April 11, 12
  20. 20. EBCDIC !=Wednesday, April 11, 12
  21. 21. Enable othersWednesday, April 11, 12
  22. 22. Modern codebaseWednesday, April 11, 12
  23. 23. C++Wednesday, April 11, 12
  24. 24. STLWednesday, April 11, 12
  25. 25. BoostWednesday, April 11, 12
  26. 26. No linked-list innovationWednesday, April 11, 12
  27. 27. CollaborationWednesday, April 11, 12
  28. 28. ParticipationWednesday, April 11, 12
  29. 29. Build businessesWednesday, April 11, 12
  30. 30. Support & ConsultingWednesday, April 11, 12
  31. 31. TestingWednesday, April 11, 12
  32. 32. TransactionalWednesday, April 11, 12
  33. 33. LoggingWednesday, April 11, 12
  34. 34. ReplicationWednesday, April 11, 12
  35. 35. Multi-source ReplicationWednesday, April 11, 12
  36. 36. AuthWednesday, April 11, 12
  37. 37. LDAPWednesday, April 11, 12
  38. 38. PAMWednesday, April 11, 12
  39. 39. In database tablesWednesday, April 11, 12
  40. 40. No bad dataWednesday, April 11, 12
  41. 41. strict_modeWednesday, April 11, 12
  42. 42. No GOTCHAsWednesday, April 11, 12
  43. 43. No implicit COMMITWednesday, April 11, 12
  44. 44. BSD Client LibrariesWednesday, April 11, 12
  45. 45. New in 7.1Wednesday, April 11, 12
  46. 46. Multi-source ReplicationWednesday, April 11, 12
  47. 47. XtraBackupWednesday, April 11, 12
  48. 48. IPv6 data typeWednesday, April 11, 12
  49. 49. ZeroMQWednesday, April 11, 12
  50. 50. Syslog is defaultWednesday, April 11, 12
  51. 51. query_log pluginWednesday, April 11, 12
  52. 52. Auth from tablesWednesday, April 11, 12
  53. 53. Server-side JavascriptWednesday, April 11, 12
  54. 54. HTTP JSON APIWednesday, April 11, 12
  55. 55. Percona InnoDB PatchesWednesday, April 11, 12
  56. 56. Wednesday, April 11, 12
  57. 57. What could possibly go wrong? Jeff MaceWednesday, April 11, 12
  58. 58. What Could Possibly Go Wrong? Je! Mace ©Continuent 2012.Wednesday, April 11, 12
  59. 59. http://mingle2.com/blog/view/web-developer-mind ©Continuent 2012 59Wednesday, April 11, 12
  60. 60. http://www.flickr.com/photos/brhefele/4026340146/ ©Continuent 2012 60Wednesday, April 11, 12
  61. 61. http://www.flickr.com/photos/themoly/4336260658/ ©Continuent 2012 61Wednesday, April 11, 12
  62. 62. ©Continuent 2012 62Wednesday, April 11, 12
  63. 63. ©Continuent 2012 62Wednesday, April 11, 12
  64. 64. What can I possibly do? • Relax • Have a plan • Keep it simple ©Continuent 2012 63Wednesday, April 11, 12
  65. 65. http://www.flickr.com/photos/devenlaney/5767271943 ©Continuent 2012 64Wednesday, April 11, 12
  66. 66. MAKE.DATABASES.FAST Nicolai PlumWednesday, April 11, 12
  67. 67. MAKE.DATABASES.FAST Nicolai Plum Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/officesWednesday, April 11, 12
  68. 68. The business • Booking.com – Leading online hotel reservation agency – Part of the Priceline Group of Companies – Sustained rapid business growth – All powered by MySQL Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/officesWednesday, April 11, 12
  69. 69. The challenge Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/officesWednesday, April 11, 12
  70. 70. The problem… • Hundreds of database servers • Many replication slaves • Few DBAs, some more sysadmins • Need to make tens of servers each day Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/officesWednesday, April 11, 12
  71. 71. If you do this manually You are doing it wrong Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/officesWednesday, April 11, 12
  72. 72. Among our weapons are… • Network OS installation • Puppet configuration management • Database cloning • and a fanatical devotion to automation! Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/officesWednesday, April 11, 12
  73. 73. Server installation • Server Database • PXE netboot • CentOS kickstart Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/officesWednesday, April 11, 12
  74. 74. System configuration • Now we have a server that can run Puppet • Puppet does – OS setup – Software installation – LVM setup and filesystem creation (tailored by role) • Prefer ongoing configuration management tools to single perfect installation tools Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/officesWednesday, April 11, 12
  75. 75. Database cloning • Now we have a server ready to run a database • Copying databases based on LVM snapshots • create_clone program – Copy via SSH and rsync – LVM snapshot – Fix my.cnf (server id!) automatically Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/officesWednesday, April 11, 12
  76. 76. Things we learned • Waiting for actions to complete takes a lot of person time – automate batches of work as much as possible • You really need a Server Database, and drive everything from it • Most hardware can’t support being a clone source and in service, you need spares • Online snapshots break in subtle ways; best stop the database. Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/officesWednesday, April 11, 12
  77. 77. Things we learned DBAs are toolsmiths, not database installers Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/officesWednesday, April 11, 12
  78. 78. Thank you • … if you like the way we work, we are hiring • http://www.booking.com/jobs Booking.com, part of priceline.com (Nasdaq: PCLN), headquartered in Amsterdam, 50+ offices worldwide. www.booking.com/officesWednesday, April 11, 12
  79. 79. Schemadoc Bill KarwinWednesday, April 11, 12
  80. 80. Schemadoc Bill Karwin, Percona Inc.Wednesday, April 11, 12
  81. 81. Schema Can Be Unclear CREATE TABLE AccountActivity ( Id INT PRIMARY KEY, Name VARCHAR(100), Status VARCHAR(20), EntityNum INT, Order INT ); www.percona.comWednesday, April 11, 12
  82. 82. Schema Can Be Unclear CREATE TABLE AccountActivity ( Id INT PRIMARY KEY, whose name? Name VARCHAR(100), Status VARCHAR(20), EntityNum INT, Order INT ); www.percona.comWednesday, April 11, 12
  83. 83. Schema Can Be Unclear CREATE TABLE AccountActivity ( Id INT PRIMARY KEY, whose name? Name VARCHAR(100), Status VARCHAR(20), what are the values? EntityNum INT, Order INT ); www.percona.comWednesday, April 11, 12
  84. 84. Schema Can Be Unclear CREATE TABLE AccountActivity ( Id INT PRIMARY KEY, whose name? Name VARCHAR(100), Status VARCHAR(20), what are the values? EntityNum INT, what’s an entity? Order INT what’s a number? ); www.percona.comWednesday, April 11, 12
  85. 85. Schema Can Be Unclear CREATE TABLE AccountActivity ( Id INT PRIMARY KEY, whose name? Name VARCHAR(100), Status VARCHAR(20), what are the values? EntityNum INT, what’s an entity? Order INT what’s a number? ); commerce, or ordinal? www.percona.comWednesday, April 11, 12
  86. 86. Comments! CREATE TABLE AccountActivity ( Id INT PRIMARY KEY, Name VARCHAR(100) COMMENT ‘person who placed the order’, Status VARCHAR(20) COMMENT ‘new, open, or closed’, EntityNum INT COMMENT ‘how many items in the order’, Order INT COMMENT ‘reference to the Orders table’ ) COMMENT ‘any change to e-commerce orders’; www.percona.comWednesday, April 11, 12
  87. 87. Length • Limits increased in MySQL 5.5.3: • Per column: 1024 characters • Per index: 1024 characters • Per table: 2048 characters • Per partition: 80 characters (MySQL 5.6 increases this to 1024) www.percona.comWednesday, April 11, 12
  88. 88. “Is there a tool for MySQL like javadoc for Java code?” www.percona.comWednesday, April 11, 12
  89. 89. development dba documentation www.percona.comWednesday, April 11, 12
  90. 90. schemadoc www.percona.comWednesday, April 11, 12
  91. 91. Usage $ schemadoc -v h=dbserver,D=EcommerceDB,u=root,p=xxxx Loading metadata... Running online analysis... done. Outputting to html. Rendering pages for tables. done. Rendering pages for views done. Rendering pages for procedures done. Rendering pages for triggers done. Rendering page for schema summary... done. Rendering page for schema link... done. Rendering page for deprecated-list... done. Rendering page for all-index... done. Rendering page for help-doc... done. Rendering page for allschema-frame... done. Rendering page for index... done. www.percona.comWednesday, April 11, 12
  92. 92. Wednesday, April 11, 12
  93. 93. hooray. www.percona.comWednesday, April 11, 12
  94. 94. But Wait, There’s More • Parses SQL dump file • Reads from a live MySQL instance • Document many schemas • Update one schema at a time www.percona.comWednesday, April 11, 12
  95. 95. Schema Analysis • Number of indexes and columns per data type • Columns with same name but different type • ENUM columns that aren’t NOT NULL • Use of FLOAT or DOUBLE • Tables with one index per column • Tables with one index over all columns • Tables with no indexes • INT(N) other than default N • Lack of INT UNSIGNED columns www.percona.comWednesday, April 11, 12
  96. 96. Schema Analysis • Tables that look like Entity-Attribute-Value • Tables that look like Polymorphic Associations • Superfluous primary key • Report table with least / most indexes • IP addresses stored in VARCHAR • Excessive use of VARCHAR(255), report actual max string length • ...And supports plugins for other checks! www.percona.comWednesday, April 11, 12
  97. 97. yes, we get it. www.percona.comWednesday, April 11, 12
  98. 98. Future of Schemadoc • Integrate with Percona Toolkit library • Read extra metadata (foreign keys) • Show table sizes and growth rates • Show SQL privileges (who has access?) • CLI for adding / updating comments • Batch-mode output • Redesign HTML output with modern look & feel www.percona.comWednesday, April 11, 12
  99. 99. Wednesday, April 11, 12
  100. 100. Future of Schemadoc www.percona.comWednesday, April 11, 12
  101. 101. launchpad.net/ schemadoc www.percona.comWednesday, April 11, 12
  102. 102. Its the application, stupid! Robert HodgesWednesday, April 11, 12
  103. 103. It’s the Application, Stupid! (Or: Multi-Master Enlightenment in 5 Minutes or Less) Robert Hodges, Continuent CEO ©Continuent 2012.Wednesday, April 11, 12
  104. 104. Multi-Master Replication Is This Easy tungsten-sandbox-2.0.4 -m 5.1.54 -i $RELDIR/tungsten-replicator-2.0.5 -n 3 --topology=star --hub=2 Spoke Hub Spoke ©Continuent 2012 99Wednesday, April 11, 12
  105. 105. Multi-Master Apps Not So Much Airline Airline Airline Reservations Reservations Reservations Spoke Hub Spoke ©Continuent 2012 100Wednesday, April 11, 12
  106. 106. Who Is in Which Seat on Flight 3523? Book Bob Bump Bob from Book Sue in Seat 11E Flight 3523 in Seat 11E Airline Airline Airline Reservations Reservations Reservations Spoke Hub Spoke ©Continuent 2012 101Wednesday, April 11, 12
  107. 107. Rules for Multi-Master Apps 1. Isolate destructive changes to a single master to avoid harmful con!icts ➡ “Change gate for !ight 3523 to C28” 2. Use non-destructive INSERTs for everything else ➡ “Please assign Sue to seat 11A” ©Continuent 2012 102Wednesday, April 11, 12
  108. 108. Rules for Multi-Master Apps 3. Coordinate across masters to aggregate results ➡ “Generate "nal passenger manifest for !ight 3523” 4. Be prepared to apologize ➡ “Add 5000 frequent !yer miles to Bob’s account” ©Continuent 2012 103Wednesday, April 11, 12
  109. 109. Multi-Master Enlightenment Scalable multi-master applications are eventually consistent 1. Think through the whole problem: Business processes are often eventually consistent already 2. Follow design rules for eventually consistent applications 3. Use appropriate replication technology for MySQL* * As in Tungsten :) ©Continuent 2012 104Wednesday, April 11, 12
  110. 110. NuoDB overview Robert BuckWednesday, April 11, 12
  111. 111. The Elastically Scalable Database™ Copyright © 2012 NuoDBWednesday, April 11, 12
  112. 112. NuoDB Summary ‣ SQL/ACID: Emergent Architecture ‣ Elastically Scale: Demand-driven, auto-balancing ‣ Single Logical Database: No sharding, no partitioning ‣ Run Anywhere: On-Premises / Private, Public, or Hybrid Cloud ‣ Store Anywhere: KV-Store backend (flexible, e.g. Files, S3, HDFS) ‣ Multi-Tenant: DBs share machines dynamically ‣ Geo-Distributed: Active/active across several data centers ‣ Extreme Availability: Massively redundant, permitting live upgrade ‣ Minimal Admin: No knobs, auto everything Copyright © 2012 NuoDB 107Wednesday, April 11, 12
  113. 113. Emergent Database Architecture “An emergent behavior can appear when a number of simple agents operate in an environment, forming more complex behaviors as a collective.” - Wikipedia Copyright © 2012 NuoDB 108Wednesday, April 11, 12
  114. 114. Instantly Elastic ‣ Second machine more than doubles TPS due to caching effects ‣ Second machine is added to live database while it is running at 1,000’s of TPS ‣ Performance increase is immediate ‣ BTW - you can take Second Machine either machine away Instant Performance Increase and the database keeps running without data loss Copyright © 2012 NuoDB 109Wednesday, April 11, 12
  115. 115. Linearly Scalable 30000" Nodes TPS MySQL 1 3,000 25000" NuoDB 1 4,500 20000" NuoDB 9 27,000 15000" Technical Details: TPS 10000" ‣ 2-9 Tx engines 5000" ‣ 1 storage manager ‣ Best sustained TPS and # 0" clients combination 1" 2" 3" 4" 5" 6" 7" 8" 9" ‣ 50% updates Number of Nodes NuoDB running on 9 nodes was approx. 9x faster than MySQL running on 1 node. Copyright © 2012 NuoDB 110Wednesday, April 11, 12
  116. 116. ... even on Amazon EC2 ‣ NuoDB scales linearly 12000" on EC2 10000" ‣ EC2 performance bound by AMZ TCP/ 8000" IP traffic shaping ‣ Linearly scales at 50% 6000" TPS of rate for physical 4000" commodity hardware ‣ Whereas, Amazon 2000" RDS runs on 1 node, and gets overloaded 0" 1" 2" 3" 4" 5" 6" 7" 8" 9" at 10+ connections Number of EC2 Nodes Copyright © 2012 NuoDB 111Wednesday, April 11, 12
  117. 117. We Work with Your SQL Tools MS Excel (and other MS tools) SQuirreLSQL DbVisualizer Copyright © 2012 NuoDB 112Wednesday, April 11, 12
  118. 118. NuoDB System Architecture TM1 TM1 SM TM1 TM1 TM1 TM1 B1 B1 TM1 B1 TM1 TM1 SM B TM1 TM1 TM1 TM1 TM1 NuoDB Brokers TM1 TE SM Application (JDBC, ODBC, SQL) NuoDB Transaction NuoDB Storage Engines Manager Tx Scalability Add as many of B, TE, or SM as you like I/O Scalability Distributed IO, e.g. Hadoop HDFS 100% Uptime No single point of failure, geo-distributed Optimal Utilization Multiple Databases on finite resources Low People Costs Single Console Management Copyright © 2012 NuoDB 113Wednesday, April 11, 12
  119. 119. NuoDB Summary ‣ 100% SQL ‣ 100% ACID ‣ 100% Elastic Scaling ‣ If you have more questions... ‣ Prove it to yourself. Try it out! ‣ Download it at: www.nuodb.com ‣ Chat with us afterward Copyright © 2012 NuoDB 114Wednesday, April 11, 12
  120. 120. NuoDB The Elastically Scalable Database™ Copyright © 2012 NuoDBWednesday, April 11, 12
  121. 121. Diagnosis and possible fixes David HoloboffWednesday, April 11, 12
  122. 122. DIAGNOSIS AND POSSIBLE FIXES: A MULTIPLE CHOICE PRIMER FOR BEGINNING DBASWednesday, April 11, 12
  123. 123. This talk is for entertainment value only. Any perceived similarity between what you will hear in the next five minutes and true life scenarios are merely coincidental, with no intent of harm or liability.Wednesday, April 11, 12
  124. 124. Wednesday, April 11, 12
  125. 125. Wednesday, April 11, 12
  126. 126. Wednesday, April 11, 12
  127. 127. Wednesday, April 11, 12
  128. 128. Are you being asked the right question?Wednesday, April 11, 12
  129. 129. Wednesday, April 11, 12
  130. 130. Wednesday, April 11, 12
  131. 131. Wednesday, April 11, 12
  132. 132. Wednesday, April 11, 12
  133. 133. Wednesday, April 11, 12
  134. 134. What do you do with a slow database server?Wednesday, April 11, 12
  135. 135. Wednesday, April 11, 12
  136. 136. What do you do if swap memory is always being used up?Wednesday, April 11, 12
  137. 137. Wednesday, April 11, 12
  138. 138. Wednesday, April 11, 12
  139. 139. Wednesday, April 11, 12
  140. 140. Wednesday, April 11, 12
  141. 141. Wednesday, April 11, 12
  142. 142. Wednesday, April 11, 12
  143. 143. Wednesday, April 11, 12
  144. 144. Wednesday, April 11, 12
  145. 145. Wednesday, April 11, 12
  146. 146. Wednesday, April 11, 12
  147. 147. Wednesday, April 11, 12
  148. 148. Wednesday, April 11, 12
  149. 149. Wednesday, April 11, 12
  150. 150. Wednesday, April 11, 12
  151. 151. Wednesday, April 11, 12
  152. 152. Wednesday, April 11, 12
  153. 153. Wednesday, April 11, 12
  154. 154. OLTP and OLAP Lawrence SchwartzWednesday, April 11, 12
  155. 155. OLTP and OLAP Have Your Cake and Eat it Too! Lawrence SchwartzWednesday, April 11, 12
  156. 156. Avoiding the B-tree Valley of Death “The real end-game for Big Data is to have transactional a analytic data on the same database.” However, Today’s database categories - OLAP, OLTP are compromises to get around dated, rigid indexing technology B-tree Valley of Usage Death OLAP OLTP Columnar DBs Hadoop MongoDB Read-intensive Write-intensiveWednesday, April 11, 12
  157. 157. B-trees Perform Poorly on Disk MySQL + InnoDB MySQL + TokuDB Based on B-Tree Indexes Based on Fractal Tree Indexes • Random Workloads Lead To Frequent • Continuous Rebalancing and Drive Head Movement Aggregation at Nodes • MySQL Can’t Take Advantage of Full • Eliminates Fragmentation, Drive I/O Potential Maximizes Disk I/OWednesday, April 11, 12
  158. 158. B-trees Can’t Handle Big Data MySQL + InnoDB MySQL + TokuDB • Performance Drops to a Trickle above • 20x Insertion Rates at One 50-100M Rows Billion Rows • Delayed Checkpoints Diminish • Stand Up More Indexes for Performance Better Query PerformanceWednesday, April 11, 12
  159. 159. B-trees Aren’t Agile MySQL + InnoDB MySQL + TokuDB • Big Database Schema Changes Take • Changes Complete in Seconds Hours To Complete • Allows Business Data Models • Changes Have to Wait for Scheduled to Immediately Adapt DowntimeWednesday, April 11, 12
  160. 160. B-trees Don’t Compress Well MySQL + InnoDB MySQL + TokuDB • Compression ~2x to ~2.5x Typical • High Compression • Compression Impacts Performance • No Impact to PerformanceWednesday, April 11, 12
  161. 161. TokuDB Eliminates Death Valley “While TokuDB® is effectively an operational database technology, it does blur the lines between operations and analytics” eCommerce Personalization Social Cloud Networks Assets Logfile analysis Fractal Tree® Indexes are the Solution Usage Real Time OLAP Analytic OLTP OLAP OLTP Read-intensive Write-intensiveWednesday, April 11, 12
  162. 162. Wednesday, April 11, 12
  163. 163. Building a scalable system Baron SchwartzWednesday, April 11, 12
  164. 164. Wednesday, April 11, 12
  165. 165. Wednesday, April 11, 12
  166. 166. Wednesday, April 11, 12
  167. 167. How to become a rock star DBA Sheeri CabralWednesday, April 11, 12
  168. 168. How to Become a Rock Star DBA Sheeri Cabral, Mozilla Oracle ACE Director MySQL Community Award Winner^2Wednesday, April 11, 12
  169. 169. Do what you love....Wednesday, April 11, 12
  170. 170. What you really loveWednesday, April 11, 12
  171. 171. Have a Twin BrotherWednesday, April 11, 12
  172. 172. CodeWednesday, April 11, 12
  173. 173. Write Tutorials / HOW-TOs MariaDB/Percona Documentation forums.mysql.com #ircWednesday, April 11, 12
  174. 174. Make MisteaksMistakesWednesday, April 11, 12
  175. 175. Teach othersWednesday, April 11, 12
  176. 176. Teach others You know more than you think you do!Wednesday, April 11, 12
  177. 177. Keep LearningWednesday, April 11, 12
  178. 178. ShareWednesday, April 11, 12
  179. 179. Organize User Group Magazine Ignite Event UnconferenceWednesday, April 11, 12
  180. 180. Connect PeopleWednesday, April 11, 12
  181. 181. Avoid burnoutWednesday, April 11, 12
  182. 182. Have a hobbyWednesday, April 11, 12
  183. 183. Have SupportWednesday, April 11, 12
  184. 184. Be tall...or notWednesday, April 11, 12
  185. 185. Find Ways Others Can Help YouWednesday, April 11, 12
  186. 186. Some Help You Can Get MySQL Community Team Volunteers – UG video: Richard Laskey “What can I help you do?” Technocation for grants - meetup.com fees - Open SQL/DB Camp - Postgres Open - Open Source Bridge - Drizzle Day (donate to www.technocation.org!)Wednesday, April 11, 12
  187. 187. It Takes a Community Permission to use pictures granted by: Ronald Bradford Francesca Simon-Carboni Photography Colin Charles Jeremy Cole James Duncan Davidson Keith Larson Gerry NarvajaWednesday, April 11, 12
  188. 188. Cruelty on database clusters Giuseppe MaxiaWednesday, April 11, 12
  189. 189. Cruel things that you can inflict on your replicated cluster ... and get away with it! Giuseppe Maxia Continuent, Inc ©Continuent 2012.Wednesday, April 11, 12
  190. 190. I must confess ©Continuent 2012 185Wednesday, April 11, 12
  191. 191. I must confess I am a serial killer ©Continuent 2012 185Wednesday, April 11, 12
  192. 192. I kill ©Continuent 2012 186Wednesday, April 11, 12
  193. 193. I kill • database servers ©Continuent 2012 186Wednesday, April 11, 12
  194. 194. I kill • database servers • client connections ©Continuent 2012 186Wednesday, April 11, 12
  195. 195. I kill • database servers • client connections • transactions ©Continuent 2012 186Wednesday, April 11, 12
  196. 196. I kill • database servers • client connections • transactions • "lesystems ©Continuent 2012 186Wednesday, April 11, 12
  197. 197. I am armed and dangerous + = "class 2" weapon ©Continuent 2012 187Wednesday, April 11, 12
  198. 198. And of course I carry ...(GASP) Perl! ©Continuent 2012 188Wednesday, April 11, 12
  199. 199. I have a criminal mindset • Like a bank robber • or an airplane hijacker • I look around, and watch for weak spots to exploit ©Continuent 2012 189Wednesday, April 11, 12
  200. 200. Whenever I get a new database cluster • I open "re :) @ @ - $_ #! #! #! #! #! #! ©Continuent 2012 190Wednesday, April 11, 12
  201. 201. when a cluster is working just "ne ... ©Continuent 2012 191Wednesday, April 11, 12
  202. 202. when a cluster is working just "ne ... • I pull the rug from under its feet ©Continuent 2012 191Wednesday, April 11, 12
  203. 203. when a cluster is working just "ne ... • I pull the rug from under its feet • I switch o# a masters database server ©Continuent 2012 191Wednesday, April 11, 12
  204. 204. when a cluster is working just "ne ... • I pull the rug from under its feet • I switch o# a masters database server • or I delete its "les ©Continuent 2012 191Wednesday, April 11, 12
  205. 205. I am a Harry Potter fan ©Continuent 2012 192Wednesday, April 11, 12
  206. 206. So I get carried away, and I use the unforgivable curses $ kill -9 ©Continuent 2012 193Wednesday, April 11, 12
  207. 207. We have di#erent standpoints ©Continuent 2012 194Wednesday, April 11, 12
  208. 208. We have di#erent standpoints • My colleagues say that I am cruel ©Continuent 2012 194Wednesday, April 11, 12
  209. 209. We have di#erent standpoints • My colleagues say that I am cruel • I call it QA ©Continuent 2012 194Wednesday, April 11, 12
  210. 210. Have you ever reported a bug ... • about a failure that happens after 9000 switches? • Or a master that slows down when handling 512 slaves? ©Continuent 2012 195Wednesday, April 11, 12
  211. 211. Have you ever reported a bug ... • about a failure that happens after 9000 switches? • Or a master that slows down when handling 512 slaves? I DID ©Continuent 2012 195Wednesday, April 11, 12
  212. 212. Things I do that make developers say "it is not fair" ©Continuent 2012 196Wednesday, April 11, 12
  213. 213. Things I do that make developers say "it is not fair" • using the installer with wrong options ©Continuent 2012 196Wednesday, April 11, 12
  214. 214. Things I do that make developers say "it is not fair" • using the installer with wrong options • running the installer without a mandatory option ©Continuent 2012 196Wednesday, April 11, 12
  215. 215. Things I do that make developers say "it is not fair" • using the installer with wrong options • running the installer without a mandatory option • Using the wrong version of Java, or Ruby, or MySQL ©Continuent 2012 196Wednesday, April 11, 12
  216. 216. Things I do that make developers say "it is not fair" • using the installer with wrong options • running the installer without a mandatory option • Using the wrong version of Java, or Ruby, or MySQL • running the installer in a system with only 200 MB of free storage ©Continuent 2012 196Wednesday, April 11, 12
  217. 217. Things I do that make developers say "it is not fair" • using the installer with wrong options • running the installer without a mandatory option • Using the wrong version of Java, or Ruby, or MySQL • running the installer in a system with only 200 MB of free storage • Counting the records after a performance test (and "nding that some are missing) ©Continuent 2012 196Wednesday, April 11, 12
  218. 218. Why do I do cruel things to replication clusters? ©Continuent 2012 197Wednesday, April 11, 12
  219. 219. Why do I do cruel things to replication clusters? • Because, most of the time: • Developers are not users • Users do what developers cant imagine ©Continuent 2012 197Wednesday, April 11, 12
  220. 220. Why do I do cruel things to replication clusters? • Because, most of the time: • Developers are not users • Users do what developers cant imagine • To make a fool-proof system you need to act like a fool ©Continuent 2012 197Wednesday, April 11, 12
  221. 221. in the end, I get away with cruelty I am not bad. I am just drawn that way (Jessica Rabbit) ©Continuent 2012 198Wednesday, April 11, 12
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×