Akiban Technologies: Renormalize
Upcoming SlideShare
Loading in...5

Like this? Share it with your network


Akiban Technologies: Renormalize

Uploaded on

Solving Performance Problems in MySQL Without Denormalization...

Solving Performance Problems in MySQL Without Denormalization

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads


Total Views
On Slideshare
From Embeds
Number of Embeds



Embeds 4

http://paper.li 4

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

    No notes for slide


  • 1. Solving Performance Problems in MySQL Without Denormalization RENORMALIZE Akiban Technologies, Inc. Confidential & Proprietary
  • 2. Problem StatementSchemas scale outData volume growsJoins become a real bottleneckAkiban Technologies, Inc. Confidential & Proprietary 2
  • 3. Two Common ManifestationsSQL Joins Queries become slower as more tables are joined.Application Object Creations Constructing an object is as expensive as SELECTing the sum of its parts Denormalize. Problem solved.Akiban Technologies, Inc. Confidential & Proprietary 3
  • 4. Application Growing Pains Web Cache Server Server V6 Release V5 V4 V3 V1 V2 Rip & ReplaceDB Shard Database Add Customers! Get Caching Replicate DB De-normalize Complexity & CostCustomers MySQL Rip & Replace Database Architecture MySQL MySQL Slaves MySQL Sharding ? MySQL Time 4
  • 5. De·nor·mal·ize[de-nawr-muh-lahyze]verb, -ized, -iz·ing.–verb (used with object)1.  the process of attempting to optimize the read performance of a database by adding redundant data or by grouping data wikipedia2.  Denormalize means to allow redundancy in a table so that the table can remain flat UCSD Blink3.  The process of restructuring a normalized data model to accommodate operational constraints or system limitations celiang.tongji.edu.cnAkiban Technologies, Inc. Confidential & Proprietary 5
  • 6. Materialized ViewsPersistent database object Contains the results of a query Store summary and pre-joined tables Require maintenance/refresh for dynamic dataSELECTDISTINCT(n.nid),n.sticky,n.title,n.createdFROM node nINNER JOIN term_node tn0 ON n.vid = tn0.vidWHERE n.status = 1 AND tn0.tid IN (77)ORDER BY n.sticky DESC, n.created DESCLIMIT 0, 25;Result: using where, using filesortAkiban Technologies, Inc. Confidential & Proprietary 6
  • 7. Drupal Materialized View ProjectCREATE TABLE `mv_drupalorg_node_by_term` ( `entity_type` varchar(64) NOT NULL, `entity_id` int(10) unsigned NOT NULL DEFAULT 0’, `term_tid` int(10) unsigned NOT NULL DEFAULT 0, `node_sticky` int(11) NOT NULL DEFAULT 0, `last_node_activity` int(11) NOT NULL DEFAULT 0, `node_created` int(11) NOT NULL DEFAULT 0, `node_title` varchar(255) NOT NULL DEFAULT ’, PRIMARY KEY (`entity_type`,`entity_id`,`term_tid`), KEY `activity` (`term_tid`,`node_sticky`,`last_node_activity`,`node_created`), KEY `creation` (`term_tid`,`node_sticky`,`node_created`) ) ENGINE=InnoDB DEFAULT CHARSET=utf8SELECT DISTINCT entity_id AS nid, node_sticky AS sticky, node_title AS title, node_created AS created FROM mv_drupalorg_node_by_term WHERE term_tid IN (77) ORDER BY node_sticky DESC, node_created DESC LIMIT 0, 25; Result: using where, using temporary tableAkiban Technologies, Inc. Confidential & Proprietary 7
  • 8. Denormalization Technique ListingTechnique Pros ConsMaterialized views Faster queries (no joins) Data explosion Manually keep synchedStore object as Blob Fast object get No modeling, or queryingDenormalize 1NF: Folding Data in one row limited # of child rowsparent-child into parent table Hard to query (UNION hell)Denormalize 2NF to 1NF: repeat Avoid join Data explosioncolumns from 1 table in M table Manually keep synched(Double writing)Adding derived columns Avoid joins, aggregation Manually keep synchedProperty bag (RDF) Schema flexibility Manage schema in app Akiban Technologies, Inc. Confidential & Proprietary Hard to index or perform 8
  • 9. RenormalizationJoin for free - Improved performance. 10-100x! - Retrieve an object in one requestAkiban Technologies, Inc. Confidential & Proprietary 9
  • 10. Introduction to Table-GroupsTraditional SQL Schema à Table à ColumnAkiban newSQL Schema à GROUP à Table à ColumnTable-Groups are first class citizensAkiban Technologies, Inc. Confidential & Proprietary 10
  • 11. Typical Relational DB SchemaAkiban Technologies, Inc. Confidential & Proprietary 11
  • 12. Typical Schema: GroupedBlockGroup User Group Node Group 12
  • 13. Table-Groups Eliminate Joins Logical PhysicalUsers Users_Roles Sessions Artist Table-group uid name pass id rid id sid timestamp 1 rriegel *** 1 1 1 19390 2011-10-01-06:02.00 2 twegner *** 1 2 2 22828 2011-10-04-22:32.10 2 1 1 49377 2011-10-04-16:07.30 Table Group Table Table bTree bTree bTree Akiban Technologies, Inc. Confidential & Proprietary 13
  • 14. Benefits of Table-groupingSQL join operations are fast -  Table Group access is equivalent to a single table access. Joins are free! -  Performance increases 10-100xApplications do not change -  Maintain the same tables and SQL -  Objects (e.g. ORM) fetched in one request -  Akiban uses standard MySQL replicationAkiban Technologies, Inc. Confidential & Proprietary 14
  • 15. Design Partner Sample QuerySELECT t1.id , t3.c1, t3.c2, t3.c3, t3.c4FROM t1INNER JOIN t2 on t2.id = t1.idLEFT JOIN t3 ON t1.id = t3.idWHERE t2.region in (1297789) AND t1.c1 = 0ORDER BY t1.latestLogin DESCLIMIT 500 Akiban Technologies, Inc. Confidential & Proprietary 15
  • 16. Typical MySQL EXPLAIN Plan 10 Project ResultsSort 9Temp Table 82 Joins 7 4 6 2 Table Accesses 2 3 5 1 3 Index Accesses Akiban Technologies, Inc. Confidential & Proprietary
  • 17. Efficiency for Speed and Scale No Joins,Project Results 3 Temp Tables or Sorts!1 Group Access 21 Group Index Access Typical MySQL EXPLAIN Project Results Sort Temp Table 1 2 Joins 2 Table Accesses 3 Index Accesses Akiban Technologies, Inc. Confidential & Proprietary
  • 18. Design Partner Acceleration: 27x Concurrent ConnectionsAkiban Technologies, Inc. Confidential & Proprietary 18
  • 19. Object Creation Query StreamSELECT * FROM t1 Where u.uid=1387SELECT * FROM t2 Where as.uid=1387SELECT * FROM t3 Where os.uid=1387SELECT * FROM t4 Where pm.uid=1387SELECT * FROM t5 Where pl.uid=1387SELECT * FROM t6 Where pa.uid=1387...... Akiban Technologies, Inc. Confidential & Proprietary 19
  • 20. Becomes Single ORM RequestSELECT * , (SELECT * FROM t2 where as.uid=u.uid), (SELECT * FROM t3 where as.uid=u.uid), ...FROM t1 Where u.uid=1387;Or simply:get my_schema:t1:uid=1387Akiban Technologies, Inc. Confidential & Proprietary 20
  • 21. Object Access in One RequestAkiban Technologies, Inc. Confidential & Proprietary 21
  • 22. Application IntegrationData replicated to Akiban Fully independent server HA Redirect Enabled MySQL Master Akiban Server MySQL adapter Replication MyISAM / InnoDB Storage Write Operations Problem Queries Akiban Technologies, Inc. Confidential & Proprietary 22
  • 23. Akiban is looking for Design Partners!Do you have•  Slow multi-join read queries?•  User concurrency or data volume challenges?http://www.akiban.com/design-partner-program Akiban Technologies, Inc. Confidential & Proprietary 23
  • 24. Ah, so you’re…Denormalizing…no. -  Schema doesn’t change -  Data is stored once, more efficientlyMaterializing Views…no. -  No triggers or post-processing -  No 2ndary logical objectsIntroducing Write Latency…no. -  Previous design partner showed 2x write improvementAkiban Technologies, Inc. Confidential & Proprietary 24
  • 25. Table-Grouping: A Closer LookArtist Each table maintains its own bTree id name gender Indexes add their own bTrees 1 Lennon M •  Covering index 2 Joplin F •  Index on frequently joined columns Covering •  Index on common sort order Index Join Cols Index Sort Order Index How many indexes do you maintain? •  Slow updates == reduced concurrency Table •  More resources == more overhead bTree •  Ongoing maintenance == high TCO Akiban Technologies, Inc. Confidential & Proprietary 25