2012 summarytables
Upcoming SlideShare
Loading in...5
×
 

2012 summarytables

on

  • 267 views

2012 SELF Conference

2012 SELF Conference

Statistics

Views

Total Views
267
Views on SlideShare
217
Embed Views
50

Actions

Likes
0
Downloads
2
Comments
0

4 Embeds 50

http://sqlhjalp.blogspot.com 43
http://sqlhjalp.blogspot.in 5
http://sqlhjalp.blogspot.it 1
http://sqlhjalp.blogspot.dk 1

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

2012 summarytables 2012 summarytables Presentation Transcript

  • 2 Copyright © 2012, Oracle and/or its affiliates. All rightsreserved.Insert Information Protection Policy Classification from Slide 8ORACLEPRODUCTLOGOMySQL: Summary TablesKeith Larsonkeith.larson@oracle.comMySQL Community Managersqlhjalp.blogspot.comsqlhjalp.com/pdf/2012_summarytables.pdf
  • 3 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.Safe Harbor StatementThe following is intended to outline our general productdirection. It is intended for information purposes only,and may not be incorporated into any contract. It is not acommitment to deliver any material, code, orfunctionality, and should not be relied upon in makingpurchasing decisions.The development, release, and timing of any features orfunctionality described for Oracle’s products remains atthe sole discretion of Oracle.
  • 4 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.localhost ~]$ whoamiKeith Larsonkeith.larson@oracle.comMySQL Community Managersqlhjalp.blogspot.comStarted with MySQL 3.23 during the dot.com days.Primary real world work was with a MySQL InnoDB delayed replicated environmentthat easily held over 4 billion rows of user data. Did a lot of data mining of that dataover the years.Numerous other sites developed on LAMP stack over the last 13 years.Also a movie buff :)Who are you?DBAs?Developers?sqlhjalp.blogspot.com/2012/02/summary-tables-with-mysql.html
  • 5 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.Program AgendaMySQL Summary Tables are not a new concept. They areexactly what they sound like, basically a summary of existingdata. We will review how to take advantage of summary tableinformation and ETL the processes to provide quick results foryour application.• A Situation• Some Solutions• Resources to learn more
  • 6 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.“I know why youre here, ... I know what youve beendoing... why you hardly sleep, why you live alone, and whynight after night, you sit by your computer. Youre lookingfor him. I know because I was once looking for the samething. And when he found me, he told me I wasnt reallylooking for him. I was looking for an answer. Its thequestion that drives us.... “-- Trinity , The Matrix“Perhaps we are asking the wrong questions.”-- Agent Brown , The Matrixhttp://keikonium.deviantart.com/art/The-Matrix-Background-32136837The Matrix copyright © 1999 - 2012, Warner Bros. Warner Bros. is the owner of all copyrights and trademark rights in The Matrix.
  • 7 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.Temet Nosce - thine own-self you must know.How well do you know your data....What if your data was as big as the matrix?http://thematrix101.com/media/still/photo-mat_temetnosce.jpgThe Matrix copyright © 1999 - 2012, Warner Bros. Warner Bros. is the owner of all copyrights and trademark rights in The Matrix.●How to sort data efficiently●How to data mine all of this data●Reporting of data trends takes a long time●How would you gather data for renderingA Situation
  • 8 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.http://www.grinningplanet.com/3001/mp3news_other/matrix-pods_222x127.jpgThe Matrix copyright © 1999 - 2012, Warner Bros. Warner Bros. is the owner of all copyrights and trademark rights in The Matrix.Table of all the structuresCREATE TABLE `structure` (`structure_id` int(5) unsigned NOT NULL auto_increment,`latitude` Float (10,6),`longitude` Float (10,6),`date_online` timestamp NOT NULL DEFAULT 0000-00-00 00:00:00,`levels` int(5) unsigned DEFAULT 0,`max_pods` int(5) unsigned DEFAULT 0,`ip` INT UNSIGNED NOT NULL,`communication_protocol`enum(OFF,IP.PIPE,IP.UDP,IP6.PIPE,IP6.UDP,IP.SPIPE,IP6.SPIPE,SNA.PIPE) default NULL,PRIMARY KEY (`structure_id`),KEY `date_online` (`date_online`)) ENGINE=InnoDB ;The Situation
  • 9 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.The Situationhttp://www.tony5m17h.net/MatrixNet.gifThe Matrix copyright © 1999 - 2012, Warner Bros. Warner Bros. is the owner of all copyrights and trademark rights in The Matrix.Table of all the podsCREATE TABLE `pod` (`pod_id` int(5) unsigned NOT NULL auto_increment,`structure_id` int(5) unsigned DEFAULT 0,`online_date` timestamp NOT NULL DEFAULT 0000-00-00 00:00:00,`offline_date` timestamp NOT NULL DEFAULT 0000-00-00 00:00:00,`level` int(5) unsigned DEFAULT 0,`position` int(5) unsigned DEFAULT 0,`ip` INT UNSIGNED NOT NULL,`communication_protocol`enum(OFF,IP.PIPE,IP.UDP,IP6.PIPE,IP6.UDP,IP.SPIPE,IP6.SPIPE,SNA.PIPE) default NULL,UNIQUE KEY `pods` (`pod_id`,`structure_id`,`online_date`),KEY `online_date` (`online_date`) ,KEY `offline_date` (`offline_date`)) ENGINE=InnoDB ;
  • 10 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.CREATE TABLE `human` (`human_id` int(5) unsigned NOT NULL auto_increment,`human_key` varchar(32) default NULL,`pod_id` int(5) unsigned DEFAULT 0,`online_date` timestamp NOT NULL DEFAULT 0000-00-00 00:00:00,`ip` INT UNSIGNED NOT NULL,`offline_date` timestamp NOT NULL DEFAULT 0000-00-00 00:00:00,PRIMARY KEY (`human_id`),UNIQUE KEY `human` (`human_id`,`pod_id`,`online_date`),KEY `online_date` (`online_date`) ,KEY `offline_date` (`offline_date`)) ENGINE=InnoDB ;www.amazon.com/ANDERSON-REEVES-Action-Figure-Accessories/dp/B003YR64HYThe Matrix copyright © 1999 - 2012, Warner Bros. Warner Bros. is the owner of all copyrights and trademark rights in The Matrix.The SituationTable of all the humans
  • 11 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.CREATE TABLE `data` (`data_id` int(5) unsigned NOT NULL auto_increment,`question` varchar(100) default NULL,`create_date` datetime default NULL,`date_updated` datetime NOT NULL default 0000-00-00 00:00:00,PRIMARY KEY (`data_id`),KEY `question` (`question`(5))) ENGINE=InnoDB ;mysql> select data_id , question from data limit 7;+---------+-------------+| data_id | question |+---------+-------------+| 1 | First Name || 2 | Last Name || 3 | Gender || 4 | Address || 5 | City || 6 | State || 7 | Zip |+---------+------------+The Situationwww.amazon.com/ANDERSON-REEVES-Action-Figure-Accessories/dp/B003YR64HYThe Matrix copyright © 1999 - 2012, Warner Bros. Warner Bros. is the owner of all copyrights and trademark rights in The Matrix.Table of all the humans
  • 12 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.CREATE TABLE `human_data` (`human_data_id` bigint(12) unsigned NOT NULL auto_increment,`human_id` int(9) unsigned default NULL,`data_id` int(5) unsigned default NULL,`value` varchar(100) default NULL,PRIMARY KEY (`human_data_id`),UNIQUE KEY `hd_id` (`human_id`,`data_id`),KEY `dv` (`data_id`,`value`(5))) ENGINE=InnoDB;Table of all the humansThe Situationwww.amazon.com/ANDERSON-REEVES-Action-Figure-Accessories/dp/B003YR64HYThe Matrix copyright © 1999 - 2012, Warner Bros. Warner Bros. is the owner of all copyrights and trademark rights in The Matrix.+---------------+----------+---------+------------------------------+| human_data_id | human_id | data_id | value |+---------------+----------+---------+------------------------------+| 1 | 1 | 1 | Thomas || 2 | 1 | 2 | Anderson || 3 | 1 | 3 | M || 13 | 1 | 4 | 1060 West Addison Street || 4 | 1 | 5 | Chicago || 5 | 1 | 6 | Illinois || 6 | 1 | 7 | 60613 || 7 | 1 | 16 | Lower Downtown, Capitol City || 8 | 1 | 17 | 03 || 9 | 1 | 18 | 11 || 10 | 1 | 19 | 1962 || 11 | 1 | 20 | Central West Junior High || 12 | 1 | 21 | Owen Paterson High School |+---------------+----------+---------+------------------------------+
  • 13 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.www.amazon.com/ANDERSON-REEVES-Action-Figure-Accessories/dp/B003YR64HYThe Matrix copyright © 1999 - 2012, Warner Bros. Warner Bros. is the owner of all copyrights and trademark rights in The Matrix.CREATE TABLE `job` (`job_id` int(5) unsigned NOT NULL DEFAULT 0,`title` varchar(15) DEFAULT NULL,`salary` decimal(5,3) default 0.000,PRIMARY KEY (`job_id`)) ENGINE=InnoDB ;CREATE TABLE `color` (`color_id` int(5) unsigned NOT NULL DEFAULT 0,`color` enum(Brown,Blue,Hazel,Green,Red,Yellow,Blonde,Gold) default NULL,PRIMARY KEY (`color_id`)) ENGINE=InnoDB ;The SituationTable of all the humans
  • 14 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.CREATE TABLE `eyes` (`eyes_id` int(5) unsigned NOT NULL auto_increment,`color_id` int(5) unsigned DEFAULT 0,`species_id` int(5) unsigned DEFAULT 0,`style_id` int(5) unsigned DEFAULT 0,`shape_id` int(5) unsigned DEFAULT 0,PRIMARY KEY (`eyes_id`)) ENGINE=InnoDB ;CREATE TABLE `hair` (`hair_id` int(5) unsigned NOT NULL auto_increment,`style_id` int(5) unsigned DEFAULT 0,`color_id` int(5) unsigned DEFAULT 0,`shape_id` int(5) unsigned DEFAULT 0,PRIMARY KEY (`hair_id`)) ENGINE=InnoDB ;The Situationwww.amazon.com/ANDERSON-REEVES-Action-Figure-Accessories/dp/B003YR64HYThe Matrix copyright © 1999 - 2012, Warner Bros. Warner Bros. is the owner of all copyrights and trademark rights in The Matrix.Table of all the humans
  • 15 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.SELECT h.human_key , hd1.value as First Name , hd2.value as Last Name , hd3.value asGender , hd4.value as Address , hd5.value as City , hd6.value as State , hd7.value asZipCode , hd16.value as Place of Birth , CONCAT(hd17.value, -,hd18.value,-, hd19.value )AS DOB, hd20.value as Junior High , hd21.value as High School, INET_NTOA(h.ip) as ipFROM human hINNER JOIN human_data hd1 ON h.human_id = hd1.human_id AND hd1.data_id = 1INNER JOIN human_data hd2 ON h.human_id = hd2.human_id AND hd2.data_id = 2INNER JOIN human_data hd3 ON h.human_id = hd3.human_id AND hd3.data_id = 3INNER JOIN human_data hd4 ON h.human_id = hd4.human_id AND hd4.data_id = 4INNER JOIN human_data hd5 ON h.human_id = hd5.human_id AND hd5.data_id = 5INNER JOIN human_data hd6 ON h.human_id = hd6.human_id AND hd6.data_id = 6INNER JOIN human_data hd7 ON h.human_id = hd7.human_id AND hd7.data_id = 7LEFT JOIN human_data hd16 ON h.human_id = hd16.human_id AND hd16.data_id = 16LEFT JOIN human_data hd17 ON h.human_id = hd17.human_id AND hd17.data_id = 17LEFT JOIN human_data hd18 ON h.human_id = hd18.human_id AND hd18.data_id = 18LEFT JOIN human_data hd19 ON h.human_id = hd19.human_id AND hd19.data_id = 19LEFT JOIN human_data hd20 ON h.human_id = hd20.human_id AND hd20.data_id = 20LEFT JOIN human_data hd21 ON h.human_id = hd21.human_id AND hd21.data_id = 21WHERE h.human_id = 1GLots of Data Across lots of TablesThe Situation
  • 16 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.*************************** 1. row ***************************human_key: 63b28bb2be0bf40085646decce81ae9fFirst Name: ThomasLast Name: AndersonGender: MAddress: 1060 West Addison StreetCity: ChicagoState: IllinoisZipCode: 60613Place of Birth: Lower Downtown, Capitol CityDOB: 03-11-1962Junior High: Central West Junior HighHigh School: Owen Paterson High Schoolip: 192.168.5.2211 row in set (0.00 sec)The SituationLots of Data Across lots of Tableswww.amazon.com/ANDERSON-REEVES-Action-Figure-Accessories/dp/B003YR64HYThe Matrix copyright © 1999 - 2012, Warner Bros. Warner Bros. is the owner of all copyrights and trademark rights in The Matrix.
  • 17 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.SELECT h.human_key , hd1.value as First Name , hd2.value as Last Name ,hd3.value as GenderFROM human hINNER JOIN human_data hd1 ON h.human_id = hd1.human_id AND hd1.data_id = 1INNER JOIN human_data hd2 ON h.human_id = hd2.human_id AND hd2.data_id = 2INNER JOIN human_data hd3 ON h.human_id = hd3.human_id AND hd3.data_id = 3WHERE hd2.value like Mc%;Lots of Data Across lots of TablesThe Situation
  • 18 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.+----------------------------------+----------------+-------------+--------+| human_key | First Name | Last Name | Gender |+----------------------------------+----------------+-------------+--------+| 5631e6ee59a4175cd06c305840562ff3 | Shunichi | McAffer | M || d28c252abf26d8ffa69ad522c66b698e | Xudong | McAffer | F || 80d4c0e3d8705d1ae4cfd19dd89fa34a | Taizo | McAffer | M || 25a714b921e3a2ba54c452debb9b5584 | Yuuichi | McAffer | M |...| 601fa6a61d57b1f8a800d15eedfbd370 | Martien | McAlpine | F |...| 657e31ff3231b847d7604f6647a2dfc9 | Shir | McClurg | M |..| 72db86e4c73b9fabb4810562b236488e | Youpyo | McClure | M || 83b7afcfe4ad452c54ea6427b18960f8 | Khaled | McClurg | F |..| 2abbe47df459c35281fb6e86f19ea85e | Maria | McDermid | F |...| 999df4ce78b966de17aee1dc87111044 | Berhard | McFarlin | M |1482 rows in set (0.15 sec)Lots of Data Across lots of TablesThe Situation
  • 19 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.Define snowflakeA snowflake schema is a logical arrangement of tablesin a multidimensional database such that the entityrelationship diagram resembles a snowflake in shape. ...-- en.wikipedia.org/wiki/Snowflake_schemahttp://thumbs.dreamstime.com/thumblarge_316/12227643929M1f6o.jpghttp://www.informix.com.ua/articles/rolap/rolap.htm
  • 20 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.SolutionsData Warehouse**Noun:A large store of data accumulated from a wide range ofsources within a company and used to guidemanagement decisions.-- Google.com** Same place, same time on Saturday for more on Data Warehouses.
  • 21 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.SolutionsBill Inmons paradigm: Data warehouse is one part of theoverall business intelligence system. An enterprise hasone data warehouse, and data marts source theirinformation from the data warehouse. In the datawarehouse, information is stored in 3rd normal form.Ralph Kimballs paradigm: Data warehouse is theconglomerate of all data marts within the enterprise.Information is always stored in the dimensional model.http://www.1keydata.com/datawarehousing/inmon-kimball.html
  • 22 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.Define Summary TableA collection of one or more data elements that areclassified into some logical structure, either asdimensions or data points-- websters dictionarySolutions
  • 23 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.CREATE TABLE `the_matrix` (`the_matrix_id` bigint(12) unsigned NOT NULL AUTO_INCREMENT,`human_id` int(9) unsigned DEFAULT NULL,`firstname` varchar(100) DEFAULT NULL,`lastname` varchar(100) DEFAULT NULL,`alias` varchar(50) DEFAULT NULL,`gender` varchar(2) DEFAULT NULL,`address` varchar(100) DEFAULT NULL,`city` varchar(100) DEFAULT NULL,`state` varchar(100) DEFAULT NULL,`zipcode` varchar(100) DEFAULT NULL,`dob_month` varchar(5) DEFAULT NULL,`dob_day` varchar(5) DEFAULT NULL,`dob_year` varchar(5) DEFAULT NULL,`hair_color` varchar(50) DEFAULT NULL,`hair_style` varchar(50) DEFAULT NULL,`eye_color` varchar(50) DEFAULT NULL,`date_updated` datetime NOT NULL DEFAULT 0000-00-00 00:00:00,PRIMARY KEY (`the_matrix_id`),UNIQUE KEY `human_id` (`human_id`),KEY `date_updated` (`date_updated`)) ENGINE=InnoDBSolutions: Summary Table
  • 24 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.DELIMITER //CREATE PROCEDURE build_the_matrix()BEGINSELECT MAX(date_updated) INTO @matrix_date FROM the_matrix ;REPLACE INTO the_matrix SELECT NULL, h.human_id, hd1.value as First Name , hd2.value as Last Name,hd13.value as alias , hd3.value as Gender , hd4.value as Address , hd5.value as City , hd6.value as State ,hd7.value as ZipCode , hd17.value as dob_month, hd18.value as dob_day, hd19.value as dob_year, hd23.value ashair_color, hd24.value as hair_style, hd25.value as eye_color, hdd.date_updatedFROM human hINNER JOIN human_data hdd ON h.human_id = hdd.human_idINNER JOIN human_data hd1 ON h.human_id = hd1.human_id AND hd1.data_id = 1INNER JOIN human_data hd2 ON h.human_id = hd2.human_id AND hd2.data_id = 2LEFT JOIN human_data hd3 ON h.human_id = hd3.human_id AND hd3.data_id = 3LEFT JOIN human_data hd4 ON h.human_id = hd4.human_id AND hd4.data_id = 4LEFT JOIN human_data hd5 ON h.human_id = hd5.human_id AND hd5.data_id = 5LEFT JOIN human_data hd6 ON h.human_id = hd6.human_id AND hd6.data_id = 6LEFT JOIN human_data hd7 ON h.human_id = hd7.human_id AND hd7.data_id = 7LEFT JOIN human_data hd13 ON h.human_id = hd13.human_id AND hd13.data_id = 13LEFT JOIN human_data hd16 ON h.human_id = hd16.human_id AND hd16.data_id = 16LEFT JOIN human_data hd17 ON h.human_id = hd17.human_id AND hd17.data_id = 17LEFT JOIN human_data hd18 ON h.human_id = hd18.human_id AND hd18.data_id = 18LEFT JOIN human_data hd19 ON h.human_id = hd19.human_id AND hd19.data_id = 19LEFT JOIN human_data hd20 ON h.human_id = hd20.human_id AND hd20.data_id = 20LEFT JOIN human_data hd21 ON h.human_id = hd21.human_id AND hd21.data_id = 21LEFT JOIN human_data hd23 ON h.human_id = hd23.human_id AND hd23.data_id = 23LEFT JOIN human_data hd24 ON h.human_id = hd24.human_id AND hd24.data_id = 24LEFT JOIN human_data hd25 ON h.human_id = hd25.human_id AND hd25.data_id = 25WHERE hdd.date_updated >= @matrix_date ;END //DELIMITER ;Solutions: ExtractLoad EL
  • 25 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.CREATE EVENT build_the_matrix_10minrunsON SCHEDULEEVERY 10 MINUTECOMMENT it is the question that drives us DOCALL build_the_matrix();show variables like %event_s%;+-----------------+-------+| Variable_name | Value |+-----------------+-------+| event_scheduler | ON |+-----------------+-------+mysql> set GLOBAL event_scheduler = ON || OFF ;mysql> show processlistG.....User: event_schedulerHost: localhostdb: NULLCommand: DaemonTime: 26State: Waiting for next activationInfo: NULLSolutions: ExtractLoad EL
  • 26 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.Solutions: Data Miningmysql> SELECT SQL_no_cache COUNT(h.human_id) , g.value , s.value , c.valueFROM human hINNER JOIN human_data g ON g.human_id = h.human_id AND g.data_id = 3LEFT JOIN human_data s ON s.human_id = h.human_id AND s.data_id = 6LEFT JOIN human_data c ON c.human_id = h.human_id AND c.data_id = 25GROUP BY g.value , s.value , c.value;+-------------------+-------+----------+-------+| COUNT(h.human_id) | value | value | value |+-------------------+-------+----------+-------+| 119932 | F | NULL | NULL |...| 14 | M | New York | Blue || 13 | M | New York | Brown || 7 | M | New York | Green || 5 | M | New York | Hazel || 7 | M | Oregon | Blue || 8 | M | Oregon | Brown || 9 | M | Oregon | Green || 4 | M | Oregon | Hazel |+-------------------+-------+------------+-------+51 rows in set (5.26 sec)
  • 27 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.Solutions: Data MiningData Mine the matrixmysql> SELECT SQL_no_cache COUNT(human_id), gender , state , hair_colorFROM the_matrixGROUP BY gender , state , hair_colorORDER BY gender , state , hair_color ;| COUNT(human_id) | gender | state | hair_color |+-----------------+--------+------------+------------+| 0 | NULL | NULL | NULL || 109196 | F | NULL | NULL || 163435 | M | NULL | NULL |.....| 1 | M | Illinois | BLACK || 25 | M | New York | BLACK || 14 | M | New York | BLOND || 12 | M | Oregon | BLACK || 16 | M | Oregon | BLOND |+-----------------+--------+------------+------------+28 rows in set (0.66 sec)
  • 28 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.Solutions: Data Miningmysql> SELECT SQL_no_cache COUNT(h.human_id) , g.value , s.value , c.valueFROM human hINNER JOIN human_data g ON g.human_id = h.human_idLEFT JOIN human_data s ON s.human_id = h.human_idLEFT JOIN human_data c ON c.human_id = h.human_idWHERE g.data_id = 3 AND s.data_id = 6 AND c.data_id = 25GROUP BY g.value , s.value , c.value;+-------------------+-------+----------+-------+| COUNT(h.human_id) | value | value | value |+-------------------+-------+----------+-------+...| 14 | M | New York | Blue || 13 | M | New York | Brown || 7 | M | New York | Green || 5 | M | New York | Hazel || 7 | M | Oregon | Blue || 8 | M | Oregon | Brown || 9 | M | Oregon | Green || 4 | M | Oregon | Hazel |+-------------------+-------+------------+-------+41 rows in set (0.14 sec)
  • 29 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.CREATE VIEW matrix.matrix_view ASSELECT SQL_NO_CACHE COUNT(h.human_id) , g.value as gender, s.value as state , c.value as hair_colorFROM human hLEFT JOIN human_data g ON g.human_id = h.human_idLEFT JOIN human_data s ON s.human_id = h.human_idLEFT JOIN human_data c ON c.human_id = h.human_idWHERE g.data_id = 3 AND s.data_id = 6 AND c.data_id = 25GROUP BY g.value , s.value , c.value;+-------------------+--------+------------+------------+| COUNT(h.human_id) | gender | state | hair_color |+-------------------+--------+------------+------------+.....| 14 | M | New York | Blue || 13 | M | New York | Brown || 7 | M | New York | Green || 5 | M | New York | Hazel || 7 | M | Oregon | Blue || 8 | M | Oregon | Brown || 9 | M | Oregon | Green || 4 | M | Oregon | Hazel |+-------------------+--------+------------+------------+41 rows in set (0.04 sec)Solutions: A View
  • 30 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.DELIMITER //CREATE PROCEDURE build_the_matrix()BEGINSELECT MAX(date_updated) INTO @matrix_date FROM the_matrix ;REPLACE INTO the_matrix SELECT NULL, h.human_id, hd1.value as First Name , hd2.value as Last Name, hd13.value as alias ,hd3.value as Gender , hd4.value as Address , hd5.value as City , hd6.value as State , hd7.value as ZipCode , hd17.value as dob_month,hd18.value as dob_day, hd19.value as dob_year, hd23.value as hair_color, hd24.value as hair_style, hd25.value as eye_color,hdd.date_updatedFROM human hINNER JOIN human_data hdd ON h.human_id = hdd.human_idINNER JOIN human_data hd1 ON h.human_id = hd1.human_id AND hd1.data_id = 1INNER JOIN human_data hd2 ON h.human_id = hd2.human_id AND hd2.data_id = 2LEFT JOIN human_data hd3 ON h.human_id = hd3.human_id AND hd3.data_id = 3LEFT JOIN human_data hd4 ON h.human_id = hd4.human_id AND hd4.data_id = 4LEFT JOIN human_data hd5 ON h.human_id = hd5.human_id AND hd5.data_id = 5LEFT JOIN human_data hd6 ON h.human_id = hd6.human_id AND hd6.data_id = 6LEFT JOIN human_data hd7 ON h.human_id = hd7.human_id AND hd7.data_id = 7LEFT JOIN human_data hd13 ON h.human_id = hd13.human_id AND hd13.data_id = 13LEFT JOIN human_data hd16 ON h.human_id = hd16.human_id AND hd16.data_id = 16LEFT JOIN human_data hd17 ON h.human_id = hd17.human_id AND hd17.data_id = 17LEFT JOIN human_data hd18 ON h.human_id = hd18.human_id AND hd18.data_id = 18LEFT JOIN human_data hd19 ON h.human_id = hd19.human_id AND hd19.data_id = 19LEFT JOIN human_data hd20 ON h.human_id = hd20.human_id AND hd20.data_id = 20LEFT JOIN human_data hd21 ON h.human_id = hd21.human_id AND hd21.data_id = 21LEFT JOIN human_data hd23 ON h.human_id = hd23.human_id AND hd23.data_id = 23LEFT JOIN human_data hd24 ON h.human_id = hd24.human_id AND hd24.data_id = 24LEFT JOIN human_data hd25 ON h.human_id = hd25.human_id AND hd25.data_id = 25WHERE hdd.date_updated >= @matrix_date ;END //DELIMITER ;Solutions: Populating Tables
  • 31 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.CREATE EVENT build_the_matrixON SCHEDULEEVERY 2 MINUTECOMMENT It is the question that drives usDOCALL build_the_matrix();mysql> show events;Db: matrixName: build_the_matrixDefiner: root@localhostTime zone: SYSTEMType: RECURRINGExecute at: NULLInterval value: 2Interval field: MINUTEStarts: 2012-05-23 23:09:24Ends: NULLStatus: ENABLEDOriginator: 1character_set_client: utf8collation_connection: utf8_general_ciDatabase Collation: latin1_swedish_cimysql> select * from information_schema.eventsGSolutions: Events
  • 32 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.Triggersdelimiter |CREATE TRIGGER some_trigger BEFORE INSERT ON Table_AFOR EACH ROW BEGININSERT INTO TEST_B SET a2 = NEW.a1;DELETE FROM TEST_C WHERE a3 = NEW.a1;UPDATE TEST_D SET b4 = b4 + 1 WHERE a4 = NEW.a1;END; |delimiter ;mysql> show triggers;Solutions: Populating Tables
  • 33 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.Transform---– This is your chance to clean data– Enable the special needs of others– Rectify Problems– Know your data....• Events• Procedures• Triggers• ScriptsSolutions: ExtractTransformLoad ETL
  • 34 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.mysql> SELECT SQL_no_cacheCOUNT(m.human_id) as Amount , UPPER(m.gender) as genderFROM human hINNER JOIN the_matrix m ON m.human_id = h.human_idINNER JOIN pod p ON p.pod_id = h.pod_idINNER JOIN structure s ON s.structure_id = p.structure_idWHERE s.date_online BETWEEN 19850101000000 AND 19890101000000GROUP BY m.gender;+--------+--------+| Amount | gender |+--------+--------+| 23783 | F || 35472 | M |+--------+--------+2 rows in set (2.08 sec)Solutions: Reporting
  • 35 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.First start with our lowest common denominator beyond your root table.CREATE TABLE `structure` (`structure_id` int(5) unsigned NOT NULL DEFAULT 0,`latitude` float(10,6) DEFAULT NULL,`longitude` float(10,6) DEFAULT NULL,`date_online` timestamp NOT NULL DEFAULT 0000-00-00 00:00:00,`levels` int(5) unsigned DEFAULT 0,`max_pods` int(5) unsigned DEFAULT 0,`ip` INT UNSIGNED NOT NULL,`communication_protocol` enum(OFF,EMAIL,FTP,BATCH POST,BATCH GET,REALTIME GET,TRACKTAGS,FTP-SSL,REAL TIME POST,CUSTOM,POST) DEFAULTNULL,PRIMARY KEY (`structure_id`),KEY `date_online` (`date_online`)) ENGINE=InnoDB DEFAULTSolutions: Reporting
  • 36 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.CREATE TABLE `report_structure_24hour` (`report_structure_24hour_id` int(5) unsigned NOT NULL AUTO_INCREMENT,`totals` int(5) unsigned DEFAULT 0,`gender` varchar(15) DEFAULT NULL,`date_online` date NOT NULL DEFAULT 0000-00-00,PRIMARY KEY (`report_structure_24hour_id`),KEY `date_online` (`date_online`)) ENGINE=MEMORY;REPLACE INTO report_structure_24hourSELECT SQL_no_cacheNULL,COUNT(m.human_id) as Amount , UPPER(m.gender) as gender ,DATE_FORMAT(s.date_online, %Y-%m-%d %h ) as dateFROM human hINNER JOIN the_matrix m ON m.human_id = h.human_idINNER JOIN pod p ON p.pod_id = h.pod_idINNER JOIN structure s ON s.structure_id = p.structure_idWHERE s.date_online >= NOW() - interval 24 hourGROUP BY m.gender, date;Solutions: Reporting
  • 37 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.CREATE TABLE `report_structure_perday` (`report_structure_perday_id` int(5) unsigned NOT NULL AUTO_INCREMENT,`totals` int(5) unsigned DEFAULT 0,`gender` varchar(15) DEFAULT NULL,`date_online` date NOT NULL DEFAULT 0000-00-00,PRIMARY KEY (`report_structure_perday_id`),KEY `date_online` (`date_online`)) ENGINE=InnoDB;REPLACE INTO report_structure_perdaySELECT SQL_no_cacheNULL,COUNT(m.human_id) as Amount , UPPER(m.gender) as gender ,DATE_FORMAT(s.date_online, %Y-%m-%d) as dateFROM human hINNER JOIN the_matrix m ON m.human_id = h.human_idINNER JOIN pod p ON p.pod_id = h.pod_idINNER JOIN structure s ON s.structure_id = p.structure_idWHERE s.date_online >= NOW() - interval 1 weekGROUP BY m.gender, date;Solutions: Reporting
  • 38 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.CREATE TABLE `report_structure_permonth` (`report_structure_permonth_id` int(5) unsigned NOT NULL AUTO_INCREMENT,`totals` int(5) unsigned DEFAULT 0,`gender` varchar(15) DEFAULT NULL,`date_online` date NOT NULL DEFAULT 0000-00-00,PRIMARY KEY (`report_structure_permonth_id`),KEY `date_online` (`date_online`)) ENGINE=InnoDB;REPLACE INTO report_structure_permonthSELECT SQL_no_cacheNULL,SUM(totals)as Amount, gender , DATE_FORMAT(date_online, %Y-%m-00) as dateFROM report_structure_perdayWHERE date_online >= NOW() - interval 2 monthGROUP BY gender, date;Solutions: Reporting
  • 39 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.CREATE TABLE `report_structure_per_year` (`report_structure_year_id` int(5) unsigned NOT NULL AUTO_INCREMENT,`totals` int(5) unsigned DEFAULT 0,`gender` varchar(15) DEFAULT NULL,`date_online` int(5) unsigned DEFAULT 0000,PRIMARY KEY (`report_structure_year_id`),KEY `date_online` (`date_online`)) ENGINE=InnoDB;REPLACE INTO report_structure_per_yearSELECT SQL_no_cacheNULL,SUM(totals)as Amount, gender , DATE_FORMAT(date_online, %Y) as dateFROM report_structure_permonthWHERE date_online >= NOW() - interval 2 YEARGROUP BY gender, date;Solutions: Reporting
  • 40 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.Populate tables with :DELIMITER //CREATE PROCEDURE reporting_perday()BEGINREPLACE INTO report_structure_perdaySELECT SQL_no_cacheNULL,COUNT(m.human_id) as Amount , UPPER(m.gender) as gender , DATE_FORMAT(s.date_online, %Y-%m-%d)as dateFROM human hINNER JOIN the_matrix m ON m.human_id = h.human_idINNER JOIN pod p ON p.pod_id = h.pod_idINNER JOIN structure s ON s.structure_id = p.structure_idWHERE s.date_online >= NOW() - interval 1 weekGROUP BY m.gender, date;END //DELIMITER ;CREATE EVENT reporting_perdayON SCHEDULEEVERY 24 HOURCOMMENT It is the question that drives usDOCALL reporting_perday();Solutions: Reporting
  • 41 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.Populate tables with :DELIMITER //CREATE PROCEDURE reporting_permonth()BEGINREPLACE INTO report_structure_permonthSELECT SQL_no_cacheNULL,SUM(totals)as Amount, gender , DATE_FORMAT(date_online, %Y-%m-00) as dateFROM report_structure_perdayWHERE date_online >= NOW() - interval 2 monthGROUP BY gender, date;END //DELIMITER ;CREATE EVENT reporting_permonthON SCHEDULE EVERY 2 WEEKCOMMENT It is the question that drives usDOCALL reporting_permonth();Solutions: Reporting
  • 42 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.Populate tables with :DELIMITER //CREATE PROCEDURE reporting_peryear()BEGINREPLACE INTO report_structure_per_yearSELECT SQL_no_cacheNULL,SUM(totals)as Amount, gender , DATE_FORMAT(date_online, %Y) as dateFROM report_structure_permonthWHERE date_online >= NOW() - interval 2 YEARGROUP BY gender, date;END //DELIMITER ;CREATE EVENT reporting_peryearON SCHEDULEEVERY 2 MONTHCOMMENT It is the question that drives usDOCALL reporting_peryear();Solutions: Reporting
  • 43 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.It all comes down to who doyou want to be?The One or just someone ?The Solutions are up to you.http://images1.wikia.nocookie.net/__cb20070215050459/uncyclopedia/images/7/75/Neo.JPGThe Matrix copyright © 1999 - 2012, Warner Bros. Warner Bros. is the owner of all copyrights and trademark rights in The Matrix.Solutions - Solutionwww.amazon.com/ANDERSON-REEVES-Action-Figure-Accessories/dp/B003YR64HYThe Matrix copyright © 1999 - 2012, Warner Bros. Warner Bros. is the owner of all copyrights and trademark rights in The Matrix.
  • 44 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.Resourceshttp://code.google.com/p/flexviews/http://code.google.com/p/flexviews/http://code.google.com/p/flexviews/http://code.google.com/p/flexviews/http://code.google.com/p/flexviews/http://code.google.com/p/flexviews/
  • 45 Copyright © 2012, Oracle and/or its affiliates. All rights reserved.