• Save
B25 Reducing redo by Julian Dyke
Upcoming SlideShare
Loading in...5
×
 

B25 Reducing redo by Julian Dyke

on

  • 564 views

 

Statistics

Views

Total Views
564
Views on SlideShare
564
Embed Views
0

Actions

Likes
0
Downloads
0
Comments
0

0 Embeds 0

No embeds

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

B25 Reducing redo by Julian Dyke B25 Reducing redo by Julian Dyke Presentation Transcript

  • 1 Reducing Redo Julian Dyke Independent Consultant DB Tech Showcase - Osaka May 2013 juliandyke.com© 2013 Julian Dyke
  • juliandyke.com© 2013 Julian Dyke2 Agenda  Introduction  Tests  Indexes  Number of columns processed  SELECT FOR UPDATE  Number of rows processed  COMMIT  Batch size  Global temporary tables  External tables  Conclusion
  • juliandyke.com© 2013 Julian Dyke3 Redo Records Redo Block Header 16 bytes Redo Block 512 or 1024 bytes Redo Block Body 496 bytes Redo Record 1 Redo Record 2 Wastage Redo Record 3 Wastage Header Body Body Header Spare Header Body Spare STOP View slide
  • juliandyke.com© 2013 Julian Dyke4 Change Vectors Redo Record Header Body Header Body Header Body Header Body Change Vectors Change Vector 3 Change Vector 1 Change Vector 2 STOP View slide
  • juliandyke.com© 2013 Julian Dyke5 Header Change Vector Header Body STOP
  • juliandyke.com© 2013 Julian Dyke6 Example  Examples in this presentation taken from Formula 1 database  Contains full details of all races from 1961 to 2012  Updated annually in November (end of season)  Currently  22 cars per race  19 races per season  Approximately 420 new rows per season
  • juliandyke.com© 2013 Julian Dyke7 Schema CLASSIFICATION SEASON GRANDPRIX RACE TEAMDRIVER COUNTRYCIRCUIT ENGINE CAR
  • juliandyke.com© 2013 Julian Dyke8 Cars  Each season has up to 20 races (19 in 2005)  Each race has up to 39 entrants (13 races in 1989)  Each car has  driver, team and engine  laps completed (may be zero)  optional notes  Results are classified as follows C Classified DNF Did not finish DNS Did not start DNQ Did not qualify DIS Disqualified
  • juliandyke.com© 2013 Julian Dyke9 Points  Points awarded to driver and team as follows 1st 2nd 3rd 4th 5th 6th 7th 8th 9th 10th Pre 1991 9 6 4 3 2 1 1991-2002 10 6 4 3 2 1 2003-2009 10 8 6 5 4 3 2 1 2010 onwards 25 18 15 12 10 8 6 4 2 1  But . . . not always straightforward  Half points awarded for incomplete races  Split races (two half point races aggregated)  Driver and / or team disqualifications e.g. Tyrrell in 1984  Up to 1980 only best scores counted for each half of season e.g.  Best 5 results from first 7 races and best 5 results from last 7 races  1982-1990 only best 11 results counted for drivers  1961-1978 only first car to finish counted for each team
  • juliandyke.com© 2013 Julian Dyke10 Input file - car.csv  Comma separated file  16181 rows  Fields are:  season_key  race_key  position  driver_key  team_key  engine_key  laps_completed  classification_key  notes (optional) 2004 17 17 ZBAU MIN FOR 41 DNF Spin 2004 17 18 DCOU MCL MER 38 DNF Accident 2004 17 19 RBAR FER FER 38 DNF Accident 2004 17 20 MWEB JAG FOR 20 DNF Overheated 2004 18 1 JMON WIL BMW 71 C 2004 18 2 KRAI MCL MER 71 C 2004 18 3 RBAR FER FER 71 C 2004 18 4 FALO REN REN 71 C 2004 18 5 RSCH WIL BMW 71 C 2004 18 6 TSAT BAR HON 71 C 2004 18 7 MSCH FER FER 71 C 2004 18 8 FMAS SAU FER 71 C 2004 18 9 GFIS SAU FER 71 C 2004 18 10 JVIL REN REN 70 C 2004 18 11 DCOU MCL MER 70 C 2004 18 12 JTRU TOY TOY 70 C 2004 18 13 RZON TOY TOY 70 C 2004 18 14 CKLI JAG FOR 69 C 2004 18 15 TGLO JOR FOR 69 C 2004 18 16 ZBAU MIN FOR 67 C 2004 18 17 GBRU MIN FOR 67 C 2004 18 18 MWEB JAG FOR 23 DNF Accident 2004 18 19 NHEI JOR FOR 15 DNF Clutch 2004 18 20 JBUT BAR HON 3 DNF Engine
  • juliandyke.com© 2013 Julian Dyke11 Input file - points.csv  Comma separated file  16181 rows  Fields are:  season_key  race_key  position  driver_points  team_points 2004 17 17 0 0 2004 17 18 0 0 2004 17 19 0 0 2004 17 20 0 0 2004 18 1 10 10 2004 18 2 8 8 2004 18 3 6 6 2004 18 4 5 5 2004 18 5 4 4 2004 18 6 3 3 2004 18 7 2 2 2004 18 8 1 1 2004 18 9 0 0 2004 18 10 0 0 2004 18 11 0 0 2004 18 12 0 0 2004 18 13 0 0 2004 18 14 0 0 2004 18 15 0 0 2004 18 16 0 0 2004 18 17 0 0 2004 18 18 0 0 2004 18 19 0 0 2004 18 20 0 0
  • juliandyke.com© 2013 Julian Dyke12 CAR table  CAR table and index definitions CREATE TABLE car ( season_key NUMBER NOT NULL, race_key NUMBER NOT NULL, position NUMBER NOT NULL, driver_key VARCHAR2(4) NOT NULL, team_key VARCHAR2(3) NOT NULL, engine_key VARCHAR2(3) NOT NULL, laps_completed NUMBER NOT NULL, classification_key VARCHAR2(4) NOT NULL, notes VARCHAR2(100), driver_points NUMBER NOT NULL DEFAULT 0, team_points NUMBER NOT NULL DEFAULT 0 ); ALTER TABLE car ADD CONSTRAINT car_pk PRIMARY KEY (season_key,race_key,position); CREATE INDEX car_driver ON car (season_key,driver_key,driver_points);
  • juliandyke.com© 2013 Julian Dyke13 CAR  CAR table relational integrity definitions ALTER TABLE car ADD CONSTRAINT car_race FOREIGN KEY (season_key,race_key) REFERENCES race (season_key,race_key); ALTER TABLE car ADD CONSTRAINT car_driver FOREIGN KEY (driver_key) REFERENCES driver (driver_key); ALTER TABLE car ADD CONSTRAINT car_team FOREIGN KEY (team_key) REFERENCES team (team_key); ALTER TABLE car ADD CONSTRAINT car_engine FOREIGN KEY (engine_key) REFERENCES engine (engine_key); ALTER TABLE car ADD CONSTRAINT car_classification FOREIGN KEY (classification_key) REFERENCES classification (classification_key);
  • juliandyke.com© 2013 Julian Dyke14 For each line in car.csv { read :season_key, :race_key, :position, :driver_key, :team_key, :engine_key, :laps_completed, :classification_key, :notes; INSERT INTO car (season_key, race_key, position, driver_key, team_key, engine_key, laps_completed, classification_key, notes) VALUES (:season_key,:race_key,:position, :driver_key, :team_key, :engine_key, :laps_completed, :classification_key, :notes) COMMIT; } Baseline - Insert
  • juliandyke.com© 2013 Julian Dyke15 Baseline - Insert Redo Generation for each Insert Statement Header 5.2 Start Transaction 5.1 (11.1)Undo Undo insert row in CAR tableINSERT 11.2Redo Insert row in CAR tableINSERT Undo Undo insert row into CAR_PK index5.1 (10.22)INSERT Redo 10.2 Insert row into CAR_PK indexINSERT 5.1 (10.22)Undo Undo insert row into CAR_DRIVER indexINSERT Redo 10.2 Insert row into CAR_DRIVER indexINSERT Commit 5.4 End TransactionCOMMIT Oracle 9.2 and below
  • juliandyke.com© 2013 Julian Dyke16 Insert Statement Redo Generation for each Insert Statement Header 5.2 Start Transaction 5.1 (11.1)Undo Undo insert row in CAR tableINSERT 11.2Redo Insert row in CAR tableINSERT Undo Undo insert row into CAR_PK index5.1 (10.22)INSERT Redo 10.2 Insert row into CAR_PK indexINSERT 5.1 (10.22)Undo Undo insert row into CAR_DRIVER indexINSERT Redo 10.2 Insert row into CAR_DRIVER indexINSERT Commit 5.4 End TransactionCOMMIT Oracle 10.1 and above
  • juliandyke.com© 2013 Julian Dyke17 Baseline - Update For each line in points.csv { read :season_key, :race_key, :position, :driver_points, :team_points; SELECT driver_key, team_key, engine_key, laps_completed, classification_key, notes INTO :driver_key, :team_key, :engine_key, :laps_completed, :classification_key, :notes FROM car WHERE season_key = :season_key AND race_key = :race_key AND position = :position FOR UPDATE; UPDATE car SET driver_key = :driver_key, team_key = :team_key,engine_key = :engine_key, laps_completed = :laps_completed,classification_key = :classification_key, notes = :notes, driver_points = :driver_points, team_points = :team_points WHERE season_key = :season_key AND race_key = :race_key AND position = :position; COMMIT; }
  • juliandyke.com© 2013 Julian Dyke18 Baseline - Update Redo Generation for each Update Statement Header 5.2 Start Transaction 5.1 (11.1)Undo Undo update row in CAR tableUPDATE 11.5Redo Update row in CAR tableUPDATE Undo Undo delete row from CAR_DRIVER index5.1 (10.22)UPDATE Redo 10.4 Delete row from CAR_DRIVER indexUPDATE 5.1 (10.22)Undo Undo insert row into CAR_DRIVER indexUPDATE Redo 10.2 Insert row into CAR_DRIVER indexUPDATE 5.1 (11.1)Undo Undo lock row in CAR tableSELECT FOR UPDATE 11.4Redo Lock row in CAR tableSELECT FOR UPDATE Commit 5.4 End TransactionCOMMIT Oracle 9.2 and below
  • juliandyke.com© 2013 Julian Dyke19 Baseline - Update Header 5.2 Start Transaction 5.1 (11.1)Undo Undo update row in CAR tableUPDATE 11.5Redo Update row in CAR tableUPDATE Undo Undo delete row from CAR_DRIVER index5.1 (10.22)UPDATE Redo 10.4 Delete row from CAR_DRIVER indexUPDATE 5.1 (10.22)Undo Undo insert row into CAR_DRIVER indexUPDATE Redo 10.2 Insert row into CAR_DRIVER indexUPDATE 5.1 (11.1)Undo Undo lock row in CAR tableSELECT FOR UPDATE 11.4Redo Lock row in CAR tableSELECT FOR UPDATE Commit 5.4 End TransactionCOMMIT Oracle 10.1 and aboveRedo Generation for each Update Statement
  • juliandyke.com© 2013 Julian Dyke20 Baseline - Results  Redo Generation in Bytes Operation INSERT (car.csv) UPDATE (points.csv) Total Baseline 20,448,852 14,409,676 34,858,528  Note  Amount of redo generated by both INSERT and UPDATE can be variable due to  Undo segment management  Recursive DDL statements e.g. extent allocation  Block cleanouts
  • juliandyke.com© 2013 Julian Dyke21 Test 1  Check for unused indexes  CAR_PK indexes columns  SEASON_KEY  RACE_KEY  POSITION  supports primary key therefore mandatory  CAR_DRIVER indexes columns  SEASON_KEY  DRIVER_KEY  DRIVER_POINTS  no longer required by current version of application DROP INDEX car_driver;
  • juliandyke.com© 2013 Julian Dyke22 Test 1 - Insert Redo Generation for each Insert Statement Header 5.2 Start Transaction 11.2Redo Insert row in CAR tableINSERT Redo 10.2 Insert row into CAR_PK indexINSERT 5.1 (10.22)Undo Undo insert row into CAR_DRIVER indexINSERT Redo 10.2 Insert row into CAR_DRIVER indexINSERT 5.1 (11.1)Undo Undo insert row in CAR tableINSERT Undo Undo insert row into CAR_PK index5.1 (10.22)INSERT Commit 5.4 End TransactionCOMMIT STOP
  • juliandyke.com© 2013 Julian Dyke23 Test 1 - Update Header 5.2 Start Transaction 5.1 (11.1)Undo Undo update row in CAR tableUPDATE 11.5Redo Update row in CAR tableUPDATE Undo Undo delete row from CAR_DRIVER index5.1 (10.22)UPDATE Redo 10.4 Delete row from CAR_DRIVER indexUPDATE 5.1 (10.22)Undo Undo insert row into CAR_DRIVER indexUPDATE Redo 10.2 Insert row into CAR_DRIVER indexUPDATE 5.1 (11.1)Undo Undo lock row in CAR tableSELECT FOR UPDATE 11.4Redo Lock row in CAR tableSELECT FOR UPDATE Commit 5.4 End TransactionCOMMIT Redo Generation for each Update Statement STOP
  • juliandyke.com© 2013 Julian Dyke24 Test 1 - Results  Redo Generation in Bytes Operation INSERT (car.csv) UPDATE (points.csv) Total Baseline 20,448,852 14,409,676 34,858,528 Test 1 14,687,756 12,467,400 27,155,156  Conclusion  Eliminating redundant index reduced  insert redo generation by 5,761,096 bytes  update redo generation by 1,942,276 bytes
  • juliandyke.com© 2013 Julian Dyke25 Test 2  In UPDATE statements  For tables undo and redo is generated for all columns in SET clause  For indexes undo and redo are only generated for index keys that have changed  Statements often update all columns to reduce parsing e.g.: UPDATE car SET driver_key = :driver_key, team_key = :team_key, engine_key = :engine_key, laps_completed = :laps_completed, classification_key = :classification_key, notes = :notes, driver_points = :driver_points, team_points = :team_points WHERE season_key = :season_key AND race_key = :race_key AND position = :position;
  • juliandyke.com© 2013 Julian Dyke Redo Generation for each Update Statement 26 Test 2 - Update Header 5.2 Start Transaction 5.1 (11.1)Undo Undo update row in CAR tableUPDATE 11.5Redo Update row in CAR tableUPDATE 5.1 (11.1)Undo Undo lock row in CAR tableSELECT FOR UPDATE 11.4Redo Lock row in CAR tableSELECT FOR UPDATE Commit 5.4 End TransactionCOMMIT Slot = 23 Col 3 = JMON Col 4 = WIL Col 5 = BMW Col 6 = 71 Col 7 = C Col 8 = <Null> Col 9 = 10 Col 10= 10 Slot = 23 Col 3 = JMON Col 4 = WIL Col 5 = BMW Col 6 = 71 Col 7 = C Col 8 = <Null> Col 9 = 0 Col 10= 0 11.5Redo 5.1 (11.1)Undo Slot = 23 Col 3 = JMON Col 4 = WIL Col 5 = BMW Col 6 = 71 Col 7 = C Col 8 = <Null> Col 9 = 10 Col 10= 10 Slot = 23 Col 3 = JMON Col 4 = WIL Col 5 = BMW Col 6 = 71 Col 7 = C Col 8 = <Null> Col 9 = 0 Col 10= 0 STOP
  • juliandyke.com© 2013 Julian Dyke27 Test 2  Only update columns which can have new values  DRIVER_POINTS  TEAM_POINTS For each line in points.csv { read :season_key, :race_key, :position, :driver_points, :team_points; SELECT ... FOR UPDATE; UPDATE car SET driver_key = :driver_key, team_key = :team_key, engine_key = :engine_key, laps_completed = :laps_completed, classification_key = :classification_key, notes = :notes, driver_points = :driver_points, team_points = :team_points WHERE season_key = :season_key AND race_key = :race_key AND position = :position; COMMIT; } For each line in points.csv { read :season_key, :race_key, :position, :driver_points, :team_points; SELECT ... FOR UPDATE; UPDATE car SET driver_key = :driver_key, team_key = :team_key, engine_key = :engine_key, laps_completed = :laps_completed, classification_key = :classification_key, notes = :notes, driver_points = :driver_points, team_points = :team_points WHERE season_key = :season_key AND race_key = :race_key AND position = :position; COMMIT; }
  • juliandyke.com© 2013 Julian Dyke28 Test 2 - Results  Redo Generation in Bytes Operation INSERT (car.csv) UPDATE (points.csv) Total Baseline 20,448,852 14,409,676 34,858,528 Test1 14,687,756 12,467,400 27,155,156 Test2 14,560,052 11,584,760 26,144,812  Conclusion  Eliminating unnecessary columns from update statements reduced update redo generation by 882,640 bytes  Would be significantly more if unchanged columns included long fields e.g. CHAR, or VARCHAR2
  • juliandyke.com© 2013 Julian Dyke29 Test 3  Eliminate unnecessary SELECT FOR UPDATE statements For each line in points.csv { read :season_key, :race_key, :position, :driver_points, :team_points; SELECT driver_key, team_key, engine_key, laps_completed, classification_key, notes INTO :driver_key, :team_key, :engine_key, :laps_completed, :classification_key, :notes FROM car WHERE season_key = :season_key AND race_key = :race_key AND position = :position FOR UPDATE; UPDATE car SET driver_points = :driver_points, team_points = :team_points WHERE season_key = :season_key AND race_key = :race_key AND position = :position; COMMIT; } For each line in points.csv { read :season_key, :race_key, :position, :driver_points, :team_points; SELECT driver_key, team_key, engine_key, laps_completed, classification_key, notes INTO :driver_key, :team_key, :engine_key, :laps_completed, :classification_key, :notes FROM car WHERE season_key = :season_key AND race_key = :race_key AND position = :position FOR UPDATE; UPDATE car SET driver_points = :driver_points, team_points = :team_points WHERE season_key = :season_key AND race_key = :race_key AND position = :position; COMMIT; }
  • juliandyke.com© 2013 Julian Dyke30 Test 3 - Update Header 5.2 Start Transaction 5.1 (11.1)Undo Undo lock row in CAR tableSELECT FOR UPDATE 11.4Redo Lock row in CAR tableSELECT FOR UPDATE Redo Generation for each Update Statement 11.5Redo Update row in CAR tableUPDATE 5.1 (11.1)Undo Undo update row in CAR tableUPDATE Commit 5.4 End TransactionCOMMIT STOP
  • juliandyke.com© 2013 Julian Dyke31 Test 3 - Results  Redo Generation in Bytes Operation INSERT (car.csv) UPDATE (points.csv) Total Baseline 20,448,852 14,409,676 34,858,528 Test 1 14,687,756 12,467,400 27,155,156 Test 2 14,560,052 11,584,760 26,144,812 Test 3 14,554,428 8,475,484 23,029,912  Conclusion  Eliminating SELECT FOR UPDATE statement reduced update redo generation by 3,109,276 bytes
  • juliandyke.com© 2013 Julian Dyke32 Test 4  Rows are inserted with default values of 0 for DRIVER_POINTS and TEAM_POINTS  Points only scored by  first ten cars – 2010 onwards  first eight cars – 2003 to 2010  first six cars - pre 2003  Only update rows with non-zero rows for DRIVER_POINTS and/or TEAM_POINTS Team Driver Points No Points Points 3514 30 No Points 324 12313 Driver Team 324 3514 30 12313 STOP
  • juliandyke.com© 2013 Julian Dyke33 Header 5.2 11.5Redo 5.1 (11.1)Undo Commit 5.4 Test 4 - Update Redo Generation for each Update Statement Header 5.2 11.5Redo 5.1 (11.1)Undo Commit 5.4 UPDATE car SET driver_points = 1 team_points = 1 WHERE ... col9 = 0 col10 = 0 col9 = 1 col10 = 1 UPDATE car SET driver_points = 0 team_points = 0 WHERE ... col9 = 0 col10 = 0 col9 = 0 col10 = 0 UPDATE car SET driver_points = 0 team_points = 0 WHERE ... col9 = 0 col10 = 0 col9 = 0 col10 = 0 UPDATE car SET driver_points = 9 team_points = 9 WHERE ... col9 = 0 col10 = 0 col9 = 9 col10 = 9 Header 5.2 11.5Redo 5.1 (11.1)Undo Commit 5.4 Header 5.2 11.5Redo 5.1 (11.1)Undo Commit 5.4 Header 5.2 11.5Redo 5.1 (11.1)Undo Commit 5.4 UPDATE car SET driver_points = 9 team_points = 9 WHERE ... col9 = 0 col10 = 0 col9 = 9 col10 = 9 STOP
  • juliandyke.com© 2013 Julian Dyke34 Test 4  Only update rows with non-zero rows for DRIVER_POINTS and/or TEAM_POINTS For each line in points.csv { read :season_key, :race_key, :position, :driver_points, :team_points; UPDATE car SET driver_points = :driver_points, team_points = :team_points WHERE season_key = :season_key AND race_key = :race_key AND position = :position; COMMIT; } For each line in points.csv { read :season_key, :race_key, :position, :driver_points, :team_points; IF driver_points != 0 OR team_points != 0 THEN { UPDATE car SET driver_points = :driver_points, team_points = :team_points WHERE season_key = :season_key AND race_key = :race_key AND position = :position; COMMIT; } }
  • juliandyke.com© 2013 Julian Dyke35 Test 4 - Results  Redo Generation in Bytes Operation INSERT (car.csv) UPDATE (points.csv) Total Baseline 20,448,852 14,409,676 34,858,528 Test 1 14,687,756 12,467,400 27,155,156 Test 2 14,560,052 11,584,760 26,144,812 Test 3 14,554,428 8,475,484 23,029,912 Test 4 14,683,408 2,070,316 16,753,724  Conclusions  Eliminating unnecessary update statements reduced update redo generation by 6,405,168 bytes
  • juliandyke.com© 2013 Julian Dyke36 For each line in car.csv { read :season_key, :race_key, :position, :driver_key, :team_key, :engine_key, :laps_completed, :classification_key, :notes; INSERT INTO car (season_key, race_key, position, driver_key, team_key, engine_key, laps_completed, classification_key, notes) VALUES (:season_key,:race_key,:position, :driver_key, :team_key, :engine_key, :laps_completed, :classification_key, :notes) COMMIT; } For each line in car.csv { read :season_key, :race_key, :position, :driver_key, :team_key, :engine_key, :laps_completed, :classification_key, :notes; INSERT INTO car (season_key, race_key, position, driver_key, team_key, engine_key, laps_completed, classification_key, notes) VALUES (:season_key,:race_key,:position, :driver_key, :team_key, :engine_key, :laps_completed, :classification_key, :notes) COMMIT; } COMMIT; Test 5  Eliminate unnecessary COMMIT statements
  • juliandyke.com© 2013 Julian Dyke37 For each line in points.csv { read :season_key, :race_key, :position, :driver_points, :team_points; IF driver_points != 0 OR team_points != 0 THEN { UPDATE car SET driver_points = :driver_points, team_points = :team_points WHERE season_key = :season_key AND race_key = :race_key AND position = :position; COMMIT; } } For each line in points.csv { read :season_key, :race_key, :position, :driver_points, :team_points; IF driver_points != 0 OR team_points != 0 THEN { UPDATE car SET driver_points = :driver_points, team_points = :team_points WHERE season_key = :season_key AND race_key = :race_key AND position = :position; COMMIT;MMIT; } } COMMIT; Test 5  Eliminate unnecessary COMMIT statements (continued)
  • juliandyke.com© 2013 Julian Dyke38 Test 5 - Insert Redo Generation for each Insert Statement Header 5.2 Start Transaction 11.2Redo Insert row in CAR tableINSERT Redo 10.2 Insert row into CAR_PK indexINSERT 5.1 (11.1)Undo Undo insert row in CAR tableINSERT Undo Undo insert row into CAR_PK index5.1 (10.22)INSERT Commit 5.4 End TransactionCOMMIT Header 5.2 Start Transaction Undo Undo insert row into CAR_PK index5.1 (10.22)INSERT 11.2Redo Insert row in CAR tableINSERT Redo 10.2 Insert row into CAR_PK indexINSERT 5.1 (11.1)Undo Undo insert row in CAR tableINSERT Commit 5.4 End TransactionCOMMIT STOP
  • juliandyke.com© 2013 Julian Dyke39 Test 5 - Results  Redo Generation in Bytes Operation INSERT (car.csv) UPDATE (points.csv) Total Baseline 20,448,852 14,409,676 34,858,528 Test 1 14,687,756 12,467,400 27,155,156 Test 2 14,560,052 11,584,760 26,144,812 Test 3 14,554,428 8,475,484 23,029,912 Test 4 14,683,408 2,070,316 16,753,724 Test 5 9,516,512 1,028,084 10,544,596  Conclusion  Eliminating COMMIT statements reduced  insert redo generation by 5,166,896 bytes  update redo generation by 1,042,232 bytes
  • juliandyke.com© 2013 Julian Dyke40 Test 6  Default batch size is 1  Test INSERT and UPDATE with different batch sizes Batch Size INSERT Redo UPDATE Redo Total Redo 1 9,517,096 1,028,084 10,545,180 2 5,654,136 1,028,084 6,682,220 4 3,927,092 1,028,440 4,955,532 8 3,011,944 1,028,084 4,040,028 16 2,588,540 1,028,636 3,617,176 32 2,375,884 1,028,172 3,404,056 64 2,254,936 1,028,040 3,282,976 128 2,195,876 1,028,084 3,223,960 256 2,179,404 1,028,440 3,207,844 512 2,163,816 1,028,084 3,191,900 1024 2,163,084 1,028,084 3,191,168 2048 2,160,012 1,028,084 3,188,096
  • juliandyke.com© 2013 Julian Dyke41 Test 6 - Results 0 1000000 2000000 3000000 4000000 5000000 6000000 7000000 8000000 9000000 10000000 1 2 4 8 16 32 64 128 256 512 1024 2048 Batch Size Redo(bytes)
  • juliandyke.com© 2013 Julian Dyke42 Test 6 - Results  Redo Generation in Bytes Operation INSERT (car.csv) UPDATE (points.csv) Total Baseline 20,448,852 14,409,676 34,858,528 Test 1 14,687,756 12,467,400 27,155,156 Test 2 14,560,052 11,584,760 26,144,812 Test 3 14,554,428 8,475,484 23,029,912 Test 4 14,683,408 2,070,316 16,753,724 Test 5 9,516,512 1,028,084 10,544,596 Test 6 2,195,876 1,028,084 3,223,960  Conclusion  Batch Size of 128  reduced insert redo generation by 7,320,636 bytes  update redo generation unaffected
  • juliandyke.com© 2013 Julian Dyke43 Test 7  Create global temporary table CREATE GLOBAL TEMPORARY TABLE temporary_car ( season_key VARCHAR2(4), race_key VARCHAR2(2), position NUMBER, driver_key VARCHAR2(4), team_key VARCHAR2(3), engine_key VARCHAR2(3), laps_completed NUMBER, classification_key VARCHAR2(4), notes VARCHAR2(100), driver_points NUMBER, team_points NUMBER ) ON COMMIT PRESERVE ROWS;
  • juliandyke.com© 2013 Julian Dyke44 Test 7  Insert rows into global temporary table For each line in car.csv { read :season_key, :race_key, :position, :driver_key, :team_key, :engine_key, :laps_completed, :classification_key, :notes; INSERT INTO temporary_car (season_key, race_key, position, driver_key, team_key, engine_key, laps_completed, classification_key, notes) VALUES (:season_key,:race_key,:position, :driver_key, :team_key, :engine_key, :laps_completed, :classification_key, :notes) COMMIT; }  Generated 64,140 bytes of redo
  • juliandyke.com© 2013 Julian Dyke45 Test 7  Update points in global temporary table For each line in points.csv { read :season_key, :race_key, :position, :driver_points, :team_points; IF driver_points != 0 OR team_points != 0 THEN { UPDATE temporary car SET driver_points = :driver_points, team_points = :team_points WHERE season_key = :season_key AND race_key = :race_key AND position = :position; } } COMMIT;  Generated 652,884 bytes of redo
  • juliandyke.com© 2013 Julian Dyke46 Test 7  Copy rows from global temporary table to permanent table INSERT INTO car ( season_key, race_key, position, driver_key, team_key, engine_key, laps_completed, classification_key, notes, driver_points, team_points ) SELECT season_key, race_key, position, driver_key, team_key, engine_key, laps_completed, classification_key, notes, driver_points, team_points FROM temporary_car;  Generated 2,166,724 bytes of redo  APPEND hint had no effect  Direct load is disabled if table has referential constraints
  • juliandyke.com© 2013 Julian Dyke47 Test 7 - Results  Redo Generation in Bytes  Conclusion  Global Temporary Table reduced total redo generation by 340,212 bytes Operation Description Total Baseline Update all rows 34,858,528 Test 1 Update affected rows 27,155,156 Test 2 Update affected columns 26,144,812 Test 3 Drop index 23,029,912 Test 4 SELECT FOR UPDATE 16,753,724 Test 5 COMMIT 10,544,596 Test 6 Increase Batch Size 3,223,960 Test 7 Global Temporary Table 2,883,748
  • juliandyke.com© 2013 Julian Dyke48 Test 8  Create external tables CREATE OR REPLACE DIRECTORY external_dir AS '/u01/app/oracle/gp'; CREATE TABLE external_points ( season_key VARCHAR2(4), race_key VARCHAR2(2), position NUMBER, driver_points NUMBER, team_points NUMBER ) ORGANIZATION EXTERNAL ( TYPE ORACLE_LOADER DEFAULT DIRECTORY external_dir ACCESS PARAMETERS ( RECORDS DELIMITED BY NEWLINE FIELDS TERMINATED BY ',' ) LOCATION ('points.csv') );
  • juliandyke.com© 2013 Julian Dyke49 Test 8 CREATE TABLE external_car ( season_key VARCHAR2(4), race_key VARCHAR2(2), position NUMBER, driver_key VARCHAR2(4), team_key VARCHAR2(3), engine_key VARCHAR2(3), laps_completed NUMBER, classification_key VARCHAR2(4), notes VARCHAR2(100) ) ORGANIZATION EXTERNAL ( TYPE ORACLE_LOADER DEFAULT DIRECTORY external_dir ACCESS PARAMETERS ( RECORDS DELIMITED BY NEWLINE FIELDS TERMINATED BY ',' MISSING FIELD VALUES ARE NULL ) LOCATION ('car.csv') );
  • juliandyke.com© 2013 Julian Dyke50 Test 8  Insert directly into permanent table joining contents of both external tables INSERT INTO car ( season_key, race_key, position, driver_key, team_key, engine_key, laps_completed, classification_key, notes, driver_points, team_points ) SELECT c.season_key, c.race_key, c.position, c.driver_key, c.team_key, c.engine_key, c.laps_completed, c.classification_key, c.notes, p.driver_points, p.team_points FROM external_car c, external_points p WHERE c.season_key = p.season_key AND c.race_key = p.race_key AND c.position = p.position";  Generated 2,166,724 bytes of redo
  • juliandyke.com© 2013 Julian Dyke51 Test 8 - Results  Redo Generation in Bytes  Conclusion  External Tables reduced total redo generation by 717,024 bytes Operation Description Total Baseline Update all rows 34,858,528 Test 1 Update affected rows 27,155,156 Test 2 Update affected columns 26,144,812 Test 3 Drop index 23,029,912 Test 4 SELECT FOR UPDATE 16,753,724 Test 5 COMMIT 10,544,596 Test 6 Increase Batch Size 3,223,960 Test 7 Global Temporary Table 2,883,748 Test 8 External Table 2,166,724
  • juliandyke.com© 2013 Julian Dyke52 Conclusion  We have seen that the following techniques can be used to reduce the amount of redo generated:  Eliminating redundant indexes  Reducing the number of columns updated  Eliminating redundant SELECT FOR UPDATE statements  Reducing the number of rows processed  Eliminating COMMIT statements  Increasing the batch size  Using Global Temporary Tables  Using External Tables
  • juliandyke.com© 2013 Julian Dyke53 Thank you for listening info@juliandyke.com