Tarik R. EssawiVeriSign Inc.Agile Database Testing Techniques
Introduction• Real world best practices• Proven Techniques• Why Test ?• How to greatly improve– PL/SQL Quality– Database S...
Why Test ?• You will spend time testing your code.– You can manually test your code every time achange is made or a build ...
Finding Defects• Defects will be found, the only question iswhen:– Options1)You can find the defects duringdevelopment2)Qu...
What is Agility• A rapid change of velocity or direction inresponse to a stimulus• Many different forms and methods– Test ...
Traditional QA role• Responsible for finding defects• If defects are not found in QA they are foundin production
A Different Approach• Guilty Until Proven Innocent• Developers must prove code is correct• QA role becomes:– Corroborate t...
Side Effects• Number of builds reduced• Amount of rework reducedWeek0Week1Week2Week3Week4Week5Week6Week7Week80510152025303...
PL/SQL Testing
UtPlsql Installation• UtPlsql is a plsql unit testing framework. It is one of many, So ifyou are not using plsql other tes...
Developing the Tests• A test package is created for each applicationpackage– For example, to test PKGORDERITEMS apackage c...
Assertions• Assertions are the heart of the testing framework• You will prove through the assertion that the results are c...
Running Tests• Compile the test package under the sameschema as the calling package• Run the test by calling utplsql.test....
Running Tests Con’t• Review the output, which will say> SSSS U U CCC CCC EEEEEEE SSSS SSSS> S S U U C C C C E S S S S> S U...
Database Schema Installs
Install Uninstall Procedures• Install and Uninstall scripts should not require any operator input• Script should be simple...
Validating Schema Installs• Schema installs are one-time tasks where quality isextremely critical• Errors during a schema ...
Validating Schema Installs con’t• Verify that every object that was expected to becreated actually exists.• Create an asse...
Schema Install Validation Example• Below is an example of a database verification script. The assertion procedure was writ...
Report Testing• Report Testing can be challenging:– Reports contain millions of rows– A single report can consist of multi...
Daily Builds
Daily Build Database• The development and test environments mustbe functionally equivalent to production• The hardware can...
DBMS_METADATA• DBMS_METADATA.SET_TRANSFORM_PARAM(DBMS_METADATA.SESSION_TRANSFORM,STORAGE,false);• DBMS_METADATA.SET_TRANSF...
DBMS_METADATA con’t• Examples of how to reverse engineer the database• SELECT DBMS_METADATA.GET_DDL(TABLE,table_name) FROM...
Daily BuildsRecipe:Daily Build DatabaseAPI to Checkout Source CodeInstall ScriptsDatabase Validation ScriptsSelf Contained...
Daily Builds con’t• At times uninstall scripts may not be completeenough to undo changes to the datadictionary• An alterna...
Logging and Monitoring
Logging and MonitoringCREATE TABLE EVENTLOG (EVENTLOGID NUMBER(9) NOT NULL,SEVERITYID NUMBER(2) DEFAULT 1 NOT NULL,LOCATIO...
Debug Flag• Example 1 – log info messageif (pkgeventlog.getdebugflag) thenpkgeventlog.createeventlog(1,PkgOrderItems.AddOr...
Summary• Everything can be tested• Developers are Guilty until Proven Innocent• Robust testing has great side effects such...
Resources• forwardthinkingdb.com– Database Architecture Resources• agiledata.org– Agile Database Development
Become a Complete Oracle Technologyand Database Professional• Join the IOUG online at www.ioug.org and get immediate acces...
Stop by the IOUG Booth• Moscone Center West – 2nd Floor– Register for Raffles and Giveaways– Learn more about IOUG– Fill o...
Questions/Comments?
Save the Date!May 3-7, 2009Orange County Convention Center WestOrlando, Florida
Tarik R. Essawitessawi@verisign.comThank You!
Upcoming SlideShare
Loading in...5
×

Agile db testing_techniques

278

Published on

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
278
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Agile db testing_techniques"

  1. 1. Tarik R. EssawiVeriSign Inc.Agile Database Testing Techniques
  2. 2. Introduction• Real world best practices• Proven Techniques• Why Test ?• How to greatly improve– PL/SQL Quality– Database Schema Installations– Reports– Build Process– Debugging and Monitoring
  3. 3. Why Test ?• You will spend time testing your code.– You can manually test your code every time achange is made or a build is deployed• Manual testing is haphazard• Code coverage is sporadic at best– You can create an automated unit test once thatcan be run every time the code changes• Tests are repeatable• Code Coverage is repeatable
  4. 4. Finding Defects• Defects will be found, the only question iswhen:– Options1)You can find the defects duringdevelopment2)Quality Assurance can find the defects3)Customers can find the defects inproduction
  5. 5. What is Agility• A rapid change of velocity or direction inresponse to a stimulus• Many different forms and methods– Test Driven Development– Extreme Programming– Scrum• Foundation of Agility is Testing– Creates confidence to change direction
  6. 6. Traditional QA role• Responsible for finding defects• If defects are not found in QA they are foundin production
  7. 7. A Different Approach• Guilty Until Proven Innocent• Developers must prove code is correct• QA role becomes:– Corroborate test results– Verify the Delivered Code meets Requirements– Focus on integration tests of corner cases
  8. 8. Side Effects• Number of builds reduced• Amount of rework reducedWeek0Week1Week2Week3Week4Week5Week6Week7Week80510152025303540TimeMan HoursDevelopment LifecycleAutomated TestingMaual Testing
  9. 9. PL/SQL Testing
  10. 10. UtPlsql Installation• UtPlsql is a plsql unit testing framework. It is one of many, So ifyou are not using plsql other testing frameworks exist thataccomplish the same goal.• Installing the package– Download zip file from• http://utplsqlsql.sourceforge.net• Unzip the file.– Create user• connect to the database as system• create user utplsql identified by utplsql default tablespace ?? temporarytablespace ??;• grant create session, create table, create procedure, create sequence,create view, create public synonym, drop public synonym to utplsql;• alter user utplsql quota unlimited on ??;• You can also just grant DBA privileges.– In the code directory where the file is unzipped, login to sqlplus asutplsql and run @ut_i_do install (you must pass in the parameter“install”).– Create public synonyms for all of the packages owned by utplsql.
  11. 11. Developing the Tests• A test package is created for each applicationpackage– For example, to test PKGORDERITEMS apackage called UT_ PKGORDERITEMS iscreated• Each package contains a ut_setup,ut_teardown, and then individual proceduresto test the code• The tests are self-contained• Tests are based on assertions
  12. 12. Assertions• Assertions are the heart of the testing framework• You will prove through the assertion that the results are correct• Below are the basic assertions that can satisfy 80% of test cases.– utassert.eqquery – compares multiple columns of two queries or result sets– utassert.eqqueryvalue – compares a single column returned from a query– utassert.eq –compares two values– utassert.throws – is used to test that exceptions are being thrown andcaught properly• For the other 20% you can also– Compare tables, pipes, refcursors, and collections for equality– Check for null values, table counts and whether previous assertions passedor failed– You can also extend the assert procedure through inheritance and createyour own
  13. 13. Running Tests• Compile the test package under the sameschema as the calling package• Run the test by calling utplsql.test. This isone of many ways to execute the testSET SERVEROUTPUT ON SIZE UNLIMITEDBEGINutplsql.test(package_in=>PKGORDERITEMS,recompile_in=>FALSE);DBMS_OUTPUT.put_line (utplsql2.runnum);END;
  14. 14. Running Tests Con’t• Review the output, which will say> SSSS U U CCC CCC EEEEEEE SSSS SSSS> S S U U C C C C E S S S S> S U U C C C C E S S> S U U C C E S S> SSSS U U C C EEEE SSSS SSSS> S U U C C E S S> S U U C C C C E S S> S S U U C C C C E S S S S> SSSS UUU CCC CCC EEEEEEE SSSS SSSSOR> FFFFFFF AA III L U U RRRRR EEEEEEE> F A A I L U U R R E> F A A I L U U R R E> F A A I L U U R R E> FFFF A A I L U U RRRRRR EEEE> F AAAAAAAA I L U U R R E> F A A I L U U R R E> F A A I L U U R R E> F A A III LLLLLLL UUU R R EEEEEEE.
  15. 15. Database Schema Installs
  16. 16. Install Uninstall Procedures• Install and Uninstall scripts should not require any operator input• Script should be simple and straightforwardspool install.lstPROMPT Calling define_var.sql...@@define_var.sql/********************************************** Creating Orders**********************************************/@cr_orders.sql/********************************************** Creating OrderItems**********************************************/@cr_order_items_table.sqlspool off;!grep ORA- *.lst!grep SP2- *.lstexit
  17. 17. Validating Schema Installs• Schema installs are one-time tasks where quality isextremely critical• Errors during a schema install can cause an entirerelease to be rolled back or cause the maintenancewindow to be missed due to debugging• Validation scripts evolved based on the followingscenarios:– Errors being written to files and never reviewed– Errors never being written to files at all– Scripts not executing, which means no error is generated– Scripts that exit part of the way through and never executethe remaining commands
  18. 18. Validating Schema Installs con’t• Verify that every object that was expected to becreated actually exists.• Create an assertion package that will verify the actualversus the expected.• Generate an error if the assertion fails.• Items to check– Tables– Indexes– Constraints– Synonyms and Grants– Reference data was inserted or updated as expected– Anything created during the install…
  19. 19. Schema Install Validation Example• Below is an example of a database verification script. The assertion procedure was writtenin-house and raises an error if the first and second parameters are not equal. Theverification script are tailored to each deployment.declarev_actual number;Begin-- Verify TABLE Tablesselect count(*) into v_actual from user_tableswhere table_name = ‘ORDER_ITEMSand tablespace_name = DATA;PkgValidation.assert(1, v_actual, Verify Order Items);-- Verify TABLE INDEXESselect count(*) into v_actual from user_indexeswhere index_name in(IDX_ORDER_ITEMS,PK_ORDER_ITEMS)and tablespace_name = INDEX;PkgValidation.assert(2, v_actual, Verify Index Creation);-- Print Validation Summarydbms_output.put_line(=======================================================);dbms_output.put_line(**Assertions passed : ||PkgValidation.v_assertions_passed);dbms_output.put_line(**Assertions Failed : ||PkgValidation.v_assertions_failed);dbms_output.put_line(**Total Assertions : ||PkgValidation.v_total_assertions);dbms_output.put_line(=======================================================);end;
  20. 20. Report Testing• Report Testing can be challenging:– Reports contain millions of rows– A single report can consist of multiple pages of SQL Code• Automating Report Testing can be accomplished:– Load the report into the database using sqlloader or externaltables– Once the report is accessible within the database the sourcedata can be compared to the report data– Assertions can be used to compare each record in the reportwith the source data and vice versa– This initial testing will identify common errors with complextable joins and date ranges– Controlled Datasets will validate the end to end testing of thesystem from the customer channel through to the reports
  21. 21. Daily Builds
  22. 22. Daily Build Database• The development and test environments mustbe functionally equivalent to production• The hardware can differ but the logicalstructure of the database must be an exactreplica• Use the DBMS_METADATA package
  23. 23. DBMS_METADATA• DBMS_METADATA.SET_TRANSFORM_PARAM(DBMS_METADATA.SESSION_TRANSFORM,STORAGE,false);• DBMS_METADATA.SET_TRANSFORM_PARAM(DBMS_METADATA.SESSION_TRANSFORM,SQLTERMINATOR,true);• DBMS_METADATA.SET_TRANSFORM_PARAM(DBMS_METADATA.SESSION_TRANSFORM,SEGMENT_ATTRIBUTES,false);• DBMS_METADATA.SET_TRANSFORM_PARAM(DBMS_METADATA.SESSION_TRANSFORM,TABLESPACE,false);
  24. 24. DBMS_METADATA con’t• Examples of how to reverse engineer the database• SELECT DBMS_METADATA.GET_DDL(TABLE,table_name) FROMUSER_TABLES;• SELECT to_char(DBMS_METADATA.GET_DDL(INDEX,index_name))FROM USER_INDEXES;• SELECT to_char(DBMS_METADATA.GET_DEPENDENT_DDL(OBJECT_GRANT,table_name)) FROM USER_TABLES;• SELECT to_char(DBMS_METADATA.GET_DEPENDENT_DDL(CONSTRAINT‘,table_name)) FROM USER_CONSTRAINTSwhere constraint_type in (P,C);• SELECT to_char(DBMS_METADATA.GET_DEPENDENT_DDL(REF_CONSTRAINT‘,table_name)) FROM USER_CONSTRAINTSwhere constraint_type = R;
  25. 25. Daily BuildsRecipe:Daily Build DatabaseAPI to Checkout Source CodeInstall ScriptsDatabase Validation ScriptsSelf Contained Unit TestsUninstall Scripts
  26. 26. Daily Builds con’t• At times uninstall scripts may not be completeenough to undo changes to the datadictionary• An alternative to uninstall scripts is usingFlashback Database• Before running the install log the current scn– DBMS_FLASHBACK.GET_SYSTEM_CHANGE_NUMBER()• When complete with all tests flashback thedatabase– FLASHBACK DATABASE TO SCN ?;
  27. 27. Logging and Monitoring
  28. 28. Logging and MonitoringCREATE TABLE EVENTLOG (EVENTLOGID NUMBER(9) NOT NULL,SEVERITYID NUMBER(2) DEFAULT 1 NOT NULL,LOCATION VARCHAR2(100 BYTE) NOT NULL,ERRORMSG VARCHAR2(4000 BYTE),ADDITIONALINFO VARCHAR2(4000 BYTE),CREATEDDATE DATE,CREATEDBY NUMBER(9))• http://log4plsql.sourceforge.net/
  29. 29. Debug Flag• Example 1 – log info messageif (pkgeventlog.getdebugflag) thenpkgeventlog.createeventlog(1,PkgOrderItems.AddOrderItems,info,Get transid,sysdate,1);end if;• Example 2 – log error messageEXCEPTION WHEN OTHERS THENvError:=substr(DBMS_UTILITY.FORMAT_ERROR_BACKTRACE ,1,100);pkgeventlog.createeventlog(4,PkgOrderItems.AddOrderItems’,error,vError,sysdate,1);RAISE;
  30. 30. Summary• Everything can be tested• Developers are Guilty until Proven Innocent• Robust testing has great side effects such as:– Reduced Rework – Spend more time developingnew code instead of fixing old code• Agility requires confidence• Confidence = Tests that prove the system iscorrect
  31. 31. Resources• forwardthinkingdb.com– Database Architecture Resources• agiledata.org– Agile Database Development
  32. 32. Become a Complete Oracle Technologyand Database Professional• Join the IOUG online at www.ioug.org and get immediate accessto:– Member Discounts and Special Offers– SELECT Journal– Library of Oracle Knowledge (LoOK)– Member Directory– Special Interest Groups– Discussion Forums:– Access to Local and Regional Users Groups:– 5 Minute Briefing: Oracle– Volunteer Opportunities
  33. 33. Stop by the IOUG Booth• Moscone Center West – 2nd Floor– Register for Raffles and Giveaways– Learn more about IOUG– Fill out a SIG interest card
  34. 34. Questions/Comments?
  35. 35. Save the Date!May 3-7, 2009Orange County Convention Center WestOrlando, Florida
  36. 36. Tarik R. Essawitessawi@verisign.comThank You!

×