Test Driven Development & testing for databases? Why, how, when and what to test in SQL and PL/SQL. Examples with utPLSQL v3.

2. https://www.slideshare.net/Kevlin/what-we-talk-about-when-we-talk-about-unit-testing

3. https://www.slideshare.net/Kevlin/what-we-talk-about-when-we-talk-about-unit-testing

4. Jacek Gębal
twitter: @GebalJacek
mail: jgebal@gmail.com
blog: oraclethoughts.com
Principal Software Engineer
@Fidelity Investments - Ireland
co-author & maintainer of: utPLSQL
Test your PL/SQL
your Database will love you

5. About me
● Software engineer since ~2000 - mainly Oracle database
● Infected with testing and automation since ~2013
● Key contributor, author and maintainer of utPLSQL v3
● Active developer, learner, testing advocate

6. Why?
Plan
What?
When?
How?

7. Why test?

8. Why do we test?
● Testing is cheaper and safer than bugs
Ariane 5 catastrophe
370 million USD burned in 37 seconds bug
Knight Capital
440 million USD loss in ~40 minutes
Toyota
2.4 mln cars recalled due to software bug

9. Why do we test?
● Make the newly written code change-able
So that others (and yourself) can touch this code safely later
● Prove that solution meets requirements
● Prove that a change didn’t break existing functionality
● Document requirements and reveal intention

10. Keep in mind
● DevOps
- you’re not doing DevOps without automated testing
● Continuous Integration
- integration is all about testing

11. What
should be
tested?

12. What should be tested (in database)?
● All logic
○ PL/SQL
○ SQL ( queries / constraints / data-types / views / calculated columns … )
○ In /out data from your code
● Exceptions / corner cases
● Data quality & integrity

13. When
should I
test?

14. When to test
● Whenever you make a change (code / data structure)
● Test
○ Early
○ Often
○ Continuously
The later you test
The harder it gets

15. How should
I test?

16. ● write only
enough code
to pass test
● get to green fast
● don’t cleanup
● write a failing test
● test is a simple example
● test becomes
documentation
● improve structure
● remove duplication
● rename, reorganize
● run tests, stay green
● don’t change behavior
RED GREEN
CLEAN
Test Driven Development (TDD)

17. F.I.R.S.T. principles
● Fast
● Isolated
● Repeatable
● Self-verifying (obvious)
● Thorough & Timely
https://github.com/ghsukumar/.../F.I.R.S.T-Principles-of...
https://pragprog.com/magazines/2012-01/unit-tests-are-first

18. PLUTO
PL/Unit
ruby-plsql-spec
DBFit
Oracle PL/SQL Testing landscape

19. ● free, open source & IDE independent
● continuously & thoroughly tested
● actively developed & maintained
● compatible with Oracle 11g r2+
● pure PL/SQL - *tests described only by annotations & expectations
● easy cursor & object data comparison
● code coverage reporting
● automatic transaction control - *no rollback / cleanup needed
● easy to Version Control
● data-type - aware comparison
Why utPLSQL v3 ?

20. utPLSQL syntax
create or replace package test_stuff as
--%suite( description )
--%test( description )
procedure my_first_test;
end;
create or replace package body test_stuff as
procedure my_first_test is
begin
ut.expect( ‘Chuck’ ).to_equal( ‘Chuck’ );
end;
end;
exec ut.run(‘test_stuff’);
Expectation
AnnotationAnnotation
Running tests

21. Function - Create room
● Creates a room with given name and returns its id
● Throws an exception when the room name is null
● Throws an exception when the room already exists

22. TDD & utPSQL
demo

23. utPLSQL & TDD
● Fast feedback loop
● Tests were tested for both Fail & Pass
● Tests were created from requirements
● Code created to satisfy the tests
● Visible progress with every test
● I can now safely change my code
- it’s covered with tests

24. Isolated
Suite
Isolated
Test
Isolated
Test
Isolated
Test
Test isolation
● create savepoint
● run the test
● rollback to savepoint
● savepoints added ar suite & test level
● can be disabled with --%rollback(manual)

25. How to organize my tests ?
3 suites in hierarchy

26. Organizing tests - suitepath
package test_add_room_content as
--%suite
--%suitepath( org.utplsql.demo.test_rooms )
...
end;
package test_remove_room_by_name as
--%suite
--%suitepath( org.utplsql.demo.test_rooms )
...
end;

27. Organizing tests - suitepath
package test_rooms as
--%suite
--%suitepath( org.utplsql.demo )
--%beforeall( shared_room_test_setup )
...
end;
package test_add_room_content as
--%suite
--%suitepath( org.utplsql.demo.test_rooms )
...
end;
package test_remove_room_by_name as
--%suite
--%suitepath( org.utplsql.demo.test_rooms )
...
end;

28. Organizing tests - contexts
package test_rooms as
--%suite
--%beforeall( shared_room_test_setup )
--%context( add_room_content )
--%test
package test_add_room;
--%endcontext
--%context( remove_room_by_name )
--%test
package test_remove_room;
--%endcontext
...
end;

29. Organizing tests
● With suitepath
○ Suites can be nested, nested, nested, nested, nested, nested
○ Suites can be grouped
● Suites can be invoked by suitepath
exec ut.run(‘:org.utplsql.demo.test_rooms’);
● Setup from parent suite is visible in child suites
● Contexts -> groups within single package

30. What is my test coverage?
http://utplsql.org/utPLSQL-coverage-html/

31. Definition
Test coverage is a measure used to describe
the degree to which the source code of a program is executed
when a particular test suite runs.
https://en.wikipedia.org/wiki/Code_coverage

32. Line coverage - from profiler
create or replace function f(a integer) return integer is
begin
if a is null then
return 0;
else
return a*a;
end if;
end;
/
3 executable lines
2 test cases -> 100% coverage

33. Block coverage - since Oracle 12.2
create or replace function f1(a integer) return integer is
begin
if a is null then return 0; else return a*a; end if;
end;
/
create or replace function f2(a integer) return integer is
begin
return nvl(a*a,0);
end;
/
1 executable line
2 executable blocks
2 test cases -> 100% coverage
1 executable line
1 executable block
1 test case -> 100% coverage

34. utPLSQL coverage
utPLSQL combines profiler line coverage
with Oracle 12.2 block coverage
to provide statement coverage

35. utPLSQL coverage interpretation
Line covered (executed)
1 time
Line partly covered
(1 of 2 statements covered)
Line not covered
(executed)

36. utPLSQL coverage - project view

37. Continuous Integration
● Script
● Version-control
● Deploy code and tests
● Run all tests
● Report
○ All good? - promote to higher environment
○ Problems? - fix them and repeat the process
● Repeat

38. Integrating with CI/CD servers (utPSLQL-cli)
utPLSQL-cli/bin/utplsql run DB_USER/DB_PASSWORD@DB_CONNECTION_STRING
--path='ut3_tester,ut3$user#'
-source_path=source -owner=ut3
-test_path=test --color
--format=ut_junit_reporter -o=junit_test_results.xml
--format=ut_coveralls_reporter -o=coverage.html
--format=ut_documentation_reporter -s
https://github.com/utPLSQL/utPLSQL-cli

39. Integration
● CI/CD servers
○ Jenkins, TeamCity
○ Travis
○ MS Azure DevOps
○ Oracle Developer cloud
○ Redgate, Flexagon, ...
● Sonar
○ test results
○ code coverage
✓ maven (plugin)
✓ SQLDeveloper ( extension )
✓ TOAD - ( coming in 13.2 )
✓ command-line
● PLSQL Developer - ( not yet )

40. Recap
● Test to prevent bugs and avoid losses
● Tests are documentation for your code
● Test as early as possible
● Group and organize your tests
● Use automatic rollback to minimize need for complex cleanup
● Analyze your test results and code coverage to measure progress
● Automate deployment and testing

41. Questions?
● Ask now :)
● utPLSQL SLACK channel
● github.com/utPLSQL/utPLSQL/issues
● @utPLSQL twitter

42. Supported data-types
● cursors
● objects
● nested tables
● varrays
● json
● number
● varchar2
● date
● boolean
● clob, blob
● timestamp (with (local) timezone)
● interval (day to second / year to month)

43. Matchers
equal( expected )
be_null / be_not_null
be_true / be_false
be_like( mask [, escape_char] )
match( pattern [, modifiers] )
contain( expected )
ut.expect( actual ).to_...
ut.expect( actual ).not_to_...
be_between( lower, upper )
be_greater_than( expected )
be_less_than( expected )
be_greater_or_equal( expected )
be_less_or_equal( expected )
have_count( count ) / be_empty
Matchers:
http://utplsql.org/utPLSQL/latest/userguide/expectations.html

44. Annotations
Package:
--%suite(<description>)
--%suitepath(<path>)
--%rollback(auto/manual)
--%disabled
--%tags(<tag>[,...])
--%context(<name>) --%endcontext
--%beforeall(<proc_name>[,...])
--%afterall(<proc_name>[,...])
--%beforeeach(<proc_name>[,...])
--%aftereach(<proc_name>[,...])
Procedure:
--%test(<description>)
--%throws(<error>[,...])
--%rollback(auto/manual)
--%disabled
--%tags(<tag>[,...])
--%beforetest(<proc_name>[,...])
--%aftertest(<proc_name>[,...])
--%beforeall --%afterall
--%beforeeach --%aftereach
http://utplsql.org/utPLSQL/latest/userguide/annotations.html

45. Resources
Source code: https://github.com/utPLSQL/utPLSQL
Downloads: https://github.com/utPLSQL/utPLSQL/releases
Documentation: http://utplsql.org/utPLSQL/
Cheat-sheet: https://www.cheatography.com/jgebal/lists/utplsql-v3-cheat-sheets/
Resources: http://utplsql.org/resources
utPLSQL-cli: https://github.com/utPLSQL/utPLSQL-cli/releases
SQLDeveloper-extension: https://github.com/utPLSQL/utPLSQL-SQLDeveloper/releases
How-to: https://www.salvis.com/blog/2019/07/06/running-utplsql-tests-in-sql-developer/
Maven plugin: https://github.com/utPLSQL/utPLSQL-maven-plugin/releases
Demo project: https://github.com/utPLSQL/utPLSQL-demo-project
Travis-CI builds: https://travis-ci.org/utPLSQL/utPLSQL
Sonar results: https://sonarcloud.io/dashboard?id=utPLSQL