How to prepare test data for regression testing.

1. Make your test workload as real
as possible.
How to prepare and test your application.
SESSION ID#:
GLEB OTOCHKIN - PRINCIPAL CONSULTANT

2. 2
Testing is not so simple as it seems to be.

3. Started to work with data in 1992
At Pythian since 2008
Area of expertise:
- Replication
- Oracle RAC
- Oracle engineered systems
- Virtualization
- Performance tuning
Gleb Otochkin
Principal Consultant

4. Live in Ottawa
Run a lot.
Run in any weather.
About me

5. Founded in 1997, Pythian is a global
leader in data consulting and managed
services specializing in planning,
optimizing, and managing mission-
critical data systems
ABOUT PYTHIAN
Top 5% talent worldwide
10 Oracle ACEs
2 Oracle ACE Directors
18 years in business
400+ employees
250+ customers worldwide

6. Why is it so important?
■ Functionality.
■ Performance.
■ Bugs.
6

7. AGENDA.
■ Sources for test dataset.
■ Data preparation.
■ Tools for testing.
■ Setting up Jmeter.
■ Apply workload.
■ Monitoring results.
■ QA
7

8. Select from dual.
■ Artificially generated data.
8

9. Trying to make it better.
■ Small difference in a way to generate the data.
9

10. Using subquery factoring.
■ Reducing memory and increase speed in generating data.
10

11. What can be wrong?
■ Data distribution in artificial dataset.
■ Data in database are different from data using for test.
11

12. Artificial data.
■ Data distribution in artificial dataset.
12

13. Real data.
■ Data distribution in real world.
13

14. Wrong test dataset.
■ For an optimized SQL.
14

15. Some examples.
■ For an optimized SQL.
15

16. Some examples.
■ Let’s disable the index.
16

17. Some examples.
■ For an non-optimized SQL.
17

18. Sources for test dataset.
■ Sample data based on public sources.
–AWS public data sets.
•https://aws.amazon.com/public-data-sets/
–IMDB resources.
•http://www.imdb.com/interfaces
–Online collections.
•https://github.com/caesar0301/awesome-public-datasets
18

19. Sources for test dataset.
■ Dataset based on real data.
–Pros:
•Realistic data distribution.
•Proper data types.
•Easy to compare.
–Cons:
•Access to real data sometimes restricted by law.
•Data masking can impact your workload.
•How we can mix our data.
19

20. How we test our code.
■ Captured real workload.
–RAT.
–HammerDB.
20

21. Data preparation.
■ Data cooking.
–Get raw data.
–Format.
–Shuffle.
–Upload to your database.
–Prepare it for a tool.
21

22. Data preparation.
■ Getting the data.
–Download actors list from IMDB.
• http://www.imdb.com/interfaces
[oracle@jmeter01 apache]$ ls -l /media/sf_shared/movie/actors.list
-rwxrwx---. 1 root vboxsf 1047896963 Jan 1 02:32
/media/sf_shared/movie/actors.list
[oracle@jmeter01 apache]$
22

23. Data preparation.
■ Formatting.
–Create a csv test file
–XML.
–Table in a database.
–Data generator.
23

24. Data preparation.
■ Formatting using a python script.
[oracle@jmeter01 movie]$ python script.py
Please, enter filename: actors.list
[oracle@jmeter01 movie]$ ls -l result.txt
-rwxrwx---. 1 root vboxsf 867430310 Jan 11 00:03 result.txt
[oracle@jmeter01 movie]$
Email me if you need it: otochkin@pythian.com
24

25. Data preparation.
■ Shuffle your data.
–Python.
–Perl
–OS standard tools.
–Database tables load/unload.
25

26. Data preparation.
■ Shuffle your data using a database table.
[oracle@localhost movie]$ cat spool_name.sql
set head off
set pages 0
spool actors_name.txt
select pk_id,name from actors order by dbms_random.random;
spool off
exit;
[oracle@localhost movie]$ nohup sqlplus movie/oracle@orcl
@spool_name.sql &
26

27. Data preparation.
■ Upload to database.
–Express mode for sql loader.
http://www.oracle.com/technetwork/database/enterprise-
edition/learnmore/sqlldr-express-mode-wp-1991038.pdf
27

28. Data preparation.
■ Add a unique ID (optional).
28

29. Tools for testing.
■ RAT.
–Pros.
•Real dataset and load.
•Support.
•Documentation.
–Cons
•Price.
•Don’t have existing load for a brand new project.
http://oracle.com
29

30. Tools for testing.
■ HammerDB.
–Pros.
•Real dataset and load.
–Cons
•Don’t have existing load for a brand new project.
http://www.hammerdb.com
30

31. Tools for testing.
■ Swingbench.
–Pros
•Interface
•Lot of options.
–Cons
•Not easy to modify some components.
http://dominicgiles.com/swingbench.html
31

32. Tools for testing.
■ JMeter.
–Pros
•Functional GUI
•Modules.
•Open source.
•Modifiable.
–Cons.
•Some limits in load manipulation.
32

33. Tools for testing.
■ Custom code.
–Pros.
•Fully flexible.
–Cons
•Need time for development.
33

34. Tools for testing.
■ Commercial tools.
–Cost.
•Good interface.
•Functionality.
•Support.
–Cons.
•Cost.
•Flexibility.
•Modifiable.
34

35. Setting up Jmeter.
■ JMeter setup.
–Prepare a server(s).
–Install OS and Java.
–Download Jmeter.
–Download drivers.
–Unzip.
–Create configuration.
–Test.
35

36. Setting up Jmeter.
■ Prepare server(s).
–Choose OS/Platform.
•Linux 86x64 works fine.
–Install OS.
•Server installation with Desktop GUI.
–Install packages to support Java.
•JDK or JRE
36

37. Setting up Jmeter.
■ Downloads Jmeter and drivers.
–Jmeter is accessible on jmeter.apache.org.
•http://jmeter.apache.org
–Oracle JDBC drivers on official site.
•http://www.oracle.com/technetwork/database/features/jdbc/in
dex-091264.html
37

38. Setting up Jmeter.
■ Prepare software.
–Uncompress the Jmeter.
–Copy Oracle JDBC drivers.
38

39. Setting up Jmeter.
■ Start Jmeter.
39

40. Setting up Jmeter.
■ Create a test plan (configuration).
40

41. Setting up Jmeter.
■ Modify configuration adding connection.
41

42. Setting up Jmeter.
■ Configure connection.
42

43. Setting up Jmeter.
■ Adding a simple query.
43

44. Setting up Jmeter.
■ Choose query and parameters.
44

45. Setting up Jmeter.
■ Check results using “Results Tree” module.
45

46. Apply workload.
■ Create a CSV file with parameters.
[oracle@localhost movie]$ cat spool_name.sql
set head off
set pages 0
spool actors_name.txt
select pk_id||','||name from actors order by dbms_random.random;
spool off
exit;
[oracle@localhost movie]$
46

47. Apply workload.
■ Create a data source for CSV in JMeter.
47

48. Apply workload.
■ Prepare SQL with parameters.
48

49. Apply workload.
■ Adjust thread parameters.
49

50. Monitoring execution.
■ JMeter metrics.
50

51. If everything has been tested properly...
51

52. QUESTIONS?
52