SlideShare a Scribd company logo
1 of 37
Performance Testing and OBIEE 
By 
Quontra Solutions
 Oracle BI specialist at Morrisons plc 
 Big IT development programme at its early 
stages implementing OBIEE, OBIA, ORDM, all 
on Oracle 11g & HP-UX
 A Performance Tuning Methodology 
 OBIEE techie stuff 
 Learn from my mistakes!
 Response times 
 Report 
 ETL batch 
 OLTP transaction 
 System impact 
 Resource usage 
 Scalability
 Check that your system performs 
 Are the users going to be happy? 
 Baseline 
 How fast is fast? 
▪ How slow is slow? 
 Validate system design 
 Do it right, first time 
 Capacity planning
 It’s never too late 
 “You’ll never catch all your problems in pre-production 
testing. That’s why you need a reliable 
and efficient method for solving the problems that 
leak through your pre-production testing 
processes.”
 Because it makes you better at your job 
 “At the very least, your performance test plan will 
make you a more competent diagnostician (and 
clearer thinker) when it comes time to fix the 
performance problems that will inevitably occur 
during production operation.”
 Quantifying response times 
 System impact 
 User expectations 
 Problem diagnosis 
 Design validation
Define 
Measure 
Analyse 
Review 
Implement 
Timebox! 
Evaluate design / 
config options 
Do it right 
Don’t “fudge it” 
Do more testing 
Redefine 
test Do more testing
 Define – what are you going to test 
• Aim of the test 
• Scope 
• Assumptions 
• Specifics 
• Data, environment, etc 
 Build – how are you going to test it 
 OBIEE specific 
Define 
Measure 
Analyse 
Review 
Implement 
•E.g. : 
•Check that the system performs 
•Baseline performance 
•Prove system capacity 
•Validate system design
Define 
Measure 
Analyse 
Review 
Implement 
MoFrew coemr cpoomnpeonntse n= tms =o reea csoiemr ptole mx a=n magoere = v maroiareb lperse =c ilsaer g=e mr moraer gefinfi coife nertror
Database 
Presentation 
Services 
BI Server 
Report / 
Dashboard 
Logical SQL 
Physical SQL 
statement(s) 
Data set(s) 
Data set 
Rendered 
report 
Excludes App/Web server & presentation 
services plug-in 
Define 
Measure 
Analyse 
Review 
Implement
Database 
Presentation 
Services 
nqcmd 
BI Server 
SQL Client 
Physical 
SQL 
LSQL 
Physical 
SQL 
User & 
Stopwatch 
Load Testing tool 
(eg. LoadRunner, 
OATS) 
Define 
Measure 
Analyse 
Review 
Implement
Define 
Measure 
Analyse 
Review 
Implement 
Usage Tracking 
or NQQuery.log 
Test 
script 
BI 
Server 
Data 
nqcmd 
Logical 
SQL 
Logical 
SQL Logical 
SQL
Master test script 
Define 
Measure 
Analyse 
Review 
Implement 
Test 
script 
BI 
Server 
Data 
nqcmd 
Logical 
SQL 
Test 
script 
nqcmd 
Test 
script 
nqcmd 
Test 
script 
nqcmd
 Simulates user interaction – HTTP traffic 
 Powerful, but can be difficult to set up 
 Ajax complicates things 
 Do you really need to use it? 
 Tools 
 Fiddler2 
 FireBug 
 Reference: 
 My Oracle Support – Doc ID 496417.1 
 http://rnm1978.wordpress.com/category/loadrunner 
Define 
Measure 
Analyse 
Review 
Implement
 Be very clear what the aim of your test is 
 You probably need to define multiple tests 
 Different points on the OBIEE stack to 
interface 
 Pick the most appropriate one 
 Write everything down! 
Define 
Measure 
Analyse 
Review 
Implement
Define 
Measure 
Analyse 
Review 
Implement
Define 
Measure 
Analyse 
Review 
Implement 
Database 
Presentation 
Services 
BI Server 
Apache log Web Server 
App Server 
Presentation 
Services plug-in 
OAS log 
Analytics 
log 
sawserver.log 
NQServer.log 
NQQuery.log 
systems 
management 
Enterprise 
Manager 
BI Management Pack 
Usage 
Tracking 
PerfMon 
(windows 
only) 
jConsole etc 
Presentation 
services 
Enterprise 
Manager 
ASH, AWR, 
SQL Monitor 
Server metrics 
e.g. : IO, CPU, Memory 
PerfMon 
(Windows) 
Oracle OS 
Watcher 
(unix) 
Enterprise 
Manager 
(Oracle)
Define 
Measure 
Analyse 
Review 
Implement
 Lots of different ways to measure 
 Build measurement into your test plan 
 Automate where possible 
▪ Easier 
▪ Less error 
Define 
Measure 
Analyse 
Review 
Implement
Define 
Measure 
Analyse 
Review 
Implement
Define 
Measure 
Analyse 
Review 
Implement
Define 
Measure 
Analyse 
Review 
Implement
Response time 
1 
1 
9 
3 
2 
10 
2 
1 
2 
3 
1 
1 
1 
2 
2 
2 
3 
3 
9 
10 
Average 
(mean) 
3.4 
50th percentile 
(Median) 
2 
90th percentile 
9.1 
Define 
Measure 
Analyse 
Review 
Implement
Define 
Measure 
Analyse 
Review 
Implement 
Dashboard 
Requests 
Logical SQL • ORA_HASH(QUERY_TEXT) 
Physical SQL • SQL IDs 
Execution plan • Execution plan hash id
S_NQ_ACCT 
START_TS 
ROW_COUNT 
TOTAL_TIME_SEC 
NUM_DB_QUERY 
QUERY_TEXT 
QUERY_SRC_CD 
SAW_SRC_PATH 
SAW_DASHBOARD 
OBIEE_REPLAY_STATEMENTS 
qt_ora_hash 
query_text 
saw_path 
dashboard 
OBIEE_REPLAY_STATS 
testid 
testenv 
qt_ora_hash 
start_ts 
response_time 
row_count 
db_query_cnt 
Define 
Measure 
Analyse 
Review 
Implement
Define 
Measure 
Analyse 
Review 
Implement 
Evaluate design / 
config options 
Do it right 
Don’t “fudge it” 
Timebox!
Define 
Measure 
Analyse 
Review 
Implement 
Redefine 
test 
Continue 
testing 
Implement
Define 
Measure 
Analyse 
Review 
Implement
Define 
Measure 
Analyse 
Review 
Implement
 You won’t get your testing right first time 
 There’s no shame in that 
 Don’t cook the books 
▪ Better to redefine your test than invalidate its results 
 Stick to the methodology 
 Don’t move the goalposts 
 Very tempting to pick off the “low-hanging fruit” 
▪ If you do, make sure you don’t get indigestion… 
 Timebox 
 Test your implementation!
Define 
Measure 
Analyse 
Review 
Implement 
Evaluate design / 
config options 
Do it right 
Don’t “fudge it” 
Do more testing 
Redefine 
test Do more testing
OBIEE Introduction By QuontraSolutions

More Related Content

More from Quontra Solutions

Java Constructors with examples - Quontra Solutions
Java Constructors with examples  - Quontra SolutionsJava Constructors with examples  - Quontra Solutions
Java Constructors with examples - Quontra SolutionsQuontra Solutions
 
Oracle-12c Online Training by Quontra Solutions
 Oracle-12c Online Training by Quontra Solutions Oracle-12c Online Training by Quontra Solutions
Oracle-12c Online Training by Quontra SolutionsQuontra Solutions
 
Test Automation Framework Online Training by QuontraSolutions
Test Automation Framework Online Training by QuontraSolutionsTest Automation Framework Online Training by QuontraSolutions
Test Automation Framework Online Training by QuontraSolutionsQuontra Solutions
 
Automation with Selenium Presented by Quontra Solutions
Automation with Selenium Presented by Quontra SolutionsAutomation with Selenium Presented by Quontra Solutions
Automation with Selenium Presented by Quontra SolutionsQuontra Solutions
 
Automated Software Testing Framework Training by Quontra Solutions
Automated Software Testing Framework Training by Quontra SolutionsAutomated Software Testing Framework Training by Quontra Solutions
Automated Software Testing Framework Training by Quontra SolutionsQuontra Solutions
 
DataMining and OLAP Technology Concepts Presented By Quontra Solutions
DataMining and OLAP Technology Concepts Presented By Quontra SolutionsDataMining and OLAP Technology Concepts Presented By Quontra Solutions
DataMining and OLAP Technology Concepts Presented By Quontra SolutionsQuontra Solutions
 
Network security by quontra solutions uk
Network security by quontra solutions ukNetwork security by quontra solutions uk
Network security by quontra solutions ukQuontra Solutions
 
Introduction to .net FrameWork by QuontraSolutions
Introduction to .net FrameWork by QuontraSolutionsIntroduction to .net FrameWork by QuontraSolutions
Introduction to .net FrameWork by QuontraSolutionsQuontra Solutions
 
Informatica Metadata Exchange Frequently Asked Questions by Quontra Solutions
Informatica Metadata Exchange Frequently Asked Questions by Quontra SolutionsInformatica Metadata Exchange Frequently Asked Questions by Quontra Solutions
Informatica Metadata Exchange Frequently Asked Questions by Quontra SolutionsQuontra Solutions
 
Informatica metadata exchange frequently asked questions by quontra solutions
Informatica metadata exchange frequently asked questions by quontra solutionsInformatica metadata exchange frequently asked questions by quontra solutions
Informatica metadata exchange frequently asked questions by quontra solutionsQuontra Solutions
 
Dataware house Introduction By Quontra Solutions
Dataware house Introduction By Quontra SolutionsDataware house Introduction By Quontra Solutions
Dataware house Introduction By Quontra SolutionsQuontra Solutions
 
Selenium overview ppt by quontra solutions
Selenium overview ppt by quontra solutionsSelenium overview ppt by quontra solutions
Selenium overview ppt by quontra solutionsQuontra Solutions
 

More from Quontra Solutions (13)

Java Constructors with examples - Quontra Solutions
Java Constructors with examples  - Quontra SolutionsJava Constructors with examples  - Quontra Solutions
Java Constructors with examples - Quontra Solutions
 
Oracle-12c Online Training by Quontra Solutions
 Oracle-12c Online Training by Quontra Solutions Oracle-12c Online Training by Quontra Solutions
Oracle-12c Online Training by Quontra Solutions
 
Test Automation Framework Online Training by QuontraSolutions
Test Automation Framework Online Training by QuontraSolutionsTest Automation Framework Online Training by QuontraSolutions
Test Automation Framework Online Training by QuontraSolutions
 
Enterprise java beans
Enterprise java beansEnterprise java beans
Enterprise java beans
 
Automation with Selenium Presented by Quontra Solutions
Automation with Selenium Presented by Quontra SolutionsAutomation with Selenium Presented by Quontra Solutions
Automation with Selenium Presented by Quontra Solutions
 
Automated Software Testing Framework Training by Quontra Solutions
Automated Software Testing Framework Training by Quontra SolutionsAutomated Software Testing Framework Training by Quontra Solutions
Automated Software Testing Framework Training by Quontra Solutions
 
DataMining and OLAP Technology Concepts Presented By Quontra Solutions
DataMining and OLAP Technology Concepts Presented By Quontra SolutionsDataMining and OLAP Technology Concepts Presented By Quontra Solutions
DataMining and OLAP Technology Concepts Presented By Quontra Solutions
 
Network security by quontra solutions uk
Network security by quontra solutions ukNetwork security by quontra solutions uk
Network security by quontra solutions uk
 
Introduction to .net FrameWork by QuontraSolutions
Introduction to .net FrameWork by QuontraSolutionsIntroduction to .net FrameWork by QuontraSolutions
Introduction to .net FrameWork by QuontraSolutions
 
Informatica Metadata Exchange Frequently Asked Questions by Quontra Solutions
Informatica Metadata Exchange Frequently Asked Questions by Quontra SolutionsInformatica Metadata Exchange Frequently Asked Questions by Quontra Solutions
Informatica Metadata Exchange Frequently Asked Questions by Quontra Solutions
 
Informatica metadata exchange frequently asked questions by quontra solutions
Informatica metadata exchange frequently asked questions by quontra solutionsInformatica metadata exchange frequently asked questions by quontra solutions
Informatica metadata exchange frequently asked questions by quontra solutions
 
Dataware house Introduction By Quontra Solutions
Dataware house Introduction By Quontra SolutionsDataware house Introduction By Quontra Solutions
Dataware house Introduction By Quontra Solutions
 
Selenium overview ppt by quontra solutions
Selenium overview ppt by quontra solutionsSelenium overview ppt by quontra solutions
Selenium overview ppt by quontra solutions
 

Recently uploaded

microwave assisted reaction. General introduction
microwave assisted reaction. General introductionmicrowave assisted reaction. General introduction
microwave assisted reaction. General introductionMaksud Ahmed
 
Accessible design: Minimum effort, maximum impact
Accessible design: Minimum effort, maximum impactAccessible design: Minimum effort, maximum impact
Accessible design: Minimum effort, maximum impactdawncurless
 
POINT- BIOCHEMISTRY SEM 2 ENZYMES UNIT 5.pptx
POINT- BIOCHEMISTRY SEM 2 ENZYMES UNIT 5.pptxPOINT- BIOCHEMISTRY SEM 2 ENZYMES UNIT 5.pptx
POINT- BIOCHEMISTRY SEM 2 ENZYMES UNIT 5.pptxSayali Powar
 
Industrial Policy - 1948, 1956, 1973, 1977, 1980, 1991
Industrial Policy - 1948, 1956, 1973, 1977, 1980, 1991Industrial Policy - 1948, 1956, 1973, 1977, 1980, 1991
Industrial Policy - 1948, 1956, 1973, 1977, 1980, 1991RKavithamani
 
Separation of Lanthanides/ Lanthanides and Actinides
Separation of Lanthanides/ Lanthanides and ActinidesSeparation of Lanthanides/ Lanthanides and Actinides
Separation of Lanthanides/ Lanthanides and ActinidesFatimaKhan178732
 
Introduction to ArtificiaI Intelligence in Higher Education
Introduction to ArtificiaI Intelligence in Higher EducationIntroduction to ArtificiaI Intelligence in Higher Education
Introduction to ArtificiaI Intelligence in Higher Educationpboyjonauth
 
Introduction to AI in Higher Education_draft.pptx
Introduction to AI in Higher Education_draft.pptxIntroduction to AI in Higher Education_draft.pptx
Introduction to AI in Higher Education_draft.pptxpboyjonauth
 
Hybridoma Technology ( Production , Purification , and Application )
Hybridoma Technology  ( Production , Purification , and Application  ) Hybridoma Technology  ( Production , Purification , and Application  )
Hybridoma Technology ( Production , Purification , and Application ) Sakshi Ghasle
 
Sanyam Choudhary Chemistry practical.pdf
Sanyam Choudhary Chemistry practical.pdfSanyam Choudhary Chemistry practical.pdf
Sanyam Choudhary Chemistry practical.pdfsanyamsingh5019
 
Advanced Views - Calendar View in Odoo 17
Advanced Views - Calendar View in Odoo 17Advanced Views - Calendar View in Odoo 17
Advanced Views - Calendar View in Odoo 17Celine George
 
Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...
Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...
Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...EduSkills OECD
 
Beyond the EU: DORA and NIS 2 Directive's Global Impact
Beyond the EU: DORA and NIS 2 Directive's Global ImpactBeyond the EU: DORA and NIS 2 Directive's Global Impact
Beyond the EU: DORA and NIS 2 Directive's Global ImpactPECB
 
A Critique of the Proposed National Education Policy Reform
A Critique of the Proposed National Education Policy ReformA Critique of the Proposed National Education Policy Reform
A Critique of the Proposed National Education Policy ReformChameera Dedduwage
 
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...Krashi Coaching
 
The basics of sentences session 2pptx copy.pptx
The basics of sentences session 2pptx copy.pptxThe basics of sentences session 2pptx copy.pptx
The basics of sentences session 2pptx copy.pptxheathfieldcps1
 
1029-Danh muc Sach Giao Khoa khoi 6.pdf
1029-Danh muc Sach Giao Khoa khoi  6.pdf1029-Danh muc Sach Giao Khoa khoi  6.pdf
1029-Danh muc Sach Giao Khoa khoi 6.pdfQucHHunhnh
 
Arihant handbook biology for class 11 .pdf
Arihant handbook biology for class 11 .pdfArihant handbook biology for class 11 .pdf
Arihant handbook biology for class 11 .pdfchloefrazer622
 
Student login on Anyboli platform.helpin
Student login on Anyboli platform.helpinStudent login on Anyboli platform.helpin
Student login on Anyboli platform.helpinRaunakKeshri1
 
URLs and Routing in the Odoo 17 Website App
URLs and Routing in the Odoo 17 Website AppURLs and Routing in the Odoo 17 Website App
URLs and Routing in the Odoo 17 Website AppCeline George
 

Recently uploaded (20)

microwave assisted reaction. General introduction
microwave assisted reaction. General introductionmicrowave assisted reaction. General introduction
microwave assisted reaction. General introduction
 
Accessible design: Minimum effort, maximum impact
Accessible design: Minimum effort, maximum impactAccessible design: Minimum effort, maximum impact
Accessible design: Minimum effort, maximum impact
 
POINT- BIOCHEMISTRY SEM 2 ENZYMES UNIT 5.pptx
POINT- BIOCHEMISTRY SEM 2 ENZYMES UNIT 5.pptxPOINT- BIOCHEMISTRY SEM 2 ENZYMES UNIT 5.pptx
POINT- BIOCHEMISTRY SEM 2 ENZYMES UNIT 5.pptx
 
Industrial Policy - 1948, 1956, 1973, 1977, 1980, 1991
Industrial Policy - 1948, 1956, 1973, 1977, 1980, 1991Industrial Policy - 1948, 1956, 1973, 1977, 1980, 1991
Industrial Policy - 1948, 1956, 1973, 1977, 1980, 1991
 
Separation of Lanthanides/ Lanthanides and Actinides
Separation of Lanthanides/ Lanthanides and ActinidesSeparation of Lanthanides/ Lanthanides and Actinides
Separation of Lanthanides/ Lanthanides and Actinides
 
Introduction to ArtificiaI Intelligence in Higher Education
Introduction to ArtificiaI Intelligence in Higher EducationIntroduction to ArtificiaI Intelligence in Higher Education
Introduction to ArtificiaI Intelligence in Higher Education
 
Introduction to AI in Higher Education_draft.pptx
Introduction to AI in Higher Education_draft.pptxIntroduction to AI in Higher Education_draft.pptx
Introduction to AI in Higher Education_draft.pptx
 
Hybridoma Technology ( Production , Purification , and Application )
Hybridoma Technology  ( Production , Purification , and Application  ) Hybridoma Technology  ( Production , Purification , and Application  )
Hybridoma Technology ( Production , Purification , and Application )
 
Sanyam Choudhary Chemistry practical.pdf
Sanyam Choudhary Chemistry practical.pdfSanyam Choudhary Chemistry practical.pdf
Sanyam Choudhary Chemistry practical.pdf
 
Advanced Views - Calendar View in Odoo 17
Advanced Views - Calendar View in Odoo 17Advanced Views - Calendar View in Odoo 17
Advanced Views - Calendar View in Odoo 17
 
Mattingly "AI & Prompt Design: The Basics of Prompt Design"
Mattingly "AI & Prompt Design: The Basics of Prompt Design"Mattingly "AI & Prompt Design: The Basics of Prompt Design"
Mattingly "AI & Prompt Design: The Basics of Prompt Design"
 
Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...
Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...
Presentation by Andreas Schleicher Tackling the School Absenteeism Crisis 30 ...
 
Beyond the EU: DORA and NIS 2 Directive's Global Impact
Beyond the EU: DORA and NIS 2 Directive's Global ImpactBeyond the EU: DORA and NIS 2 Directive's Global Impact
Beyond the EU: DORA and NIS 2 Directive's Global Impact
 
A Critique of the Proposed National Education Policy Reform
A Critique of the Proposed National Education Policy ReformA Critique of the Proposed National Education Policy Reform
A Critique of the Proposed National Education Policy Reform
 
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
Kisan Call Centre - To harness potential of ICT in Agriculture by answer farm...
 
The basics of sentences session 2pptx copy.pptx
The basics of sentences session 2pptx copy.pptxThe basics of sentences session 2pptx copy.pptx
The basics of sentences session 2pptx copy.pptx
 
1029-Danh muc Sach Giao Khoa khoi 6.pdf
1029-Danh muc Sach Giao Khoa khoi  6.pdf1029-Danh muc Sach Giao Khoa khoi  6.pdf
1029-Danh muc Sach Giao Khoa khoi 6.pdf
 
Arihant handbook biology for class 11 .pdf
Arihant handbook biology for class 11 .pdfArihant handbook biology for class 11 .pdf
Arihant handbook biology for class 11 .pdf
 
Student login on Anyboli platform.helpin
Student login on Anyboli platform.helpinStudent login on Anyboli platform.helpin
Student login on Anyboli platform.helpin
 
URLs and Routing in the Odoo 17 Website App
URLs and Routing in the Odoo 17 Website AppURLs and Routing in the Odoo 17 Website App
URLs and Routing in the Odoo 17 Website App
 

OBIEE Introduction By QuontraSolutions

  • 1. Performance Testing and OBIEE By Quontra Solutions
  • 2.  Oracle BI specialist at Morrisons plc  Big IT development programme at its early stages implementing OBIEE, OBIA, ORDM, all on Oracle 11g & HP-UX
  • 3.  A Performance Tuning Methodology  OBIEE techie stuff  Learn from my mistakes!
  • 4.  Response times  Report  ETL batch  OLTP transaction  System impact  Resource usage  Scalability
  • 5.  Check that your system performs  Are the users going to be happy?  Baseline  How fast is fast? ▪ How slow is slow?  Validate system design  Do it right, first time  Capacity planning
  • 6.  It’s never too late  “You’ll never catch all your problems in pre-production testing. That’s why you need a reliable and efficient method for solving the problems that leak through your pre-production testing processes.”
  • 7.  Because it makes you better at your job  “At the very least, your performance test plan will make you a more competent diagnostician (and clearer thinker) when it comes time to fix the performance problems that will inevitably occur during production operation.”
  • 8.  Quantifying response times  System impact  User expectations  Problem diagnosis  Design validation
  • 9. Define Measure Analyse Review Implement Timebox! Evaluate design / config options Do it right Don’t “fudge it” Do more testing Redefine test Do more testing
  • 10.  Define – what are you going to test • Aim of the test • Scope • Assumptions • Specifics • Data, environment, etc  Build – how are you going to test it  OBIEE specific Define Measure Analyse Review Implement •E.g. : •Check that the system performs •Baseline performance •Prove system capacity •Validate system design
  • 11. Define Measure Analyse Review Implement MoFrew coemr cpoomnpeonntse n= tms =o reea csoiemr ptole mx a=n magoere = v maroiareb lperse =c ilsaer g=e mr moraer gefinfi coife nertror
  • 12. Database Presentation Services BI Server Report / Dashboard Logical SQL Physical SQL statement(s) Data set(s) Data set Rendered report Excludes App/Web server & presentation services plug-in Define Measure Analyse Review Implement
  • 13. Database Presentation Services nqcmd BI Server SQL Client Physical SQL LSQL Physical SQL User & Stopwatch Load Testing tool (eg. LoadRunner, OATS) Define Measure Analyse Review Implement
  • 14. Define Measure Analyse Review Implement Usage Tracking or NQQuery.log Test script BI Server Data nqcmd Logical SQL Logical SQL Logical SQL
  • 15. Master test script Define Measure Analyse Review Implement Test script BI Server Data nqcmd Logical SQL Test script nqcmd Test script nqcmd Test script nqcmd
  • 16.  Simulates user interaction – HTTP traffic  Powerful, but can be difficult to set up  Ajax complicates things  Do you really need to use it?  Tools  Fiddler2  FireBug  Reference:  My Oracle Support – Doc ID 496417.1  http://rnm1978.wordpress.com/category/loadrunner Define Measure Analyse Review Implement
  • 17.  Be very clear what the aim of your test is  You probably need to define multiple tests  Different points on the OBIEE stack to interface  Pick the most appropriate one  Write everything down! Define Measure Analyse Review Implement
  • 18. Define Measure Analyse Review Implement
  • 19. Define Measure Analyse Review Implement Database Presentation Services BI Server Apache log Web Server App Server Presentation Services plug-in OAS log Analytics log sawserver.log NQServer.log NQQuery.log systems management Enterprise Manager BI Management Pack Usage Tracking PerfMon (windows only) jConsole etc Presentation services Enterprise Manager ASH, AWR, SQL Monitor Server metrics e.g. : IO, CPU, Memory PerfMon (Windows) Oracle OS Watcher (unix) Enterprise Manager (Oracle)
  • 20. Define Measure Analyse Review Implement
  • 21.  Lots of different ways to measure  Build measurement into your test plan  Automate where possible ▪ Easier ▪ Less error Define Measure Analyse Review Implement
  • 22. Define Measure Analyse Review Implement
  • 23. Define Measure Analyse Review Implement
  • 24.
  • 25.
  • 26.
  • 27. Define Measure Analyse Review Implement
  • 28. Response time 1 1 9 3 2 10 2 1 2 3 1 1 1 2 2 2 3 3 9 10 Average (mean) 3.4 50th percentile (Median) 2 90th percentile 9.1 Define Measure Analyse Review Implement
  • 29. Define Measure Analyse Review Implement Dashboard Requests Logical SQL • ORA_HASH(QUERY_TEXT) Physical SQL • SQL IDs Execution plan • Execution plan hash id
  • 30. S_NQ_ACCT START_TS ROW_COUNT TOTAL_TIME_SEC NUM_DB_QUERY QUERY_TEXT QUERY_SRC_CD SAW_SRC_PATH SAW_DASHBOARD OBIEE_REPLAY_STATEMENTS qt_ora_hash query_text saw_path dashboard OBIEE_REPLAY_STATS testid testenv qt_ora_hash start_ts response_time row_count db_query_cnt Define Measure Analyse Review Implement
  • 31. Define Measure Analyse Review Implement Evaluate design / config options Do it right Don’t “fudge it” Timebox!
  • 32. Define Measure Analyse Review Implement Redefine test Continue testing Implement
  • 33. Define Measure Analyse Review Implement
  • 34. Define Measure Analyse Review Implement
  • 35.  You won’t get your testing right first time  There’s no shame in that  Don’t cook the books ▪ Better to redefine your test than invalidate its results  Stick to the methodology  Don’t move the goalposts  Very tempting to pick off the “low-hanging fruit” ▪ If you do, make sure you don’t get indigestion…  Timebox  Test your implementation!
  • 36. Define Measure Analyse Review Implement Evaluate design / config options Do it right Don’t “fudge it” Do more testing Redefine test Do more testing

Editor's Notes

  1. It’s not too late to put in place a solid performance test methodology around an existing system. If you set up your performance tests now you will have a set of baselines and a full picture of how your system behaves normally Then when it breaks or someone complains you’re already set to deal with it If you don’t, then when you do have problems you have to start from scratch, finding your way through the process. Which is better – at your leisure, or with the proverbial gun of unhappy users held to your head? Performance testing isn’t optional. It’s mandatory. It’s just up to you when you do it. Even if you’re running Exadata or similar and a so confident that you don’t have performance problems and never will – how are you going to capacity plan? Do you know how your system currently behaves in terms of CPU , IO, etc? How many more users can you run? Which is easier, simulate and calculate up front, or wait until it starts creaking?
  2. Performance Testing requires an extremely thorough understanding of the system. If you do it properly there is no doubt you will come out of it better equipped to support and develop the system further
  3. Any questions so far?
  4. This is the methodology we’ve developed It’s a high level view – subsequent slides will give detail
  5. Any questions?
  6. Once you’ve defined your test you need to execute it – and measure the results Here’s the ways you should consider measuring the different parts of the stack
  7. Once you’ve decided how much of the stack you’re going to test, you need to set about designing the test and how you’re going to capture your metrics Performance tests are all about collecting metrics that allow you to make statistically valid and quantifiable conclusions about your system. The primary metric of interest is time. What’s the end-to-end response time, from request to answer, and where’s the time in between spent? If a user complains that a report take five minutes to run but the DBA says they don’t see the query hit the database for the first two, and then it executes in 30 seconds, what’s happened to the other two and a half minutes? Other metrics of interest are the environmental statistics like CPU, memory, and IO, and diagnostic statistics such as the execution plan on the database and lower-level information like buffer gets etc. So – from the top down: [click] Web server, eg. Apache log – first log of the user request coming in App server, eg. OAS Presentation Serivces plugin, Analytics – (this is where you see the error logs when you get 500 Internal Server Error from analytics) [click] sawserver.log - by default this doesn’t record that much, but by changing the logconfig.xml file you can enable extremely detailed logging. This is useful for diagnosing lots of problems, but also if you’re looking to do an accurate profile of where the time in an Answers request is spent. You can see when it receives the user request, when it sends on the logical SQL to the bI Server, and when it receives the data back See http://rnm1978.wordpress.com/category/log/ for details [click] BI Server – spoilt for choice here. For a production environment I strongly recommend enabling Usage Tracking. For performance work you should also be using NQQuery.log where the variable levels of logging show you logical and physical SQL, BI Server execution plans, response times for each database query run, etc. [click] As well as these two features there is the systemsmanagement functionality which exposes some very detailed counters through windows PerfMon or the BI Management Pack for OEM. You can also use the jmx protocol to access the data through clients like Jconsole or jManage [click] For the database all the standard monitoring practices apply, depending on what your database is. For Oracle you should be using OEM, ASH, SQL Monitor, etc. [click] And finally, for getting a complete picture of the stack’s performance -- Speak to your users! Maybe not as empirically valid as the other components, but just as important.
  8. Useful when prob is suspected on DB, only place that individual physical SQL query response times are kept Database query times & row counts
  9. EM is good For pure testing – need to capture data : SQL Tuning Sets Good for capturing behaviour of a set of SQL over time – longest running, most IO, etc Less good for focussing on individual queries because stats are aggregated SQL Monitor – export from EM (next slide) +++++++++++++++++++++++++++++++++++ SQL Server DMVs, SQL Profiler, PerfMon counters, etc Other RDBMS – pass 
  10. Got to mention this – from EM you can export a standalone HTML file that renders like this brilliant
  11. Lots of different ways to measure decide what metrics are relevant to your testing Load testing – system metrics v. Important Perf testing indiv report – maybe just response time Plan your measurements as part of the test Trigger collection scripts automagically Include manual collection in test instructions
  12. Analysis step is : Collate data store in sensible way - Raw data - label your tests better to use a non-meaningful label analyse it - visualisation - analysis will depend on aim of test - eg loadtesting – identify bottlenecks
  13. [click] raw data, [click] compared to a previous baseline – illustrate varience [click] host metrics - IO graph [click] response time, over time
  14. Testing – understand more about system as you go – prob want to redefine test – that’s part of the process! Performance Testing is an iterative process. I can’t stress this enough. You will not get it right the first time you do it Whatever you do, you’ll probably miss something or invalidate your tests. Remember that an iterative approach is entirely valid, don’t feel you “got it wrong” and have to fudge the results to cover your mistake. Better to abandon a test and learn from the mistake than produce a “perfect” test that’s complete rubbish. Stick to method Benefit of also enforces justification for changes, avoid “we’ve always done it that way” don’t move the goalposts. You might find some horrible queries as you dig into them you notice some obvious “quick wins” If you rush the fix in without completing your first round of testing, you risk invalidating it BE METHODICAL!!!! Timebox the execute/measure/analyse iterations -don’t get lost in diminishing returns It’s a good idea to timebox your work, and have regular review points Test your implementation! parallel config, not tested properly after implement in test env, nearly got to prod without realising Don’t get so bogged down in the detail that you miss the wood for the trees You can end up focussing on perfecting one element of the system at the expense of all the others.
  15. This presentation has shown you how to run big workloads against your OBIEE system But, resist the temptation to dash off and see what happens when you run a thousand users against your system at once. It’ll be fun, but ultimately a waste of time. You have to define what you’re going to do. You need to define what the ultimate aim is. Are you proving a system performs to specific user requirements? In which case your test definition is almost written for you, you just have to fill in the gaps If you’re building a performance test for best-practice and all the good reasons I spoke about before then you need to think carefully about what you’ll test. What’s a representative sample of the system’s workload? For example: - Analyse existing usage, pick the most frequently run reports SPEAK TO YOUR USERS! Which reports do they care about? Be wary of only analysing the reports that users complain about though – you want to be colleting lots and lots of good metrics. What happens when you fix the slow reports – the old “fast” reports will now appear slow in comparison, so you want to have some baselines for them too I can’t stress this strongly enough. Cary Millsap writes excellently on the whole subject of performance. I can’t recommend highly enough his paper “Thinking Clearly About Performance”, as well as many of the articles on his blog. http://carymillsap.blogspot.com/2010/02/thinking-clearly-about-performance.html There are books and books written on how you should approach performance testing and tuning, people like Mr Millsap have built their whole careers around it. It’s way outside the scope of this, but I believe it’s essential to understand the approach to follow, otherwise all your testing can be in vain. It’s not the same as dashing off an OBIEE report that you can bin and recreate next week. Imagine designing your DW schema without good modelling knowledge – or think of ones that you’ve worked with where the person who created it didn’t understand what they were doing. The wasted time and misleading results can be potentially disastrous if you don’t get it right up front. Take my word for it – time invested up front reading and understanding will repay itself ten-fold. Preaching over.
  16. Questions & Discussion