Your SlideShare is downloading. ×
0
Madrid, 26th to 28th May 2014
Reality Shock:
Performance Testing
Jose Aracil - Operations Manager
jose.aracil@globetesting...
Madrid, 26th to 28th May 2014
A radically different kind of user
System-centric User-centric
Users born after 1980
Mainfra...
Madrid, 26th to 28th May 2014
2014
A tipping point for app. delivery
“New application
projects surge in 2014
and beyond.”
...
Madrid, 26th to 28th May 2014
Changes are coming faster
Cloud and composite applications are increasingly complex
Customer...
Madrid, 26th to 28th May 2014
Dev/test lab
Developer
And the need for speed changes everything
Tester
Agile Development
“T...
Madrid, 26th to 28th May 2014
Continuous Testing for
faster app delivery
“Shift-Left”
With Confidence
Now Continuous Testi...
Madrid, 26th to 28th May 2014
Madrid, 26th to 28th May 2014
Application usage has
changed
Performance testing must change
Application development
has ch...
Madrid, 26th to 28th May 2014
You must
leave the black
box
Madrid, 26th to 28th May 2014
Sorry… I changed my mind!
Madrid, 26th to 28th May 2014
Sorry… I changed my mind!
updated
Madrid, 26th to 28th May 2014
Planning Test Strategy
Reporting Final report
Preparation
Tool setupTest Env. + Data Test ca...
Madrid, 26th to 28th May 2014
Planning
Preparation
Execution
Reporting
Test Strategy
Tool setupTest Env. + Data Test case ...
Madrid, 26th to 28th May 2014
Good for one project
Madrid, 26th to 28th May 2014
Good for one project
Madrid, 26th to 28th May 2014
New set of problems…
Multiple
projects
Continuous
Integration
Auditing
Multiple people
Deplo...
Madrid, 26th to 28th May 2014
Project based testing challenges
Project A
HP LoadRunner 8.1
1500 Vusers
Project B
HP LoadRu...
Madrid, 26th to 28th May 2014
End to end
Search PayBuy Ship Track
EXAMPLE:
The Online Shopping
Process
Product PaymentCust...
Madrid, 26th to 28th May 2014
Virtualisation Four-legged table
User
Network
Service
Data
User
• Virtualise user conditions...
Madrid, 26th to 28th May 2014
We are now able to truly apply a
agile approached to load testing
given the reduction in scr...
Madrid, 26th to 28th May 2014
Enable Agile performance Delivery
User
Test
Build Load Test scripts during Dev
Run Unit Test...
Madrid, 26th to 28th May 2014
Mobile is different
Analyze Mobile Performance on Network
Backend
monitoring
Mobile
performa...
Madrid, 26th to 28th May 2014
Existing Infrastructure
Service Virtualization Concept
Mainframe
JDBC
SOAP
RFC/IDO
C
MQ/CIC
...
Madrid, 26th to 28th May 2014
Existing Infrastructure
Service Virtualization Concept
Virtual services are created to
stand...
Madrid, 26th to 28th May 2014
Service Virtualization is the answer
Functional test
Performance Test
Regression test
Depend...
Madrid, 26th to 28th May 2014
Service Virtualization is the answer
Functional test
Performance Test
Regression test
Depend...
Madrid, 26th to 28th May 2014
Bring production data into performance
testing as baseline
• Collect, analyze and compare
pe...
Madrid, 26th to 28th May 2014
Data
Virtualise production data and
transaction responses to simulate
application request an...
Madrid, 26th to 28th May 2014
3 Keys to better
performance
• Test early
• Test often
• Monitor and leverage
data from moni...
Madrid, 26th to 28th May 2014
Q&A
Madrid, 26th to 28th May 2014
IF YOU LIKE IT
PLEASE RATE ME
Visit our blog to get all the slides
WWW.GLOBETESTING.COM
jose...
Upcoming SlideShare
Loading in...5
×

Reality shock j aracil

300

Published on

Por qué los planteamientos de pruebas de rendimiento tradicionales ya no funcionan, y cuál es el acercamiento adecuado para asegurar que los sistemas soportan la carga esperada cuando pasan a producción.

Published in: Software, Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total Views
300
On Slideshare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Transcript of "Reality shock j aracil"

  1. 1. Madrid, 26th to 28th May 2014 Reality Shock: Performance Testing Jose Aracil - Operations Manager jose.aracil@globetesting.com @pep0 - @globetesting
  2. 2. Madrid, 26th to 28th May 2014 A radically different kind of user System-centric User-centric Users born after 1980 Mainframe DevicesWebClient/Server Time
  3. 3. Madrid, 26th to 28th May 2014 2014 A tipping point for app. delivery “New application projects surge in 2014 and beyond.” “Social, mobile, analytics, cloud have reached the tipping point in 2014.” 2015 36 per app 2020 120 per app 2010 4 per app Number of releases per app per year • Apps everywhere • Accessed through multiple devices • 30x increase in application releases • Closed feedback loop with customers • Focus on time-to-value and ROI What we expect:
  4. 4. Madrid, 26th to 28th May 2014 Changes are coming faster Cloud and composite applications are increasingly complex Customers demand mobile access services The pace of change is only increasing Project and delivery teams are both agile and global Businesses need applications to connect with: • Customers • Partners • Suppliers…
  5. 5. Madrid, 26th to 28th May 2014 Dev/test lab Developer And the need for speed changes everything Tester Agile Development “The new QA is extensively automated and executed by a continuous integration process”-- Forrester Research Provision environments at “agile speed” Shift left, test continuously Agile Development, weekly drops
  6. 6. Madrid, 26th to 28th May 2014 Continuous Testing for faster app delivery “Shift-Left” With Confidence Now Continuous Testing is more critical than ever • Improve development efficiency • Reduce the test burden • Accelerate application delivery Design Develop Provision Env. Deploy Test App. Run test Test results
  7. 7. Madrid, 26th to 28th May 2014
  8. 8. Madrid, 26th to 28th May 2014 Application usage has changed Performance testing must change Application development has changed
  9. 9. Madrid, 26th to 28th May 2014 You must leave the black box
  10. 10. Madrid, 26th to 28th May 2014 Sorry… I changed my mind!
  11. 11. Madrid, 26th to 28th May 2014 Sorry… I changed my mind! updated
  12. 12. Madrid, 26th to 28th May 2014 Planning Test Strategy Reporting Final report Preparation Tool setupTest Env. + Data Test case design Monitor config. Scripting Smoke Test Execution Baseline Soak Test Load Testing Other tests Stress Performance Testing Análisis+Changes+ Report
  13. 13. Madrid, 26th to 28th May 2014 Planning Preparation Execution Reporting Test Strategy Tool setupTest Env. + Data Test case design Monitor config. Scripting Smoke Test Baseline Soak Test Load Testing Other tests Stress Final report Performance Testing Análisis+Changes+ Report But what get’s in the way? Dependencies
  14. 14. Madrid, 26th to 28th May 2014 Good for one project
  15. 15. Madrid, 26th to 28th May 2014 Good for one project
  16. 16. Madrid, 26th to 28th May 2014 New set of problems… Multiple projects Continuous Integration Auditing Multiple people Deployment of LoadGenerators Dependencies Asset sharing Reporting Security Collaboration Best practices Network conditions Cloud Low application maturity Agile (good and bad)
  17. 17. Madrid, 26th to 28th May 2014 Project based testing challenges Project A HP LoadRunner 8.1 1500 Vusers Project B HP LoadRunner 11.0 1500 Vusers Project C Other load test SW 2500 Vusers Project D No load test budget “Hope for the best” Unrealistic “code freeze” No sharing of best practices & skills – each project is unique and isolated Duplicated resources (HW and SW) in separate projects Some test cases not fully working (missing components) End-to-end behaviour not fully tester
  18. 18. Madrid, 26th to 28th May 2014 End to end Search PayBuy Ship Track EXAMPLE: The Online Shopping Process Product PaymentCustomer Profiles Shipping Track package Public CloudTraditionalManaged CloudComposite APIs enable integration Each have SLAs and performance dependencies Components anywhere– in-house or cloud
  19. 19. Madrid, 26th to 28th May 2014 Virtualisation Four-legged table User Network Service Data User • Virtualise user conditions to simulate real world conditions Network • Virtualise network conditions to simulate production architecture and real user conditions Service • Virtualise services and service interactions to simulate E2E business processes Data • Virtualise production data and transaction responses to simulate application request and responses
  20. 20. Madrid, 26th to 28th May 2014 We are now able to truly apply a agile approached to load testing given the reduction in scripting and build time. Our scripting time is reduced by more than 75% and resource productivity across projects has improved by more that 50%. Source: TechValidate. TVID: B40-ED8-557 User
  21. 21. Madrid, 26th to 28th May 2014 Enable Agile performance Delivery User Test Build Load Test scripts during Dev Run Unit Tests in Load Test Scenarios Add Performance Tests as build steps in continuous integration tools
  22. 22. Madrid, 26th to 28th May 2014 Mobile is different Analyze Mobile Performance on Network Backend monitoring Mobile performance automation Mobile Network characteristics Network Identify bottlenecks before end users Test in both good and BAD network connections Understand impact of mobile on entire system
  23. 23. Madrid, 26th to 28th May 2014 Existing Infrastructure Service Virtualization Concept Mainframe JDBC SOAP RFC/IDO C MQ/CIC S Third Party Application Under Test Mobile App Web browser Composite Application API SAP System Existing database Web service and Legacy application REST Pay-per-transaction Under construction Service
  24. 24. Madrid, 26th to 28th May 2014 Existing Infrastructure Service Virtualization Concept Virtual services are created to stand in for the real service when developers or testers need access Mainframe JDBC SOAP RFC/IDO C MQ/CIC S Third Party Application Under Test Mobile App Web browser Composite Application API SAP System Existing database Web service and Legacy application REST Pay-per-transaction Under construction Service Virtualization SOAP REST JDBC MQ RFC CICS Simulatio n Data Perf. Config Service
  25. 25. Madrid, 26th to 28th May 2014 Service Virtualization is the answer Functional test Performance Test Regression test Dependencies
  26. 26. Madrid, 26th to 28th May 2014 Service Virtualization is the answer Functional test Performance Test Regression test Dependencies Virtualization Allow frequent access to app components, APIs and services Remove constraints with virtual version No access to a component? Use virtual component Enable performance test teams to set up simulated services
  27. 27. Madrid, 26th to 28th May 2014 Bring production data into performance testing as baseline • Collect, analyze and compare performance data from production • Updated log file analysis of production data for IIS • Integrate with Google Analytics DataPerformance application lifecycle Value: Performance testing closer to reality with accurate, realistic, reliable test: reduce risk and improve test results
  28. 28. Madrid, 26th to 28th May 2014 Data Virtualise production data and transaction responses to simulate application request and responses Data
  29. 29. Madrid, 26th to 28th May 2014 3 Keys to better performance • Test early • Test often • Monitor and leverage data from monitoring as baseline for test The NEW Reality User Network Service Data
  30. 30. Madrid, 26th to 28th May 2014 Q&A
  31. 31. Madrid, 26th to 28th May 2014 IF YOU LIKE IT PLEASE RATE ME Visit our blog to get all the slides WWW.GLOBETESTING.COM jose.aracil@globetesting.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×