EXTENT Conference October 2011 - Test Automation for Trading Systems Presentation by Sibgatullina Taya QA Solution Manager HP Software CEE

1. Cost of Quality
How to Save money
Sibgatullina Taisiya
Solution Consultant, CEE

2. Part 1: Quality assurance
2 HP Application Lifecycle Management & HP Quality Center 11.00

3. Cost Of Quality
3 HP Application Lifecycle Management & HP Quality Center 11.00

4. Application Lifecycle model
4 HP Application Lifecycle Management & HP Quality Center 11.00

5. ALM
5 HP Application Lifecycle Management & HP Quality Center 11.00
5

6. Market Leading Solutions
IDC Worldwide Automated Software Quality Revenue by Vendor in „09
“As the market revenue leader, with a 41.1% share, HP is in a
position to disproportionately influence the overall ASQ market.”
Others
HP
(41.1%)
(Combined)
32.1%
Empirix
1.9%
Borland
2.9%
Compuware
5.3%
Microsoft IBM (Rational)
5.6% 11.1%
Source: Melinda Ballou IDC, “Worldwide Automated Software Quality 2009-2013 Forecast Update and 2008 Vendor Shares:
Evolving Quality in a Challenging Market” (Doc # 219695E) October 2009

7. HP ALM #1 AGAIN
New Gartner report just published in January 2011
7 ©2009 HP Confidential

8. HP ALM 11.00 is a Unified Platform
Designed for the End-to-End Application Lifecycle
Application Lifecycle Management
Integrations into Strategy and Planning
Project Planning & Tracking Enterprise Release Management
Integrations into Run
(Deployment, APM, ITSM)
Requirements Development Quality
Management Management Management
(PPM, Gov)
IDE, SCM, Defect Secure
Requirements Requirements
Build Mgmt Develop- Functional Performance Security
Definition Management
Integrations ment
Foundation
Process Standardization Extensibility
Traceability (i.e. Waterfall, Agile) Reporting (Open APIs)
Unified for the Core App Lifecycle, Integrated with the Complete App Lifecycle

9. HP Application Lifecycle Management
A single platform for managing the application lifecycle
Dashboard
HP Application Lifecycle Management
Project Planning Release Requirements Business Test Plan Test Lab Defect
& Tracking Management Management Models Management
HP Unified Functional Performance Optimization Application Business Application Services
Testing Security Testing Validation
HP QuickTest HP Service HP Performance HP HP Business Process Service Test
HP QAInspect
Professional Test Center LoadRunner Testing Management
Test Data Management
ALM Platform
Process Standardization
Single Data Repository Asset Sharing & Reuse Reporting Open APIs (REST)
(i.e. Waterfall, Agile)

10. Track & measure project milestones and KPIs
Project Planning & Tracking
Defects High 10
Defects Med 25
Requirements Test Auth 93%
Defects High 45
Progress Test Exec 98%
Defects Med 95
Tests Passed 89%
Defects High 0
Req Reviewed 92% Test Exec 78%
Req Approved Tests Passed E2E Testing Defects Med 15
Development
75% 90%
Test Exec 98%
Progress Testing Tests Passed 92%
Req Implemented 85% Progress Readiness
Tests Reviewed 95%
Assessment
Development
Requirements Test Integration User
Authoring Execution E2E Testing Acceptance
Test Authoring Testing
Functional, Performance and Security
10 HP Application Lifecycle Management & HP Quality Center 11.00

11. Requirements Management
Integrated solution for defining, managing and tracking requirements
– Business analyst can collaborate on application
requirements without fear of overriding other‟s
work
– Assess overall risk to the business through real-
time visibility of test coverage and associated
defects against business needs
– Assess the impact of change before changing
requirements
– Easy adoption by Business Analysts
– Define and track multiple requirement types
– Establish templates to ensure higher quality
requirements
– Manage requirements dependencies
– Bi-directional traceability from requirements to
requirements, tests and defects
– Manage change requests through formal
approval process
– Familiar rich test editor interface
– Version Control and Baselining
– Leverage existing assets in MS Word
11

12. Baselining
Track changes made to entities in your project over time
– Compare releases and cycles
– A snapshot of the project state at a
particular point in time
– Enable rollback of assets to key points in the
application lifecycle
– Compare baselines to assess changes
– Ability to use baselines in test execution
– Create baselines – including traceability
between assets
– Compare baselines both at high level and
drill down to specific changes including
traceability
– Report and print baselines for sign-off
– Control who can create and modify
baselines
12

13. Release & Cycle Management
Track and measure the progress and quality of releases and cycles
– Manage projects and assets against release
and cycles
– Track release progress
– Gain visibility into release readiness and
release quality
– Compare releases and cycles for continuous
process improvement
– Manage requirements & tests according to
application releases and dev/QA cycles
– Compare planned vs. actual progress for
each application release and cycle
– Assess release quality across multiple cycles
13

14. Project Planning & Tracking
Track and measure project milestones and KPIs
– Visibility into application projects progress
and status
– Visibility into key project milestones
– Identify delays or poor quality before it
impacts the schedule
– Proactively adjust plans when a project is
not on track
– Define and track project milestones
– Define and measure key performance
indicators across the application lifecycle
– Automatically updates project status against
milestones
– Drill down on KPIs for more details
14

15. Test Plan
Define, manage and track all test scripts
– Create test cases to adequately test the
requirements
– Manage all test types (Functional,
Performance, Security) in one place
– Ensure requirements coverage through
traceability to requirements
– Manage change associated with
requirements changes
– Design, build and manage manual and
automated test cases
– Link test cases to requirements
– Leverage test cases from MS Word and MS
Excel
– Version Control and Baselining
15

16. HP Sprinter
The next generation of manual testing
– Accelerate manual testing
– Improved testing accuracy
– Better communication of defects to development
(accelerated defect remediation)
– Unobtrusive, easy to use user interface
– Supports exploratory testing
– Reduce time and resources by testing multiple
environments at once
– Toolkit – Screen capture and annotations,
measure objects position etc.
– Record user actions to support exploratory
testing
– Automatic data injection
– Mirror testing
– Log defects with associated information
directly into HP Quality Center/ALM
16

17. Defect Management
Track defects across the entire application lifecycle
– Manage, track and enforce defects across
the application lifecycle
– Create defects from manually or directly
from the execution of manual and
automated tests
– Clearly communicate bugs to developers
with context from requirements and tests
– Integrated into developers IDE
– Enforce defect lifecycle process through
custom fields and workflow
– Log defects directly from test runs with test
details (test step, result, screenshots,
attachments) to ensure the developers can
reproduce the issue
– Traceability to requirements, tests and other
defects
– Alert key stakeholders through automated
emails
17

18. Reports, Graphs & Dashboard
Clear communication and visibility into your application projects
– Make informed business decisions
– Clearly communicate to all stakeholders
– Understand the real-time status of an
application
– Full trending analysis and insight into
application projects
– Understand release and cycle progress
– Fully customizable project reports (export to
MS Word)
– Integrated Dashboard
– Out of the box or customizable analysis
reports and graphs
– Report to MS Excel
– Embed graphs and charts into web portals
– Requirements traceability matrix
18

19. Development tools integration
©2010 Hewlett-Packard Development Company, L.P. The information contained herein is subject to change without notice

20. Working with defects in Developer
various tools
• Traceability between Quality Center Other tools
requirements, code
changes and defects Release &
JIRA
DEMAND
• Adjustable processes Requirements STRATEGY/
Strategic
• Side-tools integrations Test Plan
BugZilla
demand
• New
applications
• New services
StarTeam
• Application
Test Lab integrations
Operational
demand
CCE Tracker
Defects
VersionOne

21. Development tools integrations
• Tasktop & • CollabNet • CollabNet
CollabNet • AccuRev • Hudson
• Eclipse & Eclipse- • Perforce • Electric Cloud
based clients •
• CVS AnthillPro
• Visual Studio
• ClearCase

22. HP Application Lifecycle Intelligence
Waterfall Scrum
TFS Hudson
TFS SVN
RUP Hybrid
HP ALM
Req Build Test Electric
Jenkins
SVN CVS Cloud
Single Platform

23. The Decision Support System for ALM…
Waterfall Scrum
TFS Hudson
TFS SVN
RUP Hybrid
HP ALM
Req Build Test Electric
Jenkins
SVN CVS Cloud
Single Platform

24. Key ALI Features
Code changes
from Subversion
Code changes
from TFS
New
Module
Drill down

25. Key ALI Features
End-to-end app
delivery Report

26. HP ALM 1 and TFS integration
1
Work
Items
Defects
Reqs
Bugs &
TFS adapter for QC Reqs
Synchronizer 1.4
QC
TFS

27. Part 2: Performance validation
28 HP Application Lifecycle Management & HP Quality Center 11.00

28. HP Performance Center
Solutions for management and automation of performance optimization
HP Performance Center Family
Performance Testing: LoadRunner / Performance Center
VuGen Controller Load Generator Monitors Analysis
Management Framework: Performance Center
User/privilege Infrastructure Central Global access & Requirements & Resource pooling
management management repository collaboration Defects & scheduling
Diagnostics
J2EE .NET SOA SAP Oracle

29. HP LoadRunner
The market leading solution for performance validation
Reduce risk of
Enables a consistent, deploying systems that do
repeatable process to: not meet performance
requirements
• Emulate production
workload to test a broad
range of applications Reduce cost by
predicting scalability and
• Identify performance capacity and eliminating
over-deployment
bottlenecks, diagnose the
root cause of issues
• Optimize application Reduce time by
shortening test cycle time.
performance before Accelerate delivery of
deployment high quality applications

30. The challenge: Scripting process
Current scripting process
Record a Parameterize Add
Correlations Add logic
script data transactions
– Sequential process takes time
– Scripting expertise needed (especially for complex apps)
– Protocol level scripts break with minor app changes

31. The challenge: Scripting for web 2.0 apps
New Web 2.0 technologies
Web Data
Client Server App Server
Web Server Cloud
Server Services
Networks Web EDW
App Data
Server
Server Web Server
Web Services
Server
– Many technologies, frameworks and toolkits
– Very customizable – almost no “vanilla” usage
– Dynamic, asynchronous behavior
– Client-side processing and logic
– Most testing solutions do not support all the customizations

32. LoadRunner VuGen
Record an application business process into an automated script
Add end-user transactions, parameterize input data, add verification
checkpoints
Apply user behaviors such as network connectivity or pacing
Business process
Browser
Application
Scripts
VuGen Recorder

33. LoadRunner Controller
 Design and execute load scenarios that mimic production conditions
 Emulate hundreds or thousands of concurrent users
 Gather end-to-end transaction response times
 Collect application, network and system data across all architectural tiers and components
Diagnostics
Controller Load Network Load Web App Database
generators balancer server servers
Europe
VUsers
US
Monitors

34. INTRODUCING: LoadRunner TruClient
The revolutionary way of scripting modern apps
Scripting engine
embedded within
browser
No correlations
required!
Data can be
parameterized
while scripting
Interactive,
dynamic
Transactions, and scripting!
logic can be added
while scripting
Supports all
Ajax
Automatic applications
suggestion of
automated steps

35. Monitoring and Diagnostics
Diagnose performance bottlenecks
Key benefits
• Monitor system
performance during tests
• Diagnose performance
bottlenecks as they occur
• Identify application code
level problems
• Resolve issues faster
Key capabilities
• Sitescope monitoring for
system performance
metrics
• Diagnostics to drill down to
the method level
• Data available during run
time as well as for post-test
analysis
36
6 November 2011
 LoadRunner  Performance Center

36. Extensible Platform
Customize and extend the industry-standard platform
Key benefits
• Build niche/custom
protocols
• Export analysis data for HP 
Partners
--> •HP supports 46% of protocols customers need.
custom reports
•New Demand = 30% of current usage
• Include custom monitoring •25% of customers requested AJAX, RDP and VoIP
•Smaller demand for multiple protocols sums up to
and diagnostics big opportunity
Key capabilities
• Custom protocol
development via a SDK
• Analysis API to export data
for custom reports
• Extensible monitoring and
diagnostics framework
37 6 November 2011
 LoadRunner  Performance Center

37. Performance Center 11.0
Performance Requirements & Defects, Coverage reports
Requirement
Performance coverage
Requirement
Requirements status
New PC Lifecycle User to access performance requirements and defects

38. Performance Center 11.0
Next-gen platform: based on the ALM platform
Performance Performance Performance
Performance
Lab Mgmt Requirements & Test
Testing
(scheduling) Defects Management
 Collaboration  Admin Console  Authentication
 Traceability  Users & license  Authorization
 Reporting  Projects  Workflow
 CoE
 Version Control  APIs, Integrations
ALM Platform
Performance Center

39. Performance Center 11.0
Topology
Drag & drop from Define machine
list to build your names and
topology diagram properties
Right click to set up
monitors

40. Performance Center 11.0
Miscellaneous
– Timeslots
• Redone in Outlook fashion
• More information available regarding booked timeslots
• Allows for maintenance windows
Shows when you can start
a test based on your
requirements

41. Performance Center 11.0
Timeslots
Gives information
on used timeslots

42. Performance Center 11.0
Miscellaneous
– Timeslots
• Redone in Outlook fashion
• More information available regarding booked timeslots
• Allows for maintenance windows
Shows when you can start
a test based on your
requirements

43. Performance Center 11.0
Timeslots
Gives information
on used timeslots

44. Performance Center – fully integrated
into the application lifecycle
A repeatable quality management process mitigates risk
Align with management and stakeholders
STRATEGY/ REQUIREMENTS RISK-BASED TEST MANAGEMENT Go/
No Go OPERATIONS
DEMAND MANAGEMENT TEST PLANNING AND EXECUTION
Business Create manual
Strategic demand
Connect to production
Connect to production
requirements test cases
Integrate with demand
Execute functional
• New tests
applications Assess and Automate
• New services Functional
regression test
• Application requirements Analyze risk cases
integrations
Establish Operational
Operational Performance
testing Identify and security
demand requirements Execute security
priorities customize security management
scans
policies
• Defects
• Enhancements Security Create Production
• Change requests requirements test plans Create monitoring
Execute tests,
performance
Enterprise diagnose and
scripts and
Architecture and resolve problems
Other non- scenarios
Policies functional Service desk
requirements DEFECT MANAGEMENT
• SOA
• Security
45
45 6 November 2011
6 November 2011 Collaborate with design and development teams

45. Part 3: HP UFT

46. Part 3.1 Functional Testing and Service Test

47. A Modern testing team has new needs
People
• Expand skills to test beyond the GUI
• Extend testing to more members of the team
• Increased collaboration with developers and business analysts
Processes
• Automate as early as possible
• Accommodate faster change
Technology
• Complex integrated systems
• Test GUIs, APIs, web services and vertical integration
• Test RIA, Web 2.0

48. Late stage testing is too expensive…
Iteration 1 Iteration 2 Iteration 3 Test
When it was injected way back here? Why wait to find a bug here?
Cost to Fix
Remember, the age old QA principle…
Time between injection
and identification

49. Test & automate as early as possible
C1 C1 C1 C1 C1 C1 C1
C2 C2 C2 C2 C2 C2
G1 G1 G1 G1 G1
C3 C3 C3 C3
… …
Automate tests when
G2 G2 G2
available components
become stable G3 G3
BP1
BP2
Regression test at the end of each iteration or as part of continuous build
C GUI-less G GUI BP business stable
component element process component

50. Modern applications have many layers
End to end Process Layer
business A new transaction
process
Graphical Web 2.0 Billing SAP Exchange
user interface Screen 1
(Ajax)
Screen 2
(Java)
Screen 3
(.NET)
Screen 4
(Ajax)
Screen 5 Screen 6
(Java) (.NET)
layer
Headless Component 1 Component 2 Component 3 Component 4 Component 5 Component 6
(.Net API) (Java API) (web service) (JMS service) (web service) (REST service)
Layer
Interact with modern GUI interfaces, Web 2.0
Continue the scenario to the headless layer; test the integration between the layers
Report on the quality of the entire end to end process

51. HP Software solutions
End to end Process Layer
business A new transaction
process
Graphical Web 2.0 Billing SAP Exchange
user interface HP Unified Functional Testing
Screen 1
(Ajax)
Screen 2
(Java)
Screen 3
(.NET)
Screen 4
(Ajax)
Screen 5
(Java)
Screen 6
(.NET)
layer
Headless Component 1 Component 2 Component 3 Component 4 Component 5 Component 6
(.Net API) (Java API) (web service) (JMS service) (web service) (REST service)
Layer
HP Unified Functional Testing includes the ability to test the
GUI layer + the Headless layer + the end to end process.

52. What‟s new with
HP Unified Functional Testing 11.0?
Unified Functional Testing
HP Functional Testing HP Service Test
Integration
(HP QuickTest Pro
+ all add-ins)
Call activities from
one product to the
other
Transfer complex
data structures
Unified report

53. Report for a Unified Functional Testing scenario
Shows activities from both
HP Functional Testing &
HP Service Test
Multiple views of the flow
Full traceability for easier
debugging

54. Testing Web 2.0 GUIs
OOB Web technology support
• Ajax toolkits including Dojo, GWT, YUI
• RIA such as Flex, Silverlight v2.0, v3.0,
(v4.0 soon), WPF extensibility
• New support for Firefox
Community
• Wizard-like tool helps users extend support to
more web 2.0 controls
• Partners and customers can share code created to
customize support for add‟l toolkits

55. Opening web extensibility to the online
community: The Extensibility Accelerator
Easy, wizard-like interface
Extends HP Functional Testing
fast; uses Intellisense
Automated deployment
Free for anyone? Yes, FREE!

56. HP Service Test 11.0 - new Visual Test
Designer
Modern, intuitive test design
interface
Reusable modules improve
maintainability
Visual test designer extends
testing to non-developers
Flexible and extensible service
11/6/2011 processing

57. BPT Positioning
What is BPT?
Business Process Testing is a framework for QA organizations to build and
reuse structured testing components in both manual and automated testing.
Target users
Non-technical QA personnel
Subject Matter Experts
BPT is ideal to
Build frameworks to maximize reuse of testing components
Mitigate risks by aligning business requirements with end-to-end test
scenarios

58. Part 3.2 BPT

59. BPT 1 1.0
Simpler. Faster. Better.

60. What‟s new in BPT 11
Significantly improved performance
Built an all new simple and intuitive user interface
Provided the ability to leverage QC 1 Capabilities
1.0
Enhanced Traceability via Component Criteria
Rapid Manual Testing with Sprinter Integration

61. BPT 11 Exponential Performance Boost
Performanc
e
Number of components
More Components = Better Performance

62. Test Script view – New look & Feel
Cleaner UI showing all BPT Resources and Dependencies
Clean and Simple Iteration View
•Tree structure instead of a flat list
• Improved Iterations dialog (multi-
iterations view & Single Iterations view)
• new ‘Comments’ column for each
Component instance in the test
• Search option inside the Iterations
dialog

63. One-step Component Creation Process
Simplified the component creation process from three steps to just one
step

64. Enhanced Drag and Drop Feature
Proving the ability to drag and drop multiple test components into the test
script area

65. Summary of Key Enhancements and Features
• Major performance improvements with a redesigned execution
engine
• All new BPT UI built using QC web toolkit for seamless look and feel
• Simple and easy to understand all new test scrip view
• Ability to create a manual testing component in just one step
• Ability to drag and drop multiple components into test script pane
• Enhanced test iteration view to show different combinations
• Cleaner look and feel of QC resource dependencies in BPT
• Application area is a standard test resource in BPT
• BPT integration with QC Sprinter
• Enhanced traceability to map requirements to component level

66. HP Lifecycle Solutions & Operating Principles
PLAN DELIVER RUN RETIRE
COMPLETE
LIFECYCLE Project & HP ALM Business Records Modular
COVERAGE Portfolio Availability Management
Management Quality
Integrated
Center
Deployment Archiving
Automation Heterogeneous
Perform
Center
SOA Systinet Best-in-class
App. Service
Security Management
CORE LIFECYCLE
Center
EXCELLENCE
HP APPLICATION LIFECYCLE MANAGEMENT
Project Planning & Tracking Enterprise Release Management Traceability & Metrics
Single Platform
Requirements Development Quality
Management Management Management Simplified Web Access
Versioning & Link to Defect Secure
Definition
Change Control IDE, SCM Mgmt. Code
Function Perform Secure Technology Agnostic
Powerful Automation
ALM Foundation
Process standardization Reporting Extensibility Quality at the Center

67. Why HP?
Integrated management & automation: instead
of loosely coupled point tools, HP provides a cohesive
platform for lifecycle management and automation.
Technology & methodology agnostic: broadest
support for heterogeneous environments (.NET, Java,
SAP, Oracle, etc.) – covers more than 70 environments.
Covers traditional and Agile methods in single solution.
Project ready, enterprise scale: whether for a team
of ten, or an enterprise of tens of thousands, HP solutions
offer proven configurability and scalability.
Complete application lifecycle: no other company
offers the integrated product coverage of HP to support
applications from beginning to end.