ASUG 2010 - Structuring your Testing Chaos with Solution Manager

Session 0411: Structuring Your Testing Chaos
with Solution Manager at Weill Cornell Medical College ]
Sabine Margolis – Enterprise Architect
Weill Cornell Medical College
Ajay Vonkarey – Solution Mgr Cons.
Alpha Sirius Inc.

[ ERIC SYNSTELIEN
ASUG INSTALLATION MEMBER
MEMBER SINCE: 2000

[ RICHARD RAUCH
MEMBER SINCE: 1999

[ JENNIFER BUKO
MEMBER SINCE: 2006

[ Learning Points
How Solution Manager improves quality and
testing duration through business process
relevance, standards enforcement and
repeatable processes.

How Solution Manager fosters transparency
between Business and IT through meaningful
business to technology mapping.

The benefits of using SAP Solution Manager as
an integration tool, from business process
design through upgrades and production
optimization … and how it’s never too late to
implement.

Real Experience. Real Advantage. 2

[ What We Will Cover ….
Introduction: Stakeholders & Implementation
Industry Issues & Functional Testing
Solution Manager Revisited
Why is adoption lagging
What is required for testing
Solution Manager Test Workbench
Test Planning & Execution
EhP4 Project Turnaround
Wrap up / Q&A


[ Intro: Alpha Sirius Inc.
Founded in 1998
One of first SAP certified partners for Run SAP Methodology.
Expertise in providing end-to-end solutions to disparate
functional and technical implementation processes – including
tools, such as SAP Solution Manager (www.alphasirius.com).
Core Competencies:
Program Management
Portfolio Management
Application Life Cycle Management
Change Management
Training & Development
Systems Management
Offshore Support/Development

[ Intro: Weill Cornell Medical College (WCMC)
Est. in 1898 as Cornell University Medical College
Affiliated with New York-Presbyterian Hospital since 1927
Among US top-ranked clinical /medical research centers
Providing degrees in medicine and bio-medicine
Parent Inst.: Cornell University, Ithaca (Kuali/PeopleSoft)
WCMC Loc: New York, NY med.cornell.edu
Doha, Qatar qatar-med.cornell.edu
Why SAP: Grants Mgmt, SRM & BI
Solution Name: Weill Business Gateway (WBG)
Initial Go Live: January 2009
Users: ~ 1000 SAP Portal users (departments)
~ 60 Central users (administration)
~ 40 COE users (support)
Real Experience. Real Advantage.

[ Intro: WBG - Implementation Scope

FIN: Finance (Public Sector Accounting/Higher Education):
Plan to Report [FM,(+ OPS),CO,AP,AR,GL,TR]
Award to Report [GM,PS,FM,(+ OPS),CO,AP,AR,GL,TR]
Capital Projects to Report [PS,AM,(+ OPS),CO,AP,AR,GL,TR]
Fixed Assets to Report [AM, (+ OPS),AP,AR,GL,TR]
OPS: Operational
Procure to Pay [SRM,MM/IM,(+ FIN)]
Hire to Pay [OM,PA,TM*,PBC,PY,BN,(+ FIN)]
* SAP TM + TimeLink Front End
Enroll to Graduate [StAcc,FM,(+ FIN) … more to come]
Other
Business Intelligence
SAP Portal
SAP NetWeaver Process Integration PI/XI

[ Intro: WBG - Implementation Complexity
Extensive Master Data Setup
Long Process Chains
All transactions tie back to either Funds - Grants - WBS
FM/GM Encumbrances
3rd Financial Transaction Layer (Encumbrance - Actual = Budget)
Heavy Customization
Custom labor info types = many custom programs & user exits
FM/GM Departmental Security Model
Departmental user may only see their assigned funds/grants
No of: Funds 8,477 / Grants 3,012 / WBS 5,258
BI enforces security model


[ Industry Reality – The Shared Pain of Testing

On the difficulties of SAP functional testing:
1) Functional tests in heterogeneous system landscapes
2) Lack of analysis concerning the impact of changes
3) Provision of test environments and test data
4) Limited test automation
[ASUG and DSAG 2008 Survey]


[ Industry Reality: Test Effort Equal Regardless of Initiative


[ Industry Reality: Insuff. Tools f. Managerial Oversight
Scope
Technical: am I addressing custom objects?
Functional: do I have sufficient coverage?
Management
Reusability: can we use past test cases / where are they?
Predictability: how long will this take?
Accountability: who will test – who signs off?
Monitoring
Level of Completion: what has been tested/what’s outstanding?
Level of Quality: what is the quality of my system?
Number of Defects: what are the issues?
Note: Mgmt has never has the same reporting as business users have.
Business Inclusion
Are the test scenarios realistic
What is the platform for review and sign-off

[ Weill Business Gateway: September 2009
X
Sept 2009 EHP4 Upgrade Project
Idealistic test guidelines
No system to reinforce
XLS based scripts/rpts
Catalog distributed
across many locations
Scope/Coverage unclear
Last minute planning
Business cannot relate
No single point if truth

[ Solution Manager Revisited
Take just one thing
out of the universe …
and you will find …
it is connected to everything else.
[MRP Proverb]


[ Solution Manager Revisited: The Components
ChaRM: Change
Request Mgmt
• Change/Proj. Mgmt
Service Desk • Change Logistics
(CTS+) BPM:
• Incident Mgmt (CRM ) • Business Process
• Link to SAP Support Mgmt & Monitoring

Test Workbench: Maintenance
Optimizer
End-to-End Testing
• Manual
• Automated * (eCATT)

Root Cause
Analysis
Implementation
Content
• Blueprint
SOLUTION
• Realization Job Mgmt*
(Config & Dev) MANAGER

* Requires additional application

Real Experience. Real Advantage.

[ Solution Manager Revisited: ‘10 Panaya Adoption Survey
The top reason cited for using SAP Solution Manager:
Central System Administration (19%)
Monitoring (14%)
Project Management (13%)
Key function: Solution Documentation
Respondents who have not documented any solution in SM (42%)
Respondents who have fully documented solution in SM (3%)
SAP Solution Manager features adoption
do not use (51%)
used rarely (10%)
used partially (16%)
use them quite a bit (12%)
used fully (8%).
http://m.zdnet.com/blog/howlett/sap-solution-manager-continues-to-confuse/2200

[ Solution Manager Revisited: Component Hierarchy
Key Integration:
Business Process Hierarchy
Implementation
Content Affects all Other Components
Best Practices: 16 Standards
Methodology: Run SAP

End-to-End
BPM
Testing

Service Desk ChaRM

Maintenance Optimizer

Root Cause Analysis

Job Mgmt


[ Solution Manager Revisited: Testing Requirements
Testing Strategy
List of E2E Business Scenarios to start Implementation Content
Sol Man: Implementation Content [Test Content]
Blueprint (solar01) E2E Business Process Hierarchy
TCodes
Config (solar02) Test Cases
RICEFW Objects
Solution Manager Business Process Change Analyzer
Sol Man: Test Workbench [Test Cases/Results]
Sol Man: Service Desk (light) [Test Issue Mgmt]
Optional – but recommended:
Config (solar02) TBOM - technical bill of materials
Test Workbench integration with 3rd party testing tool for automation
HP Quality Center, WorkSoft, SAP TAO …

[ Solution Manager Revisited: Business Process Architecture

Process Definition via ‘Folders’ – Defined in Blueprint
• Level 1: E2E Business Scenario Procure to Pay – Line of Credit
• Level 2: Business Process Process Requ, Process PO
• Level 3: Process Step Create Requ., Approve Requ
• Attributes – very useful for reporting (e.g. flag core processes)

[ Solution Manager Revisited: Business Process Structure
1

2
3
1

2

GM: Focus on Generic Processes GM: Focus on End-to-End Scenarios
= Disconnected 2 Tier Structure = Connected 3 Tier Structure

[ Solution Manager Revisited: Lessons Learned
Lack of adoption less due to complexity …
more due to lack of clearly defined internal IT processes
leaving it up to BASIS vs. getting functional side involved
Don’t implement one component without understanding how it
affects the others.
Business Process Hierarchy is the gold …
Key to reusability across every project & change effort
Key to process/technology – technology/process mapping
Define in terms the business understands (E2E vs SAP Module)
Create your own hierarchies (SAP’s not recommended)
SAP provides modular view – with good documentation and great
selection of TCodes … but there is no universal BPA
… use them as inspiration only.


[ Solution Manager: Test Workbench

You can’t expect to meet the challenges of today –
with yesterday's tools
and expect to be the business of tomorrow.
[Anonymous]


[ Testing: Workbench Components (Reuse of Hierarchy)
Test Set (Test Cycle)
Business HierarchySubset Test Plan Test Plan
TCodes Scripts (E2E)
Subset Subset (E2E)
Subset Subset

Test Test Test Test
Package Package Package Package
(Scenario) (Scenario) (Scenario) (Scenario)

Test Note Test Note Test Note Test Note
(Results) (Results) (Results) (Results)


[ Testing: Convergence of Projects and Change
Scope for Testing
Test Planning
Test Set
Test Plan
Test Package
Test Script
Assignments
Test Execution
Issue Management
Reporting


[ Testing: Scope - Project Analysis Reports / Cross Tab

Testing Scope/Requirements: Business Owner Review
E2E Scenarios – Reflecting Complexity
• Flow (Scenarios, Business Processes, Process Steps)
• Transactions
• Development Objects
• Systems

[ Testing: Scope – Bus. Process Change Analyzer (BPCA)

Additional Requirements: Change between 2 Projects
• Many options for comparison reporting (e.g. process prioritization )
• Traffic Light (G-Y-R) indicates areas of risk – testing suggested for Y & R


[ Testing: Planning Effort - Test Plan to Test Package

One Test Plan Many Test Packages
(subset of hierarchy) (subset of Test Plan)

Inherit Scenarios from Hierarchy Select scope of a test scenario

Caveat: include Test Cases that match Caveat: activate both the appropriate
Cycle & TCodes. script and TCodes within the script –
results in better reporting.


[ Testing: Planning Effort - Tester Assignment

Package Assignment

Tester Worklist


[ Testing: Execution & Results (Manual Testing)
Note:
Results are captured in test notes within test script Automated
testing
Test and TCodes are marked as pass or fail uses test
(variety of statuses) workbench
in the same
Testing issues are logged as messages
way.
Messages are addressed as Service Desk tickets
Note: better status reporting –
one script / many TCodes


[ Testing: Execution – Issue Management

Test Messages
• Shows issues by …
• Test Set / Test Plan / Test Package
• Links to Service Desk Ticket
• Note: service desk ticket link back to test – not provided

[ Testing: Execution – Reporting

Test Workbench Information System
• Review Real-Time Status
• Review Test Issues
• Generate Test Reports


[ Testing: Lessons Learned
With and established business structure, test
planning/development is fast.

Without adequate generation of TBOMs – Business Process
Change Analyzer provides limited results.

Always include Scripts and TCodes in the Test Packages for
best results in progress reporting.

Resource assignment is at Test Package level - can be difficult
to manage hand-off. Fine tune with ‘Sequences’ feature.

If test should be changed – evaluate if this is a business
hierarchy change or merely a test structure change.

[ EHP4 Upgrade Project – Taming the Beast

I love deadlines … especially the whooshing sound they
make as they fly by.
[Douglas Adams]


[ Project: Testing Approach + Solution Manager
Heavily Correlates with Landscape Design
SBX Unstructured assessment & validation [Sept - Nov]
MAJOR PROJECT INTERRUPTIONS (Year End SP)
D Recov Sol Man: Roll-Back Testing in QA [Mid March]
MAINT Sol Man: System Val. (parts of disaster recov.)
QA Sol Man: Sys Val., Unit, Integration, UAT, Stress
PRD Sol Man: Sys Val. [Mid May]
TRN Sol Man: Integration (skipped Sys Val. – not recommended)
Lack of test scripts allowed for other efforts to interfere with
EHP4 upgrade – we had to rethink the strategy Sol Man


[ Project: Sol. Manager Build & EHP4 Upgrade Timeline


[ Project: Milestones – 9 Weeks Testing (Plan to Execute)!
Milestone Dur.
1 Define Requirements 3 wks
Testing Process and Implementation Content
2 Configure Solution Manager 5 wks
Document Templates, Statuses, Serv. Desk (Light),
Logical Components
3 Develop Implementation Content Training Materials
4 Document Implementation Content 5 wks
5 Develop Test Workbench Training Materials
6 Prepare Test Scripts, Test Plans, Test Packages 4 wks
7 Execute Testing: 5 wks
Unit – Integr. – Dis. Recovery – Final Sys Validation
8 Go Live N/A
Total Duration 22 wks /
5.5 mos -


[ Project: Resources
Resources Count
COE Business Analysts w. team leads (dictated by discipline) 23
COE Developers 5
Subject Matter Experts (dictated by discipline) 14
Additional Business Testers (minimal UAT testing) 26
Offshore Testers 4
Testing Coordinator 1
Solution Manager Consultant 1
Solution Manager PMO Owner 1
Training Resources (Mgr + Trainers) 3
COE PMO/Senior Mgmt 4
Total Count 82


[ Project: Artifacts / Heuristics
Solution Manager Objects Count
E2E Scenarios 108
Unique Transactions 700
Unique Development Objects (RICEFW) 70
Unit Test Scripts 750
Integration Test Scripts 100
Disaster Recovery Test Scripts / Used also as systems validation 80
Test Sets 7
(Test Cycles: Dis.Recov./Unit/Integr./Validation/UAT/Stress/Training)
Test Plans (generally 1 per E2E genre x Test Set) 69
Test Packages 330
Service Desk Messages 47


[ Benefits: Solution Manager & Managerial Confidence
Scope
Technical: clear – ability to address and assess custom dev.
Functional: clear – can generate BPML to work with business
Management
Reusability: all components reusable – hierarchy, test scripts, plans …
Predictability: plans can log duration as KPI – did not use
Accountability: can track tester and business sign-off
Repeatability: hierarchy has enabled repeatable testing processes
Monitoring – 100 % Transparency
Level of Completion: status always real-time – single point of truth
Level of Quality: measured by quality of TCode
Number of Defects: easy to find in Service Desk & associated to test
Note: Mgmt is still getting used to reporting vs XLS.
Business Inclusion – 100% Transparency
Business currently reviewing business structure to improve processes
Appreciated the ability to review results as an auditable feature

[ Wrap Up

This is not the end. It is not even the beginning of the end.
But it is perhaps the end of the beginning.
[Winston Churchill]


[ The Change: What Lies Ahead
Service Desk
Phase 1 – COE [2nd week of go live!]
Phase II – roll out to user community (Portal enabled interface)
Test Workbench – Automated Testing [in progress]
ChaRM
Adjust Implementation Content
Revise E2E with Business [Currently Underway]
Develop Solution Directory
Additional updates through Solution Documentation Assistant
Enhance with Business Process Attributes
Capture Configuration
Work Centers
Extended Reporting from within BI

[ Solution Manager Testing - Return on Investment
Process-oriented Business Model Reduced Testing Efforts
Scope: clear IT and business understanding of what is needed
Prep: significant reduction in prep time for upgrade testing
Execution:
Faster turn-around between issue ident. and resolution
No wasted Management time on outdated status reports
Reduction in documentation and review efforts.

Solution Manager functionality can now be easily expanded

Best of All: Software Acquisition Costs = 0


[ Solution Manager Testing – Implemented Best Practices
Document Implementation Content
Process and System Components of Solution
Custom Development of Solution
Keep solution up-to-date
Use Structured Business Process Architecture
E2E vs. Module Based
Leverage SAP’s Business Process Repository
Use Business Process Change Analyzer
Risk-based test planning
Integrate with Incident and Problem Management
Leverage the Service Desk integration with Test Workbench


[ Solution Manager Testing – Key Learnings
Solution Manager – not just for Basis … and not just out of the box

Requires adequate methodology and process definition.

Design your business hierarchy to E2E test scenarios – better re-
usability in the long-run.

Get every one involved in BPA – Steering committee to end user

Central dash board view – invaluable for managing deadlines.

Provided reports are utilitarian – however, include additional fields
for ALV and generate global display variants for ease of use.

Link Test Workbench to Service Desk – faster resolution.

[ Now it’s Your Turn …

Questions?
Sabine Margolis, PMP
ssm2005@med.cornell.edu

Ajay Vonkarey, PMP
avonkarey@alphasirius.com


[
Thank you for participating.
Please remember to complete and return your
evaluation form following this session.
For ongoing education on this area of focus, visit the Year-Round
Community page at www.asug.com/yrc
]
[SESSION CODE:
0411


[ Addendum I: Helpful Links
SAP Solution Manager & Tools:
http://service.sap.com/solutionmanager
http://www.solutionmanagerexpert.com/
Application Lifecycle Management (ALM)
Solution Manager = Technical Enabler:
http://service.sap.com/alm-processes
http://www.sdn.sap.com/irj/sdn/alm-technical-enablement
http://www.sdn.sap.com/irj/sdn/alm-test-management
Business Process Change Analyzer (BPCA) Document
https://websmp104.sap-ag.de/~sapidb/011000358700000932192009E
SAP Solution Manager eLearning Material:
http://service.sap.com/rkt-solman
Business Process Monitoring / Management
http://service.sap.com/bpm

[ Addendum II: 16 Operational Standards/Best Practices
1. Incident Management describes the process of incident resolution
2. Exception Handling explains how to define a model and procedures to manage exceptions and
error situations during daily business operations
3. Data Integrity avoids data inconsistencies in end-to-end solution landscapes
4. Change Request Management enables efficient and punctual implementation of changes with minimal risks
5. Upgrade guides customers and technology partners through upgrade projects
6. eSOA Readiness covers both technical and organizational readiness for enterprise service-
oriented architectures (eSOA)
7. Root Cause Analysis defines how to perform root cause analysis end-to-end across different
support levels and different technologies
8. Change Control Management covers the deployment and the analysis of changes
9. Solution Documentation defines the required documentation and reporting regarding the customer
solution
10. Remote Supportability contains five basic requirements that have to be met to optimize the
supportability of customer solutions
11. Bus. Proc. & Interf. Monitoring describes the monitoring and supervision of the mission critical business
processes
12. Data Volume Management defines how to manage data growth
13. Job Scheduling Management explains how to manage the planning, scheduling, and monitoring of
background jobs
14. Transactional Consistency safeguards data synchronization across applications in distributed system
landscapes
15. System Administration describes how to administer SAP technology in order to run a customer
solution efficiently
16. System Monitoring covers monitoring and reporting of the technical status of IT solutions
https://websmp103.sap-ag.de/~form/sapnet?_SHORTKEY=01100035870000711194&_SCENARIO=01100035870000000202&


[ Addendum III: SDLC in Solution Manager
Project Prep Blue Print Realization Go Live Prep Go-Live

Solution Manager Tool

Blueprint Blueprint Config. Testing Solution

Business Business Business Business
BPR Scenario Scenario Scenario Scenario

Business Business Business Business
Process Process Process Process

Step Step Step Step

Solar01 Solar01 Solar02 STWB_WORK DSWP
• Structure Tab • Configuration
• General Data Tab • Development Tab • Test packages • All Solution Manager
• Project Documentation Tab • Test Cases Tab • Test results documentation
• Transactions Tab
• Graphic Tab


[ Addendum IV: Implementation Content

The beginning is the most important part of the work.
[Plato]


[ IC: Scope – Solution Documentation Assistant (SolDocA)

Reverse Business Engineering: Establish initial BPML
• Batch Job: compares Implementation Content to System Usage
• If no structure - use SAP BPR as initial Implementation Content
• Not possible for BI, SRM, Portal Systems
• Baseline - Early Watch Reports -> typically 3 month retention
• Should be used periodically to ensure business structure is up to date

[ IC: 3 Tiered Business Process Architecture

Process Definition via ‘Folders’ – Defined in Blueprint
• Level 1: E2E Business Scenario Procure to Pay – Line of Credit
• Level 2: Business Process Process Requ, Process PO
• Level 3: Process Step Create Requ., Approve Requ
• Attributes – very useful for reporting (e.g. flag core processes)

[ IC: Process to Technology Mapping - Transaction Codes

Details of the Process Step – Defined in Blueprint
• Place TCodes at Level 3 and Level 3 only (process step).
• Represents one critical path TCode (standard – typically create)
and several support TCodes (typically change/display).
• Include worklists to support transaction review.
• Business Process Master List (BPML) via Project Analysis Reports

[ IC: Process to Technology Mapping – Dev. Objects

Main objects supporting the process – Defined in Config
• Place objects at appropriate level – typically Level 3.
• Objects easy to find using search features.
• Project Analysis Reports allow you to search any process to object
relationship – e.g. list of processes using a particular program.


[ IC: Process to Technology Mapping -Tech. Bill of Materials

Requ. Business Process Change Analyzer – Defined in Config
• Created as an “attribute” for each TCode
• Easily generated by triggering the TCode (Logical Comp. important!)
• Catalogs all critical technical information from TCode and Dev Objects.
• Project Analysis Reports help find generated and missing TBOMs by process.


[ IC: Process to Technology Mapping - Manual Test Cases

Unit, Functional, Integration – Defined in Config
• Level 1: Integration Tests
• Level 2: Functional Tests
• Level 3: Unit Tests
• Project Analysis Reports support test coverage review & test catalog.

[ IC: Development Object Catalog (Optional)

Use Bus. Hierarchy Tool – Defined in Blueprint & Config
• Level 1: Type of object R I C E F W
• Level 2: Module (vs. E2E - better for internal reporting)
• Level 3: Main Development Object – include TCode(s) & Dev Obj
• Development Objects List (RICEFW) via Project Analysis Reports
Note: Interface Scenarios section is used for BPM

[ IC: Stay Informed – Real Time Project Analysis Reports

Dynamic ALV Reports – Accessible in Blueprint & Config
• Use both ‘Assignments’ menu and ‘Test Menu’ [SAPLSPROJECT_EVAL_GENERAL]
• Business Process Master List (BPML)
• Development Objects List (RICEFW)
• TBOM Analysis, Test Catalog & Test Coverage … and many more
Note: ABAP Developers can pull in additional fields in ALV

[ Implementation Content: Lessons Learned
3 Tiered Business Process Architecture
Clearly define E2E naming conventions and train your team
Took a while to grasp by team, need many reviews …
Create an E2E integr. role to coach team and keep data clean
Create your own hierarchies (SAP’s not recommended)
SAP provides modular view – with good documentation and
great selection of TCodes … but there is no universal BPA
… use them as inspiration only.
Don’t forget the Mgmt and the Business Owners
Train them as well
Get business owner sign-off on hierarchy design
Start using the naming of the E2E in your meetings
Defining the BPA is part Art part Science – in the end it must
make business sense.

ASUG 2010 - Structuring your Testing Chaos with Solution Manager

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to ASUG 2010 - Structuring your Testing Chaos with Solution Manager

Similar to ASUG 2010 - Structuring your Testing Chaos with Solution Manager (20)

ASUG 2010 - Structuring your Testing Chaos with Solution Manager