User expectations have changed over the last decade. Customers today expect access to their applications and data from all devices (mobile, laptop, desktop, tablet, etc.) with similar performance from any of those devices at all times of the day. In a world of growing complexity where architects and application designers are dependent on 3rd party providers to delivering part (or at time entire) of the application how does one ensure consistent delivery of performance. This presentation provides a view of some of the challenges involved and how not to make costly mistakes.

1. ANZ Testing Symposium 2015
Trevor Warren & Jason Koch, ATA/PE, ANZ
Delivering A Great End User Experience

2. ANZ Testing
Symposium 2015
Trevor Warren, Tech Arch/Performance EngineeringPresenter Bio
2Copyright © 2015 Accenture All rights reserved.
• Trevor is part of the ATA (Advanced Technology Architecture)
Group within Technology Consulting and leads the Performance
Engineering Capability for ANZ
• In his current role Trevor works closely with accounts across
ANZ advocating Proactive Performance Management with a
view to delivering applications that scale first time around.
• Trevor Warren is passionate about challenging the status-quo
and finding reasons to innovate.
Name : Trevor Warren
Position : PE Capability Lead, ANZ
Organisation : Tech Arch, Performance Engineering
Contact # : 0423 607 688
Email : trevor.warren@Accenture.com
Name : Jason Koch
Position : Senior Manager, Tech Arch
Organisation : Tech Arch, Performance Engineering
Contact # : 0423 607 688
Email : Jason.Koch@Accenture.com
• Jason Koch is part of the ATA (Advanced Technology
Architecture) Group within Technology Consulting and has
collaborated on various Performance Engineering projects here
in ANZ
• Jason is an experienced Technical Architect with a focus on
Architecture and Systems Optimization.
• Jason is passionate about optimizing systems to scale and is
fiercely focused on driving efficiencies through complex
enterprise systems.

3. ANZ Testing
Symposium 2015
What Are The Challenges We Face
3Copyright © 2015 Accenture All rights reserved.

4. ANZ Testing
Symposium 2015
What Constitutes A Great End User Experience
4Copyright © 2015 Accenture All rights reserved.
Source : KissMetrics, Akmai, DynaTrace
“A 1 second delay in page response
times can result in a 7% reduction in
conversions, 11% fewer page views and
16% decrease in customer satisfaction”
• Intuitiveness : A User Interface with functionality that’s intuitive and easy to access.
• Availability : A system which provides the functionality you need when you need it most.
• Responsiveness : A system which responds to your requests in a reasonable timeframe.
• Are there any others that come to mind ?

5. ANZ Testing
Symposium 2015
Human Perception Of Page Load Times
5Copyright © 2015 Accenture All rights reserved.
• 0.1 second is about the limit for having the user feel that the system
is reacting instantaneously.
• 1.0 second is about the limit for the user's flow of thought to stay
uninterrupted, even though the user will notice the delay.
• 3-5 seconds is about the limit for keeping the user's attention focused
on the dialogue. The user’s flow of thought is interrupted and the user
doesn’t feel that the system is reacting instantaneously.
• After 5s, users get impatient and notice that they're waiting for a slow
computer to respond.
• The longer the wait, the more this impatience grows; after about 8
seconds, the average attention span is maxed out.
Source : Nielsen Norman Group
Source : KissMetrics, Akmai, DynaTrace

6. ANZ Testing
Symposium 2015
Trends Which Are Shaping The Way We Think & Work
6Copyright © 2015 Accenture All rights reserved.

7. ANZ Testing
Symposium 2015
How Are These Trends Impacting Delivery Of Customer Experience
7Copyright © 2015 Accenture All rights reserved.
• Applications components spread across multiple data centers make obtaining an End to End view of performance tough.
Distributed setup requires smart distributed tooling.
• Performance reporting when available does not provide a view of the physical layer which is challenging when it comes to
identifying performance issues.
• Ownership of Performance SLA’s is “generally” limited to the data center but you are expected to manage SLA’s End to End.
• Noisy Neighbour issues can cause you a lot of grief. Performance SLA’s mostly not available for Shared infrastructure i.e. IaaS,
PaaS.

8. ANZ Testing
Symposium 2015
How Are These Trends Impacting Delivery Of Customer Experience
8Copyright © 2015 Accenture All rights reserved.
• Users expectations on performance of interfaces haven’t changed, but volume and rate of data flows have increased ten fold
• Customers expect to have a consistent experience across multiple device types and interfaces.
• Slashdot Effect : Being featured on social media may result in very fast massive increases in traffic that may crash a system that is
sized for typical daily workload.

9. ANZ Testing
Symposium 2015
Complex Application Delivery Chain
9Copyright © 2015 Accenture All rights reserved.
Traditional Inside
Out View of
Performance
Image Source : DynaTrace

10. ANZ Testing
Symposium 2015
Performance : At The Core Of A Digital World
10Copyright © 2015 Accenture All rights reserved.
“In the new Digital world, performance is a key enabler of the customer
experience. Poor performance will make or break a Digital business.
Faster systems allow IT spend to be moved from license and hardware
spend to services spend.”

11. ANZ Testing
Symposium 2015
Application Delivery Chain : Optimized
11Copyright © 2015 Accenture All rights reserved.
• Poor performing JavaScript
• Large Page Sizes
• Device Compatibility
• Poor performing Code
• Storage bottlenecks
• Database locking
• Oversubscribed Virtual Layer
• Sub optimal network routes
• Network latency, jitter, etc.
• Bandwidth contention
• CDN Configuration issues
• Oversubscribed CDN’s
• Bandwidth bottlenecks
• Routing optimization issues
• 3rd Party scripts that block
• Chatty Social Media apps
• Consuming precious client side resources
during load
Resolution : APM
Resolution : NPM
Resolution : NPM
Resolution : APM
Resolution : APM
Image Source : DynaTrace
Outside In
View of
Performance
Traditional Inside Out
View of Performance

12. ANZ Testing
Symposium 2015
Silo Performance Management
12Copyright © 2015 Accenture All rights reserved.
Image Source : DynaTrace

13. ANZ Testing
Symposium 2015
Where Did We Go Wrong
13Copyright © 2015 Accenture All rights reserved.
• 4 Weeks to Go live : The application is now ready. Let’s start addressing Performance.
• No Non Functional Requirements. Can you please come in and Performance Test my system.
• No Workload Models. Can you please come in and Performance Test my system.
• No PoC required for the green field solution. Let’s go build our fancy solution!!!.
• No Consideration, planning or budgeting for performance early on.
• Lack of appropriate tooling for Performance Testing, Diagnostics, Monitoring, etc.
• Focus on system monitoring (post go live). Lack of proper application monitoring tools. No view into
business workload responsible for performance of the system.

14. ANZ Testing
Symposium 2015
End User Experience Matters
14Copyright © 2015 Accenture All rights reserved.

15. ANZ Testing
Symposium 2015
Case Study : Arch Review, Diagnostics, Tuning & SVT
15Copyright © 2015 Accenture All rights reserved.
Business Drivers for the engagement: Delivery of new Digital Streaming Video On Demand solution, delayed
due to platform performance and scalability, including functional and integration issues.
Challenges
• Unstable solution, significant performance and scalability issues
• Lack of understanding and agreement on NFR’s among Business, IT, Development & Support teams
• Lack of industry standard Performance Testing & Diagnostics tools
• Lack of process for Performance Engineering i.e. from Design to Test
• Lack of defect management, triage and defect resolution
• Significant architectural inefficiencies, application bottlenecks & infra bottlenecks
Objectives
• Performance As-Is assessment, identify gaps and recommend overall PE approach
• Model Workload, determine NFR’s and recommend Performance Testing Strategy
• Manage SVT, Triage & Defect Management. Responsible for Workload & NFR’s.
• Holistic architecture review architecture for reliability, performance, and scalability

16. ANZ Testing
Symposium 2015
16Copyright © 2015 Accenture All rights reserved.
Case Study : Arch Review, Diagnostics, Tuning & SVT
Solution
• Work with program leadership, provide guidance on the overall Performance Engineering approach
• Design End to End Performance NFR’s and Workload Models for SVT
• Manage on-going Performance Test and co-ordinate on-going Performance Testing
• Lead triage, use diagnostics tools to identify bottlenecks across the application
• Work with Engineering teams on identifying defects and defect fixes – deep dive code reviews and specific parameter tuning recommendations
• Work with Tech Arch on identifying areas of optimization across the system
• Present roadmap to business to improve customer credibility with business users
Value Delivered & Business Benefits
 Initial successful 4-week engagement to diagnose and tune key stability issues; led to follow-on PE and SI work
 Delivered custom workload models & NFR models based on analysis of existing production data providing insight into customer behaviour during
peak workload
 Optimized Application Design, Diagnosed bottlenecks and tuned system to meet overall production volumes
 Delivered an application that met documented NFR’s
 Delivered an application that supports the customers growing business workload
 Delivered product to successful launch date

17. ANZ Testing
Symposium 2015
Performance Engineering Framework
17Copyright © 2015 Accenture All rights reserved.
Performance Engineering defined within Systems Engineering encompasses a set of roles, responsibilities, activities, deliverables and
outcomes that are defined at every phase of the Systems Development Life Cycle which ensures that the solution will be designed to meet the
documented Non Functional Requirements. Performance Engineering can be said to include the following activities:
Performance Modelling &
Capacity Management
Application & System
Performance Monitoring &
Management
Performance Testing
Fine Tune Application
Capacity
Planning
Performance
Forecasting
Profiling & Diagnostics
Performance
Components Building
Application & System
Component Tuning
Perform Code Review
Performance
Testing
Strategy
Performance Modeling
Performance Design
Reviews
Performance Components
Design
Performance
Requirements Analysis
(NFR’s)
Performance Management
Strategy
Analysis Design Build Test Run
Proactive Performance Management across the Application Life Cycle
Capacity Modeling
Application &
System Performance
Monitoring & Management

18. Copyright © 2015 Accenture All rights reserved.
Thank You
18