The Digital Revolution is transforming the way every industry engages with customers, making the user experience a critical competitive differentiator.
From retail to banking to media, business results and reputation depend on consistently delivering good performance. So who are the Best of the Web?
Dynatrace reveals the companies in retail, banking, insurance, brokerage, news media, air travel, and hotels who exceeded their peers in web performance, availability, and user experience in 2015.
• Learn which companies earned the distinction “Best of the Web”.
• Discover unique industry performance standards.
• Hear the best practices that enabled leaders to outpace their competitors.
• Analyze user experience trends – how did 2015 compare with prior results?
Learn what it takes to deliver the best digital performance to customers!
2. 2 #Dynatrace
Insert
image here
David Jones
Director, Sales Engineering
APM Evangelist
David Jones is the Director of Sales Engineering and APM
Evangelism for Dynatrace. He has been with Dynatrace for
8 years, and has 20 years’ experience working with web and
mobile technologies from the first commercial HTML editor to
the latest web delivery platforms and architectures. He has
worked with scores of Fortune 500 organizations providing
them the most recent industry best practices for web and
mobile application delivery.
3. 3 #Dynatrace
Insert
image here
Seth Voorhies
Director, Digital Performance
Insights, Dynatrace
Seth Voorhies directs the overall Digital Performance
Insights team and having been at Keynote/Dynatrace for 8
years has seen a lot of evolution in the performance
space. With a passion for performance, technology, and
driving customer experience Seth has helped customers
realize the importance of performance as feature no 1 for
customer experience, not just for technical organizations but
across the business.
4. 4 #Dynatrace4 #Dynatrace
• What is Best of the Web?
• Our methodology
• Top performers
– Banking
– Brokerage
– Insurance
– Digital news media
– Travel – airlines
– Travel – hotels
– Retail distribution
• How did they do it? Retail industry leaders’ deep dive
• Your comprehensive Digital Performance Management approach
5. 5 #Dynatrace
• This is Dynatrace’s 7th annual Best of the Web Award
• Recognizes firms that excel in web and mobile delivery
• Our methodology has evolved
– Additional changes to recognize multiple leaders in categories
– To expand relevance and validity
• Introducing deeper analysis highlighting Retail industry
6. 6 #Dynatrace
Methodology
Dynatrace Benchmarks Methodology:
www.dynatrace.com/
content/dam/en/general/benchmark_methodology.pdf
ReliabilityIndex
Performance Index
Weighted average of a site’s
Availability rate and
Consistency rate.
Weighted average of a site’s
Load Time across multiple
channels.
Sites placing higher
and farther right
deliver a faster and
more reliable digital
experience for
today’s consumer.
• Leverages Dynatrace Benchmarks
• Based on measurements collected
throughout 2015
• “Performance Index”
– Combines last mile, mobile, transaction
and backbone performance
– Weighted towards last mile, mobile and
Chrome browser
• “Reliability Index” combines availability
and consistency
• Metrics are collected and analyzed by
Dynatrace Synthetic Monitoring
7. 7 #Dynatrace7 #Dynatrace
• Top performers
– Banking
– Brokerage
– Insurance
– Digital news media
– Travel – airlines
– Travel – hotels
– Retail distribution
16. 16 #Dynatrace
Key Performance Issues
• Lack of process/culture that is supportive of performance
• Lack of visibility into performance issues
• Lack of performance budget understanding – balancing
business requirements with user expectations
17. 17 #Dynatrace
Key Performance Issues
• Third parties and lack of visibility/process and control
• Overuse of JS and CSS
• Endless scrolling with delays to initial render
• Serialization of all types
• Missing the basics
– Persistent connections
– Cache control
– Large images sizes
– Etc.
18. 18 #Dynatrace
Performance Comparison – Firefox Backbone
• Looking at the Firefox comparison of Apple (one of the top scorers) vs one of
those outside of the top right quadrant we see a
19. 19 #Dynatrace
Page Construction – Page Size
• Apple has a significantly smaller site across all of the pages, especially on Home
and Checkout
20. 20 #Dynatrace
Page Construction – Element Count
• In terms of element count comparison we see an even greater disparity
between the two sites
22. 22 #Dynatrace
Key Performance Issues – Third Parties
• Know your third parties
– Analytics
– Personalization, A/B Testing
– Web services
– Social
– Display ads
– Tracking
• Understand the value to the business
– How critical are these to the customer experience and to the
business?
– Are they worth a performance hit?
23. 23 #Dynatrace
Key Performance Issues – Third Parties
• Measure, Move Down and Audit
– Measure the performance of each – focus on consistency (less than 100MS
each)
– Move down out of render, below site content and ideally after onload
– Audit the number of usage quarterly – are they still needed?
• Special focus on rendering third parties
– Personalization and A/B testing
– Required to be high in the render
– Can have a massive impact on performance (completely outside of the site’s
control)
– Use sparingly
– Realize these vendors are not focused on performance
24. 24 #Dynatrace
Mobile - Responsive Design Challenge
Responsive design has created a slower mobile web experience for all of us…
25. 25 #Dynatrace
Responsive Design Challenge
• Solves UI/maintainability problem
but creates a unique performance
challenge
• Typically means taking a bloated
desktop site and “making it
responsive”
• All the best practices you have
ignored on desktop matter
massively on mobile
• Pervasive myth that I only need to
measure “one access” method
26. 26 #Dynatrace
Performance Comparison – Mobile Home
• If we look at the Mobile Home for both sites, both responsive, we again see a
substantial difference in performance
28. 28 #Dynatrace
Responsive Design Plan
• Understand the massive variability of 3G/4G/LTE/Tower capacity, etc
• Measure all three access methods but focus on “over the air”
• Expand your “view” using Real User (RUM) data
• Get serious about optimization – follow all the best practices you
have been ignoring
• Keep an eye on custom font files sizes and usage
• Consider server side detection
• Render quickly (might use inline code)
• Limit third parties and domains
31. 31 #Dynatrace
Digital
Outcomes
SaaS Vendors
CDNs
3rd Party Services
Rapid Root Cause Analysis
All Problem Patterns
All Applications
All the Time
DevOPs
CI/CD
Readiness
Customer Experience
Competitive Insight
Omni-Channel
I N N O V A T I O N
O P E R A T I O N A L
E F F E C T I V E
E F F I C I E N T
TODAY
TOMORROW
Data Center
Public/Private Cloud
PaaS/Containers
SDNs
Dynatrace publishes more than 300 global mobile and web benchmarks that include thousands of companies in more than 26 countries and millions of measurements each month, making this the most comprehensive source of industry-based mobile and web performance information.
Measured across channels that reflect how customers browse the Web today
Based on a rigorous and well-defined set of online transactions and homepage measurements, Dynatrace Benchmarks competitively rank web and mobile web sites over specific time periods and locations. This data allows organizations to analyze the performance and best practices used by their competitors and fellow industry leaders.
“Best of the Web” companies distinguish themselves from their peers in both performance and reliability.
Dynatrace Benchmark metrics are collected and analyzed by Dynatrace Synthetic Monitoring, which measures websites’ availability and response time for identified pages and transactions. Leveraging the largest and most comprehensive performance-testing network in the world, Dynatrace’s integrated platform makes it easy to monitor the performance and reliability of websites from the end user perspective.
Transition to Seth – role of services in developing an analysis for a specific org
↓ 24% (1.3 sec slower): Last Mile Load Time over previous year (comparative members)
Highest overall Availability of any industry
Banking: Homepage -> Login -> Username and Password -> Security Questions -> Account Details -> Logout
Home page, backbone load time:
Average for all: 3.13 sec
Top 3 average: 1.89 sec
Bottom 3 average: 5.19
Mobile Page Weight:
Average: 795 KB,
Fastest 3: 308 KB,
Slowest 3: 1,288 KB
Fastest overall Last Mile and Transaction Load Time of any industry
Brokerage: Homepage -> Login -> Security Questions -> Trade -> Cancel Trade -> Logout
Home page, backbone load time:
Average for all: 2.77 sec
Top 3 average: 1.63 sec
Bottom 3 average: 3.83
↓ 19% (1.1 sec slower): Last Mile Load Time over previous year (comparative members)
Home page, backbone load time:
Average for all: 3.29 sec
Top 3 average: 1.64 sec
Bottom 3 average: 4.42
Mobile Page Weight
Average: 656 KB,
Fastest 3: 343 KB,
Slowest 3: 1,100 KB
Fastest overall Mobile Load Time of any industry
Airlines: Homepage -> Search -> Choose Flight
Home page, backbone load time:
Average for all: 5.32 sec
Top 3 average: 2.56 sec
Bottom 3 average: 9.56
Mobile Page Weight
Average: 616 KB,
Fastest 3: 233 KB,
Slowest 3: 964 KB
News: Homepage -> Search -> Article Details
Home page, backbone load time:
Average for all: 7.14 sec
Top 3 average: 2.14 sec
Bottom 3 average: 14.23
Mobile Page Weight
Average: 1,953 KB,
Fastest 3: 534 KB,
Slowest 3: 4,271 KB
Average Booking Transaction Load Time 7 sec less than Airlines Booking Transaction
Hotels: Homepage -> Hotel Page -> Search -> Room Details
Home page, backbone load time:
Average for all: 3.92 sec
Top 3 average: 2.42 sec
Bottom 3 average: 5.10
Booking Transaction Page Weight
Average: 3,436 KB,
Fastest 3: 2,079 KB,
Slowest 3: 6,242 KB
↑ 27% (2.0 sec faster): Mobile Load Time over previous year (comparative members)
Retail: Homepage -> Search -> Product Details -> Select Item -> Add to Cart -> Checkout -> Login -> Review Order and Payment
Home page, backbone load time:
Average for all: 4.22 sec
Top 3 average: 1.40 sec
Bottom 3 average: 7.59
Mobile Page Weight
Average: 1,106 KB,
Fastest 3: 371 KB,
Slowest 3: 1,629 KB
A New Way: Commit to full Digital Outcomes
It starts with….
Building a foundation that makes you operationally sound. Within the Digital Transformation, Operational Excellence is a MUST and incorporates (keeping the lights on) …
Performance and Rapid Root Cause Analysis across all problem patterns, all applications (modern + legacy), all the time (synthetic + real)
Technology Management – effective governance and control of 3rd parties (frontend and backend), Clouds, CDN’s, etc …
Agility across an ever evolving and increasingly complexity application delivery portfolio
But as true Digital Transformation requires full excellence below the line. Above the line is where Digital Outcome innovations, cultural change where everyone owns User Experience below the line in conjunction with the …
The customer experience via multi channels and commit to rapid innovation so that you not only are catching, testing for issues, but anticipating the future needs of your customers.
Without both disciplines, you are missing an opportunity to transform into a digital business that is focused on digital outcomes.
How can you achieve all of this today?
Companies today understand that the delivery of a great digital experience is very important. Inludes all channels.
Your customers evaluate you with every digital touchpoint, compared with their expectations set by their interactions with all companies – not just those I your industry.