Eight wpo apm best practices for the holiday season


Published on

eBook can be downloaded here: http://www.yottaa.com/ebook-eight-wpo-apm-best-practices-for-the-holiday-season

  • Be the first to comment

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Eight wpo apm best practices for the holiday season

  1. 1. Ensuring Success &Avoiding Site Disruptions:Tips for WebOps, WebDev andMarketingEight WPO/APM Best Practicesfor the Holiday SeasonYottaa, Inc.101 Summer StreetBoston MA 02110http://www.yottaa.com© 2012 Yottaa, Inc. All Rights Reserved.
  2. 2. Contents• Introduction• Tip #1: Monitor performance & set up alerts• Tip #2: Use real browsers to monitor user experience• Tip #3: Monitor key pages• Tip #4: Trend user experience• Tip #5: See what your users see• Tip #6: Establish performance & SLA goals• Tip #7: Capture & diagnose site & app issues• Tip #8: Carry out statistically meaningful sampling• Appendix: Key site performance and user experience metrics 2
  3. 3. Introduction• Holiday shopping is expected to rise nearly 17% this season, and sites large and small want the competitive edge to capture as much revenue as possible.• Is your site ready for the holiday rush? How do the best sites monitor and manage their site performance and availability?• In this eBook, we identify the best practices to help you be proactive and prevent site problems during the holiday season. 3
  4. 4. 24/7 Monitoring & Alerting• Set up 24/7 monitoring• Set up Email/text/SMS alertsGet started for free here
  5. 5. Best Practices• Configure issue definitions • Customize Alert Escalations to different people Critical Error: Recommend Senior Management: network connectivity and page Consider text messaging status code>400 as critical for critical errors not errors (this is the default resolved after a certain setting) duration Error: Recommend Development Team: HTML content check Consider email alerting for as an error critical errors, errors and maybe warnings Warning: Recommend treat page asset issues, user Operations Team: Consider experience issues and text messaging for critical network performance issues errors and errors as warnings
  6. 6. Real Browser Monitoring• Only real browser monitoring provides visibility into your users’ experience• With Firefox, IE and Chrome browsers from locations across the planet, you can capture your users’ experience and see exactly what your users see
  7. 7. Best Practices• Select monitor agent • Monitoring • Select the Agent: frequency: locations your • Use HTTP agent for APIs HTTP and HTTP services – For availability users are from • Use HTTP agent if you monitoring, try are only interested in 5 min or less • To monitor service availability and – For Operations logged in pages, network performance Team, try 5 min • Use browser agents or less use Yottaa’s if you are interested in front end as well as – For Development “Advanced” tab backend performance and availability, including Team, try 10 min (more later) to 30 min monitoring third party assets/widgets Browser Agent Choice: • Pick the most popular browser for your site • IE9 is still the most popular browser in general, but your site may differ
  8. 8. Monitor Pages Key for Your BusinessMonitor:• App pages• Pages behind a log-in• Customer sites(specific examples on next page) Using cookies and other methods, monitor pages behind log-in, user sessions such as shopping cart, etc.
  9. 9. Best Practices: Which Pages to Monitor Web Sites & User Mobile Apps SaaS RegistrationWeb Sites & SaaS Apps:Mobile Apps: • User dashboard page• Home page • Key pages behind• Key landing pages user login (e.g., User registration page)• Pages using different e-Commerce technologies (e.g., User forum, knowledge base)• Dynamically-generated e-Commerce Sites: pages • Product catalog page• Pages from different • Product detail page origin servers • Customer shopping cart page • Customer checkout page
  10. 10. Best Practice: Performance Dashboard
  11. 11. Track User Experience Trending Pages behind login can be really slow, and perform inconsistently
  12. 12. Best Practices: What Trends to Look For?• Look at the Summary Section for Average Values – If the average values are way out of the range, consider you have problems• Look at the Trending Curves: – Trending curves should be as flat as possible (consistent) – If there are sudden spikes, these spikes are typically problems • Click into them to look at the data samples to see what happened – If the trending curve goes up and down too much, you may have a consistency issue • A consistent experience means confidence and reliability• Trending Curves to look into: – Time to Interact – Time to Start Render – Time to Last Byte (see appendix for definitions)
  13. 13. See What Your Users See• Capture user experience as video and screenshot filmstrips• Correlate with user experience & site performance metrics
  14. 14. Best Practices• See What Your • Analyze the • Stats to Think Users See Page Load About: – Look at the film strip, Waterfall – How many requests would you be happy are there? Too many as a user? – Are there any requests can mean particularly slow a slow experience – Play the film strip loading assets? as a video to get – How many third party a realistic replay – Are there any HTTP assets? Too many of the actual user status code errors? of them can mean experience – Are their obvious an unreliable user gaps in the waterfall? experience • Which typically means heavy client side JavaScript execution blocking the entire browser HTTP
  15. 15. Ensure Performance & SLA ObjectivesGet started for free here
  16. 16. Capture & Diagnose Site & App Issues• Define and monitor the issues that matter to your business• Gain 360 visibility into all factors impacting your site: – Server issues – Page errors – Network problems – 3rd-party asset issues
  17. 17. Capture & Diagnose Site & App Issues
  18. 18. Best Practices: Dealing with Issues• Configure issue definitions • If you are only interested in monitoring certain domains, Critical Error: Recommend network connectivity and page you can do so status code>400 as critical (simply configure it) errors (this is the default setting) • Check the Issue Listing Page to Error: Recommend HTML content check see how many as an error events each Warning: Recommend treat issue has page asset issues, user generated experience issues and – The more events, the higher network performance issues as warnings priority to tackle the issue
  19. 19. Statistically Meaningful Testing• Eliminate random noise• Schedule tests across geographies, browsers, last mile connectivitiesGet started for free here
  20. 20. Best Practices• Monitoring & Testing • Operations Team: – Use WebsiteTest.com Site Monitor: to diagnose Is good for continuous monitoring and issue issues alerting WebSite Test: Is great for on-demand testing • Development Team: and diagnosis – Use WebsiteTest.com • Statistically to analyze & improve site Meaningful Testing: Collect enough number of samples to improve signal to noise ratio
  21. 21. Improve Web & Mobile Experience• Explore every asset’s impact on performance• Identify bottlenecks using waterfall chart (In this example – the Facebook widget is slowing down the page load!)
  22. 22. Best Practices• Analyze • Check Data • Analyze Individual Trending First Samples Related Data Samples – Use “Edit Filter” to “Odd” Trending – Does the page loading to compare key filmstrip look ok? – To determine the root metrics such as Time – What does the page cause for “odd” to Last Byte, Time to load waterfall show? Start Render and – Any error or warning? Any particular slow Time to Interact assets, HTTP errors? – Identify “Odd” data points from trending lines – Identify if any issue is a consistent issue or only temporary
  23. 23. Appendix: A Primer onUser Experience &Site Performance Metrics 23
  24. 24. Characterizing User ExperienceGet started for free here
  25. 25. Time to Title • Time at which the browser is able to display the title of the site in the title bar. • The first visual indicator to the user that the loading process is working. • Typically, ~1 sec.Get started for free here
  26. 26. Time to Start Render • (Also know as “Time to First Paint”) • The time at which browser draws the first visual element of a web page. • Up to that point, a blank screen  high risk of abandonment if takes too long. • Typically, ~2 sec.Get started for free here
  27. 27. Time to Display• time at which browser has finished parsing the HTML web page, constructed the Document Object Model (DOM) and displayed the HTML document.• (However, the assets in the document such as images may not have been loaded yet.)• Typically, ~5 sec.Get started for free here
  28. 28. Time to Interact• The length of time it took before a user could interact with the page (type into a field, click on a link, etc.).• Typically, ~6 sec.
  29. 29. Yottaa Site Monitor• 24/7 Site Monitoring is always on alert & ready to help• Count on Yottaa to monitor user experience & identify problems• Multivariate testing & web performance troubleshooting with real-browsers• See what your users see from any device, any location, & any browserGet started for free here 29