The Truth About Your Web App's Performance

644 views

Published on

The performance of your web app is obviously important. But how do you know your web app is performing well for all of your users? Out of the box tools provide us metrics, but most only provide an overall view. This case study of building the XFINITY X1 single-page web app will demonstrate what frontend performance data you should be gathering, how to gather it, and how to make sense of all that data.

Existing tools provide insight into the performance of our web applications, but there is not a single tool that gives you the full picture. You can fill these gaps by gathering the performance data of your actual users. In this talk, we'll walk through the parts of the W3C Navigation Timing, High Resolution Time & User Timing recommendations that you can easily take advantage of right now to collect important metrics (with the help of Open Source software). We'll determine the "types" of users you need to focus on to understand your web app, as well as what other factors could impact those individual users' experiences. And we'll make sure "Average Response Time" is never the primary focus of your metrics dashboard

Published in: Software
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
644
On SlideShare
0
From Embeds
0
Number of Embeds
15
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Hello everyone. I'm John Riviello and I current work a few blocks from here at Comcast as the lead frontend developer for the customer entertainment websites
    As you can see I'm going to be talking about web app & website performance.

    Specifically I’m going to be discussing
    - What web performance data you should be gathering
    - How to gather that data, and
    - How to interpret & analyze that data so you can focus your efforts to improve your WEB APP’S PERFORMANCE on the areas that will provide the BIGGEST IMPACT to your users

    And of course, no surprise that you can do all of that with Open Source Software

    This talk is based on the work I did for a product known as X1 CloudTV.
  • So to provide some quick background on the product,

    you may have already heard of X1, which is the newest set top box from Comcast.

    CloudTV is the ability to store your DVR recordings in the cloud, which is nice because you get more space and then you can stream those recordings to any device in the home, as well as download them to your iOS or Android devices.

    It also supports Live TV and On Demand content.

    The idea is to have the same experience across all your devices, and I was one of the developers working on the web experience.

    We ended up building it as a SINGLE PAGE APP so that you can continue to STREAM VIDEO while you perform other activities, just like you do on your TV.

    It uses BackboneJS on the frontend with ruby on the server side talking to a hypermedia API, but what I'm going to talk about applies to all single page apps regardless of framework, and websites as well.

    So we're essentially turning your web browser into a TV, which was a lot of fun to build.

    Since I was recreating the TV experience, I had to think about how people typically experience TV.

    Think about being at home. You sit on the couch, pickup your remote (or your smartphone or your smartwatch), and turn on your TV.
    What happens? (pause)

    Video starts playing immediately. In the sense of startup speeds, televisions have always been fast. The slowest aspect was when your old tube television had to warm up for a few seconds.

    So it’s pretty clear that the PERFORMANCE of the web app I was building is itself a FEATURE, and a very important one at that.

    But that’s not only the case for this app, it’s the case for any app or website, including yours.

    So I knew we had to make our web app fast, and in our quest to do so…
  • …we've asked ourselves these questions that I'll be covering throughout this talk:

    What do existing tools provide?
    What data do you care about?
    How do you gather that data?
    What do you do with the data?

    Now before you can fulfill the super-specific requirement of "make it fast", you need to know:
    1. how fast or slow your app currently is, and
    2. a way to consistently monitor those metrics
    so that when you get a target either that you set or that is given to you, you can see your work helping (or not) and make sure you're focusing on the right things
  • So let's start off by looking at some existing tools, of which there are many.
    I’m going to touch on a few of them…
  • webpagetest.org
    - This tool was originally developed by AOL & then they open-sourced it and Google has since taken the reigns on it and done an excellent job
  • - This provides some great data & suggestions
    - Get to see the performance of the page on first load and then a repeat view when assets are cached
  • - You get a full waterfall showing in detail the timing information
  • - There's the connection view which shows each TCP socket
  • - You get a nice PageSpeed Optimization check with actions you can take to improve performance
  • - But here's one spot where it falls a bit short:
    - I assure you that this is not what our app looks like when it is "Fully Loaded".
    This is of course a loading screen, which we want to display for as short of a time as possible, but we need to know when the app is actually ready to use to understand the actual performance
    - now, there is a way to fix that in some browsers, but I'll get into those details a little later
  • If running your own metrics-collecting server with the help of Vagrant or Docker sounds like fun, then SiteSpeed.io provides all the details that WebPageTest does and much more, as well as support for custom metrics.
    (if that doesn’t, talk to the person in your organization that is into DevOps)

    You can deploy to various clouds (such as Digital Ocean) and collect data from different parts of the country & around the world

    3 Docker images: One containing Graphite, one with Grafana and one running sitespeed.io.

    This allows you to collect metrics, store the data, and then graph them

    The run.sitespeed.io allows you to test it out online
  • Here’s a peak at what it looks like. This is their dashboard that tracks WebPageTest runs over time
  • It provides an overall summary of the site you’re testing over time by testing multiple URLs & fetching them every 30 minutes

    These boxes at the bottom are colored based on performance budgets that you can configure
  • It will also provide metrics for an individual page
  • It’s also setup to compare performance metrics across multiple sites so you can track your competitors

    …Also, it has the capability to add your own metrics and graph them
  • One other great feature is it has a grunt plugin as well that will tell you if you’ve exceeded your performance budget
  • Phantomas is another tool, with a much cooler name
    Which as you can see is a phantomJS-based web performance metrics collector and monitoring tool.

    I find it most valuable when used with a Continuous Integration server where you push new code, deploy it, and then run this tool so you get a history of performance related changes and can tie them to a commit.

    I'd recommend running it via grunt-devperf
    1. The grunt integration makes for an easy setup, and
    2. Phantomas provides a TON of data, and grunt-devperf has a nice initial opening screen that looks like this…
  • to give you a high level overview of TimeToFirstByte, OnDOMReadyTIme, WindowOnLoadTime
    I'll switch over to my browser real quick to show you just how extensive these phantomas reports are
    [BROWSER]
  • For a more overall, single-run, score-based report, the author of Grunt-DevPerf has released an amazing online tool called
    Yellow Lab Tools
    Which is available at yellowlab.tools
  • It gives you an overall score, and you can dig down into the specifics

    Let’s switch over to the browser again and take a look
  • It has a grunt plugin as well, which is for running against a website in a CI environment (ideally a test or staging environment before deploying to production)

    So similar to sitespeed.io, it can fail the build based on your YellowLabTools scores
  • I really love their grunt configuration since it uses natural language
  • First you define the URLs you want to test
  • You can have it fail based on the overall global score

    'fail if at least one url has a global score < 80/100',
  • If can fail if any rule is below a certain score

    'fail if at least one url has a rule score < 50/100',
  • You can ignore certain rules

    'ignore iframesCount',
  • And you can fail against specific metric numbers instead of the score as well
    'fail if at least one url has a domElementsCount > 2000’
  • grunt-yellowlabtools is not currently capable of keeping an history but this is definitely something the author wants to work on in the next months!
    So for now you ideally would run both grunt-devperf and grunt-yellow-lab-tools
    The author’s plan is:
    - Adding the ability to send data to Graphite (which can then be called by Kibana for dashboard creation).
    - Then work on including history directly inside YellowLabTools, which should be followed by adding this to the grunt task too.
    - This is a lot of work, he needs help, so check it out and contribute! \\more below//

    And this is very useful, but this has it's limitations as well.

    For one, no one is actually using PhantomJS to USE your web app
    different users may have different experiences,

    Tools such as Phantomas & WebPageTest provide what is known as “Synthetic Monitoring” because it doesn’t reference actual user data.
    In the case of Phantomas it’s phantomJS running on either your machine or some server you’ve setup, and for WebPageTest it’s run from one of their servers they maintain around the world.

    so what you really want is …
  • RUM!!!

    I'm not actually talking about the booze, I'm actually talking about...
  • Real user monitoring (Real User Measurement or Metrics)

    It's not the "M" that's important, it's the "REAL USER" part that is important

    So let’s talk about some services that provide that…
  • There are lots of paid services to collect RUM data. I’ve listed a few of them here. You’ll find a lot of the same features with these.

    I’ve used or at least researched a good number of these, and I’ve learned that if you are a company that is going to offer Real User Monitoring, the FIRST feature you build looks something like this… (new slide)
  • You’ll often find a main dashboard where the center of attention will be a world map, like this one from AppDynamics.
    If you’re a global product then that may make sense, but for someone like Comcast where I’m working on products for customers that are 100% based on the United States, these world graphs are rather uninteresting to me.

    The charts are around it are where the good data is, such as overall response times by Average or various percentiles
  • Pretty much all provide a break down of the page load time over time. This graph from New Relic contains:
    Request Queuing
    Server Application Time
    Network Time
    DOM Processing
    Page Rendering time
  • New Relic and others also offer the ability to get around that issue of the fully loaded not actually being fully loaded be manually marking the spot that your app is “fully loaded”
  • Since they’re collecting real client-side data, many of these services also offer JavaScript error reporting which is pretty nice.

    There are actually some companies products that focus exclusively on capturing errors, and thankfully,
  • …their JavaScript code is open source, so you can see how they go about capturing stack traces from JavaScript by listening to window.onerror.
    So you can use these libraries from BugSnag & Sentry and tweak them to send the data to your own servers to analyze
  • OK so now we know about some existing tools,
    so let's talk about the data you care about
  • So again, we care about real users.

    And those services I just mentioned are useful, but they’re all paid services so you may not be using it, and if you are, there are some pieces of data I find it best to gather yourself to analyze

    What more could I possibly care about?

    Let's talk about a feature that Phantomas & these paid services rely on:
  • And that is navigation timing, which is a recommendation from the W3C

    If you want to see what it is, open up your favorite web developer tools console
    And run “window.performance.timing”…
  • and you'll get a bunch of numbers like this

    Now these are individual timestamps of various events of when a page was loaded by the browser

    And from those, we can calculate what was going on at different points of that http request
  • This illustration from the W3C spec explains that flow nicely.

    So you can see that covers:
    - unloading any previous document
    - checking HTML5 Application Cache
    - the DNS lookup, and
    - TCP handshake
    - then we get to the actual HTTP request
    - which is followed by the server actually responding
    - and then various DOM events as the document is built until finally,
    - the onLoad event fires \\//

    This is some super useful stuff.

    You can take a wider view…
  • and see how this provides info on:
    - Network Latency
    - Processing Time
    and those make up the majority of the
    - Full HTTP Request

    But then of course, there's this:...
  • The app may not actually be usable when the onLoad event fires...
  • so you need to account for that loading screen time...
  • and mark the end of it so you can determine how long the loading screen is displayed

    And if you do gather that data, you can build nice charts…
  • like this one that show the breakdown of Network Latency, Processing Time and the Loading Screen time

    But there are a bunch of things that could be affecting those response times & they really can't be represented in just one graph

    So let's talk about some of those
  • - User authentication state [CLICK THROUGH!]
    - signed in users most likely will be loading more data than users that are not signed in

    - “Type” of user
    - once a user is signed in, there may be different types of users
    - For example, in our app, you may or may not have DVR
    - you may subscribe to certain premium networks
    - or you may have a bunch of parental controls configured
    - or for users not signed in, perhaps you're doing some A/B testing and segmenting users that way to deliver different experiences

    - Number of “items” returned
    - For example, if you do have a DVR, you could have
    - a couple recordings
    - or hundreds
    - or even a thousand
    - Also the number of channels will vary by region

    - SWF dependency
    - this is probably the case for those delivering video (other than HTML5 video) or games (which are the typical uses of Flash these days)

    - since flash is tied to video and that is so critical to the app, we handle some parts of authentication through a SWF file...
  • Now that typically is embedded fairly quickly as I've pictured here, but it can bleed into the loading screen time
  • Taking a step outside of the CloudTV app and looking elsewhere, here's an example of this happening with Gmail:

    Notice that in this case, Gmail was slow when these 3 factors were true:
    - the user had the people widget enabled
    - they were using the website (not a mobile app)
    - and they opened an email with a large number of participants

    So you can see, without that level of detail in your performance metrics, it's going to be MUCH more difficult to determine the cause of performance issues especially if it’s only affecting a certain portion of your user base
  • So take a moment to consider what other unique factors in your app may impact performance
  • OK so now that we know what data you want,
    how can we go about gathering that data if we don’t have access to one of those paid services,
    or we want more control of the data?
  • So as I said, we need other marks, such as an "appReady" when the app is truly ready for user interaction

    So how do we mark that? ...
  • - Back in the day we did this
    - ECMAscript 5.1 gave us Date.now() [shorter to write & faster to execute]
    - but most modern browsers have a better option...
  • And that is High Resolution Time

    THAT SOUNDS COOL! What is it?
  • [Read bullet points]

    Let's see why that is useful…
  • First, the sub-millisecond accuracy

    I'm going to write a function using the old Date.now method that does something pretty simple, and you'll see it takes 0 milliseconds to complete!

    https://github.com/getify/You-Dont-Know-JS/blob/master/async%20%26%20performance/ch6.md
    Some platforms don't have single millisecond precision, but instead only update the timer in larger increments. For example, older versions of windows (and thus IE) had only 15ms precision, which means the operation has to take at least that long for anything other than 0 to be reported!

    when I use High Resolution time, I get a crazy- accurate response

    The spec says it should be accurate to the thousandth of a millisecond. Some browsers are even more accurate, and some on Windows are only accurate to the millisecond, but they at least use the monotonic clock...
  • Now as far as the Monotonic clock goes, let's go right to the person who reviewed the commit in chromium that added that for what that actually means:

    “Most systems run a daemon which regularly synchronizes the time. It is common for the clock to be tweaked a few milliseconds every 15-20 minutes.
    At that rate about 1% of 10 second intervals measured would be inaccurate.”

    So now we see the benefit of that
  • It's also nice because now our appReady mark is the actual time for the full App to be ready.

    No math necessary!

    But to figure out the loading screen, we do need math.

    So we know we have the LoadEventEnd time, but loadEventEnd & appReady have different sources of 0
    loadEventEnd = january 1, 1970
    appReady = hopefully only a few seconds ago

    So we can't even do math with them
  • so instead, we need to mark LoadEventEnd with a High Resolution timestamp when the page’s onload event fires,
    And then we have the same zero source so we can do math

    This is kind of a bummer, isn't it?

    Luckily, the W3C has realized this, and they're working on a new spec...
  • called Navigation Timing Level 2
    which uses the same zero source as High Resolution time
    and this will replace Navigation Timing Level 1

    but it's still in a Draft state, so no one can take advantage of that yet

    bummer again

    But there is something else...
  • and that's User Timing
    this is great for 2 reasons:
    - gives high resolution durations between user marks as well as a user mark and a navigation timing mark
    - webpagetest.org will not declare a document "fully loaded" until all user timing marks have been set

    Now, it’s not supported by all browsers, but there is a polyfill available so you can use the same methods, you just get limited functionality
  • So with that in mind, let's talk about Browser Support
  • As you can see, Navigation Timing Level 1 is in most modern browsers.

    IE added it back in Version 9.0

    iOS Safari added it in 8.0, but in their 8.1 update, if you scroll to the very bottom of the release notes, they mentioned they removed it due to “performance issues”, which is ironic and unfortunate
    Luckily, it appears to be back in v9
  • If we take a look at High Resolution Time, it’s the same for the current stable releases.

    IE didn't add it until Version 10,

    Android recently added it and again we have iOS teasing us with adding it in 8.0 and removing it in 8.1, but it’s back in 9.0 as well
  • User Timing is even less support, Safari hasn’t implemented it in any version.

    Good news though is Firefox added support for User Timing a few months ago
  • So that means essentially for every mark you make, you need a standard timestamp and a high resolution one (if the browser supports it) and a User Timing mark to be able to get as much info as possible

    So if you mark
    - Page Start as soon as you possibly can (that is, basically as the first line of code after the opening head tag)
    - LoadEventEnd when the onLoad event fires
    - And "AppReady" when the app is ready...
  • You can actually calculate your users' experience regardless of the browser their on

    BLUE = supports Navigation Timing
    RED = does not
    GREEN = all browsers

    Just make sure when you look at that data, you split it up by browser support

    So we do all of this in the CloudTV app. We:
    - set these marks and
    - do these calculations and
    - also log the info about the user and
    - the number of items returned by certain AJAX calls and
    - take browser support all into account, and
    and so on and so forth

    And when I went to do this work, I first searched for some open source project to handle all these inconsistencies and gather the extra data I needed, and I didn't find one that did that.

    So I opened sourced my library that handles that…
  • It’s called Surf And Perf

    You can check it out on Comcast's GitHub page
    It’s available as an NPM module
    and also as a RUBY GEM for it so you can easily integrate it into Rails & other sprockets-based ruby web applications like Middleman

    This micro library will handle:

    - Marks
    - Events (start & end mark, as well as additional data such as success/failure & number of items)
    - Custom Data (e.g. user state)
    - Calculations for the durations

    and package it all up into a nice javascript object that you can then send to your servers to log
  • Let’s talk about how to use Surf-N-Perf by comparing it to the User Timing API
  • Now remember, Surf-N-Perf provides this functionality to all browsers using the best available API

    I’d like to show you how you would use the User Timing API in the browsers that support it vs using Surf-N-Perf
    To show how Surf-N-Perf makes it easier & more useful to set & get performance data

    Duration doesn’t make much sense here, but since a PerformanceMark extends the PerformanceEntry interface from the PerformanceTimeline spec (which is leveraged by the Navigation Timing & User Timing, among others, and that part of a PerformanceEntry, it’s always there for marks, and it’s always set to 0
  • API for measure: window.performance.measure(MEASURE_NAME, MARK_1, MARK_2);

    - loadEventEnd -> the browser’s onload event fires
    Here the duration makes sense
    startTime is the first mark in the measure, which isn’t high-resolution time, but you do get a high-resolution duration
  • We know how to do this based on what I’ve showed so far

    Create 2 marks

    Surf-N-Perf takes the concept of events & makes it easier
    surfnperf.eventStart('bar'); will mark that start for you
  • Shifting to a few Surf-N-Perf-specific functions…

    We know how to do eventStart

    eventEnd actually takes an optional custom data object as its 2nd argument

    Then getEventData takes the event key as its 1st argument and the data you want as the 2nd

    An example of how we use this is to include the number of recordings a user has stored in their DVR
  • // mark the start of the AJAX call

    // mark the end of the AJAX call, set its status to 'success' and the number of items returned as 'items’

    // mark the end of the AJAX call, set its status to 'error'
  • Takes a key & a value
    surfnperf.setCustom('initialUrl', window.location.pathname);

    Then you can get it by the key
    surfnperf.getCustom('initialUrl');


    This is a relevant example, as time for the initial load of the app is based on what they requested, as the recent page will respond differently than the TV Grid, for example
  • Surf-N-Perf also has helper methods for calculating common durations between navigation timing marks
  • getNetworkTime(); // fetchStart to connectEnd
  • getServerTime(); // requestStart to responseEnd

  • getNetworkLatency(); // fetchStart to responseEnd


    Note it is NOT Network Time + Server Time, as there is usually a few milliseconds between connectEnd & requestStart
  • getProcessingLoadTime(); // responseEnd to loadEventEnd


    responseEnd (or pageStart for older browsers)
  • getFullRequestLoadTime(); // navigationStart to loadEventEnd


    navigationStart (or pageStart for older browsers) to loadEventEnd
  • OK so now you have a bunch of real user performance data filling up your logs. So the question is what to do with it.
  • We build various charts like this one I showed earlier using Splunk,
    but there are also Open Source tools such as
    Graphite
    Kibana and
    Graylog2
    that can generate similar charts from your logs, and all of them have Docker images available on Docker Hub

    Now the Y-axis here is response time and
    the X-axis is time (by hour),
    but my question to you is,
    what do you think is the value to chart for each hour?

    One thing that may pop into your head is average response times, and I'd argue you don't want to chart that, or at the very least, that shouldn't be your primary focus. Why?
  • Because Average response times are for average products

    And I know you don’t build average products, because you’re here at this conference on a sunny Saturday morning, and you wouldn’t be here if you didn’t want to build the best products

    Think about it. If you say your app response in under a second on average, does that really sound good?

    Would you want the servers you host your web apps on, be it AWS or your own data center or dream host or whatever,
    to be up and running "on average"

    of course not, you want them always to be running

    that's why you hear that industry talk about stuff like five 9s uptime (meaning they're up 99.999 percent of the time, which is less than 1 hour of downtime a month)

    So what you want to focus on is..
  • the 99 percentile
    And if your traffic is very large, you may want to even go further, to 3, 4 or 5 9s

    This way you're looking at what the VAST MAJORITY of your users are experiencing

    And if half are having a painful experience, staring at averages can mask that.
  • So let's dig down more into the types of 99th percentile charts that we look at:

    - First you can see we break response time down not only by network latency, processing and the loading screen
    but we build a chart of each of those depending on what part of the app the user loads first
    RECENT, GRID, UNAUTH, OTHER
  • - Here you can see how the number of recordings in a user's DVR can affect the response time of our API call for that
  • - And here you can see how, interestingly, the number of channels does NOT appear to have an affect
  • Now I mentioned earlier how we have this SWF that handles Auth, and it can bleed into the loading screen time.
    Now I've never actually seen that happen on my computer, but how do I know that happens?
    Here's the 99th percentile chart for that SWF
  • And this chart compares it to the Loading Screen 99th percentile
    As you can see, they follow each other,
    With the SWF taking more time than the Loading Screen sometimes
    which means that to bring down the 99th percentile, we're going to need to focus on that AuthSWF
  • A 95th percentile chart shows that we should see benefits there a well
  • At 90th the relationship isn't as strong but it could help
  • And then at 50th they're not that related
    So it's important to check a range of percentiles so you can estimate what the impact will be for a specific performance improvement

    Because although you want everyone to have a fast experience, you also want your performance tweaks to improve the most amount of users that are experiencing the bad performance issues
  • And just to help show how averages aren't as useful, here is a chart comparing the auth swf average to the loading screen average.

    If I was just looking at that, I wouldn't have realized the impact of the AuthSWF

    IF NO TIME LEFT:
    1-0-5-ENTER
  • I have a few minutes left, so there are a few other things I’d like to quickly share with you regarding what other front-end performance data you should be gathering
  • Time to First Paint = Perceived Performance
  • Internet Explorer actually has a metric for this

    It’s a timestamp measured in milliseconds from the Epoch

    Chrome also has a method called loadTimes() that includes a number of things…
  • One of which is firstPaintTime

    Issues:
    Only supported in those 2 browsers
    May actually report first paint for a white screen
  • Window.requestAnimationFrame() = requests that the browser call a specified function to update an animation before the next repaint. The argument of the callback function is a high-res timestamp
    It’s supported by IE10+, pretty much everything except for Opera Mini
  • Resource Timing is another W3C performance working group spec, and this provides similar timing information for resources loaded by the main page
  • This is accessible via window.performance.getEntriesByType(“resource”)

    So as you can see, you get:
    Redirect Time
    App Cache
    DNS
    TCP
    Request &
    Response times

    BUT, by default, this is only for resources loaded from the same origin… 3rd parties just provide this:
  • All you get is the duration of the request, and in this case, that unfortunately is more than download time.
    It also includes “blocking time” - the delay between when the browser realizes it needs to download a resource to the time that it actually starts downloading the resource.
    Typically this happens when there are more resources than TCP connections (6-12, depending on the browser).

    Now, there is a way around this if 3rd-parties add a header:
  • If a vendor adds that header, then you get the full picture.

    Only about 5% currently do, most notably, Google, Facebook, Disqus, and mPulse

    but this is growing.

    Also, this won’t be a big deal with HTTP/2 since you shouldn’t have to worry about blocking
  • Even though it’s still a draft, a lot of browsers have adopted it already

    IE10 added it, Safari is the only modern desktop browser that doesn’t have it

    Mobile Safari & Opera mini don’t either
  • TO RECAP:
  • I’m JohnRiv on Twitter,
    And there’s the URL for Surf-N-Perf on Github

    THANK YOU!
  • The Truth About Your Web App's Performance

    1. 1. The Truth About Your Web App’s Performance John Riviello @JohnRiv Distinguished Engineer, Comcast Interactive Media FOSSCON – August 22, 2015
    2. 2. John Riviello – The Truth Behind Your Web App’s Performance2
    3. 3. Frontend Performance Data 1. What do existing tools provide? 2. What data do you care about? 3. How do you gather that data? 4. What do you do with the data? John Riviello – The Truth Behind Your Web App’s Performance3
    4. 4. 1. Existing Tools 2. The Data You Care About 3. Gathering the Data 4. Analyzing the Data
    5. 5. Frontend Performance Data – Existing Tools WebPagetest.org John Riviello – The Truth Behind Your Web App’s Performance5
    6. 6. Frontend Performance Data – Existing Tools http://www.sitespeed.io/ https://run.sitespeed.io/ John Riviello – The Truth Behind Your Web App’s Performance11
    7. 7. Frontend Performance Data – Existing Tools https://github.com/sitespeedio/grunt-sitespeedio John Riviello – The Truth Behind Your Web App’s Performance16
    8. 8. Frontend Performance Data – Existing Tools https://github.com/macbre/phantomas https://github.com/gmetais/grunt-devperf John Riviello – The Truth Behind Your Web App’s Performance17
    9. 9. http://yellowlab.tools/
    10. 10. Frontend Performance Data – Existing Tools https://github.com/gmetais/grunt-yellowlabtools John Riviello – The Truth Behind Your Web App’s Performance21
    11. 11. Frontend Performance Data – Existing Tools John Riviello – The Truth Behind Your Web App’s Performance22 grunt.initConfig({ yellowlabtools: { production: { urls: [ 'https://xtv.comcast.net' ], failConditions: [ // The global score is the one calculated by Yellow Lab Tools 'fail if at least one url has a global score < 80/100', // Every single rule has its own score 'fail if at least one url has a rule score < 50/100', // You can ignore certain rules 'ignore iframesCount', // You can check a metric instead of the score by omitting '/100' 'fail if at least one url has a domElementsCount > 2000' ] } } });
    12. 12. Frontend Performance Data – Existing Tools John Riviello – The Truth Behind Your Web App’s Performance23 grunt.initConfig({ yellowlabtools: { production: { urls: [ 'https://xtv.comcast.net' ], failConditions: [ // The global score is the one calculated by Yellow Lab Tools 'fail if at least one url has a global score < 80/100', // Every single rule has its own score 'fail if at least one url has a rule score < 50/100', // You can ignore certain rules 'ignore iframesCount', // You can check a metric instead of the score by omitting '/100' 'fail if at least one url has a domElementsCount > 2000' ] } } });
    13. 13. Frontend Performance Data – Existing Tools John Riviello – The Truth Behind Your Web App’s Performance24 grunt.initConfig({ yellowlabtools: { production: { urls: [ 'https://xtv.comcast.net' ], failConditions: [ // The global score is the one calculated by Yellow Lab Tools 'fail if at least one url has a global score < 80/100', // Every single rule has its own score 'fail if at least one url has a rule score < 50/100', // You can ignore certain rules 'ignore iframesCount', // You can check a metric instead of the score by omitting '/100' 'fail if at least one url has a domElementsCount > 2000' ] } } });
    14. 14. Frontend Performance Data – Existing Tools John Riviello – The Truth Behind Your Web App’s Performance25 grunt.initConfig({ yellowlabtools: { production: { urls: [ 'https://xtv.comcast.net' ], failConditions: [ // The global score is the one calculated by Yellow Lab Tools 'fail if at least one url has a global score < 80/100', // Every single rule has its own score 'fail if at least one url has a rule score < 50/100', // You can ignore certain rules 'ignore iframesCount', // You can check a metric instead of the score by omitting '/100' 'fail if at least one url has a domElementsCount > 2000' ] } } });
    15. 15. Frontend Performance Data – Existing Tools John Riviello – The Truth Behind Your Web App’s Performance26 grunt.initConfig({ yellowlabtools: { production: { urls: [ 'https://xtv.comcast.net' ], failConditions: [ // The global score is the one calculated by Yellow Lab Tools 'fail if at least one url has a global score < 80/100', // Every single rule has its own score 'fail if at least one url has a rule score < 50/100', // You can ignore certain rules 'ignore iframesCount', // You can check a metric instead of the score by omitting '/100' 'fail if at least one url has a domElementsCount > 2000' ] } } });
    16. 16. Frontend Performance Data – Existing Tools John Riviello – The Truth Behind Your Web App’s Performance27 grunt.initConfig({ yellowlabtools: { production: { urls: [ 'https://xtv.comcast.net' ], failConditions: [ // The global score is the one calculated by Yellow Lab Tools 'fail if at least one url has a global score < 80/100', // Every single rule has its own score 'fail if at least one url has a rule score < 50/100', // You can ignore certain rules 'ignore iframesCount', // You can check a metric instead of the score by omitting '/100' 'fail if at least one url has a domElementsCount > 2000' ] } } });
    17. 17. Contribute to Open Source: https://github.com/gmetais/YellowLabTools
    18. 18. "RUM" by Tom B is licensed under CC BY-NC-SA 2.0
    19. 19. Real User Monitoring "yes, that's a rum filled coconut." by Brandon King is licensed under CC BY-NC 2.0 / Color adjusted from original
    20. 20. Frontend Performance Data – Existing Tools John Riviello – The Truth Behind Your Web App’s Performance31
    21. 21. http://www.appdynamics.com/product/browser-real-user-monitoring/
    22. 22. http://docs.newrelic.com/docs/new-relic-browser/page-load-timing-process
    23. 23. https://docs.newrelic.com/docs/browser/new-relic-browser/browser-agent-apis/manually-reporting-page-load-timing-data
    24. 24. https://ruxit.com/features/real-user-monitoring.html
    25. 25. https://github.com/bugsnag/bugsnag-js https://github.com/getsentry/raven-js
    26. 26. 1. Existing Tools 2. The Data You Care About 3. Gathering the Data 4. Analyzing the Data
    27. 27. Real User Monitoring "yes, that's a rum filled coconut." by Brandon King is licensed under CC BY-NC 2.0 / Color adjusted from original
    28. 28. Illustration from http://www.w3.org/TR/navigation-timing/#processing-model John Riviello – The Truth Behind Your Web App’s Performance41
    29. 29. Illustration from http://www.w3.org/TR/navigation-timing/#processing-model John Riviello – The Truth Behind Your Web App’s Performance42
    30. 30. John Riviello – The Truth Behind Your Web App’s Performance44
    31. 31. John Riviello – The Truth Behind Your Web App’s Performance45
    32. 32. Frontend Performance Data – The Data You Care About John Riviello – The Truth Behind Your Web App’s Performance46
    33. 33. Frontend Performance Data – The Data You Care About Example Factors That Impact Performance: John Riviello – The Truth Behind Your Web App’s Performance47 • User authentication state •“Type” of user • Number of “items” returned • Flash SWF dependency
    34. 34. John Riviello – The Truth Behind Your Web App’s Performance48
    35. 35. http://mashable.com/2014/01/31/gmail-slow/ John Riviello – The Truth Behind Your Web App’s Performance49 “Gmail’s People Widget appears to be the cause of the sluggishness, which is, only affecting Gmail on the web Google says…This is noticeable when users open an email conversation with a large number of participants…”
    36. 36. Consider what other unique factors in your app may impact performance
    37. 37. 1. Existing Tools 2. The Data You Care About 3. Gathering the Data 4. Analyzing the Data
    38. 38. John Riviello – The Truth Behind Your Web App’s Performance52
    39. 39. Frontend Performance Data – Gathering the Data Marking Timestamps John Riviello – The Truth Behind Your Web App’s Performance53 Back in the day: > new Date().getTime(); 1399754123456 ECMAScript 5.1: > Date.now(); 1399754123456 Most modern browsers have a better option…
    40. 40. Frontend Performance Data – Gathering the Data W3C High Resolution Time John Riviello – The Truth Behind Your Web App’s Performance55 •DOMHighResTimeStamp is available via window.performance.now() •Provides the time with sub-millisecond accuracy •Measured relative to the navigationStart attribute of the PerformanceTiming interface •Not subject to system clock skew or adjustments (uses a monotonically increasing clock)
    41. 41. Frontend Performance Data – Gathering the Data W3C High Resolution Time – Sub-ms Example John Riviello – The Truth Behind Your Web App’s Performance56 > var dateTest = function() { var start = Date.now(), area = window.innerWidth*window.innerHeight; return Date.now() - start; }; dateTest(); 0 > var highResTest = function() { var start = window.performance.now(), area = window.innerWidth*window.innerHeight; return window.performance.now() - start; }; highResTest(); 0.01200000406242907
    42. 42. Frontend Performance Data – Gathering the Data W3C High Resolution Time – Monotonic Clock John Riviello – The Truth Behind Your Web App’s Performance57 Why do we care? “Most systems run a daemon which regularly synchronizes the time. It is common for the clock to be tweaked a few milliseconds every 15-20 minutes. At that rate about 1% of 10 second intervals measured would be inaccurate.” Source: Tony Gentilcore http://gent.ilcore.com/2012/06/better-timer-for-javascript.html
    43. 43. John Riviello – The Truth Behind Your Web App’s Performance58
    44. 44. John Riviello – The Truth Behind Your Web App’s Performance59
    45. 45. Browser Support
    46. 46. John Riviello – The Truth Behind Your Web App’s Performance66
    47. 47. John Riviello – The Truth Behind Your Web App’s Performance67
    48. 48. https://github.com/Comcast/Surf-N-Perf
    49. 49. User Timing vs. Surf-N-Perf "Hurricane Earl's Waves" by John Riviello is licensed under CC BY-NC-SA 2.0 / Color adjusted from original
    50. 50. Frontend Performance Data – Gathering the Data with Surf-N-Perf Setting & Getting a Mark John Riviello – The Truth Behind Your Web App’s Performance70 // User Timing API > window.performance.mark('foo'); > window.performance.getEntriesByName('foo'); [PerformanceMark ] duration: 0 entryType: "mark" name: "foo" startTime: 3323.620999988634 > window.performance.getEntriesByName('foo')[0].startTime; 3323.620999988634 // Surf-N-Perf > surfnperf.mark('foo'); > surfnperf.getMark('foo'); 3323.620999988634
    51. 51. Frontend Performance Data – Gathering the Data with Surf-N-Perf Navigation Timing Mark & User Mark Duration John Riviello – The Truth Behind Your Web App’s Performance71 // User Timing API > window.performance.mark('foo'); > window.performance.measure('page_load_to_foo', 'loadEventEnd', 'foo'); > window.performance.getEntriesByName('page_load_to_foo'); [PerformanceMeasure ] duration: 3201.620999988634 entryType: "measure" name: "page_load_to_foo" startTime: 122 > window.performance.getEntriesByName('page_load_to_foo')[0].duration; 3201.620999988634 // Surf-N-Perf > surfnperf.mark('foo'); > surfnperf.duration('loadEventEnd','foo'); 3202 > surfnperf.duration('loadEventEnd','foo',{decimalPlaces:3}); 3201.621
    52. 52. Frontend Performance Data – Gathering the Data with Surf-N-Perf Event Duration (i.e. 2 User Marks) John Riviello – The Truth Behind Your Web App’s Performance72 // User Timing API > window.performance.mark('barStart'); > window.performance.mark('barEnd'); > window.performance.measure('barEvent', 'barStart', 'barEnd'); > window.performance.getEntriesByName(’barEvent')[0].duration; 3512.499000004027 // Surf-N-Perf > surfnperf.eventStart('bar'); > surfnperf.eventEnd('bar'); > surfnperf.eventDuration('bar'); 3512 > surfnperf.eventDuration('bar',{decimalPlaces:12}); 3512.499000004027
    53. 53. Frontend Performance Data – Gathering the Data with Surf-N-Perf Custom Event Data John Riviello – The Truth Behind Your Web App’s Performance73 // Surf-N-Perf > surfnperf.eventStart('bar'); // surfnperf.eventEnd(KEY, CUSTOM_DATA_OBJECT); > surfnperf.eventEnd('bar', {baz:'qux'}); > surfnperf.getEventData('bar', 'baz'); "qux"
    54. 54. Frontend Performance Data – Gathering the Data with Surf-N-Perf Tying It Together with Backbone.fetch() John Riviello – The Truth Behind Your Web App’s Performance74 // Surf-N-Perf > var collection = new Backbone.Collection(); collection.url = '/get-data/'; surfnperf.eventStart('getData'); collection.fetch({ success: function(collection) { surfnperf.eventEnd('getData', {status:'success', items: collection.length}); }, error: function() { surfnperf.eventEnd('getData', {status:'error'}); }, }); > surfnperf.eventDuration('getData', {decimalPlaces:2}); 1464.75 > surfnperf.getEventData('getData', 'status'); "success" > surfnperf.getEventData('getData', 'items'); 33
    55. 55. Frontend Performance Data – Gathering the Data with Surf-N-Perf Custom Data (not tied to an event) John Riviello – The Truth Behind Your Web App’s Performance75 // Surf-N-Perf > surfnperf.setCustom('initialUrl', window.location.pathname); > surfnperf.getCustom('initialUrl'); "https://xtv.comcast.net/recent"
    56. 56. Frontend Performance Data – Gathering the Data with Surf-N-Perf Common Navigation Timing Measurements John Riviello – The Truth Behind Your Web App’s Performance76 // Surf-N-Perf
    57. 57. Frontend Performance Data – Gathering the Data with Surf-N-Perf Common Navigation Timing Measurements John Riviello – The Truth Behind Your Web App’s Performance77 > surfnperf.getNetworkTime(); // fetchStart to connectEnd
    58. 58. Frontend Performance Data – Gathering the Data with Surf-N-Perf Common Navigation Timing Measurements John Riviello – The Truth Behind Your Web App’s Performance78 > surfnperf.getServerTime(); // requestStart to responseEnd
    59. 59. Frontend Performance Data – Gathering the Data with Surf-N-Perf Common Navigation Timing Measurements John Riviello – The Truth Behind Your Web App’s Performance79 > surfnperf.getNetworkLatency(); // fetchStart to responseEnd
    60. 60. Frontend Performance Data – Gathering the Data with Surf-N-Perf Common Navigation Timing Measurements John Riviello – The Truth Behind Your Web App’s Performance80 > surfnperf.getProcessingLoadTime(); // responseEnd to loadEventEnd
    61. 61. Frontend Performance Data – Gathering the Data with Surf-N-Perf Common Navigation Timing Measurements John Riviello – The Truth Behind Your Web App’s Performance81 > surfnperf.getFullRequestLoadTime(); // navigationStart to loadEventEnd
    62. 62. 1. Existing Tools 2. The Data You Care About 3. Gathering the Data 4. Analyzing the Data
    63. 63. Frontend Performance Data – Analyzing The Data John Riviello – The Truth Behind Your Web App’s Performance83
    64. 64. Average Response Times are for Average Products
    65. 65. PERCENTILE (at a minimum)
    66. 66. Frontend Performance Data – Analyzing The Data John Riviello – The Truth Behind Your Web App’s Performance86
    67. 67. Frontend Performance Data – Analyzing The Data John Riviello – The Truth Behind Your Web App’s Performance87
    68. 68. Frontend Performance Data – Analyzing The Data John Riviello – The Truth Behind Your Web App’s Performance88
    69. 69. Frontend Performance Data – Analyzing The Data John Riviello – The Truth Behind Your Web App’s Performance89
    70. 70. Frontend Performance Data – Analyzing The Data John Riviello – The Truth Behind Your Web App’s Performance90
    71. 71. Frontend Performance Data – Analyzing The Data John Riviello – The Truth Behind Your Web App’s Performance91
    72. 72. Frontend Performance Data – Analyzing The Data John Riviello – The Truth Behind Your Web App’s Performance92
    73. 73. Frontend Performance Data – Analyzing The Data John Riviello – The Truth Behind Your Web App’s Performance93
    74. 74. Frontend Performance Data – Analyzing The Data John Riviello – The Truth Behind Your Web App’s Performance94
    75. 75. BONUS! What other front-end performance data should you focus on?
    76. 76. Frontend Performance Data – Additional Metrics Time to First Paint John Riviello – The Truth Behind Your Web App’s Performance96 ”Love for paint" by Derek Gavey is licensed under CC BY 2.0
    77. 77. Frontend Performance Data – Additional Metrics Time to First Paint John Riviello – The Truth Behind Your Web App’s Performance97 Internet Explorer: > window.performance.timing.msFirstPaint; 1434341969277 Google Chrome: > window.chrome.loadTimes(); Object { commitLoadTime: 1434341655.700179 connectionInfo: "http/1" finishDocumentLoadTime: 1434341656.208713 finishLoadTime: 1434341656.733739 ...
    78. 78. Frontend Performance Data – Additional Metrics Time to First Paint John Riviello – The Truth Behind Your Web App’s Performance98 Google Chrome (continued): firstPaintAfterLoadTime: 1434341657.201959 firstPaintTime: 1434341655.978471 navigationType: "Other" npnNegotiatedProtocol: "unknown" requestTime: 1434341655.141803 startLoadTime: 1434341655.570092 wasAlternateProtocolAvailable: false wasFetchedViaSpdy: false wasNpnNegotiated: false }
    79. 79. Frontend Performance Data – Additional Metrics Time to First Paint John Riviello – The Truth Behind Your Web App’s Performance99 Alternatives: -window.requestAnimationFrame() - Load of last non-async resource in <head> - Custom Metric (First Tweet, Hero Image, etc.) http://www.stevesouders.com/blog/2015/05/12/hero-image-custom-metrics/
    80. 80. Frontend Performance Data – Additional Metrics John Riviello – The Truth Behind Your Web App’s Performance101 Illustration from http://www.w3.org/TR/resource-timing/#processing-model window.performance.getEntriesByType("resource")
    81. 81. Frontend Performance Data – Additional Metrics John Riviello – The Truth Behind Your Web App’s Performance102 3rd Party Resource
    82. 82. Frontend Performance Data – Additional Metrics John Riviello – The Truth Behind Your Web App’s Performance103 3rd Party Resource with Timing-Allow-Origin: *
    83. 83. Recap
    84. 84. Frontend Performance Data – Recap John Riviello – The Truth Behind Your Web App’s Performance106 • Performance is a feature! • Measure first—at the 99th percentile • Leverage W3C Performance APIs • Log network latency, browser processing time, and the full webpage request & response • Log major app-specific events with details For Further Info & Feedback: Twitter: @JohnRiv GitHub: https://github.com/Comcast/Surf-N-Perf

    ×