Analysis of user experience is typically done by taking a random sample of users, measuring their experiences and extracting a single number from that sample. In terms of web performance, the experience we need to measure is user perceived page load time, and the single number we need to extract depends on the distribution of measurements across the sample.
There are a few contenders for what the magic number should be. Do you use the mean, median, mode, or something else? How do you determine the correctness of this number or whether your sample size is large enough? Is one number sufficient?
This talk covers some of the statistics behind figuring out which numbers one should be looking at and how to go about extracting it from the sample.
Analysis of user experience is typically done by taking a random sample of users, measuring their experiences and extracting a single number from that sample. In terms of web performance, the experience we need to measure is user perceived page load time, and the single number we need to extract depends on the distribution of measurements across the sample.
There are a few contenders for what the magic number should be. Do you use the mean, median, mode, or something else? How do you determine the correctness of this number or whether your sample size is large enough? Is one number sufficient?
This talk covers some of the statistics behind figuring out which numbers one should be looking at and how to go about extracting it from the sample.
An Approach to Data Quality for Netflix Personalization SystemsDatabricks
Personalization is one of the key pillars of Netflix as it enables each member to experience the vast collection of content tailored to their interests.
[MicroCPH 2019] Airbnb's Great Migration: Building Services at ScaleJessica Tai
MicroCPH 2019 conference in Copenhagen (https://microcph.dk/) abstract:
So you’ve decided to migrate from monolith to microservices, what next? Such a redesign to service-oriented architecture (SOA) is a long, arduous journey that benefits from an incremental, iterative approach. Yet, such a migration often must be done while still shipping new features, accelerating developer velocity, and growing the team in addition to ensuring there are no performance regressions.
This talk will focus on how Airbnb is building, operating, and scaling its expanding network of services. Though our re-architecture to SOA is still ongoing, we are already seeing various benefits including improved performance, developer productivity, build and deploy times, and site reliability.
Key takeaways:
* Understand design principles for building scalable, performant services
* Plan for dependencies: how to sequence decomposition into services and an API gateway
* Learn best practices for standardization, reliability, and performance when migrating architecture
* Identify ways to shift product culture to empower migration work
* Recognize tradeoffs with operating microservices
Performance Quality Metrics for Mobile Web and Mobile Native - Agile Testing ...Andreas Grabner
5 Real Life Examples on why Mobile Web and Mobile Native Apps failed and Which Metrics would have shown the problem early on.
Using these metrics along your delivery chain allows you go get closer to full automated deployment pipeline but also making sure performance criteria is met
Frontend Performance: Beginner to Expert to Crazy PersonPhilip Tellis
There’s no such thing as fast enough. You can always make your website faster. This talk will show you how. The very first requirement of a great user experience is actually getting the bytes of that experience to the user before they they get tired and leave.In this talk we’ll start with the basics and get progressively insane. We’ll go over several frontend performance best practices, a few anti-patterns, the reasoning behind the rules, and how they’ve changed over the years. We’ll also look at some great tools to help you.
Frontend Performance: De débutant à Expert à Fou FurieuxPhilip Tellis
Frontend Performance Beginner to Expert to Crazy Person
The very first requirement of a great user experience is actually getting the bytes of that experience to the user before they they get tired and leave.
In this talk we'll start with the basics and get progressively insane. We'll go over several frontend performance best practices, a few anti-patterns, the reasoning behind the rules, and how they've changed over the years. We'll also look at some great tools to help you.
La performance front-end de débutant, à expert, à fou furieux !
La toute première condition nécessaire à une bonne expérience utilisateur est de pouvoir obtenir les octets de cette expérience avant que l'utilisateur ne se lasse et parte.
Nous débuterons cette conférence avec les bases pour progressivement devenir démentiel. Nous aborderons plusieurs des meilleurs pratiques de la performance front-end, quelques anti-patterns à éviter, le raisonnement derrière les règles, et comment ces dernières ont changé au fil des ans. Nous regarderons d'un peu plus près quelques très bon outils qui peuvent vous aider.
RUM isn’t just for page level metrics anymore. Thanks to modern browser updates and new techniques we can collect real user data at the object level, finding slow page components and keeping third parties honest.
In this talk we will show you how to use Resource Timing, User Timing, and other browser tricks to time the most important components in your page. We’ll also share recipes for several of the web’s most popular third parties. This will give you a head start on measuring object level performance on your own site.
Frontend Performance: Beginner to Expert to Crazy PersonPhilip Tellis
Boston Web Performance Meetup, April 22, 2014
The very first requirement of a great user experience is actually getting the bytes of that experience to the user before they they get fed up and leave. In this talk we'll start with the basics and get progressively insane. We'll go over several front-end performance best practices, a few anti-patterns, the reasoning behind the rules, and how they've changed over the years. We'll also look at some great tools to help you.
Schedule: 6:30, pizza
7:15: talk
Frontend Performance: Beginner to Expert to Crazy PersonPhilip Tellis
The very first requirement of a great user experience is actually getting the bytes of that experience to the user before they they get fed up and leave.
In this talk we'll start with the basics and get progressively insane. We'll go over several frontend performance best practices, a few anti-patterns, the reasoning behind the rules, and how they've changed over the years. We'll also look at some great tools to help you.
When we built boomerang at Yahoo!, we planned on it being a generic beaconing system with different payloads attached by plugins. We published an API, and wrote plugins to measure page roundtrip time, network throughput and latency. We received other plugins from Yahoo! to measure IPv6 and DNS latency, and then nothing happened...
Until one day, a certain Mr. Brewer submitted a NavTiming plugin. As it turns out, people were using boomerang in-house, and creating their own plugins that were never published.
In this talk, we’ll go over the basics of writing a boomerang plugin to measure anything you need, some best practices involved with writing plugins, and examples of third party plugins that others have written.
Abusing JavaScript to measure Web Performance, or, "how does boomerang work?"Philip Tellis
While building boomerang, we developed many interesting methods to measure network performance characteristics using JavaScript running in the browser. While the W3C's NavigationTiming API provides access to many performance metrics, there's far more you can get at with some creative tweaking and analysis of how the browser reacts to certain requests.
In this talk, I'll go into the details of how boomerang works to measure network throughput, latency, TCP connect time, DNS time and IPv6 connectivity. I'll also touch upon some of the other performance related browser APIs we use to gather useful information.
http://www.nywebperformance.org/events/78566362/
An Approach to Data Quality for Netflix Personalization SystemsDatabricks
Personalization is one of the key pillars of Netflix as it enables each member to experience the vast collection of content tailored to their interests.
[MicroCPH 2019] Airbnb's Great Migration: Building Services at ScaleJessica Tai
MicroCPH 2019 conference in Copenhagen (https://microcph.dk/) abstract:
So you’ve decided to migrate from monolith to microservices, what next? Such a redesign to service-oriented architecture (SOA) is a long, arduous journey that benefits from an incremental, iterative approach. Yet, such a migration often must be done while still shipping new features, accelerating developer velocity, and growing the team in addition to ensuring there are no performance regressions.
This talk will focus on how Airbnb is building, operating, and scaling its expanding network of services. Though our re-architecture to SOA is still ongoing, we are already seeing various benefits including improved performance, developer productivity, build and deploy times, and site reliability.
Key takeaways:
* Understand design principles for building scalable, performant services
* Plan for dependencies: how to sequence decomposition into services and an API gateway
* Learn best practices for standardization, reliability, and performance when migrating architecture
* Identify ways to shift product culture to empower migration work
* Recognize tradeoffs with operating microservices
Performance Quality Metrics for Mobile Web and Mobile Native - Agile Testing ...Andreas Grabner
5 Real Life Examples on why Mobile Web and Mobile Native Apps failed and Which Metrics would have shown the problem early on.
Using these metrics along your delivery chain allows you go get closer to full automated deployment pipeline but also making sure performance criteria is met
Frontend Performance: Beginner to Expert to Crazy PersonPhilip Tellis
There’s no such thing as fast enough. You can always make your website faster. This talk will show you how. The very first requirement of a great user experience is actually getting the bytes of that experience to the user before they they get tired and leave.In this talk we’ll start with the basics and get progressively insane. We’ll go over several frontend performance best practices, a few anti-patterns, the reasoning behind the rules, and how they’ve changed over the years. We’ll also look at some great tools to help you.
Frontend Performance: De débutant à Expert à Fou FurieuxPhilip Tellis
Frontend Performance Beginner to Expert to Crazy Person
The very first requirement of a great user experience is actually getting the bytes of that experience to the user before they they get tired and leave.
In this talk we'll start with the basics and get progressively insane. We'll go over several frontend performance best practices, a few anti-patterns, the reasoning behind the rules, and how they've changed over the years. We'll also look at some great tools to help you.
La performance front-end de débutant, à expert, à fou furieux !
La toute première condition nécessaire à une bonne expérience utilisateur est de pouvoir obtenir les octets de cette expérience avant que l'utilisateur ne se lasse et parte.
Nous débuterons cette conférence avec les bases pour progressivement devenir démentiel. Nous aborderons plusieurs des meilleurs pratiques de la performance front-end, quelques anti-patterns à éviter, le raisonnement derrière les règles, et comment ces dernières ont changé au fil des ans. Nous regarderons d'un peu plus près quelques très bon outils qui peuvent vous aider.
RUM isn’t just for page level metrics anymore. Thanks to modern browser updates and new techniques we can collect real user data at the object level, finding slow page components and keeping third parties honest.
In this talk we will show you how to use Resource Timing, User Timing, and other browser tricks to time the most important components in your page. We’ll also share recipes for several of the web’s most popular third parties. This will give you a head start on measuring object level performance on your own site.
Frontend Performance: Beginner to Expert to Crazy PersonPhilip Tellis
Boston Web Performance Meetup, April 22, 2014
The very first requirement of a great user experience is actually getting the bytes of that experience to the user before they they get fed up and leave. In this talk we'll start with the basics and get progressively insane. We'll go over several front-end performance best practices, a few anti-patterns, the reasoning behind the rules, and how they've changed over the years. We'll also look at some great tools to help you.
Schedule: 6:30, pizza
7:15: talk
Frontend Performance: Beginner to Expert to Crazy PersonPhilip Tellis
The very first requirement of a great user experience is actually getting the bytes of that experience to the user before they they get fed up and leave.
In this talk we'll start with the basics and get progressively insane. We'll go over several frontend performance best practices, a few anti-patterns, the reasoning behind the rules, and how they've changed over the years. We'll also look at some great tools to help you.
When we built boomerang at Yahoo!, we planned on it being a generic beaconing system with different payloads attached by plugins. We published an API, and wrote plugins to measure page roundtrip time, network throughput and latency. We received other plugins from Yahoo! to measure IPv6 and DNS latency, and then nothing happened...
Until one day, a certain Mr. Brewer submitted a NavTiming plugin. As it turns out, people were using boomerang in-house, and creating their own plugins that were never published.
In this talk, we’ll go over the basics of writing a boomerang plugin to measure anything you need, some best practices involved with writing plugins, and examples of third party plugins that others have written.
Abusing JavaScript to measure Web Performance, or, "how does boomerang work?"Philip Tellis
While building boomerang, we developed many interesting methods to measure network performance characteristics using JavaScript running in the browser. While the W3C's NavigationTiming API provides access to many performance metrics, there's far more you can get at with some creative tweaking and analysis of how the browser reacts to certain requests.
In this talk, I'll go into the details of how boomerang works to measure network throughput, latency, TCP connect time, DNS time and IPv6 connectivity. I'll also touch upon some of the other performance related browser APIs we use to gather useful information.
http://www.nywebperformance.org/events/78566362/
The Statistics of Web Performance AnalysisPhilip Tellis
If you're interested in measuring real user web performance, you'll find tools like boomerang or episodes quite handy. Some popular web frameworks even have modules that make it easy to add them to your site. However, what does one do once one has collected the data? How do you filter out the noise and get meaningful insights from the data?
In this talk, I'll go over the techniques we've picked up by analyzing millions of datapoints daily. I'll cover some simple rules to filter out invalid data, and the statistics to analyze and make sense of what's left. Do you use the mean, median or mode? What about the geometric mean and standard deviation? How confident are we in the results? And finally, why should we care?
This talk should help you gain useful insights from a histogram, or at the very least point you in the right direction for further analysis.
Abusing JavaScript to Measure Web PerformancePhilip Tellis
While building boomerang, we developed many interesting methods to measure network performance characteristics using JavaScript running in the browser. While the W3C's NavigationTiming API provides access to many performance metrics, there's far more you can get at with some creative tweaking and analysis of how the browser reacts to certain requests.
In this talk, I'll go into the details of how boomerang works to measure network throughput, latency, TCP connect time, DNS time and IPv6 connectivity. I'll also touch upon some of the other performance related browser APIs we use to gather useful information. I will NOT be covering the W3C Navigation Timing API since that's been covered by Alois Reitbauer in a previous Boston Web Perf talk.
Real user monitoring is one of the best ways of learning “the truth” about what visitors experience on your web site, but it comes at a cost. The real world is messy and noisy making it hard to know exactly what’s going on. Filtering your data, splitting it along multiple dimensions, and determining what to discard are important second steps on the path to insightful RUM analysis, and in this session, we’ll go into some of the details.
A tale of scale & speed: How the US Navy is enabling software delivery from l...sonjaschweigert1
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdfPeter Spielvogel
Building better applications for business users with SAP Fiori.
• What is SAP Fiori and why it matters to you
• How a better user experience drives measurable business benefits
• How to get started with SAP Fiori today
• How SAP Fiori elements accelerates application development
• How SAP Build Code includes SAP Fiori tools and other generative artificial intelligence capabilities
• How SAP Fiori paves the way for using AI in SAP apps
Climate Impact of Software Testing at Nordic Testing DaysKari Kakkonen
My slides at Nordic Testing Days 6.6.2024
Climate impact / sustainability of software testing discussed on the talk. ICT and testing must carry their part of global responsibility to help with the climat warming. We can minimize the carbon footprint but we can also have a carbon handprint, a positive impact on the climate. Quality characteristics can be added with sustainability, and then measured continuously. Test environments can be used less, and in smaller scale and on demand. Test techniques can be used in optimizing or minimizing number of tests. Test automation can be used to speed up testing.
Removing Uninteresting Bytes in Software FuzzingAftab Hussain
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex ProofsAlex Pruden
This paper presents Reef, a system for generating publicly verifiable succinct non-interactive zero-knowledge proofs that a committed document matches or does not match a regular expression. We describe applications such as proving the strength of passwords, the provenance of email despite redactions, the validity of oblivious DNS queries, and the existence of mutations in DNA. Reef supports the Perl Compatible Regular Expression syntax, including wildcards, alternation, ranges, capture groups, Kleene star, negations, and lookarounds. Reef introduces a new type of automata, Skipping Alternating Finite Automata (SAFA), that skips irrelevant parts of a document when producing proofs without undermining soundness, and instantiates SAFA with a lookup argument. Our experimental evaluation confirms that Reef can generate proofs for documents with 32M characters; the proofs are small and cheap to verify (under a second).
Paper: https://eprint.iacr.org/2023/1886
The Metaverse and AI: how can decision-makers harness the Metaverse for their...Jen Stirrup
The Metaverse is popularized in science fiction, and now it is becoming closer to being a part of our daily lives through the use of social media and shopping companies. How can businesses survive in a world where Artificial Intelligence is becoming the present as well as the future of technology, and how does the Metaverse fit into business strategy when futurist ideas are developing into reality at accelerated rates? How do we do this when our data isn't up to scratch? How can we move towards success with our data so we are set up for the Metaverse when it arrives?
How can you help your company evolve, adapt, and succeed using Artificial Intelligence and the Metaverse to stay ahead of the competition? What are the potential issues, complications, and benefits that these technologies could bring to us and our organizations? In this session, Jen Stirrup will explain how to start thinking about these technologies as an organisation.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Enhancing Performance with Globus and the Science DMZGlobus
ESnet has led the way in helping national facilities—and many other institutions in the research community—configure Science DMZs and troubleshoot network issues to maximize data transfer performance. In this talk we will present a summary of approaches and tips for getting the most out of your network infrastructure using Globus Connect Server.
Enhancing Performance with Globus and the Science DMZ
Measuring the web with boomerang
1. Why measure?
boomerang
data data data
Measuring the web with boomerang
Philip Tellis / philip@bluesmoon.info
SF Performance Meetup / 2010-08-11
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
2. Why measure?
boomerang
data data data
$ finger philip
Philip Tellis
philip@bluesmoon.info
@bluesmoon
yahoo
geek
http://bluesmoon.info/
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
3. Why measure? The slow web
boomerang Measurements
data data data Measuring with javascript
Where does all the time go?
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
4. Why measure? The slow web
boomerang Measurements
data data data Measuring with javascript
Who controls it?
Some of this we control and some of it we don’t
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
5. Why measure? The slow web
boomerang Measurements
data data data Measuring with javascript
Back end
Measuring and improving back end performance can be done
during development
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
6. Why measure? The slow web
boomerang Measurements
data data data Measuring with javascript
80-20
Turns out that less than 20% of the time is spent on the back
end
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
7. Why measure? The slow web
boomerang Measurements
data data data Measuring with javascript
Front end
It’s what we can’t control that bites us
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
8. Why measure? The slow web
boomerang Measurements
data data data Measuring with javascript
Too many variations
browsers
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
9. Why measure? The slow web
boomerang Measurements
data data data Measuring with javascript
Too many variations
plugins
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
10. Why measure? The slow web
boomerang Measurements
data data data Measuring with javascript
Too many variations
OSes
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
11. Why measure? The slow web
boomerang Measurements
data data data Measuring with javascript
Too many variations
viruses
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
12. Why measure? The slow web
boomerang Measurements
data data data Measuring with javascript
Too many variations
antiviruses
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
13. Why measure? The slow web
boomerang Measurements
data data data Measuring with javascript
Too many variations
microwaves
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
14. Why measure? The slow web
boomerang Measurements
data data data Measuring with javascript
Too many variations
baby monitors
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
15. Why measure? The slow web
boomerang Measurements
data data data Measuring with javascript
Too many variations
naughty neighbours
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
16. Why measure? The slow web
boomerang Measurements
data data data Measuring with javascript
Too many variations
file shares
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
17. Why measure? The slow web
boomerang Measurements
data data data Measuring with javascript
Too many variations
governments
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
18. Why measure? The slow web
boomerang Measurements
data data data Measuring with javascript
Too many variations
rodents
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
19. Why measure? The slow web
boomerang Measurements
data data data Measuring with javascript
Too many variations
Try simulating all that in the lab!
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
20. Why measure? The slow web
boomerang Measurements
data data data Measuring with javascript
We need to measure real end-user performance
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
21. Why measure? The slow web
boomerang Measurements
data data data Measuring with javascript
We need to measure it from the real end-user’s box
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
22. Why measure? The slow web
boomerang Measurements
data data data Measuring with javascript
Ask the user?
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
23. Why measure? The slow web
boomerang Measurements
data data data Measuring with javascript
Bias
While this might work, it isn’t necessarily representative
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
24. Why measure? The slow web
boomerang Measurements
data data data Measuring with javascript
A/B testing
You also want to be able to dynamically tune which users get
which tests
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
25. Why measure? The slow web
boomerang Measurements
data data data Measuring with javascript
Phone home
It’s most useful if you can send these measurements back to
your server for analysis
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
26. Why measure? The slow web
boomerang Measurements
data data data Measuring with javascript
Mostly ubiquitous
We know that javascript is available on almost every browser
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
27. Why measure? The slow web
boomerang Measurements
data data data Measuring with javascript
Rich pages
We really want to measure the performance of rich pages
which depend on javascript already
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
28. Why measure? The slow web
boomerang Measurements
data data data Measuring with javascript
Limited
But javascript can’t measure everything... we get as close as
we can
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
29. Why measure? What?
boomerang How does it work?
data data data Accuracy
A piece of javascript that you add to your web page where it
measures and beacons back to you the end user’s perceived
performance of your page
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
30. Why measure? What?
boomerang How does it work?
data data data Accuracy
How?
<script src="boomerang.js" type="text/javascript"></
<script type="text/javascript">
BOOMR.init({
user_ip: "<user’s ip address>",
beacon_url: "http://yoursite.com/path/to/beacon.php"
});
</script>
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
31. Why measure? What?
boomerang How does it work?
data data data Accuracy
What does it do?
About once a week, measures user’s bandwidth and
latency to your server
On (almost) every request, measures the time it took to
load the current page
Beacons these results back to your server
Other stuff based on plugins
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
32. Why measure? What?
boomerang How does it work?
data data data Accuracy
How does it do it?
Let’s take that one at a time
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
33. Why measure? What?
boomerang How does it work?
data data data Accuracy
How do we measure latency?
Download a 32 byte gif 10 times in sequence
Measure the time to download each
Discard the first measurement because it’s overpriced
Calculate the arithmetic mean, standard deviation and
margin of error of the remaining
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
34. Why measure? What?
boomerang How does it work?
data data data Accuracy
How do we measure bandwidth?
After the latency test is done, we download progressively
larger images
Stop at the first image that times out
Redownload that image a few more times
Calculate the median, standard deviation and margin of
error of the largest images
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
35. Why measure? What?
boomerang How does it work?
data data data Accuracy
How do we measure page load time?
In the onbeforeunload event, measure the time and
store it in a cookie
In the onload event, check the cookie, and measure the
difference with the current time
We also make sure that the page that set the cookie is the
referrer of the current page
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
36. Why measure? What?
boomerang How does it work?
data data data Accuracy
What? Two pages?
Yes, this needs two pages and cookies. If those aren’t
supported, we try to use the WebTiming API.
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
37. Why measure? What?
boomerang How does it work?
data data data Accuracy
How accurate is it?
Latency measurements are very accurate (±1%)
Bandwidth is to an order of magnitude. For bad
connections can be ±30%
Page load time sometimes has outliers, you need
post-filtering
The margin of error tells you how good your data is
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
38. Why measure? Filtering
boomerang Grouping
data data data More...
What do we do with the data?
Sanity checking to:
Remove fake data
Remove abusive data
Maybe just rate limiting
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
39. Why measure? Filtering
boomerang Grouping
data data data More...
What do we do with the data?
Statistical analysis to:
Remove outliers
Aggregate based on bandwidth blocks
Measure trends over time and correlate them with code
changes
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
40. Why measure? Filtering
boomerang Grouping
data data data More...
Bandwidth slabs
0-100 kbps
100-300 kbps
300-2000 kbps
2-6 Mbps
6+ Mbps
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
41. Why measure? Filtering
boomerang Grouping
data data data More...
Bandwidth slabs
Group page load times based on bandwidth slab
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
42. Why measure? Filtering
boomerang Grouping
data data data More...
Bandwidth slabs
Data points from some countries may require smaller slabs
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
43. Why measure? Filtering
boomerang Grouping
data data data More...
Geographic data
Looking at latency from different geographic locations can tell
you where to put your next CDN
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
44. Why measure? Filtering
boomerang Grouping
data data data More...
ISPs
Grouping data by ISP can tell you who’s behaving badly
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
45. Why measure? Filtering
boomerang Grouping
data data data More...
More data
Write plugins to get more performance data
We already have a DNS plugin
I’m thinking of an IPv6 v/s IPv4 plugin
What about a full WebTiming plugin?
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
46. Why measure? Filtering
boomerang Grouping
data data data More...
You decide
Once you have the data, you can do anything with it
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
47. Why measure? Filtering
boomerang Grouping
data data data More...
Thank you
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
48. Why measure? Filtering
boomerang Grouping
data data data More...
Photo credits
flickr.com/photos/21233184@N02/4389412851
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
49. Why measure? Filtering
boomerang Grouping
data data data More...
Contact me
Philip Tellis
yahoo
geek
@bluesmoon
http://bluesmoon.info/
slideshare.net/bluesmoon
philip@bluesmoon.info
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang
50. Why measure? Filtering
boomerang Grouping
data data data More...
References
github.com/yahoo/boomerang
More bandwidth doesn’t matter (much) – Mike Belshe
Analysing Bandwidth & Latency – YUI Blog
It’s the latency, stupid – Stuart Cheshire
The statistics of web performance
SF Performance Meetup / 2010-08-11 Measuring the web with boomerang