SlideShare a Scribd company logo
Bots vs. Real Users – Page Load Times
You want/need to know about the Load Times and Core Web Vitals of
your Digital Brand or offerings for your real users?
Cool, let’s get there but be aware of the Bots. They can ruin
everything.
Best possible Source: Browser of your User
Best possible Analytics Engine =
Get Browser Performance Metrics into New
Relic
(Browser performance metrics arrived in New Relic)
1.) Create a free Account at https://newrelic.com/signup
2.) Follow the instructions on the start page – in case you are already a New Relic user – just click on
Be aware: This includes every User Agent
Make sense of the of the data
• Get an overview
Once you have collected at least a day of data you are good to start identifying your
real Real Users. Lets click on “Dashboards” create a new one and do some queries:
FROM PageView SELECT count(*) where domain = ' mainDomain.com ' SINCE 1 day AGO limit 20 timeseries
FROM PageView SELECT average(duration) where domain= mainDomain.com' SINCE 1 day AGO limit 20 timeseries
Make sense of the of the data : Find the real users
95% of Real Users are using up to date Browsers (more or less) – 80% of Bots (or
Machines) visit you with outdated Browser Versions – How to get there? Let’s group
them!
FROM PageView
select count(userAgentName) where domain
= ‘mainDomain.com' facet userAgentName
, userAgentVersion
limit 20
These are pretty
outdated
Versions
Make sense of the of the data : Proof for Bots
Bots behave different from Real People – they either appear in spikes (crawls) or
permanent (i. e. Synthetic Monitors). Let’s check how these Browserversions behave
FROM PageView
select count(userAgentName) where domain
= ‘mainDomain.com' facet userAgentName
, userAgentVersion
limit 20 since 1 day ago timeseries
Linear Traffic over the course of
the Day = not real users
Make sense of the of the data : Proof for Humans
Bots behave different from Real People – they either appear in spikes (crawls) or
permanent (i. e. Synthetic Monitors). Let’s check how these Browserversions behave
FROM PageView
select count(userAgentName) where domain
= ‘mainDomain.com' facet userAgentName
, userAgentVersion
limit 20 since 1 day ago timeseries
Normal Traffic – Human sleep at
Night. International variances
possible
Make sense of the data: Now the performance!
• We combine the learnings and check what’s going on. Bots often call from Data Centers while Real
People often struggle with bandwidth variances.
FROM PageView
SELECT
filter(average(duration), where userAgentVersion >= '83' and userAgentName like 'Chrome') as 'Chrome Real User‘,
filter(average(duration), where userAgentName not like 'Chrome' and userAgentVersion not like '13.2.3' and userAgentVersion not like '9.1' and userAgentVersion != '10') as 'Other Real User Brow
filter(average(duration), where domain = 'newrelic.com‘) as 'Duration Unfiltered‘,
filter(average(duration), where userAgentVersion < '83' and userAgentName like 'Chrome') as 'Bots‘
WHERE domain = ‘mainDomain.com‘ SINCE 7 days AGO timeseries 15 minutes
Here we have proof that:
• Bots are served way faster than humans
• The high amount of Bots (in this extreme
example) make the Performance appear solid
and stable
• It is helpful to understand what you are
looking for

More Related Content

Similar to Bots vs real_users

High Speed Web Sites At Scale
High Speed Web Sites At ScaleHigh Speed Web Sites At Scale
High Speed Web Sites At Scale
Buddy Brewer
 
Google Analytics tutorial by Jay Murphy
Google Analytics tutorial by Jay Murphy Google Analytics tutorial by Jay Murphy
Google Analytics tutorial by Jay Murphy
Newport Interactive Marketers
 
Clickstream Analysis
Clickstream AnalysisClickstream Analysis
Clickstream Analysis
intuitiv.de
 
Google Analytics
Google Analytics Google Analytics
High Speed Web Sites At Scale (Buddy Brewer)
High Speed Web Sites At Scale (Buddy Brewer)High Speed Web Sites At Scale (Buddy Brewer)
High Speed Web Sites At Scale (Buddy Brewer)
Ontico
 
DataDome's winning deck for 2019 FIC (Cybersecurity International Forum) "Pri...
DataDome's winning deck for 2019 FIC (Cybersecurity International Forum) "Pri...DataDome's winning deck for 2019 FIC (Cybersecurity International Forum) "Pri...
DataDome's winning deck for 2019 FIC (Cybersecurity International Forum) "Pri...
DataDome
 
Digital Ad Glossary.pdf
Digital Ad Glossary.pdfDigital Ad Glossary.pdf
Digital Ad Glossary.pdf
awhite18
 
Google Analytics Training
Google Analytics TrainingGoogle Analytics Training
Google Analytics Training
vishalsaxena952267
 
Omniture 101 - Digital Analytics - iProspect Canada
Omniture 101 - Digital Analytics - iProspect CanadaOmniture 101 - Digital Analytics - iProspect Canada
Omniture 101 - Digital Analytics - iProspect Canada
Lemesle Gautier
 
Google Analytics Workshop 2013
Google Analytics Workshop 2013Google Analytics Workshop 2013
Google Analytics Workshop 2013
Milad Safarzadeh
 
E Commerce Analytics Demandware
E Commerce Analytics DemandwareE Commerce Analytics Demandware
E Commerce Analytics Demandware
loripelletier
 
Which watcher watches CloudWatch
Which watcher watches CloudWatch Which watcher watches CloudWatch
Which watcher watches CloudWatch
David Lutz
 
17 Web Performance Metrics You Should Care About
17 Web Performance Metrics You Should Care About17 Web Performance Metrics You Should Care About
17 Web Performance Metrics You Should Care About
Evgeny Tsarkov
 
An Introduction to Web Analytics
An Introduction to Web AnalyticsAn Introduction to Web Analytics
An Introduction to Web Analytics
iexpertsforum
 
Google analytics
Google analyticsGoogle analytics
Google analytics
Hemant Mali
 
Micro services from scratch - Part 1
Micro services from scratch - Part 1Micro services from scratch - Part 1
Micro services from scratch - Part 1
Azrul MADISA
 
Complete Google analytics document
Complete Google analytics documentComplete Google analytics document
Complete Google analytics document
Parshuram Yadav
 
Www usenix-org
Www usenix-orgWww usenix-org
Www usenix-org
Ouzza Brahim
 
Load Speed PSI development of webcore vitals
Load Speed PSI development of webcore vitalsLoad Speed PSI development of webcore vitals
Load Speed PSI development of webcore vitals
rahmathidayat471220
 
web metric glossary omniture
web metric glossary omnitureweb metric glossary omniture
web metric glossary omniture
Lalitendu Sahoo
 

Similar to Bots vs real_users (20)

High Speed Web Sites At Scale
High Speed Web Sites At ScaleHigh Speed Web Sites At Scale
High Speed Web Sites At Scale
 
Google Analytics tutorial by Jay Murphy
Google Analytics tutorial by Jay Murphy Google Analytics tutorial by Jay Murphy
Google Analytics tutorial by Jay Murphy
 
Clickstream Analysis
Clickstream AnalysisClickstream Analysis
Clickstream Analysis
 
Google Analytics
Google Analytics Google Analytics
Google Analytics
 
High Speed Web Sites At Scale (Buddy Brewer)
High Speed Web Sites At Scale (Buddy Brewer)High Speed Web Sites At Scale (Buddy Brewer)
High Speed Web Sites At Scale (Buddy Brewer)
 
DataDome's winning deck for 2019 FIC (Cybersecurity International Forum) "Pri...
DataDome's winning deck for 2019 FIC (Cybersecurity International Forum) "Pri...DataDome's winning deck for 2019 FIC (Cybersecurity International Forum) "Pri...
DataDome's winning deck for 2019 FIC (Cybersecurity International Forum) "Pri...
 
Digital Ad Glossary.pdf
Digital Ad Glossary.pdfDigital Ad Glossary.pdf
Digital Ad Glossary.pdf
 
Google Analytics Training
Google Analytics TrainingGoogle Analytics Training
Google Analytics Training
 
Omniture 101 - Digital Analytics - iProspect Canada
Omniture 101 - Digital Analytics - iProspect CanadaOmniture 101 - Digital Analytics - iProspect Canada
Omniture 101 - Digital Analytics - iProspect Canada
 
Google Analytics Workshop 2013
Google Analytics Workshop 2013Google Analytics Workshop 2013
Google Analytics Workshop 2013
 
E Commerce Analytics Demandware
E Commerce Analytics DemandwareE Commerce Analytics Demandware
E Commerce Analytics Demandware
 
Which watcher watches CloudWatch
Which watcher watches CloudWatch Which watcher watches CloudWatch
Which watcher watches CloudWatch
 
17 Web Performance Metrics You Should Care About
17 Web Performance Metrics You Should Care About17 Web Performance Metrics You Should Care About
17 Web Performance Metrics You Should Care About
 
An Introduction to Web Analytics
An Introduction to Web AnalyticsAn Introduction to Web Analytics
An Introduction to Web Analytics
 
Google analytics
Google analyticsGoogle analytics
Google analytics
 
Micro services from scratch - Part 1
Micro services from scratch - Part 1Micro services from scratch - Part 1
Micro services from scratch - Part 1
 
Complete Google analytics document
Complete Google analytics documentComplete Google analytics document
Complete Google analytics document
 
Www usenix-org
Www usenix-orgWww usenix-org
Www usenix-org
 
Load Speed PSI development of webcore vitals
Load Speed PSI development of webcore vitalsLoad Speed PSI development of webcore vitals
Load Speed PSI development of webcore vitals
 
web metric glossary omniture
web metric glossary omnitureweb metric glossary omniture
web metric glossary omniture
 

Recently uploaded

Experts live - Improving user adoption with AI
Experts live - Improving user adoption with AIExperts live - Improving user adoption with AI
Experts live - Improving user adoption with AI
jitskeb
 
End-to-end pipeline agility - Berlin Buzzwords 2024
End-to-end pipeline agility - Berlin Buzzwords 2024End-to-end pipeline agility - Berlin Buzzwords 2024
End-to-end pipeline agility - Berlin Buzzwords 2024
Lars Albertsson
 
原版一比一多伦多大学毕业证(UofT毕业证书)如何办理
原版一比一多伦多大学毕业证(UofT毕业证书)如何办理原版一比一多伦多大学毕业证(UofT毕业证书)如何办理
原版一比一多伦多大学毕业证(UofT毕业证书)如何办理
mkkikqvo
 
Palo Alto Cortex XDR presentation .......
Palo Alto Cortex XDR presentation .......Palo Alto Cortex XDR presentation .......
Palo Alto Cortex XDR presentation .......
Sachin Paul
 
原版一比一利兹贝克特大学毕业证(LeedsBeckett毕业证书)如何办理
原版一比一利兹贝克特大学毕业证(LeedsBeckett毕业证书)如何办理原版一比一利兹贝克特大学毕业证(LeedsBeckett毕业证书)如何办理
原版一比一利兹贝克特大学毕业证(LeedsBeckett毕业证书)如何办理
wyddcwye1
 
06-12-2024-BudapestDataForum-BuildingReal-timePipelineswithFLaNK AIM
06-12-2024-BudapestDataForum-BuildingReal-timePipelineswithFLaNK AIM06-12-2024-BudapestDataForum-BuildingReal-timePipelineswithFLaNK AIM
06-12-2024-BudapestDataForum-BuildingReal-timePipelineswithFLaNK AIM
Timothy Spann
 
ViewShift: Hassle-free Dynamic Policy Enforcement for Every Data Lake
ViewShift: Hassle-free Dynamic Policy Enforcement for Every Data LakeViewShift: Hassle-free Dynamic Policy Enforcement for Every Data Lake
ViewShift: Hassle-free Dynamic Policy Enforcement for Every Data Lake
Walaa Eldin Moustafa
 
一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理
一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理
一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理
nyfuhyz
 
The Ipsos - AI - Monitor 2024 Report.pdf
The  Ipsos - AI - Monitor 2024 Report.pdfThe  Ipsos - AI - Monitor 2024 Report.pdf
The Ipsos - AI - Monitor 2024 Report.pdf
Social Samosa
 
一比一原版(harvard毕业证书)哈佛大学毕业证如何办理
一比一原版(harvard毕业证书)哈佛大学毕业证如何办理一比一原版(harvard毕业证书)哈佛大学毕业证如何办理
一比一原版(harvard毕业证书)哈佛大学毕业证如何办理
taqyea
 
Open Source Contributions to Postgres: The Basics POSETTE 2024
Open Source Contributions to Postgres: The Basics POSETTE 2024Open Source Contributions to Postgres: The Basics POSETTE 2024
Open Source Contributions to Postgres: The Basics POSETTE 2024
ElizabethGarrettChri
 
原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样
原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样
原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样
ihavuls
 
一比一原版(Unimelb毕业证书)墨尔本大学毕业证如何办理
一比一原版(Unimelb毕业证书)墨尔本大学毕业证如何办理一比一原版(Unimelb毕业证书)墨尔本大学毕业证如何办理
一比一原版(Unimelb毕业证书)墨尔本大学毕业证如何办理
xclpvhuk
 
A presentation that explain the Power BI Licensing
A presentation that explain the Power BI LicensingA presentation that explain the Power BI Licensing
A presentation that explain the Power BI Licensing
AlessioFois2
 
Predictably Improve Your B2B Tech Company's Performance by Leveraging Data
Predictably Improve Your B2B Tech Company's Performance by Leveraging DataPredictably Improve Your B2B Tech Company's Performance by Leveraging Data
Predictably Improve Your B2B Tech Company's Performance by Leveraging Data
Kiwi Creative
 
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...
Social Samosa
 
Udemy_2024_Global_Learning_Skills_Trends_Report (1).pdf
Udemy_2024_Global_Learning_Skills_Trends_Report (1).pdfUdemy_2024_Global_Learning_Skills_Trends_Report (1).pdf
Udemy_2024_Global_Learning_Skills_Trends_Report (1).pdf
Fernanda Palhano
 
Global Situational Awareness of A.I. and where its headed
Global Situational Awareness of A.I. and where its headedGlobal Situational Awareness of A.I. and where its headed
Global Situational Awareness of A.I. and where its headed
vikram sood
 
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...
sameer shah
 
原版一比一弗林德斯大学毕业证(Flinders毕业证书)如何办理
原版一比一弗林德斯大学毕业证(Flinders毕业证书)如何办理原版一比一弗林德斯大学毕业证(Flinders毕业证书)如何办理
原版一比一弗林德斯大学毕业证(Flinders毕业证书)如何办理
a9qfiubqu
 

Recently uploaded (20)

Experts live - Improving user adoption with AI
Experts live - Improving user adoption with AIExperts live - Improving user adoption with AI
Experts live - Improving user adoption with AI
 
End-to-end pipeline agility - Berlin Buzzwords 2024
End-to-end pipeline agility - Berlin Buzzwords 2024End-to-end pipeline agility - Berlin Buzzwords 2024
End-to-end pipeline agility - Berlin Buzzwords 2024
 
原版一比一多伦多大学毕业证(UofT毕业证书)如何办理
原版一比一多伦多大学毕业证(UofT毕业证书)如何办理原版一比一多伦多大学毕业证(UofT毕业证书)如何办理
原版一比一多伦多大学毕业证(UofT毕业证书)如何办理
 
Palo Alto Cortex XDR presentation .......
Palo Alto Cortex XDR presentation .......Palo Alto Cortex XDR presentation .......
Palo Alto Cortex XDR presentation .......
 
原版一比一利兹贝克特大学毕业证(LeedsBeckett毕业证书)如何办理
原版一比一利兹贝克特大学毕业证(LeedsBeckett毕业证书)如何办理原版一比一利兹贝克特大学毕业证(LeedsBeckett毕业证书)如何办理
原版一比一利兹贝克特大学毕业证(LeedsBeckett毕业证书)如何办理
 
06-12-2024-BudapestDataForum-BuildingReal-timePipelineswithFLaNK AIM
06-12-2024-BudapestDataForum-BuildingReal-timePipelineswithFLaNK AIM06-12-2024-BudapestDataForum-BuildingReal-timePipelineswithFLaNK AIM
06-12-2024-BudapestDataForum-BuildingReal-timePipelineswithFLaNK AIM
 
ViewShift: Hassle-free Dynamic Policy Enforcement for Every Data Lake
ViewShift: Hassle-free Dynamic Policy Enforcement for Every Data LakeViewShift: Hassle-free Dynamic Policy Enforcement for Every Data Lake
ViewShift: Hassle-free Dynamic Policy Enforcement for Every Data Lake
 
一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理
一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理
一比一原版(UMN文凭证书)明尼苏达大学毕业证如何办理
 
The Ipsos - AI - Monitor 2024 Report.pdf
The  Ipsos - AI - Monitor 2024 Report.pdfThe  Ipsos - AI - Monitor 2024 Report.pdf
The Ipsos - AI - Monitor 2024 Report.pdf
 
一比一原版(harvard毕业证书)哈佛大学毕业证如何办理
一比一原版(harvard毕业证书)哈佛大学毕业证如何办理一比一原版(harvard毕业证书)哈佛大学毕业证如何办理
一比一原版(harvard毕业证书)哈佛大学毕业证如何办理
 
Open Source Contributions to Postgres: The Basics POSETTE 2024
Open Source Contributions to Postgres: The Basics POSETTE 2024Open Source Contributions to Postgres: The Basics POSETTE 2024
Open Source Contributions to Postgres: The Basics POSETTE 2024
 
原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样
原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样
原版制作(unimelb毕业证书)墨尔本大学毕业证Offer一模一样
 
一比一原版(Unimelb毕业证书)墨尔本大学毕业证如何办理
一比一原版(Unimelb毕业证书)墨尔本大学毕业证如何办理一比一原版(Unimelb毕业证书)墨尔本大学毕业证如何办理
一比一原版(Unimelb毕业证书)墨尔本大学毕业证如何办理
 
A presentation that explain the Power BI Licensing
A presentation that explain the Power BI LicensingA presentation that explain the Power BI Licensing
A presentation that explain the Power BI Licensing
 
Predictably Improve Your B2B Tech Company's Performance by Leveraging Data
Predictably Improve Your B2B Tech Company's Performance by Leveraging DataPredictably Improve Your B2B Tech Company's Performance by Leveraging Data
Predictably Improve Your B2B Tech Company's Performance by Leveraging Data
 
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...
4th Modern Marketing Reckoner by MMA Global India & Group M: 60+ experts on W...
 
Udemy_2024_Global_Learning_Skills_Trends_Report (1).pdf
Udemy_2024_Global_Learning_Skills_Trends_Report (1).pdfUdemy_2024_Global_Learning_Skills_Trends_Report (1).pdf
Udemy_2024_Global_Learning_Skills_Trends_Report (1).pdf
 
Global Situational Awareness of A.I. and where its headed
Global Situational Awareness of A.I. and where its headedGlobal Situational Awareness of A.I. and where its headed
Global Situational Awareness of A.I. and where its headed
 
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...
 
原版一比一弗林德斯大学毕业证(Flinders毕业证书)如何办理
原版一比一弗林德斯大学毕业证(Flinders毕业证书)如何办理原版一比一弗林德斯大学毕业证(Flinders毕业证书)如何办理
原版一比一弗林德斯大学毕业证(Flinders毕业证书)如何办理
 

Bots vs real_users

  • 1.
  • 2. Bots vs. Real Users – Page Load Times You want/need to know about the Load Times and Core Web Vitals of your Digital Brand or offerings for your real users? Cool, let’s get there but be aware of the Bots. They can ruin everything.
  • 3. Best possible Source: Browser of your User Best possible Analytics Engine =
  • 4. Get Browser Performance Metrics into New Relic (Browser performance metrics arrived in New Relic) 1.) Create a free Account at https://newrelic.com/signup 2.) Follow the instructions on the start page – in case you are already a New Relic user – just click on
  • 5. Be aware: This includes every User Agent
  • 6. Make sense of the of the data • Get an overview Once you have collected at least a day of data you are good to start identifying your real Real Users. Lets click on “Dashboards” create a new one and do some queries: FROM PageView SELECT count(*) where domain = ' mainDomain.com ' SINCE 1 day AGO limit 20 timeseries FROM PageView SELECT average(duration) where domain= mainDomain.com' SINCE 1 day AGO limit 20 timeseries
  • 7. Make sense of the of the data : Find the real users 95% of Real Users are using up to date Browsers (more or less) – 80% of Bots (or Machines) visit you with outdated Browser Versions – How to get there? Let’s group them! FROM PageView select count(userAgentName) where domain = ‘mainDomain.com' facet userAgentName , userAgentVersion limit 20 These are pretty outdated Versions
  • 8. Make sense of the of the data : Proof for Bots Bots behave different from Real People – they either appear in spikes (crawls) or permanent (i. e. Synthetic Monitors). Let’s check how these Browserversions behave FROM PageView select count(userAgentName) where domain = ‘mainDomain.com' facet userAgentName , userAgentVersion limit 20 since 1 day ago timeseries Linear Traffic over the course of the Day = not real users
  • 9. Make sense of the of the data : Proof for Humans Bots behave different from Real People – they either appear in spikes (crawls) or permanent (i. e. Synthetic Monitors). Let’s check how these Browserversions behave FROM PageView select count(userAgentName) where domain = ‘mainDomain.com' facet userAgentName , userAgentVersion limit 20 since 1 day ago timeseries Normal Traffic – Human sleep at Night. International variances possible
  • 10. Make sense of the data: Now the performance! • We combine the learnings and check what’s going on. Bots often call from Data Centers while Real People often struggle with bandwidth variances. FROM PageView SELECT filter(average(duration), where userAgentVersion >= '83' and userAgentName like 'Chrome') as 'Chrome Real User‘, filter(average(duration), where userAgentName not like 'Chrome' and userAgentVersion not like '13.2.3' and userAgentVersion not like '9.1' and userAgentVersion != '10') as 'Other Real User Brow filter(average(duration), where domain = 'newrelic.com‘) as 'Duration Unfiltered‘, filter(average(duration), where userAgentVersion < '83' and userAgentName like 'Chrome') as 'Bots‘ WHERE domain = ‘mainDomain.com‘ SINCE 7 days AGO timeseries 15 minutes Here we have proof that: • Bots are served way faster than humans • The high amount of Bots (in this extreme example) make the Performance appear solid and stable • It is helpful to understand what you are looking for