SlideShare a Scribd company logo
Ondřej Surý <ondrej@isc.org>
Internet Systems Consortium
2024-05-21
Measuring DNS Latency
DNS Performance Metrics (quick intro) Measuring DNS Latency
• Performance under normal conditions
‣
• The data is right-skewed
‣ The usual descriptive statistics are useless (average, mean, …)
‣ Most of the queries are answered very quickly
‣ In fact, 95% of the queries are answered under 2 milliseconds
‣ The tails makes it interesting
Ondřej Surý <ondrej@isc.org> 2024-05-21 1 / 8
Logarithmic Percentile Histogram Measuring DNS Latency
Ondřej Surý <ondrej@isc.org> 2024-05-21 2 / 8
Logarithmic Percentile Histogram Measuring DNS Latency
• Both axes are logarithmic
‣ x-axis: slowest percentile
‣ y-axis: average latency
• It makes the tail more visible
• Variant of Complementary Cumulative Distribution Function
• Very robust, can be used for monitoring (1% slowest percentile)
• Introduced by the good folks at PowerDNS
See more: https://blog.powerdns.com/2017/11/02/dns-performance-
metrics-the-logarithmic-percentile-histogram
Ondřej Surý <ondrej@isc.org> 2024-05-21 3 / 8
Compare DNS Latencies
DNS Performance for Developers Compare DNS Latencies
• Comparing two branches of BIND 9
‣ Did we improve the code?
‣ Did we made things worse?
‣ Currently, we compare the graphs by looking at them;
‣ And then running more tests;
‣ And then some wishful thinking…
• Sending thanks to Python’s numpy and scipy developers!
Ondřej Surý <ondrej@isc.org> 2024-05-21 4 / 8
Pick the right statistics Compare DNS Latencies
• The distribution is not normal
• Non-parametrical test then?
‣ Kolmogorov-Smirnov test didn’t really work
• Normalize the data?
‣ Box Cox Transformation didn’t really work
• Maybe look only at the tail then?
Ondřej Surý <ondrej@isc.org> 2024-05-21 5 / 8
Looking at the tail Compare DNS Latencies
• Pick the 95% (99%) percentile complement
‣ Either return the lowest bucket needed for 5% of responses
‣ Or count the answers in (1.9-2.0 second buckets)
• Have at least 3 runs for each group
• Yay! The data are normal and the group variances are equal
‣ Shapiro-Wilk test
– first group (𝑊 = 0.905, 𝑝 = 0.436)
– second group (𝑊 = 0.970, 𝑝 = 0.874)
‣ Brown-Forsyth test (𝐹 = 0.070, 𝑝 = 0.798)
Ondřej Surý <ondrej@isc.org> 2024-05-21 6 / 8
Parametrical test (ANOVA) Compare DNS Latencies
• We can test more than two branches
• One-way ANOVA reports difference between branches
‣ 𝐹 = 9244.090, 𝑝 < .001
• Two-sample T-Test (for confirmation)
‣ 𝑇 = −96.146, 𝑝 < .001
Ondřej Surý <ondrej@isc.org> 2024-05-21 7 / 8
More ideas?
Other tests? More ideas?
• Is this even correct? Or am I crazy? (I’m not a statistician)
• Can we just compare two data sets (1x baseline with 1x branch)?
• Can we use the full (right-skewed) population?
• Are there any other non-parametrical tests I can try/use?
• Are there any other suitable statistical methods?
• Is this useful for other Internet measurements?
Ondřej Surý <ondrej@isc.org> 2024-05-21 8 / 8
Thank you!

More Related Content

Similar to Statistical Analysis of DNS Latencies.pdf

Optimizing Uptime in SOA
Optimizing Uptime in SOAOptimizing Uptime in SOA
Optimizing Uptime in SOA
Matthew Barlocker
 
Finding Bugs Faster with Assertion Based Verification (ABV)
Finding Bugs Faster with Assertion Based Verification (ABV)Finding Bugs Faster with Assertion Based Verification (ABV)
Finding Bugs Faster with Assertion Based Verification (ABV)
DVClub
 
Matsunaga crowdsourcing IEEE e-science 2014
Matsunaga crowdsourcing IEEE e-science 2014Matsunaga crowdsourcing IEEE e-science 2014
Matsunaga crowdsourcing IEEE e-science 2014
Andrea Matsunaga
 
Effective monitoring of distributed systems
Effective monitoring of distributed systemsEffective monitoring of distributed systems
Effective monitoring of distributed systems
Mohanraj Nagasamy
 
Daamen r 2010scwr-cpaper
Daamen r 2010scwr-cpaperDaamen r 2010scwr-cpaper
Daamen r 2010scwr-cpaper
John B. Cook, PE, CEO
 
NDOR Research Conference: Dr. Rilett
NDOR Research Conference: Dr. RilettNDOR Research Conference: Dr. Rilett
NDOR Research Conference: Dr. Rilett
Mid-America Transportation Center
 
Performance Issue? Machine Learning to the rescue!
Performance Issue? Machine Learning to the rescue!Performance Issue? Machine Learning to the rescue!
Performance Issue? Machine Learning to the rescue!
Maarten Smeets
 
Clustering - ACM 2013 02-25
Clustering - ACM 2013 02-25Clustering - ACM 2013 02-25
Clustering - ACM 2013 02-25
MapR Technologies
 
Sean Kandel - Data profiling: Assessing the overall content and quality of a ...
Sean Kandel - Data profiling: Assessing the overall content and quality of a ...Sean Kandel - Data profiling: Assessing the overall content and quality of a ...
Sean Kandel - Data profiling: Assessing the overall content and quality of a ...
huguk
 
Bridging the Gap: Machine Learning for Ubiquitous Computing -- Evaluation
Bridging the Gap: Machine Learning for Ubiquitous Computing -- EvaluationBridging the Gap: Machine Learning for Ubiquitous Computing -- Evaluation
Bridging the Gap: Machine Learning for Ubiquitous Computing -- Evaluation
Thomas Ploetz
 
Learning to Search Henry Kautz
Learning to Search Henry KautzLearning to Search Henry Kautz
Learning to Search Henry Kautz
butest
 
Learning to Search Henry Kautz
Learning to Search Henry KautzLearning to Search Henry Kautz
Learning to Search Henry Kautz
butest
 
(BDT207) Real-Time Analytics In Service Of Self-Healing Ecosystems
(BDT207) Real-Time Analytics In Service Of Self-Healing Ecosystems(BDT207) Real-Time Analytics In Service Of Self-Healing Ecosystems
(BDT207) Real-Time Analytics In Service Of Self-Healing Ecosystems
Amazon Web Services
 
Decision Forests and discriminant analysis
Decision Forests and discriminant analysisDecision Forests and discriminant analysis
Decision Forests and discriminant analysis
potaters
 
Real World Performance - OLTP
Real World Performance - OLTPReal World Performance - OLTP
Real World Performance - OLTP
Connor McDonald
 
Parallel DNA Sequence Alignment
Parallel DNA Sequence AlignmentParallel DNA Sequence Alignment
Parallel DNA Sequence Alignment
Giuliana Carullo
 
System and User Aspects of Web Search Latency
System and User Aspects of Web Search LatencySystem and User Aspects of Web Search Latency
System and User Aspects of Web Search Latency
Telefonica Research
 
Beyond Averages - Web Performance Meetup
Beyond Averages - Web Performance MeetupBeyond Averages - Web Performance Meetup
Beyond Averages - Web Performance Meetup
Dan Kuebrich
 
Towards Detecting Performance Anti-patterns Using Classification Techniques
Towards Detecting Performance Anti-patterns Using Classification TechniquesTowards Detecting Performance Anti-patterns Using Classification Techniques
Towards Detecting Performance Anti-patterns Using Classification Techniques
James Hill
 
Splunk Enterprise for Information Security Hands-On Breakout Session
Splunk Enterprise for Information Security Hands-On Breakout SessionSplunk Enterprise for Information Security Hands-On Breakout Session
Splunk Enterprise for Information Security Hands-On Breakout Session
Splunk
 

Similar to Statistical Analysis of DNS Latencies.pdf (20)

Optimizing Uptime in SOA
Optimizing Uptime in SOAOptimizing Uptime in SOA
Optimizing Uptime in SOA
 
Finding Bugs Faster with Assertion Based Verification (ABV)
Finding Bugs Faster with Assertion Based Verification (ABV)Finding Bugs Faster with Assertion Based Verification (ABV)
Finding Bugs Faster with Assertion Based Verification (ABV)
 
Matsunaga crowdsourcing IEEE e-science 2014
Matsunaga crowdsourcing IEEE e-science 2014Matsunaga crowdsourcing IEEE e-science 2014
Matsunaga crowdsourcing IEEE e-science 2014
 
Effective monitoring of distributed systems
Effective monitoring of distributed systemsEffective monitoring of distributed systems
Effective monitoring of distributed systems
 
Daamen r 2010scwr-cpaper
Daamen r 2010scwr-cpaperDaamen r 2010scwr-cpaper
Daamen r 2010scwr-cpaper
 
NDOR Research Conference: Dr. Rilett
NDOR Research Conference: Dr. RilettNDOR Research Conference: Dr. Rilett
NDOR Research Conference: Dr. Rilett
 
Performance Issue? Machine Learning to the rescue!
Performance Issue? Machine Learning to the rescue!Performance Issue? Machine Learning to the rescue!
Performance Issue? Machine Learning to the rescue!
 
Clustering - ACM 2013 02-25
Clustering - ACM 2013 02-25Clustering - ACM 2013 02-25
Clustering - ACM 2013 02-25
 
Sean Kandel - Data profiling: Assessing the overall content and quality of a ...
Sean Kandel - Data profiling: Assessing the overall content and quality of a ...Sean Kandel - Data profiling: Assessing the overall content and quality of a ...
Sean Kandel - Data profiling: Assessing the overall content and quality of a ...
 
Bridging the Gap: Machine Learning for Ubiquitous Computing -- Evaluation
Bridging the Gap: Machine Learning for Ubiquitous Computing -- EvaluationBridging the Gap: Machine Learning for Ubiquitous Computing -- Evaluation
Bridging the Gap: Machine Learning for Ubiquitous Computing -- Evaluation
 
Learning to Search Henry Kautz
Learning to Search Henry KautzLearning to Search Henry Kautz
Learning to Search Henry Kautz
 
Learning to Search Henry Kautz
Learning to Search Henry KautzLearning to Search Henry Kautz
Learning to Search Henry Kautz
 
(BDT207) Real-Time Analytics In Service Of Self-Healing Ecosystems
(BDT207) Real-Time Analytics In Service Of Self-Healing Ecosystems(BDT207) Real-Time Analytics In Service Of Self-Healing Ecosystems
(BDT207) Real-Time Analytics In Service Of Self-Healing Ecosystems
 
Decision Forests and discriminant analysis
Decision Forests and discriminant analysisDecision Forests and discriminant analysis
Decision Forests and discriminant analysis
 
Real World Performance - OLTP
Real World Performance - OLTPReal World Performance - OLTP
Real World Performance - OLTP
 
Parallel DNA Sequence Alignment
Parallel DNA Sequence AlignmentParallel DNA Sequence Alignment
Parallel DNA Sequence Alignment
 
System and User Aspects of Web Search Latency
System and User Aspects of Web Search LatencySystem and User Aspects of Web Search Latency
System and User Aspects of Web Search Latency
 
Beyond Averages - Web Performance Meetup
Beyond Averages - Web Performance MeetupBeyond Averages - Web Performance Meetup
Beyond Averages - Web Performance Meetup
 
Towards Detecting Performance Anti-patterns Using Classification Techniques
Towards Detecting Performance Anti-patterns Using Classification TechniquesTowards Detecting Performance Anti-patterns Using Classification Techniques
Towards Detecting Performance Anti-patterns Using Classification Techniques
 
Splunk Enterprise for Information Security Hands-On Breakout Session
Splunk Enterprise for Information Security Hands-On Breakout SessionSplunk Enterprise for Information Security Hands-On Breakout Session
Splunk Enterprise for Information Security Hands-On Breakout Session
 

Recently uploaded

留学挂科(UofM毕业证)明尼苏达大学毕业证成绩单复刻办理
留学挂科(UofM毕业证)明尼苏达大学毕业证成绩单复刻办理留学挂科(UofM毕业证)明尼苏达大学毕业证成绩单复刻办理
留学挂科(UofM毕业证)明尼苏达大学毕业证成绩单复刻办理
uehowe
 
manuaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaal
manuaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaalmanuaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaal
manuaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaal
wolfsoftcompanyco
 
假文凭国外(Adelaide毕业证)澳大利亚国立大学毕业证成绩单办理
假文凭国外(Adelaide毕业证)澳大利亚国立大学毕业证成绩单办理假文凭国外(Adelaide毕业证)澳大利亚国立大学毕业证成绩单办理
假文凭国外(Adelaide毕业证)澳大利亚国立大学毕业证成绩单办理
cuobya
 
国外证书(Lincoln毕业证)新西兰林肯大学毕业证成绩单不能毕业办理
国外证书(Lincoln毕业证)新西兰林肯大学毕业证成绩单不能毕业办理国外证书(Lincoln毕业证)新西兰林肯大学毕业证成绩单不能毕业办理
国外证书(Lincoln毕业证)新西兰林肯大学毕业证成绩单不能毕业办理
zoowe
 
存档可查的(USC毕业证)南加利福尼亚大学毕业证成绩单制做办理
存档可查的(USC毕业证)南加利福尼亚大学毕业证成绩单制做办理存档可查的(USC毕业证)南加利福尼亚大学毕业证成绩单制做办理
存档可查的(USC毕业证)南加利福尼亚大学毕业证成绩单制做办理
fovkoyb
 
[HUN][hackersuli] Red Teaming alapok 2024
[HUN][hackersuli] Red Teaming alapok 2024[HUN][hackersuli] Red Teaming alapok 2024
[HUN][hackersuli] Red Teaming alapok 2024
hackersuli
 
一比一原版(USYD毕业证)悉尼大学毕业证如何办理
一比一原版(USYD毕业证)悉尼大学毕业证如何办理一比一原版(USYD毕业证)悉尼大学毕业证如何办理
一比一原版(USYD毕业证)悉尼大学毕业证如何办理
k4ncd0z
 
办理新西兰奥克兰大学毕业证学位证书范本原版一模一样
办理新西兰奥克兰大学毕业证学位证书范本原版一模一样办理新西兰奥克兰大学毕业证学位证书范本原版一模一样
办理新西兰奥克兰大学毕业证学位证书范本原版一模一样
xjq03c34
 
Design Thinking NETFLIX using all techniques.pptx
Design Thinking NETFLIX using all techniques.pptxDesign Thinking NETFLIX using all techniques.pptx
Design Thinking NETFLIX using all techniques.pptx
saathvikreddy2003
 
可查真实(Monash毕业证)西澳大学毕业证成绩单退学买
可查真实(Monash毕业证)西澳大学毕业证成绩单退学买可查真实(Monash毕业证)西澳大学毕业证成绩单退学买
可查真实(Monash毕业证)西澳大学毕业证成绩单退学买
cuobya
 
Gen Z and the marketplaces - let's translate their needs
Gen Z and the marketplaces - let's translate their needsGen Z and the marketplaces - let's translate their needs
Gen Z and the marketplaces - let's translate their needs
Laura Szabó
 
成绩单ps(UST毕业证)圣托马斯大学毕业证成绩单快速办理
成绩单ps(UST毕业证)圣托马斯大学毕业证成绩单快速办理成绩单ps(UST毕业证)圣托马斯大学毕业证成绩单快速办理
成绩单ps(UST毕业证)圣托马斯大学毕业证成绩单快速办理
ysasp1
 
Should Repositories Participate in the Fediverse?
Should Repositories Participate in the Fediverse?Should Repositories Participate in the Fediverse?
Should Repositories Participate in the Fediverse?
Paul Walk
 
Search Result Showing My Post is Now Buried
Search Result Showing My Post is Now BuriedSearch Result Showing My Post is Now Buried
Search Result Showing My Post is Now Buried
Trish Parr
 
Ready to Unlock the Power of Blockchain!
Ready to Unlock the Power of Blockchain!Ready to Unlock the Power of Blockchain!
Ready to Unlock the Power of Blockchain!
Toptal Tech
 
不能毕业如何获得(USYD毕业证)悉尼大学毕业证成绩单一比一原版制作
不能毕业如何获得(USYD毕业证)悉尼大学毕业证成绩单一比一原版制作不能毕业如何获得(USYD毕业证)悉尼大学毕业证成绩单一比一原版制作
不能毕业如何获得(USYD毕业证)悉尼大学毕业证成绩单一比一原版制作
bseovas
 
重新申请毕业证书(RMIT毕业证)皇家墨尔本理工大学毕业证成绩单精仿办理
重新申请毕业证书(RMIT毕业证)皇家墨尔本理工大学毕业证成绩单精仿办理重新申请毕业证书(RMIT毕业证)皇家墨尔本理工大学毕业证成绩单精仿办理
重新申请毕业证书(RMIT毕业证)皇家墨尔本理工大学毕业证成绩单精仿办理
vmemo1
 
制作毕业证书(ANU毕业证)莫纳什大学毕业证成绩单官方原版办理
制作毕业证书(ANU毕业证)莫纳什大学毕业证成绩单官方原版办理制作毕业证书(ANU毕业证)莫纳什大学毕业证成绩单官方原版办理
制作毕业证书(ANU毕业证)莫纳什大学毕业证成绩单官方原版办理
cuobya
 
Azure EA Sponsorship - Customer Guide.pdf
Azure EA Sponsorship - Customer Guide.pdfAzure EA Sponsorship - Customer Guide.pdf
Azure EA Sponsorship - Customer Guide.pdf
AanSulistiyo
 
学位认证网(DU毕业证)迪肯大学毕业证成绩单一比一原版制作
学位认证网(DU毕业证)迪肯大学毕业证成绩单一比一原版制作学位认证网(DU毕业证)迪肯大学毕业证成绩单一比一原版制作
学位认证网(DU毕业证)迪肯大学毕业证成绩单一比一原版制作
zyfovom
 

Recently uploaded (20)

留学挂科(UofM毕业证)明尼苏达大学毕业证成绩单复刻办理
留学挂科(UofM毕业证)明尼苏达大学毕业证成绩单复刻办理留学挂科(UofM毕业证)明尼苏达大学毕业证成绩单复刻办理
留学挂科(UofM毕业证)明尼苏达大学毕业证成绩单复刻办理
 
manuaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaal
manuaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaalmanuaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaal
manuaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaal
 
假文凭国外(Adelaide毕业证)澳大利亚国立大学毕业证成绩单办理
假文凭国外(Adelaide毕业证)澳大利亚国立大学毕业证成绩单办理假文凭国外(Adelaide毕业证)澳大利亚国立大学毕业证成绩单办理
假文凭国外(Adelaide毕业证)澳大利亚国立大学毕业证成绩单办理
 
国外证书(Lincoln毕业证)新西兰林肯大学毕业证成绩单不能毕业办理
国外证书(Lincoln毕业证)新西兰林肯大学毕业证成绩单不能毕业办理国外证书(Lincoln毕业证)新西兰林肯大学毕业证成绩单不能毕业办理
国外证书(Lincoln毕业证)新西兰林肯大学毕业证成绩单不能毕业办理
 
存档可查的(USC毕业证)南加利福尼亚大学毕业证成绩单制做办理
存档可查的(USC毕业证)南加利福尼亚大学毕业证成绩单制做办理存档可查的(USC毕业证)南加利福尼亚大学毕业证成绩单制做办理
存档可查的(USC毕业证)南加利福尼亚大学毕业证成绩单制做办理
 
[HUN][hackersuli] Red Teaming alapok 2024
[HUN][hackersuli] Red Teaming alapok 2024[HUN][hackersuli] Red Teaming alapok 2024
[HUN][hackersuli] Red Teaming alapok 2024
 
一比一原版(USYD毕业证)悉尼大学毕业证如何办理
一比一原版(USYD毕业证)悉尼大学毕业证如何办理一比一原版(USYD毕业证)悉尼大学毕业证如何办理
一比一原版(USYD毕业证)悉尼大学毕业证如何办理
 
办理新西兰奥克兰大学毕业证学位证书范本原版一模一样
办理新西兰奥克兰大学毕业证学位证书范本原版一模一样办理新西兰奥克兰大学毕业证学位证书范本原版一模一样
办理新西兰奥克兰大学毕业证学位证书范本原版一模一样
 
Design Thinking NETFLIX using all techniques.pptx
Design Thinking NETFLIX using all techniques.pptxDesign Thinking NETFLIX using all techniques.pptx
Design Thinking NETFLIX using all techniques.pptx
 
可查真实(Monash毕业证)西澳大学毕业证成绩单退学买
可查真实(Monash毕业证)西澳大学毕业证成绩单退学买可查真实(Monash毕业证)西澳大学毕业证成绩单退学买
可查真实(Monash毕业证)西澳大学毕业证成绩单退学买
 
Gen Z and the marketplaces - let's translate their needs
Gen Z and the marketplaces - let's translate their needsGen Z and the marketplaces - let's translate their needs
Gen Z and the marketplaces - let's translate their needs
 
成绩单ps(UST毕业证)圣托马斯大学毕业证成绩单快速办理
成绩单ps(UST毕业证)圣托马斯大学毕业证成绩单快速办理成绩单ps(UST毕业证)圣托马斯大学毕业证成绩单快速办理
成绩单ps(UST毕业证)圣托马斯大学毕业证成绩单快速办理
 
Should Repositories Participate in the Fediverse?
Should Repositories Participate in the Fediverse?Should Repositories Participate in the Fediverse?
Should Repositories Participate in the Fediverse?
 
Search Result Showing My Post is Now Buried
Search Result Showing My Post is Now BuriedSearch Result Showing My Post is Now Buried
Search Result Showing My Post is Now Buried
 
Ready to Unlock the Power of Blockchain!
Ready to Unlock the Power of Blockchain!Ready to Unlock the Power of Blockchain!
Ready to Unlock the Power of Blockchain!
 
不能毕业如何获得(USYD毕业证)悉尼大学毕业证成绩单一比一原版制作
不能毕业如何获得(USYD毕业证)悉尼大学毕业证成绩单一比一原版制作不能毕业如何获得(USYD毕业证)悉尼大学毕业证成绩单一比一原版制作
不能毕业如何获得(USYD毕业证)悉尼大学毕业证成绩单一比一原版制作
 
重新申请毕业证书(RMIT毕业证)皇家墨尔本理工大学毕业证成绩单精仿办理
重新申请毕业证书(RMIT毕业证)皇家墨尔本理工大学毕业证成绩单精仿办理重新申请毕业证书(RMIT毕业证)皇家墨尔本理工大学毕业证成绩单精仿办理
重新申请毕业证书(RMIT毕业证)皇家墨尔本理工大学毕业证成绩单精仿办理
 
制作毕业证书(ANU毕业证)莫纳什大学毕业证成绩单官方原版办理
制作毕业证书(ANU毕业证)莫纳什大学毕业证成绩单官方原版办理制作毕业证书(ANU毕业证)莫纳什大学毕业证成绩单官方原版办理
制作毕业证书(ANU毕业证)莫纳什大学毕业证成绩单官方原版办理
 
Azure EA Sponsorship - Customer Guide.pdf
Azure EA Sponsorship - Customer Guide.pdfAzure EA Sponsorship - Customer Guide.pdf
Azure EA Sponsorship - Customer Guide.pdf
 
学位认证网(DU毕业证)迪肯大学毕业证成绩单一比一原版制作
学位认证网(DU毕业证)迪肯大学毕业证成绩单一比一原版制作学位认证网(DU毕业证)迪肯大学毕业证成绩单一比一原版制作
学位认证网(DU毕业证)迪肯大学毕业证成绩单一比一原版制作
 

Statistical Analysis of DNS Latencies.pdf

  • 1. Ondřej Surý <ondrej@isc.org> Internet Systems Consortium 2024-05-21
  • 3. DNS Performance Metrics (quick intro) Measuring DNS Latency • Performance under normal conditions ‣ • The data is right-skewed ‣ The usual descriptive statistics are useless (average, mean, …) ‣ Most of the queries are answered very quickly ‣ In fact, 95% of the queries are answered under 2 milliseconds ‣ The tails makes it interesting Ondřej Surý <ondrej@isc.org> 2024-05-21 1 / 8
  • 4. Logarithmic Percentile Histogram Measuring DNS Latency Ondřej Surý <ondrej@isc.org> 2024-05-21 2 / 8
  • 5. Logarithmic Percentile Histogram Measuring DNS Latency • Both axes are logarithmic ‣ x-axis: slowest percentile ‣ y-axis: average latency • It makes the tail more visible • Variant of Complementary Cumulative Distribution Function • Very robust, can be used for monitoring (1% slowest percentile) • Introduced by the good folks at PowerDNS See more: https://blog.powerdns.com/2017/11/02/dns-performance- metrics-the-logarithmic-percentile-histogram Ondřej Surý <ondrej@isc.org> 2024-05-21 3 / 8
  • 7. DNS Performance for Developers Compare DNS Latencies • Comparing two branches of BIND 9 ‣ Did we improve the code? ‣ Did we made things worse? ‣ Currently, we compare the graphs by looking at them; ‣ And then running more tests; ‣ And then some wishful thinking… • Sending thanks to Python’s numpy and scipy developers! Ondřej Surý <ondrej@isc.org> 2024-05-21 4 / 8
  • 8. Pick the right statistics Compare DNS Latencies • The distribution is not normal • Non-parametrical test then? ‣ Kolmogorov-Smirnov test didn’t really work • Normalize the data? ‣ Box Cox Transformation didn’t really work • Maybe look only at the tail then? Ondřej Surý <ondrej@isc.org> 2024-05-21 5 / 8
  • 9. Looking at the tail Compare DNS Latencies • Pick the 95% (99%) percentile complement ‣ Either return the lowest bucket needed for 5% of responses ‣ Or count the answers in (1.9-2.0 second buckets) • Have at least 3 runs for each group • Yay! The data are normal and the group variances are equal ‣ Shapiro-Wilk test – first group (𝑊 = 0.905, 𝑝 = 0.436) – second group (𝑊 = 0.970, 𝑝 = 0.874) ‣ Brown-Forsyth test (𝐹 = 0.070, 𝑝 = 0.798) Ondřej Surý <ondrej@isc.org> 2024-05-21 6 / 8
  • 10. Parametrical test (ANOVA) Compare DNS Latencies • We can test more than two branches • One-way ANOVA reports difference between branches ‣ 𝐹 = 9244.090, 𝑝 < .001 • Two-sample T-Test (for confirmation) ‣ 𝑇 = −96.146, 𝑝 < .001 Ondřej Surý <ondrej@isc.org> 2024-05-21 7 / 8
  • 12. Other tests? More ideas? • Is this even correct? Or am I crazy? (I’m not a statistician) • Can we just compare two data sets (1x baseline with 1x branch)? • Can we use the full (right-skewed) population? • Are there any other non-parametrical tests I can try/use? • Are there any other suitable statistical methods? • Is this useful for other Internet measurements? Ondřej Surý <ondrej@isc.org> 2024-05-21 8 / 8