The document discusses tools and techniques for debugging CDN issues, emphasizing the use of Fastly's debugging capabilities and CURL commands to diagnose problems effectively. It highlights features such as fast setup, clear output, and the importance of data for reproducibility in troubleshooting. Additionally, it describes best practices for using diagnostic tools like Fastly Debug Header and practical examples of their application.

2. D E B U G G I N G Y O U R C D N
A U S T I N S P I R E S
FA S T LY

3. I ’ M A U S T I N S P I R E S
A S P I R E S O R A U S T I N S P I R E S
O N T H E I N T E R N E T
C U S T O M E R E N G I N E E R I N G
T E A M L E A D
W O R K O N S U P P O R T A N D
D O C U M E N TA T I O N
M Y FA V O R I T E P I C T U R E O N
T H E I N T E R N E T R I G H T N O W
I S T H I S C A T - >

4. W H O ’ S F I L E D A T I C K E T ?

5. W H O ’ S T E A M M E M B E R
H A S F I L E D A T I C K E T ?

6. 8 T O O L S F O R B E T T E R
C D N D E B U G G I N G

7. A LT E R N AT E T I T L E :
T H E T H I N G S I W I S H
M O R E P E O P L E K N E W
A B O U T FA S T LY A N D
N E T W O R K S I N G E N E R A L

8. W H Y T H E S E S P E C I F I C T H I N G S
• They’re fast
• They can be set up quickly
• Diagnostic tools can be run quickly

9. W H Y T H E S E S P E C I F I C T H I N G S
• They’re fast
• They can be set up quickly
• Diagnostic tools can be run quickly
• They’re clear
• Predictable output
• Our support team can run with the results
immediately, or be pulled in ASAP

10. W H Y T H E S E S P E C I F I C T H I N G S
• They’re fast
• They can be set up quickly
• Diagnostic tools can be run quickly
• They’re clear
• Predictable output
• Our support team can run with the results
immediately, or be pulled in ASAP
• They’re easy
• Limited screwups in a stressful time

11. Y O U M AY K N O W S O M E
O F T H E S E , B U T S H A R E
T H E M W I T H Y O U R T E A M

12. 1 — D E B U G . FA S T LY. C O M

13. D E B U G . FA S T LY. C O M
• For when _your_ users report problems

14. D E B U G . FA S T LY. C O M
• For when _your_ users report problems
• Reports ISP information, DNS information, etc. we can
use to debug

15. D E B U G . FA S T LY. C O M
• For when _your_ users report problems
• Reports ISP information, DNS information, etc. we can
use to debug
• Information bundled in an encrypted block

16. D E B U G . FA S T LY. C O M
• For when _your_ users report problems
• Reports ISP information, DNS information, etc. we can
use to debug
• Information bundled in an encrypted block
B O N U S :
1 0 0 % G E N E R A T E D W I T H S Y N T H E T I C
R E S P O N S E S , S O S U I TA B L E F O R
P R O G R A M A T I C U S E

17. 2 — C U R L - S V O / D E V / N U L L
O R I T D I D N ’ T H A P P E N

18. C U R L - S V O / D E V / N U L L
• More than a header check
• Runs request though the CDN
• Pipes output to null

19. C U R L - S V O / D E V / N U L L
• More than a header check
• Runs request though the CDN
• Pipes output to null
• “I’m seeing odd caching behavior”
• Saying this without data is worthless

20. C U R L - S V O / D E V / N U L L
• More than a header check
• Runs request though the CDN
• Pipes output to null
• “I’m seeing odd caching behavior”
• Saying this without data is worthless
• If we can reproduce the problem, we can start fixing
• Investigation and reproduction is ~60% of a ticket’s
time

21. C U R L - S V O / D E V / N U L L
D O N ’ T U S E C U R L - I

22. C U R L - S V O / D E V / N U L L
D O N ’ T U S E C U R L - I
P L E A S E , D O N ’ T U S E C U R L - I

23. C U R L - S V O / D E V / N U L L
• `
D O N ’ T U S E C U R L - I
I T ’ S N O T A R E A L R E Q U E S T

24. 3 — FA S T LY D E B U G H E A D E R

25. FA S T LY D E B U G H E A D E R
T R A C E R E Q U E S T S
T H R O U G H T H E C D N

26. FA S T LY D E B U G H E A D E R
$ C U R L - S V O / D E V / N U L L E X A M P L E . C O M
- H “ F A S T L Y - D E B U G : 1 ”

27. FA S T LY D E B U G H E A D E R
$ C U R L - S V O / D E V / N U L L E X A M P L E . C O M
- H “ F A S T L Y - D E B U G : 1 ”
< H T T P / 1 . 1 2 0 0 O K
< …
< F A S T L Y - D E B U G - P A T H : ( D C A C H E - S J C 3 1 3 5 -
S J C 1 4 3 5 1 1 2 9 5 1 ) ( F C A C H E - S J C 3 1 2 2 - S J C
1 4 3 5 1 0 6 3 6 6 ) ( D C A C H E - S E A 1 9 2 2 - S E A
1 4 3 5 1 0 6 3 6 6 ) ( F C A C H E - S E A 1 9 2 1 - S E A
1 4 3 5 1 0 6 3 6 6 )
< F A S T L Y - D E B U G - T T L : ( H C A C H E - S J C 3 1 3 5 - S J C
- - 6 5 8 4 ) ( M C A C H E - S E A 1 9 2 2 - S E A - - 0 )

28. FA S T LY D E B U G H E A D E R
T O D E C I P H E R T H E WA L L
O F T E X T…

29. FA S T LY D E B U G H E A D E R
F A S T L Y - D E B U G - P A T H :
( D C A C H E - S J C 3 1 3 5 - S J C 1 4 3 5 1 1 2 9 5 1 )
E D G E C A C H E I N S A N J O S E , T I M E

30. FA S T LY D E B U G H E A D E R
F A S T L Y - D E B U G - P A T H :
( D C A C H E - S J C 3 1 3 5 - S J C 1 4 3 5 1 1 2 9 5 1 )
E D G E C A C H E I N S A N J O S E , T I M E
( F C A C H E - S J C 3 1 2 2 - S J C 1 4 3 5 1 0 6 3 6 6 )
F E T C H C A C H E I N S A N J O S E , T I M E

31. FA S T LY D E B U G H E A D E R
F A S T L Y - D E B U G - P A T H :
( D C A C H E - S J C 3 1 3 5 - S J C 1 4 3 5 1 1 2 9 5 1 )
E D G E C A C H E I N S A N J O S E , T I M E
( F C A C H E - S J C 3 1 2 2 - S J C 1 4 3 5 1 0 6 3 6 6 )
F E T C H C A C H E I N S A N J O S E , T I M E
( D C A C H E - S E A 1 9 2 2 - S E A 1 4 3 5 1 0 6 3 6 6 )
E D G E C A C H E I N S E A T T L E , T I M E

32. FA S T LY D E B U G H E A D E R
F A S T L Y - D E B U G - P A T H :
( D C A C H E - S J C 3 1 3 5 - S J C 1 4 3 5 1 1 2 9 5 1 )
E D G E C A C H E I N S A N J O S E , T I M E
( F C A C H E - S J C 3 1 2 2 - S J C 1 4 3 5 1 0 6 3 6 6 )
F E T C H C A C H E I N S A N J O S E , T I M E
( D C A C H E - S E A 1 9 2 2 - S E A 1 4 3 5 1 0 6 3 6 6 )
E D G E C A C H E I N S E A T T L E , T I M E
( F C A C H E - S E A 1 9 2 1 - S E A 1 4 3 5 1 0 6 3 6 6 )
F E T C H C A C H E I N S E A T T L E , T I M E

33. FA S T LY D E B U G H E A D E R
F A S T L Y - D E B U G - P A T H :
( D C A C H E - S J C 3 1 3 5 - S J C 1 4 3 5 1 1 2 9 5 1 )
E D G E C A C H E I N S A N J O S E , T I M E
( F C A C H E - S J C 3 1 2 2 - S J C 1 4 3 5 1 0 6 3 6 6 )
F E T C H C A C H E I N S A N J O S E , T I M E
( D C A C H E - S E A 1 9 2 2 - S E A 1 4 3 5 1 0 6 3 6 6 )
E D G E C A C H E I N S E A T T L E , T I M E
( F C A C H E - S E A 1 9 2 1 - S E A 1 4 3 5 1 0 6 3 6 6 )
F E T C H C A C H E I N S E A T T L E , T I M E

34. FA S T LY D E B U G H E A D E R
F A S T L Y - D E B U G - P A T H :
( D C A C H E - S J C 3 1 3 5 - S J C 1 4 3 5 1 1 2 9 5 1
E D G E C A C H E I N S A N J O S E , T I M E
( F C A C H E - S J C 3 1 2 2 - S J C 1 4 3 5 1 0 6 3 6 6
F E T C H C A C H E I N S A N J O S E , T I M E
( D C A C H E - S E A 1 9 2 2 - S E A 1 4 3 5 1 0 6 3 6 6
E D G E C A C H E I N S E A T T L E , T I M E
( F C A C H E - S E A 1 9 2 1 - S E A 1 4 3 5 1 0 6 3 6 6
F E T C H C A C H E I N S E A T T L E , T I M E
D I F F E R E N T E D G E ,
S A M E F E T C H /
S H I E L D
=
C A C H E H I T

35. F A S T L Y - D E B U G - T T L :
( H C A C H E - S J C 3 1 3 5 - S J C - 6 5 8 4 )
H I T I N S A N J O S E # 3 1 3 5 , A G E
FA S T LY D E B U G H E A D E R

36. F A S T L Y - D E B U G - T T L :
( H C A C H E - S J C 3 1 3 5 - S J C - 6 5 8 4 )
H I T I N S A N J O S E # 3 1 3 5 , A G E
( M C A C H E - S E A 1 9 2 2 - S E A - - 0 )
M I S S I N S E A T T L E # 1 9 2 2 , A G E
FA S T LY D E B U G H E A D E R

37. 4 — W H AT X - T I M E R
A C T U A L LY M E A N S

38. W H AT X - T I M E R A C T U A L LY M E A N S
$ C U R L - S V O / D E V / N U L L E X A M P L E . C O M

39. $ C U R L - S V O / D E V / N U L L E X A M P L E . C O M
…
< X - T I M E R : S 1 4 3 5 1 1 2 9 5 1 . 0 6 9 2 5 2 , V S 0 , V E 1
…
W H AT X - T I M E R A C T U A L LY M E A N S

40. $ C U R L - S V O / D E V / N U L L E X A M P L E . C O M
…
< X - T I M E R : S 1 4 3 5 1 1 2 9 5 1 . 0 6 9 2 5 2 , V S 0 , V E 1
…
U N I X T I M E S TA M P
W H AT X - T I M E R A C T U A L LY M E A N S

41. $ C U R L - S V O / D E V / N U L L E X A M P L E . C O M
…
< X - T I M E R : S 1 4 3 5 1 1 2 9 5 1 . 0 6 9 2 5 2 , V S 0 , V E 1
…
VA R N I S H S TA R T T I M E R
W H AT X - T I M E R A C T U A L LY M E A N S

42. $ C U R L - S V O / D E V / N U L L E X A M P L E . C O M
…
< X - T I M E R : S 1 4 3 5 1 1 2 9 5 1 . 0 6 9 2 5 2 , V S 0 , V E 1
…
VA R N I S H E N D T I M E R
W H AT X - T I M E R A C T U A L LY M E A N S

43. $ C U R L - S V O / D E V / N U L L E X A M P L E . C O M
…
< X - T I M E R : S 1 4 3 5 1 1 2 9 5 1 . 0 6 9 2 5 2 , V S 0 , V E 1
…
VA R N I S H E N D T I M E R
0 - 1 M S = = H I T
W H AT X - T I M E R A C T U A L LY M E A N S

44. $ C U R L - S V O / D E V / N U L L E X A M P L E . C O M
…
< X - T I M E R : S 1 4 3 5 1 1 2 9 5 1 . 0 6 9 2 5 2 , V S 0 , V E 1
…
VA R N I S H E N D T I M E R
> 1 M S = = M I S S
W H AT X - T I M E R A C T U A L LY M E A N S

45. $ C U R L - S V O / D E V / N U L L E X A M P L E . C O M
…
< X - T I M E R : S 1 4 3 5 1 1 2 9 5 1 . 0 6 9 2 5 2 , V S 0 , V E 1
…
VA R N I S H E N D T I M E R
R E A L LY H I G H = = P R O B L E M
W H AT X - T I M E R A C T U A L LY M E A N S

46. 5 — $ M T R - C 2 0 - W - R

47. M T R - C 2 0 - W - R
• MTR
• Network testing tool
• Combines Ping and Traceroute

48. M T R - C 2 0 - W - R
• MTR
• Network testing tool
• Combines Ping and Traceroute
• Runs 20 network pings and traces
• Outputs a minimized latency and hop report

49. M T R - C 2 0 - W - R
• MTR
• Network testing tool
• Combines Ping and Traceroute
• Runs 20 network pings and traces
• Outputs a minimized latency and hop report
• Spot network problems
• High latency
• Bad routes

50. M T R - C 2 0 - W - R
~ mtr -c 20 -r -w www.example.com

51. M T R - C 2 0 - W - R
~ mtr -c 20 -r -w www.example.com
HOST: aspirestrabajo.local Loss% Snt Last Avg Best Wrst
StDev
1.|-- 10.100.20.2 0.0% 20 2.3 2.7 1.6 14.1
2.6
2.|-- 8.18.217.1 0.0% 20 1.9 2.5 1.8 8.0
1.2
3.|-- ae3-464.bar2.sanfrancisco1.level3.net 0.0% 20 2.1 5.3 2.1 13.5
3.7
4.|-- fastly-inc.bar2.sanfrancisco1.level3.net 0.0% 20 4.0 4.1 3.7 5.4
0.2
5.|-- ??? 100.0 20 0.0 0.0 0.0 0.0
0.0
6.|-- 23.235.47.73 0.0% 20 4.3 7.5 3.7 53.9
11.3

52. M T R - C 2 0 - W - R
~ mtr -c 20 -r -w www.example.com
HOST: aspirestrabajo.local Loss% Snt Last Avg Best Wrst
StDev
1.|-- 10.100.20.2 0.0% 20 2.3 2.7 1.6 14.1
2.6
2.|-- 8.18.217.1 0.0% 20 1.9 2.5 1.8 8.0
1.2
3.|-- ae3-464.bar2.sanfrancisco1.level3.net 0.0% 20 2.1 5.3 2.1 13.5
3.7
4.|-- fastly-inc.bar2.sanfrancisco1.level3.net 0.0% 20 4.0 4.1 3.7 5.4
0.2
5.|-- ??? 100.0 20 0.0 0.0 0.0 0.0
0.0
6.|-- 23.235.47.73 0.0% 20 4.3 7.5 3.7 53.9
11.3
H O S T N A M E S :
O F T E N C O N TA I N
T R A N S I T V E N D O R ,
R E G I O N D ATA

53. M T R - C 2 0 - W - R
~ mtr -c 20 -r -w www.example.com
HOST: aspirestrabajo.local Loss% Snt Last Avg Best Wrst
StDev
1.|-- 10.100.20.2 0.0% 20 2.3 2.7 1.6 14.1
2.6
2.|-- 8.18.217.1 0.0% 20 1.9 2.5 1.8 8.0
1.2
3.|-- ae3-464.bar2.sanfrancisco1.level3.net 0.0% 20 2.1 5.3 2.1 13.5
3.7
4.|-- fastly-inc.bar2.sanfrancisco1.level3.net 0.0% 20 4.0 4.1 3.7 5.4
0.2
5.|-- ??? 100.0 20 0.0 0.0 0.0 0.0
0.0
6.|-- 23.235.47.73 0.0% 20 4.3 7.5 3.7 53.9
11.3
PA C K E T D ATA
T R A C K F O R
PA C K E T L O S S

54. M T R - C 2 0 - W - R
~ mtr -c 20 -r -w www.example.com
HOST: aspirestrabajo.local Loss% Snt Last Avg Best Wrst
StDev
1.|-- 10.100.20.2 0.0% 20 2.3 2.7 1.6 14.1
2.6
2.|-- 8.18.217.1 0.0% 20 1.9 2.5 1.8 8.0
1.2
3.|-- ae3-464.bar2.sanfrancisco1.level3.net 0.0% 20 2.1 5.3 2.1 13.5
3.7
4.|-- fastly-inc.bar2.sanfrancisco1.level3.net 0.0% 20 4.0 4.1 3.7 5.4
0.2
5.|-- ??? 100.0 20 0.0 0.0 0.0 0.0
0.0
6.|-- 23.235.47.73 0.0% 20 4.3 7.5 3.7 53.9
11.3
L AT E N C Y D ATA
T R A C K F O R
S L O W H O P S

55. 6 — P U R G E S H AV E I D ’ S

56. P U R G E I D S
• Every purge is logged in our systems
• If there’s a problem in purging, we can track it down
with the ID
~ C U R L - X P U R G E W W W . E X A M P L E . C O M
{ " S T A T U S " : " O K " ,
“ I D ” : " 2 1 6 - 1 4 3 2 0 6 0 4 1 7 - 7 8 7 7 3 3 5 " }

57. 7 — W E B PA G E T E S T

58. W E B PA G E T E S T
• www.webpagetest.org
• You might know about it

59. W E B PA G E T E S T
• www.webpagetest.org
• You might know about it
• Built with browser technology to simulate end user activity
• All sorts of browser versions and traffic simulations

60. W E B PA G E T E S T
• www.webpagetest.org
• You might know about it
• Built with browser technology to simulate end user activity
• All sorts of browser versions and traffic simulations
• Highly featured
• Scriptable
• Side-by-side comparisons

61. E V E RY O N E O N Y O U R T E A M
S H O U L D B E FA M I L I A R
W I T H W E B PA G E T E S T

62. 8 — I R C

63. I R C
• #fastly on Freenode
• We’re there at most hours
• Obviously, there’s more people there M-F during
daylight

64. I R C
• #fastly on Freenode
• We’re there at most hours
• Obviously, there’s more people there M-F during
daylight
• We’ll set you up a private room if you ask

65. I R C
• #fastly on Freenode
• We’re there at most hours
• Obviously, there’s more people there M-F during
daylight
• We’ll set you up a private room if you ask
• All IRC rooms push into the Fastly Slack account
• “Poor man’s ZNC bouncer”

66. ~ 2 0 O F T H E S E G O T C U T F O R
T I M E , S O F I N D U S L AT E R

67. W H AT ’ S T H E P O I N T ?

68. “ T H E B E S T T I C K E T S A R E
T H E O N E S N O T F I L E D ”

69. H E L P S U S D I A G N O S E
W H E N Y O U C A L L U S I N

70. S E L F S E R V I C E I S T H E
FA S T E S T F E E D B A C K L O O P

71. T H E FA S T E R Y O U ’ R E U P
A N D R U N N I N G , T H E
B E T T E R

72. S U P P O R T @ FA S T LY. C O M
“ T O S P E A K W I T H A H U M A N ”