In this talk I will walk through multiple tools/resources available to help you handle large datasets from log files to Google Analytics. These new techniques will empower you to find more valuable insights and help you avoid the annoyance of crashing Excel spreadsheets.

1. @haydenroche3
Technical SEO at
Scale
What to do when your Excel keeps
crashing
1
Senior Analyst, Red Ventures
@haydenroche3
#BristolSEO

2. What we’ll walk through
● Brief intro to SQL and BigQuery
2
@haydenroche3
#BristolSEO

3. What we’ll walk through
● Brief intro to SQL and BigQuery
● Merging large data sets and the power you’ll gain
3
@haydenroche3
#BristolSEO

4. What we’ll walk through
● Brief intro to SQL and BigQuery
● Merging large data sets and the power you’ll gain
● Projects that YOU can do with “minimal-ish”
analytical effort
4
@haydenroche3
#BristolSEO

5. 5

6. Intro to BigQuery and
SQL 6#BristolSEO

7. BigQuery 101
7
@haydenroche3
#BristolSEO
Online database where you
can quickly pull data using
SQL

8. Joining Data Sources w/ SQL
8
@haydenroche3
#BristolSEO
http://www.sql-join.com/sql-join-types

9. FREE SQL Resources
9
1. https://www.w3schools.com/sql/default.asp
2. https://www.udemy.com/topic/sql/free/
3. https://www.codecademy.com/learn/learn-sql
4. Your colleagues and technical SEO community
@haydenroche3
#BristolSEO

10. BigQuery and SQL for SEOs
10
@haydenroche3
#BristolSEO
linkedin.com/in/areejabuali/
@areej_abuali
Find Areej’s slides here!

11. Challenge 11#BristolSEO

12. 12
“My website has 10K → 100K → 1M → 10M pages.
Doing analysis takes a decade without a data
expert.”
“Impossible” Technical SEO
@haydenroche3
#BristolSEO

13. 13
“Two stars in the universe have the same chance of
colliding as 4 bumblebees randomly flying around
the United States.”
Seemingly Impossible Statistic
@haydenroche3
#BristolSEO

14. 14
“My website has 10K → 100K → 1M → 10M pages.
Doing analysis takes a decade without a data
expert.”
Not So “Impossible” Technical SEO
@haydenroche3
#BristolSEO

15. Case Study:
15
“StockX is the world’s
first stock market for
things – a live ‘bid/ask’
marketplace.”
@haydenroche3
#BristolSEO

16. Starting with a Crawl
16
Find all the URLs!
Over 100K found
URLs in a starter
crawl
@haydenroche3
#BristolSEO

17. Export All URLs
17
@haydenroche3
#BristolSEO

18. Get Everything Into
BigQuery
18#BristolSEO

19. 1. Getting a CSV in BigQuery
19
@haydenroche3
Upload from local storage (your desktop)
→ File size limitations
#BristolSEO

20. 1. Getting a CSV in BigQuery
20
@haydenroche3
Better strategy: upload csv to
Cloud Storage
#BristolSEO

21. 2. Creating a Storage Bucket
21
@haydenroche3
https://cloud.google.com/storage/docs/
creating-buckets
#BristolSEO

22. 3. Create a Data Table
22
Create Table within your
“techseo” project
@haydenroche3
#BristolSEO

23. 3. Create a Data Table
23
Find your Screaming Frog CSV from
Cloud Storage
@haydenroche3
#BristolSEO

24. Trust the Process
24#BristolSEO

25. Starting with a Crawl
25
sf.URL sf.Status_Code
/ 200
/supreme 200
/supreme/jackets 200
/supreme-duffle-bag-ss18-black 200
/nike/airmax/95 200
@haydenroche3
#BristolSEO

26. Starting with a Crawl
26
SELECT
sf.URL,
sf.Status_Code
FROM `techseo.site_audit.screaming_frog_crawl` sf
LIMIT 5
@haydenroche3
#BristolSEO

27. Starting with a Crawl
27
sf.URL sf.Status_Code
/ 200
/supreme 200
/supreme/jackets 200
/supreme-duffle-bag-ss18-black 200
/nike/airmax/95 200
SELECT
sf.URL,
sf.Status_Code
FROM
`techseo.site_audit.screaming_frog_
crawl` sf
LIMIT 5
@haydenroche3
#BristolSEO

28. Now Let’s Add
More Data
28#BristolSEO

29. Adding in Google Analytics
29
Option 1:
Export Data
From UI
Pros: Easy, Quick
Cons: Manual
@haydenroche3
#BristolSEO

30. Adding in Google Analytics
30
Option 1:
Export Data
From UI
Pros: Easy, Quick
Cons: Manual
Option 2:
BigQuery + GA Data
Integration
Pros: Automated & granular
Cons: API setup knowledge*
@haydenroche3
#BristolSEO

31. Not real data!
31#BristolSEO@haydenroche3

32. Adding in Google Analytics
32
ga.Landing_Page ga.Sessions ga.Sales ga.Revenue
/ 800,000 12,800 £576,000
/supreme 29,000 145 £11,165
/supreme/jackets 6,700 52 £11,759
/supreme-duffle-bag-ss18-black 4,400 48 £19,360
/nike/airmax/95 18,800 47 £3,055
@haydenroche3
#BristolSEO

33. Adding in Google Analytics
33
ga.Landing_Page ga.Sessions ga.Sales ga.Revenue
/ 800,000 12,800 £576,000
/supreme 29,000 145 £11,165
/supreme/jackets 6,700 52 £11,759
/supreme-duffle-bag-ss18-black 4,400 48 £19,360
/nike/airmax/95 18,800 47 £3,055
SELECT
ga.Landing_Page,
ga.Sessions,
ga.Sales,
ga.Revenue
FROM
`techseo.site_audit.google_analyti
cs` ga
@haydenroche3
#BristolSEO

34. Join SF and GA
34#BristolSEO

35. Joining SF & GA
35
@haydenroche3
SELECT
sf.URL,
sum(ga.Sessions) AS Sessions,
sum(ga.Revenue) AS Revenue,
round(ga.Revenue/ga.Sessions, 2) AS RPS
FROM
`techseo.site_audit.screaming_frog_crawl` sf
LEFT JOIN `techseo.site_audit.google_analytics` ga ON (sf.URL = ga.Landing_Page)
GROUP BY
sf.URL
#BristolSEO

36. Joining SF & GA
36
@haydenroche3
SELECT
sf.URL,
sum(ga.Sessions) AS Sessions,
sum(ga.Revenue) AS Revenue,
round(ga.Revenue/ga.Sessions, 2) AS RPS
FROM
`techseo.site_audit.screaming_frog_crawl` sf
LEFT JOIN
`techseo.site_audit.google_analytics` ga ON
(sf.URL = ga.Landing_Page)
GROUP BY
sf.URL
#BristolSEO
sf.URL ga.Sessions ga.Revenue ga.RPS
/ 800,000 £576,000 £0.72
/supreme 29,000 £11,165 £0.39
/supreme/jackets 6,700 £11,759 £1.76
/supreme-duffle-bag-
ss18-black
4,400 £19,360 £4.40
/nike/airmax/95 18,800 £3,055 £0.16

37. Adding in Search
Console
37#BristolSEO

38. Once again…
Not real data!
38#BristolSEO@haydenroche3

39. Adding in Search Console
39
gsc.Landing_Page gsc.Query gsc.Clicks gsc.Impressions gsc.Position
/ stockx 658,400 823,000 1.00
/supreme supreme 79,300 1,220,000 4.00
/supreme/jackets supreme windbreaker 1,260 3,600 1.00
/supreme-duffle-bag-ss18-black supreme duffle bag 2,268 8,100 1.20
/nike/airmax/95 nike airmax 95 925 74,000 6.50
@haydenroche3
#BristolSEO

40. Joining SF, GA & GSC
40
SELECT
sf.URL,
sum(ga.Sessions) AS Sessions,
sum(ga.Revenue) AS Revenue,
count(gsc.Query) AS Ranking_KWs,
avg(gsc.Position) AS Avg_Pos
FROM `techseo.site_audit.screaming_frog_crawl` sf
LEFT JOIN `techseo.site_audit.google_analytics` ga ON (sf.URL = ga.Landing_Page)
LEFT JOIN `techseo.site_audit.google_search_console` gsc ON (sf.URL = gsc.Landing_Page)
GROUP BY
sf.URL
@haydenroche3
#BristolSEO

41. Joining SF, GA & GSC
41
sf.URL ga.Sessions ga.Revenue gsc.Ranking_KWs gsc.Avg_Pos
/ 800,000 £576,000 30 1.00
/supreme 29,000 £11,165 7,900 4.00
/supreme/jackets 6,700 £11,759 610 1.00
/supreme-duffle-bag-ss18-black 4,400 £19,360 10 1.20
/nike/airmax/95 18,800 £3,055 4 6.50
@haydenroche3
#BristolSEO

42. 42#BristolSEO@haydenroche3

43. Additional Data Sources
43
● Off-page factors → backlinks and referring domains
@haydenroche3
#BristolSEO

44. Additional Data Sources
44
● Off-page factors → backlinks and referring domains
● Log files → Googlebot requests and frequency
@haydenroche3
#BristolSEO

45. Additional Data Sources
45
● Off-page factors → backlinks and referring domains
● Log files → Googlebot requests and frequency
● Pagespeed insights → TTI, FCP, FMP
@haydenroche3
#BristolSEO

46. Additional Data Sources
46
● Off-page factors → backlinks and referring domains
● Log files → Googlebot requests and frequency
● Pagespeed insights → TTI, FCP, FMP
● GSC Coverage Report → indexed pages
@haydenroche3
#BristolSEO

47. Additional Data Sources
47
● Off-page factors → backlinks and referring domains
● Log files → Googlebot requests and frequency
● Keywords → 3rd party ranking data
● Pagespeed insights → TTI, FCP, FMP
● GSC Coverage Report → indexed pages
@haydenroche3
#BristolSEO

48. 48
“Let’s Take A Quick Breather”
@haydenroche3
#BristolSEO

49. 49
“If you flip a coin 100 times, there’s less than an 8%
chance that it ends 50/50.”
Seemingly Impossible Statistic
@haydenroche3
#BristolSEO

50. Time to
Harness the
Data Power
50#BristolSEO

51. Site Consolidation 51#BristolSEO

52. Site Consolidation
52
Goal:
Identify low-value pages/subfolders
→ redirect or remove
@haydenroche3
#BristolSEO

53. Site Consolidation
53
@haydenroche3
URL
/adidas-yeezy-boost-350-v2-black
/adidas-yeezy-boost-350-v2-core-black-green
/palace-pro-tool-t-shirt-black
/palace-pro-tool-t-shirt-white
/supreme-waist-bag-ss19-black
/supreme-waist-bag-ss19-light-blue
/jordan-4-retro-black-cement-2012
/jordan-4-retro-white-cement-2012
Start with a
sample → expand
to everything
#BristolSEO

54. Site Consolidation
54
@haydenroche3
Use a CASE Statement to
group URLs at scale
#BristolSEO

55. Site Consolidation
55
@haydenroche3
CASE
WHEN URL like ‘/adidas%’ THEN ‘Adidas’
WHEN URL like ‘/palace%’ THEN ‘Palace’
WHEN URL like ‘/supreme%’ THEN ‘Supreme’
WHEN URL like ‘/jordan%’ THEN ‘Air Jordan’
ELSE ‘’
END as Brand
#BristolSEO
Wildcard (%)
allows any
character/num
ber to follow
‘/adidas’

56. Site Consolidation
56
@haydenroche3
Time to let go of
the manual Excel
filtering +
labelling
#BristolSEO

57. Site Consolidation
57
@haydenroche3
URL
/adidas-yeezy-boost-350-v2-black
/adidas-yeezy-boost-350-v2-core-black-green
/palace-pro-tool-t-shirt-black
/palace-pro-tool-t-shirt-white
/supreme-waist-bag-ss19-black
/supreme-waist-bag-ss19-light-blue
/jordan-4-retro-black-cement-2012
/jordan-4-retro-white-cement-2012
CASE
WHEN URL like ‘/adidas%’ THEN ‘Adidas’
WHEN URL like ‘/palace%’ THEN ‘Palace’
WHEN URL like ‘/supreme%’ THEN ‘Supreme’
WHEN URL like ‘/jordan%’ THEN ‘Air Jordan’
ELSE ‘’
END as Brand
#BristolSEO

58. Site Consolidation
58
@haydenroche3
Landing_Page Brand
/adidas-yeezy-boost-350-v2-black Adidas
/adidas-yeezy-boost-350-v2-core-black-green Adidas
/palace-pro-tool-t-shirt-black Palace
/palace-pro-tool-t-shirt-white Palace
/supreme-waist-bag-ss19-black Supreme
/supreme-waist-bag-ss19-light-blue Supreme
/jordan-4-retro-black-cement-2012 Air Jordan
/jordan-4-retro-white-cement-2012 Air Jordan
#BristolSEO

59. Site Consolidation
59
@haydenroche3
Landing_Page Brand Type
/adidas-yeezy-boost-350-v2-black Adidas Shoes
/adidas-yeezy-boost-350-v2-core-black-green Adidas Shoes
/palace-pro-tool-t-shirt-black Palace T-Shirt
/palace-pro-tool-t-shirt-white Palace T-Shirt
/supreme-waist-bag-ss19-black Supreme Bag
/supreme-waist-bag-ss19-light-blue Supreme Bag
/jordan-4-retro-black-cement-2012 Air Jordan Shoes
/jordan-4-retro-white-cement-2012 Air Jordan Shoes
#BristolSEO

60. Site Consolidation
60
@haydenroche3
Landing_Page Brand Type Colour
/adidas-yeezy-boost-350-v2-black Adidas Shoes Black
/adidas-yeezy-boost-350-v2-core-black-green Adidas Shoes Black-Green
/palace-pro-tool-t-shirt-black Palace T-Shirt Black
/palace-pro-tool-t-shirt-white Palace T-Shirt White
/supreme-waist-bag-ss19-black Supreme Bag Black
/supreme-waist-bag-ss19-light-blue Supreme Bag Light-Blue
/jordan-4-retro-black-cement-2012 Air Jordan Shoes Black
/jordan-4-retro-white-cement-2012 Air Jordan Shoes White
#BristolSEO

61. Site Consolidation
61
@haydenroche3
Combining CASE statements
with what we learned before
#BristolSEO

62. Site Consolidation
62
@haydenroche3
Brand Type Indexed Pages
Adidas Shoes 150
Palace T-Shirt 15,000
Supreme Bag 10,000
Air Jordan Shoes 50
#BristolSEO

63. Site Consolidation
63
@haydenroche3
Brand Type Indexed Pages Revenue
Adidas Shoes 150 £750,000
Palace T-Shirt 15,000 £1,000
Supreme Bag 10,000 £300,000
Air Jordan Shoes 50 £125,000
#BristolSEO

64. Site Consolidation
64
@haydenroche3
Brand Type Indexed Pages Revenue Backlinks
Adidas Shoes 150 £750,000 1,400
Palace T-Shirt 15,000 £1,000 5
Supreme Bag 10,000 £300,000 100
Air Jordan Shoes 50 £125,000 350
#BristolSEO

65. Site Consolidation
65
@haydenroche3
Brand Type Indexed Pages Revenue Backlinks
Adidas Shoes 150 £750,000 1,400
Palace T-Shirt 15,000 £1,000 5
Supreme Bag 10,000 £300,000 100
Air Jordan Shoes 50 £125,000 350
#BristolSEO

66. Site Consolidation
66
@haydenroche3
Save and reuse all CASE
statements for the future!
#BristolSEO

67. 67
Technical SEO and BQ insights

68. Let’s Recap 68#BristolSEO

69. What Have We Learned?
69
● Large CSVs can live in BigQuery, not your desktop
@haydenroche3
#BristolSEO

70. What Have We Learned?
70
● Large CSVs can live in BigQuery, not your desktop
● The power of the LEFT JOIN
@haydenroche3
#BristolSEO

71. What Have We Learned?
71
● Large CSVs can live in BigQuery, not your desktop
● The power of the LEFT JOIN
● CASE statements and data groupings
@haydenroche3
#BristolSEO

72. What Have We Learned?
72
● Large CSVs can live in BigQuery, not your desktop
● The power of the LEFT JOIN
● CASE statements and data groupings
● How to find quick insights from combining data sources
@haydenroche3
#BristolSEO

73. Additional Resources
73
1. Uploading CSVs to BigQuery →
https://cloud.google.com/bigquery/docs/loading-data-cloud-storage-csv
2. BigQuery Pricing Scheme →
https://cloud.google.com/bigquery/pricing
3. Set up BigQuery export →
https://support.google.com/analytics/answer/3416092
4. Creating a storage bucket in GCS →
https://cloud.google.com/storage/docs/creating-buckets
@haydenroche3
#BristolSEO

74. Thank You!
74
@haydenroche3
#BristolSEO
Questions/Comments?
➔ @haydenroche3
➔ linkedin.com/in/haydenroche