The document discusses crawl budget optimization for enterprise websites, highlighting the importance of crawl demand, crawl rate, and effective sitemap management. It outlines a three-step approach to optimize crawl budgets by identifying low-quality pages, adjusting sitemaps, and improving page load speed. Additionally, strategies for increasing crawl budget through link building and page rank sculpting are presented, emphasizing collaboration between content and PR teams.

1. Master Crawl Budget
Optimization for
Enterprise Websites
Manick Bhan | LinkGraph
SLIDESHARE.NET/MANICKBHAN1
@madmanick

2. Hi, I’m
Manick👋
Founder/CTO
Scalable SEO for agencies +
brands
- 89,000 client link placements
secured
- 25,000 landing pages + blog posts
published and ranked
- 8.8 billion search results analyzed
Creator of SearchAtlas
SEO software suite
Formerly Founder & CEO
Rukkus.com (acquired by Stubhub +
TickPick)
Avid researcher of Google ranking
factors (10+ years)
🌐
🚀
🎟
📊
📈

4. What is Crawl Budget?

5. ● 367 million
domains on the
internet
● 130 trillion
pages in the
Google Index
● 20 billion pages
per day crawled
A Map of Internet

6. A Map of Internet
DA vs DR & OT

7. Crawl Budget
The amount of time and pages
Google is going to crawl on your
site.

8. Crawl Demand
How popular your pages are.
High OT + Ranking Keywords = high
Crawl Demand.

9. Crawl Rate
The speed at which Google is
crawling your site.

11. Does every website need to worry
about their crawl budget?

12. Enterprise-level and ecommerce
sites are at risk for maxing out their
budget
- Ecommerce Brands with 1K+
SKUs: sometimes up to 50% of
products get 0 organic traffic

13. Sitemaps are great for making sure
Google can see all of your pages.
Can also hurt you if they’re filled
with lower-value or under-
performing pages.

15. What Tools can we use to analyze
Google’s Crawl Behavior

16. Google Search Console - Crawl Stats

17. Log File Analyzer (ScreamingFrog)

18. So, how can enterprise sites
perform crawl budget optimization?

19. Step 1
Identify which pages Google is
actually crawling on your site

20. Step 2: Pruning
Eliminate low quality pages, fix
technical errors, 404ing pages.
use the robot tags [noindex,
nofollow] or a 301 redirect

21. Case Study: $20M/Year Ecommerce Retailer
36% of Pages (4K pages) are Low Quality
Word Count vs Search Impressions

22. Identifying Low Quality Pages
Search Impressions vs Total Ranking Keywords

23. Step 3:
Adjust your sitemap
a)remove low value pages from
sitemap
b)increase “priority” and
“changefreq” of high-value pages

24. TIP
Improve Page Load Speed so that
Google can index your pages faster

25. TIP
Accept that not all your landing
pages need to rank in Google

27. How can you increase your
Crawl Budget?

28. Building Site Authority with Link
Building and Digital PR

30. Step 1:
Use a tool like SearchAtlas to find
sites linking to your competitors but
not linking to you, and outreach
them with Digital PR

31. TIP #1
Create free tools, resources, guides, and
other high value products you can give
away to increase natural link acquisition
This is how I got hundreds free DA 60+
backlinks, including from Neil Patel,
Universities, Wordpress, and Google

32. TIP #2
Reclaim lost PageRank via Broken
Backlinks - use “Broken Links” in
Ahrefs to find backlinks that are
pointing to broken pages on your
site.

33. TIP #3
Use HARO to help/pitch journalists
looking for insightful sources in
your industry

35. Strategy
Work with content & PR teams on
executing a guest author program
for company leadership

37. Let’s talk about PageRank…

41. Pages with more PageRank get
crawled more frequently by Google

42. Visualizing PageRank and Depth
(SearchAtlas)

43. Visualizing PageRank and Depth
(SearchAtlas)

44. How do you determine which pages
would benefit most from PageRank
sculpting?

45. Strategy #1
Find pages that have good traffic
but don’t have enough PageRank

46. Find ways to get those pages more
internal links and send more Page
Rank there

47. Strategy #2 - Wasted PageRank
Focus on the pages that have a lot
of internal links, but don’t get much
traffic, search impressions, and
rank for very few keywords

48. Benchmarking & SEO A/B Testing

49. Use a Google Search Console tool
to track changes and see if crawl
budget optimization is working

50. GSC Insights

51. Bravo!

52. THANK YOU!
manick@linkgraph.io
@madmanick
Say hi