Wait! Exclusive 60 day trial to the world's largest digital library.
The SlideShare family just got bigger. You now have unlimited* access to books, audiobooks, magazines, and more from Scribd.Cancel anytime.
Francois Goube during BrightonSEO on September 2019 about the best kept secrets from robots.txt
Explore all the ways search engines are exploiting your robots.txt. Francois will help you understand how Google is using the robots.txt file to understand your website. Spam detection, optimizing crawl budget, prioritizing crawling queues, we will dive into the most strategic use cases to show how to leverage the use of robots.txt.