OK Google, Whats next? - OMT Wiesbaden 2018Bastian Grimm
My talk from Online Marketing Tag (OMT) Wiesbaden 2018 covering the ever-changing landscape of search and some of the stuff that I think will have a significant impact on SEO in the very near future!
Migration Best Practices - Search Y 2019, ParisBastian Grimm
My talk from SEARCHY 2019 in Paris covering best practices on how to successfully navigate through the various types of migrations (protocol migrations, frontend migrations, website migration, cms migration, etc.) from an SEO perspective - mainly focussing on all things technical SEO.
Migration Best Practices - SMX West 2019Bastian Grimm
My talk from SMX West 2019 in San Jose covering best practices on how to successfully navigate through the various types of migrations (protocol migrations, frontend migrations, website migration, cms migration, etc.) from an SEO perspective - mainly focussing on all things technical SEO.
Migration Best Practices - SEOkomm 2018Bastian Grimm
My talk from SEOkomm 2018 in Salzburg covering best practices on how to successfully naviate through the various types of migrations (protocal migrations, frontend migrations, etc.) from an SEO perspective - mainly focussing on all things tech.
Technical SEO vs. User Experience - Bastian Grimm, Peak Ace AGBastian Grimm
My kick-off talk for a webinar titled "Technical SEO vs. UI/UX" which featured a panel of speakers discussing if and how SEO should work (more closely) together with UX. Enjoy!
Migration Best Practices - SMX London 2018Bastian Grimm
My talk from SMX 2018 in London covering best practices on how to successfully naviate through the various types of migrations (protocal migrations, frontend migrations, etc.) from an SEO perspective - mainly focussing on all things tech.
The need for Speed: Advanced #webperf - SEOday 2018Bastian Grimm
My deck on #webperf from SEOday 2018 in Cologne. Especially in a mobile-first world, fast loading websites are of outmost importance. Also, Google has been very vocal about anything web performance in the last years and is pushing hard to innovate repeatedly. But performance is so much more! User satisfaction should be the main goal because expectations are clear: You’ve got two seconds maximum to deliver, so make it count. In this deck I will be walking you through various advanced topics around web performance optimisation going way beyond Accelerated Mobile Pages (and other short-term solutions) to make any website really, really fast.
OK Google, Whats next? - OMT Wiesbaden 2018Bastian Grimm
My talk from Online Marketing Tag (OMT) Wiesbaden 2018 covering the ever-changing landscape of search and some of the stuff that I think will have a significant impact on SEO in the very near future!
Migration Best Practices - Search Y 2019, ParisBastian Grimm
My talk from SEARCHY 2019 in Paris covering best practices on how to successfully navigate through the various types of migrations (protocol migrations, frontend migrations, website migration, cms migration, etc.) from an SEO perspective - mainly focussing on all things technical SEO.
Migration Best Practices - SMX West 2019Bastian Grimm
My talk from SMX West 2019 in San Jose covering best practices on how to successfully navigate through the various types of migrations (protocol migrations, frontend migrations, website migration, cms migration, etc.) from an SEO perspective - mainly focussing on all things technical SEO.
Migration Best Practices - SEOkomm 2018Bastian Grimm
My talk from SEOkomm 2018 in Salzburg covering best practices on how to successfully naviate through the various types of migrations (protocal migrations, frontend migrations, etc.) from an SEO perspective - mainly focussing on all things tech.
Technical SEO vs. User Experience - Bastian Grimm, Peak Ace AGBastian Grimm
My kick-off talk for a webinar titled "Technical SEO vs. UI/UX" which featured a panel of speakers discussing if and how SEO should work (more closely) together with UX. Enjoy!
Migration Best Practices - SMX London 2018Bastian Grimm
My talk from SMX 2018 in London covering best practices on how to successfully naviate through the various types of migrations (protocal migrations, frontend migrations, etc.) from an SEO perspective - mainly focussing on all things tech.
The need for Speed: Advanced #webperf - SEOday 2018Bastian Grimm
My deck on #webperf from SEOday 2018 in Cologne. Especially in a mobile-first world, fast loading websites are of outmost importance. Also, Google has been very vocal about anything web performance in the last years and is pushing hard to innovate repeatedly. But performance is so much more! User satisfaction should be the main goal because expectations are clear: You’ve got two seconds maximum to deliver, so make it count. In this deck I will be walking you through various advanced topics around web performance optimisation going way beyond Accelerated Mobile Pages (and other short-term solutions) to make any website really, really fast.
Super speed around the globe - SearchLeeds 2018Bastian Grimm
My talk covering some of the very latest in web performance optimisation (paint timings, critical rendering path, custom web fonts, etc.) for technical marketers & SEOs from SearchLeeds 2018.
Welcome to a new reality - DeepCrawl Webinar 2018Bastian Grimm
My webinar with DeepCrawl talking about mobile-friendliness, assessing keyword targeting on mobile, finding content inconsistencies across devices and much, much more!
Migration Best-Practices: Successfully re-launching your website - SMX New Yo...Bastian Grimm
My talk from SMX 2017 in New York covering best practices on how to successfully naviate through the various types of migrations (protocal migrations, frontend migrations, etc.) from an SEO perspective.
An SEO's Guide to Website Migrations | Faye Watt | BrightonSEO's Advanced Tec...Faye Watt
In this presentation, Faye will take you through the necessary steps you need to take to ensure a successful website migration, how to avoid the loss of organic traffic and search visibility, and why migrations often fail.
This presentation includes:
How to redirect a domain
How to set up a local server
A migration timeline template
A migration checklist
The slides of my talk at the seokomm in Salzburg, Austria (November 2015).
It covers the basics of web crawling, with focus of search engine bots. It can be settled in the SEO space aswell as in the general webmaster world. Besides quotes of important influencers of Bing, Yandex and Google, it gives actionable advices on how you can influence the Crawl Budget allocation of search engine spiders.
In context of ajax / javascript crawling there is a minor excursion into the world of angularjs.
The talk closes on some insights on how we wrote our own CMS in order to cover all the SEO needs we are facing (multilanguage, multi-template, caching, if-modified-since, etc.).
If you like this talk, please follow me on twitter:
http://twitter.com/jhmjacob
And to miss to sign up for the Free Account of OnPage.org:
http://onpa.ge/V141p
Last but not least -> visit the seokomm next year if you are around - its worth it :)
http://www.seokomm.at/
SearchLove London 2016 | Dom Woodman | How to Get Insight From Your LogsDistilled
In the SEO industry, we obsess on everything Google says, from John Mueller dropping a hint in a Webmaster Hangout, to the ranking data we spend £1000s to gather. Yet we ignore the data Google throws at us every day, the crawling data. For the longest time, site crawls, traffic data, and rankings have been the pillars of SEO data gathering. Log files should join them as something everyone is doing. We'll go through how to get everything set-up, look at some of the tools to make it easy and repeatable and go through the kinds of analysis you can do to get insights from the data.
Jamie Alberico — How to Leverage Insights from Your Site’s Server Logs | 5 Ho...Semrush
These slides were presented at the SEMrush webinar "How to Leverage Insights from Your Site’s Server Logs | 5 Hours of Technical SEO". Video replay and transcript are available at https://www.semrush.com/webinars/how-to-leverage-insights-from-your-site-s-server-logs-or-5-hours-of-technical-seo/
Technical SEO: Crawl Space Management - SEOZone Istanbul 2014Bastian Grimm
My talk at #SEOZone 2014 in Istanbul covering various aspects of crawl space optimization such as crawler control & indexation strategies as well as site speed.
SEO for Large/Enterprise Websites - Data & Tech SideDominic Woodman
There is a lot to cover about SEO for large websites/enterprise.
In this talk we'll cover primarily the data analysis and the technical SEO side of things. In future presentations we'll look at more.
Super speed around the globe - SearchLeeds 2018Bastian Grimm
My talk covering some of the very latest in web performance optimisation (paint timings, critical rendering path, custom web fonts, etc.) for technical marketers & SEOs from SearchLeeds 2018.
Welcome to a new reality - DeepCrawl Webinar 2018Bastian Grimm
My webinar with DeepCrawl talking about mobile-friendliness, assessing keyword targeting on mobile, finding content inconsistencies across devices and much, much more!
Migration Best-Practices: Successfully re-launching your website - SMX New Yo...Bastian Grimm
My talk from SMX 2017 in New York covering best practices on how to successfully naviate through the various types of migrations (protocal migrations, frontend migrations, etc.) from an SEO perspective.
An SEO's Guide to Website Migrations | Faye Watt | BrightonSEO's Advanced Tec...Faye Watt
In this presentation, Faye will take you through the necessary steps you need to take to ensure a successful website migration, how to avoid the loss of organic traffic and search visibility, and why migrations often fail.
This presentation includes:
How to redirect a domain
How to set up a local server
A migration timeline template
A migration checklist
The slides of my talk at the seokomm in Salzburg, Austria (November 2015).
It covers the basics of web crawling, with focus of search engine bots. It can be settled in the SEO space aswell as in the general webmaster world. Besides quotes of important influencers of Bing, Yandex and Google, it gives actionable advices on how you can influence the Crawl Budget allocation of search engine spiders.
In context of ajax / javascript crawling there is a minor excursion into the world of angularjs.
The talk closes on some insights on how we wrote our own CMS in order to cover all the SEO needs we are facing (multilanguage, multi-template, caching, if-modified-since, etc.).
If you like this talk, please follow me on twitter:
http://twitter.com/jhmjacob
And to miss to sign up for the Free Account of OnPage.org:
http://onpa.ge/V141p
Last but not least -> visit the seokomm next year if you are around - its worth it :)
http://www.seokomm.at/
SearchLove London 2016 | Dom Woodman | How to Get Insight From Your LogsDistilled
In the SEO industry, we obsess on everything Google says, from John Mueller dropping a hint in a Webmaster Hangout, to the ranking data we spend £1000s to gather. Yet we ignore the data Google throws at us every day, the crawling data. For the longest time, site crawls, traffic data, and rankings have been the pillars of SEO data gathering. Log files should join them as something everyone is doing. We'll go through how to get everything set-up, look at some of the tools to make it easy and repeatable and go through the kinds of analysis you can do to get insights from the data.
Jamie Alberico — How to Leverage Insights from Your Site’s Server Logs | 5 Ho...Semrush
These slides were presented at the SEMrush webinar "How to Leverage Insights from Your Site’s Server Logs | 5 Hours of Technical SEO". Video replay and transcript are available at https://www.semrush.com/webinars/how-to-leverage-insights-from-your-site-s-server-logs-or-5-hours-of-technical-seo/
Technical SEO: Crawl Space Management - SEOZone Istanbul 2014Bastian Grimm
My talk at #SEOZone 2014 in Istanbul covering various aspects of crawl space optimization such as crawler control & indexation strategies as well as site speed.
SEO for Large/Enterprise Websites - Data & Tech SideDominic Woodman
There is a lot to cover about SEO for large websites/enterprise.
In this talk we'll cover primarily the data analysis and the technical SEO side of things. In future presentations we'll look at more.
On-Page SEO EXTREME - SEOZone Istanbul 2013Bastian Grimm
My presentation from #SEOZone Istanbul 2013 covering advanced On-Page SEO optimization aspects such as crawl-ability, semantics, duplicate content issues as well as performance optimization stragies.
This is the slide deck on how to perform log analysis with BigQuery. The companion guide is here which has most of this information in written format. https://www.distilled.net/resources/guide-to-log-analysis-with-big-query/
SearchLove Boston 2017 | Dom Woodman | How to Get Insight From Your LogsDistilled
In the SEO industry, we obsess on everything Google says, from John Mueller dropping a hint in a Webmaster Hangout, to the ranking data we spend $1000s to gather. Yet we ignore the data Google throws at us every day, the crawling data. For the longest time, site crawls, traffic data, and rankings have been the pillars of SEO data gathering. Log files should join them as something everyone is doing. We'll go through how to get everything set-up, look at some of the tools to make it easy and repeatable and go through the kinds of analysis you can do to get insights from the data.
Learn advanced SEO tactics and strategies in this second installment of my Demand Quest course. Topics include local SEO, link building, and international SEO.
Web scraping tools are software developed specifically to simplify the process of extracting data from websites. Data mining is a rather useful and commonly used process, but it can also easily turn into a complicated and messy activity and take a lot of time and effort.
Read This FirstAnnielytics.com@AnnieCushingNOTES ABOUT THIS WORKBO.docxsodhi3
Read This FirstAnnielytics.com@AnnieCushingNOTES ABOUT THIS WORKBOOKPLEASE DO NOT REQUEST ACCESS TO THIS GDOC. MY COPY IS ALREADY IN YOUR GDOCS AS SOON AS YOU OPEN IT, WHICH IS WHY BOTH GOOGLE DOCS START WITH MY NAME - SO YOU KNOW WHICH IS MINE (WHICH WILL BE UPDATED REGULARLY). IF YOU WANT YOUR OWN COPY TO EDIT, LOG IN AND CHOOSE FILE > MAKE A COPY. I WON'T BE ABLE TO REPLY TO FURTHER REQUESTS TO SHARE THE DOC B/C IT'S GETTING TOO TIME-CONSUMING. THANK YOU FOR YOUR UNDERSTANDING. This audit list, though thorough, is not exhaustive. If you'd like to suggest a checkpoint or suggest a correction, please email me at [email protected]Want more? You can now buy my site self-guided audit template here. 90+ explainer graphics, boilerplate text explaining all 18 sections, 140 pages, 156 checkpoints.And my self-guided analytics audit: 146 pages, 45 checkpoints, 6 sections, 84 explainer graphicsIf a cell has an orange triangle in the upper-right corner, hover over it to see a comment.You can sort or order tables by choosing options from the drop-down menu in each heading cell. I tried to make each checkpoint a stand-alone element, so they could be reordered. I also have a Tools for Marketers Google Doc with an extensive list of tools, which you can use as a complement to this doc.http://bit.ly/tools-galoreLearn to create sexy, dynamic dashboards using the Google Analytics API.See Spanish translationRead more about the fab guy who translated this doc!http://bit.ly/tools-galore
Initial Data PullsAnnielytics.com@AnnieCushingWant more? You can now buy my site self-guided audit template here. 90+ explainer graphics, boilerplate text explaining all 18 sections, 140 pages, 156 checkpoints.And my self-guided analytics audit: 146 pages, 45 checkpoints, 6 sections, 84 explainer graphics.Learn to create sexy, dynamic dashboards using the Google Analytics API.CheckToolScreaming FrogPotential custom filters to include:
snippet of GATC to make sure they're tracking every page
YouTube embed snippet to see what videos they're embedding
</iframe> to see how they're using iframes
pageTracker._trackPageview to see if they have remnants of GA's traditional tracking codeGWTAll errorsCrawl TestMozbar > SEOmoz Tools (wrench icon) > Crawl Test or
http://pro.seomoz.org/tools/crawl-testFind out keywords site is ranking forSEMRush or Keyword Spy
IndexingAnnielytics.com@AnnieCushingWant more? You can now buy my site self-guided audit template here. 90+ explainer graphics, boilerplate text explaining all 18 sections, 140 pages, 156 checkpoints.And my self-guided analytics audit: 146 pages, 45 checkpoints, 6 sections, 84 explainer graphics.Learn to create sexy, dynamic dashboards using the Google Analytics API.CheckInstructionsObservationsRecommendationPriority (1–3)How many pages are indexed by Google?Search for [site:yoursite.com] in Google
GWT: Google Index > Index Status2820?When you do a site: search, does the homepage come up first?YesDoes the site have index bloat?Check ra ...
My talk [DE] from SEOday 2020 in Cologne titled: "Surprise, Surprise - Fünf Dinge, die du über technische Suchmaschinenoptimierung bisher nicht wusstest". Enjoy!
Meine Präsentation von der SEOkomm 2018 mit vielen Tipps und Empfehlungen rund um das Thema Logfile Analyse - inkl. Übersicht Marktübersicht der Tools, vielen Logfile Auditing Workflows, etc.
Web Performance Madness - brightonSEO 2018Bastian Grimm
My talk from brightonSEO 2018 covering various web performance strategies, this time mainly focussing on critical rendering path, various image optimisation strategies as well as how to handle custom web fonts.
How fast is fast enough - SMX West 2018Bastian Grimm
My talk on web performance optimizations (including critical rendering path, better measurements, paint timings, custom font optimization, chrome performance observer and much more) from SMX West 2018 in San Jose.
Meine Session von der OMX 2017 in Salzburg rund um das Thema "Digitale Assistenten", inklusive vieler Tipps zur Optimierung von Voice Search Systemen, primär für den Google Assistant bzw. Google Home.
Welcome to a New Reality - SEO goes Mobile First in 2017Bastian Grimm
Crawling, Indexing & Ranking in the age of a mobile first world. Meine Präsentation von der OMT 2017 in Wiesbaden rund um das Thema "Mobile First Indexing" inkl. Performance Optimierungen, Responsive Design & vielem mehr.
Welcome to a New Reality - SEO goes Mobile First in 2017Bastian Grimm
Crawling, Indexing & Ranking in the age of a mobile first world. Meine Präsentation von der OMT 2017 in Wiesbaden rund um das Thema "Mobile First Indexing" inkl. Performance Optimierungen, Responsive Design & vielem mehr.
HTTPs Migration How To - SMX München 2017Bastian Grimm
Meine Präsentation der SMX 2017 in München (Tag 2) rund um das Thema "HTTPs Site Migration" inkl. ausführlicher Checkliste, diversen Tools für Monitoring, Testing, etc. sowie vielem mehr!
Keyword Strategie: Do's & Don'ts bei der Keyword Recherche - SMX München 2017Bastian Grimm
Meine Präsentation der SMX 2017 in München rund um das Thema Keyword Architektur, inkl. Searchers Intent, Desktop vs. Mobile Use-Cases, lokalisierten SERPs sowie diversen Tools.
International Site Speed Tweaks - ISS 2017 BarcelonaBastian Grimm
Talking international site speed optimization at International Search Summit 2017 in Barcelona, London as well as in Munich covering a broad variety of performance optimization strategies.
Technical SEO: 2017 Edition - SEO & Love Verona 2017Bastian Grimm
My presentation from SEO&Love in Verona covering the latest in technical SEO for 2017, including performance optimization strategies, AMP, rich snippets & cards as well as mobile first indexing.
APNIC Foundation, presented by Ellisha Heppner at the PNG DNS Forum 2024APNIC
Ellisha Heppner, Grant Management Lead, presented an update on APNIC Foundation to the PNG DNS Forum held from 6 to 10 May, 2024 in Port Moresby, Papua New Guinea.
This 7-second Brain Wave Ritual Attracts Money To You.!nirahealhty
Discover the power of a simple 7-second brain wave ritual that can attract wealth and abundance into your life. By tapping into specific brain frequencies, this technique helps you manifest financial success effortlessly. Ready to transform your financial future? Try this powerful ritual and start attracting money today!
1.Wireless Communication System_Wireless communication is a broad term that i...JeyaPerumal1
Wireless communication involves the transmission of information over a distance without the help of wires, cables or any other forms of electrical conductors.
Wireless communication is a broad term that incorporates all procedures and forms of connecting and communicating between two or more devices using a wireless signal through wireless communication technologies and devices.
Features of Wireless Communication
The evolution of wireless technology has brought many advancements with its effective features.
The transmitted distance can be anywhere between a few meters (for example, a television's remote control) and thousands of kilometers (for example, radio communication).
Wireless communication can be used for cellular telephony, wireless access to the internet, wireless home networking, and so on.
# Internet Security: Safeguarding Your Digital World
In the contemporary digital age, the internet is a cornerstone of our daily lives. It connects us to vast amounts of information, provides platforms for communication, enables commerce, and offers endless entertainment. However, with these conveniences come significant security challenges. Internet security is essential to protect our digital identities, sensitive data, and overall online experience. This comprehensive guide explores the multifaceted world of internet security, providing insights into its importance, common threats, and effective strategies to safeguard your digital world.
## Understanding Internet Security
Internet security encompasses the measures and protocols used to protect information, devices, and networks from unauthorized access, attacks, and damage. It involves a wide range of practices designed to safeguard data confidentiality, integrity, and availability. Effective internet security is crucial for individuals, businesses, and governments alike, as cyber threats continue to evolve in complexity and scale.
### Key Components of Internet Security
1. **Confidentiality**: Ensuring that information is accessible only to those authorized to access it.
2. **Integrity**: Protecting information from being altered or tampered with by unauthorized parties.
3. **Availability**: Ensuring that authorized users have reliable access to information and resources when needed.
## Common Internet Security Threats
Cyber threats are numerous and constantly evolving. Understanding these threats is the first step in protecting against them. Some of the most common internet security threats include:
### Malware
Malware, or malicious software, is designed to harm, exploit, or otherwise compromise a device, network, or service. Common types of malware include:
- **Viruses**: Programs that attach themselves to legitimate software and replicate, spreading to other programs and files.
- **Worms**: Standalone malware that replicates itself to spread to other computers.
- **Trojan Horses**: Malicious software disguised as legitimate software.
- **Ransomware**: Malware that encrypts a user's files and demands a ransom for the decryption key.
- **Spyware**: Software that secretly monitors and collects user information.
### Phishing
Phishing is a social engineering attack that aims to steal sensitive information such as usernames, passwords, and credit card details. Attackers often masquerade as trusted entities in email or other communication channels, tricking victims into providing their information.
### Man-in-the-Middle (MitM) Attacks
MitM attacks occur when an attacker intercepts and potentially alters communication between two parties without their knowledge. This can lead to the unauthorized acquisition of sensitive information.
### Denial-of-Service (DoS) and Distributed Denial-of-Service (DDoS) Attacks
Multi-cluster Kubernetes Networking- Patterns, Projects and GuidelinesSanjeev Rampal
Talk presented at Kubernetes Community Day, New York, May 2024.
Technical summary of Multi-Cluster Kubernetes Networking architectures with focus on 4 key topics.
1) Key patterns for Multi-cluster architectures
2) Architectural comparison of several OSS/ CNCF projects to address these patterns
3) Evolution trends for the APIs of these projects
4) Some design recommendations & guidelines for adopting/ deploying these solutions.
Bridging the Digital Gap Brad Spiegel Macon, GA Initiative.pptxBrad Spiegel Macon GA
Brad Spiegel Macon GA’s journey exemplifies the profound impact that one individual can have on their community. Through his unwavering dedication to digital inclusion, he’s not only bridging the gap in Macon but also setting an example for others to follow.
Advanced data-driven technical SEO - SMX London 2019
1. @basgr from @peakaceag1
Bastian Grimm, Peak Ace AG | @basgr
Merging your logfiles, GA, GSC & web crawl data
for better SEO insights
Advanced data-driven technical SEO
3. pa.ag@peakaceag @basgr from @peakaceag3
I am a big fan of the various crawling tools, but…
It’s only the access log files that demonstrate
how a search engine’s crawler is behaving on
your site; all crawling tools are simply trying to
simulate their behaviour!
4. @basgr from @peakaceag4
You need to see which pages are being prioritised by the search
engines and should therefore be considered the most important
1. Understand crawl priorities
5. @basgr from @peakaceag5
Google may reduce its crawling behaviour/frequency & eventually rank
you lower if you are constantly providing a large amount of errors
2. Prevent reduced crawling
6. @basgr from @peakaceag6
It’s essential to identify any crawl shortcomings
(such as hierarchy or internal link structure)
with potential site-wide implications
3. Understand global issues
7. @basgr from @peakaceag7
You need to ensure that Google crawls everything important:
primarily ranking, relevant content, but also fresh & older
items
4. Ensure proper crawling
8. @basgr from @peakaceag8
It’s important to ensure that any gained link equity will
always be passed using proper links and/or redirects
5. Ensure proper linking
10. @basgr from @peakaceag10
…depending on your webserver (Apache, nginx, IIS, etc.), caching
and its configuration. Make sure to understand your setup first!
Content & structure can vary…
11. pa.ag@peakaceag @basgr from @peakaceag11
What does a log file usually look like?
Server IP/host name1
Timestamp (date & time)2
Method (GET/POST/HEAD)3
Request URL4
HTTP status code5
Size in bytes6
Referrer7
User-agent8
188.65.114.xxx [21/May/2019:02:00:00 -0100]
/resources/whitepapers/seo-whitepaper/
HTTP/1.1" 200 512 "http://www.wikipedia.org/"
"Mozilla/5.0 (compatible; Googlebot/2.1;
+http://www.google.com/bot.html)"
"GET
12. @basgr from @peakaceag12
Log file data can be quite overwhelming because you can do so
many different things; make sure you’ve got
your questions prepared!
You need to ask the right questions!
13. pa.ag@peakaceag @basgr from @peakaceag13
Log file data can be different e.g. to Google Analytics data
While log files are direct, server-side pieces of information, Google Analytics uses client-
side code. As the data sets are coming from two different sources, they can be
different!
The configuration within Google Analytics also leads to data differences when
compared to the log files, i.e. filters!
15. @basgr from @peakaceag15
We only care about crawlers such as Google and Bing; no need for any
user data (operating system, browser, phone number, usernames, etc.)
1. Personal information in logs?
16. @basgr from @peakaceag16
If you are running a cache server and/or a CDN which
creates logs elsewhere, we will also need these logs
2. Separate multi-location logs?
19. pa.ag@peakaceag @basgr from @peakaceag19
Do-it-yourself solution based on Excel
You’d have to manually build filtering, cross-references, etc. – it just doesn’t scale!
20. pa.ag@peakaceag @basgr from @peakaceag20
Screaming Frog Log File Analyser
Beginners’ level, desktop-based log file auditing with pre-defined reports.
21. @basgr from @peakaceag21
No sharing capabilities, log files need to be manually up/downloaded,
which is usually problematic for larger files, etc.
Desktop solutions are limited
22. pa.ag@peakaceag @basgr from @peakaceag22
Splunk or Sumo Logic: proprietary, paid software solutions
Enterprise tools such as Splunk usually come with a hefty (volume-based) price tag.
In all fairness though: these solutions offer features way beyond log file monitoring!
Image sources: https://pa.ag/2srgTZu (splunk) & https://pa.ag/2JcuiLt (sumologic)
23. pa.ag@peakaceag @basgr from @peakaceag23
The Elastic Stack (ELK): Elasticsearch, Logstash & Kibana
Elasticsearch: search & analytics engine, Logstash: server-side data processing
pipeline, Kibana: data visualisation (charts, graphs, etc.) – all open source.
Image source: https://pa.ag/2JbFUhP
24. pa.ag@peakaceag @basgr from @peakaceag24
Other SaaS solutions: logrunner.io, logz.io (ELK) & Loggly
Especially logrunner.io, which has a strong focus on SEO-based auditing (dashboards
etc.).
25. pa.ag@peakaceag @basgr from @peakaceag25
crawlOPTIMIZER: SaaS Logfile Auditing, made in Vienna
BRPs (Business Relevant Pages) with dedicated evaluations of these as top USP.
26. @basgr from @peakaceag26
No messing around with exports, up/downloads, easy sharing
capabilities and the ability to deal with massive volumes, etc.
The beauty of SaaS: almost real time
27. @basgr from @peakaceag27
For an easy start: trend monitoring (over time) & gathering insights
Let’s have a look at some data
28. pa.ag@peakaceag @basgr from @peakaceag28
Most obvious approach: spotting anomalies vs. time frame
Tip: this is why it makes a lot of sense to check your log files regularly (e.g. daily).
This looks unusual; take it
as a starting point for
further investigation.
29. pa.ag@peakaceag @basgr from @peakaceag29
User crawling frequencies over time
Understanding patterns and irregularities can be very helpful - always look at the crawl
behaviour of individual users over time.
30. @basgr from @peakaceag30
Use log files to look for spam bots or scrapers to block!
What other ”bots“ access your site?
31. pa.ag@peakaceag @basgr from @peakaceag31
Not everyone is who they claim to be!
The easiest way to detect if Googlebot really is Googlebot: run a reverse DNS lookup.
Bingbot can also be verified via *.search.msn.com.
Source: https://pa.ag/2JqOk8d
32. pa.ag@peakaceag @basgr from @peakaceag32
What are the most crawled Googlebot pages?
Also, verify if they coincide with your domains’ most important ones.
Understand if these are really
your most valuable pages?
33. pa.ag@peakaceag @basgr from @peakaceag33
Breakdown of crawl requests & status codes per directory
You’d easily see if one of your main directories encountered crawling/response issues.
Tip: establish this on a regular basis to ensure continued performance of top directories.
36. pa.ag@peakaceag @basgr from @peakaceag36
Identify any kind of ”wrong“ redirect: 302/304/307/308
Action: change to 301 (except geo redirects); also watch out for redirect chains!
Investigate further to
see what’s in there
38. pa.ag@peakaceag @basgr from @peakaceag38
4xx client errors: too many are a sign of poor site health
Action: recover (200), redirect (301) or kill off entirely (410)
39. pa.ag@peakaceag @basgr from @peakaceag39
Googlebot can‘t login… (403: forbidden)
If it‘s linked, Google will try to crawl it – they are greedy!
40. pa.ag@peakaceag @basgr from @peakaceag40
5xx server errors: usually infrastructure-related
Action: watch closely and/or talk to IT (server availability, high load, etc.)
Check consistency; what
happens when re-trying?
42. pa.ag@peakaceag @basgr from @peakaceag42
Understanding the most/least crawled URLs and folders
Action: highly crawled pages/folders could be used e.g. for additional internal linking
(add link hubs), low crawled areas need to be linked more prominently.
Can be used for additional, internal linking (improve
discovery of other content)
Clearly weak, either irrelevant (remove) or requires
more attention
44. pa.ag@peakaceag @basgr from @peakaceag44
Investigate if (new) URLs have been crawled at all
Action: if relevant URLs haven’t been discovered/crawled at all, your internal linking is
probably too weak. Consider XML sitemaps, better/more prominent linking, etc.
If these are important URLs,
you might have a problem!
47. pa.ag@peakaceag @basgr from @peakaceag47
This is what the Google Webmaster Central blog says:
Source: https://pa.ag/2HhsYoz
Wasting server resources on pages […] will
drain crawl activity from pages that do actually
have value, which may cause a significant
delay in discovering great content on a site.
48. pa.ag@peakaceag @basgr from @peakaceag48
If you have ever had to deal with sites like these…
Properly dealing with >30,000,000 crawlable URLs (due to parameter usage) certainly
makes a difference in organic performance!
49. pa.ag@peakaceag @basgr from @peakaceag49
URL parameters cause most problems
(Combined) URL parameters often generate millions of unnecessary URLs, especially for
large domains, which Googlebot diligently crawls (once found).
50. pa.ag@peakaceag @basgr from @peakaceag50
URL parameter behaviour over time
Constantly be on the lookout for new parameters as well as significantly increased
crawling for known parameters.
51. pa.ag@peakaceag @basgr from @peakaceag51
A brief overview: #SMXInsights
01
No one-size-fits-all
solution
Log file size, quantity
& availability are all
decisive with regards
to tool selection.
02
Preparation is key
Concrete questions
help to generate
efficient analysis.
03
Crawl data only
Be precise with your
requests (to the IT
department) - you
just want to know
what the search
engines are doing!
04
Reverse DNS use
Not every crawler is
who they pretend to
be - do not "blindly"
trust in the user-agent
string
05
URL parameters
These are almost
always the biggest
problem
(combinations, order,
consistency) - audit
them first.
53. @basgr from @peakaceag53
I want: no IT involvement, unlimited scalability, flexible reporting, multiple
(API) data sources and ease of use!
There's got to be another way!
55. pa.ag@peakaceag @basgr from @peakaceag55
Say hello to the Peak Ace log file auditing stack
Log files are stored in Google Cloud Storage, processed in Dataprep, exported to BigQuery and
visualised in Data Studio via the BigQuery Connector.
Google Data Studio
Data
transmission
Display
dataImport
Google Dataprep Google BigQuery
1
Log files
GSC
API v3
GA
API v4
GA
GSC
2
3
65
Google Apps Script
DeepCrawl
API
4
86 7
61. pa.ag@peakaceag @basgr from @peakaceag61
#1 Log file data from web servers, CDN, cache, etc.
How often do bots actually crawl? What do they crawl and when?
Source: https://pa.ag/2zs9lcY
Goal: improve site architecture by
analysing real bot crawling data.
▪ Amount of crawls/requests by bot type
▪ Identification of crawling patterns
▪ Overview of errors
▪ 3xx
▪ 4xx
▪ 5xx
Log filesGoogle Cloud
Storage
Import as text files
(exclude IP addresses!)
64. pa.ag@peakaceag @basgr from @peakaceag64
#2 Google Analytics API
Enrich reports with traffic, engagement, behavioural and page speed data
Goal: compare crawling behaviour with user & loading
time data.
URL-based data on important engagement metrics:
▪ Sessions
▪ Users
▪ Bounce rate
▪ Session duration
▪ Avg. time on page
▪ Avg. server response time
▪ Avg. page load time
▪ …
Google Analytics
Reporting API v4
65. pa.ag@peakaceag @basgr from @peakaceag65
#3 Google Search Console API
Organic search performance data directly from Google
Goal: compare crawling behaviour with organic click
data & e.g. retrieve reported crawling errors.
Organic click data
▪ Clicks
▪ Impressions
▪ Device
▪ …
URL-based server response data
▪ Status code
Google Search
Console API v3
66. pa.ag@peakaceag @basgr from @peakaceag66
#4 DeepCrawl API
Website architecture, status codes, indexing directives, etc.
Goal: capture indexing directives, response codes and
more.
DeepCrawl
API
67. pa.ag@peakaceag @basgr from @peakaceag67
#5 Google Apps Scripts for GA, GSC & DeepCrawl
API access: capture multiple dimensions and metrics from GA, retrieve GSC crawl and
search analysis data and DeepCrawl crawl & analysis data
Source: https://pa.ag/2OWnjJa
Goal: send data (via/from the respective API) to BigQuery
and store the data there.
Google Apps Script
68. pa.ag@peakaceag @basgr from @peakaceag68
#6 Google Cloud Dataprep
Clean and process the data. Afterwards, combine these various sources with
several joins so that they‘re ready for visualisation.
Source: https://pa.ag/2Q6rEde
Goal: combine data from log files, GSC, GA & DeepCrawl
within/by using processing flows.
Dataprep: “Excel with super rocket fuel“
▪ Amazing RegEx support
▪ Select data, receive automated
proposals for processing
▪ Join data sources by e.g. full
inner/outer join, left/right outer join…
Google Apps Script
74. pa.ag@peakaceag @basgr from @peakaceag74
Log file auditing is not a project, but a process!
Integrate log file auditing into your regular
SEO workflow; one-off audits are good to
begin with, but they really become invaluable
if you combine them with web crawl data and
perform them on an on-going basis.
75. pa.ag@peakaceag @basgr from @peakaceag75
bg@pa.ag
Slides? No problem:
https://pa.ag/smxl19logs
You want our log file setup (for free)?
e-mail us > hi@pa.ag
Bastian Grimm
twitter.com/peakaceag
facebook.com/peakaceag
www.pa.ag
ALWAYS LOOKING FOR TALENT! CHECK OUT JOBS.PA.AG
WINNER