2. Who Am I?
1
2
3
4
Hi, my name is Chris!
I’ve worked in SEO for nearly 15 years
I have an unhealthy interest in breaking
things and making things do things they
aren’t supposed to
I’m a member of the Professional Services
team at Deepcrawl, working with some of
the biggest websites on earth, finding,
diagnosing and fixing issues from the really
really mundane to the really really weird
1
2
3
4
3. 60% of Small Businesses
close within 6 months of a data breach
Why should I be concerned about security?
😞
4. 60% of Small Businesses
close within 6 months of a data breach
As well as direct financial damage,
damage to reputation and customer confidence can be long term
Why should I be concerned about security?
👤
5. 60% of Small Businesses
close within 6 months of a data breach
As well as direct financial damage,
damage to reputation and customer confidence can be long term
You don’t have to be targeted
to be a victim of malicious activity, just vulnerable
Why should I be concerned about security?
🤷
♂️
7. I am not a security expert!
I’m just an SEO who is either cursed or blessed
with the ability to find these things.
This talk is about preventing issues where
possible, and learning how to find problems to
report to your Secops/Dev teams
Disclaimer:
11. Three Ways You Can Provide Security Benefits
Prevent risks
12. Three Ways You Can Provide Security Benefits
Prevent risks
Identify weaknesses
13. Three Ways You Can Provide Security Benefits
Prevent risks
Identify weaknesses
Identify Malicious Activity
both successful and attempted
14. Robots.txt
● Robots.txt is a great way of keeping Google out
of folders and files you don’t want it getting into
● But consider whether you want to announce their
existence to the whole world
15. Robots.txt
● Instead, consider using the X-Robots-Tag header
to prevent indexation and limit crawling if you don’t
want the urls known - or better yet, block non-
verified visits
● As an aside, if you allow UGC, consider what could
happen if a user is allowed to create a robots.txt slug
16. Google Alerts
● Set up an alert for ‘site:github.com “[your-website.com]”’
● Catch devs accidentally storing private
keys etc in public github repos
● Catch other nefarious actors who might
be targeting these domains with scripts/code
17. Google Alerts
● Keep an eye out on what shows up for an image
search for your brand - what can you see in the
background of office photos from news stories?
● This also applies to social media -
has your new starter taken a photo
of their pass?
18. Crawl Your Site As Google
● This will help you see if your site returns anything
weird or untoward when it thinks you are not a
“normal” user
● Don’t worry too much if the crawl crashes! Your
security team might already be one step ahead
19. Monitor your SERPs
● Wordpress sites in particular are susceptible to
compromise due to their off the shelf nature
● A famous hack, known as “The Pharma Hack”
(Recently overtaken by “The Japanese Keyword
Hack”) can serve spammy content to Google -
but not to users
20. Question Things That Look Weird
● Look into outliers - go down rabbitholes,
● and always think laterally about how or why
something has ended up a specific way
● Just because something says it’s Googlebot,
don’t believe it on face value
21. Question Things That Look Weird
● Look into outliers - go down rabbitholes, and
always think laterally about how or why
something has ended up a specific way
● Just because something says its Googlebot,
don’t believe it on face value
22. Search Console
● Search Console will straight up tell you if Google
believes your site has been compromised
● Keep an eye on all those subdomains that are no
longer used - a malicious actor can tank an entire
domain’s traffic by 90% via DMCA takedowns
● Make sure the owner inbox is monitored
23. Summary
● Get to know your site
○ How big is it?
○ What do your SERPs look like?
● Be vigilant of change - especially changes you
haven’t made
● Set up alerts
● Automate crawls
● Spend time in Search Console!
● Anything you really don’t want Google or users
to find should not be in your robots.txt
● Go down rabbitholes, ask questions, investigate
anomalies
24. Thanks for Coming.
Resources: https://linktr.ee/chrisspann
Chris Spann, Senior Technical SEO at Deepcrawl
@marqueetag
Editor's Notes
In our survey, we asked them. Understanding the importance of your website and the real business impact it can provide is only half the battle. When it came time to execute, we found that many marketing leaders were struggling.
Here’s why:
People: 40% said that they did not have the right people (or enough people) on their teams who could carry out the work necessary to succeed in website health and organic search.
Delays in implementing website changes: 39% said there were significant delays when it came to implementing changes on their sites that would benefit SEO.
Poor collaboration across teams: 23% said that there wasn’t the necessary level of collaboration happening across teams — and 23% also said that their tech/IT/development teams did not prioritize organic search — likely leading to the delays in implementation mentioned earlier!
A lack of inclusion in strategy: 29%, meanwhile, said that improving their websites’ health was not seen as part of their organizations’ strategic priorities — despite the fact they themselves understood the impact that website performance and organic search could have on larger goals such as revenue and awareness-building.
A lack of leadership buy-in: 23% also called out leadership specifically as creating blockers when it came to getting the resources they needed to implement website health
In our survey, we asked them. Understanding the importance of your website and the real business impact it can provide is only half the battle. When it came time to execute, we found that many marketing leaders were struggling.
Here’s why:
People: 40% said that they did not have the right people (or enough people) on their teams who could carry out the work necessary to succeed in website health and organic search.
Delays in implementing website changes: 39% said there were significant delays when it came to implementing changes on their sites that would benefit SEO.
Poor collaboration across teams: 23% said that there wasn’t the necessary level of collaboration happening across teams — and 23% also said that their tech/IT/development teams did not prioritize organic search — likely leading to the delays in implementation mentioned earlier!
A lack of inclusion in strategy: 29%, meanwhile, said that improving their websites’ health was not seen as part of their organizations’ strategic priorities — despite the fact they themselves understood the impact that website performance and organic search could have on larger goals such as revenue and awareness-building.
A lack of leadership buy-in: 23% also called out leadership specifically as creating blockers when it came to getting the resources they needed to implement website health
In our survey, we asked them. Understanding the importance of your website and the real business impact it can provide is only half the battle. When it came time to execute, we found that many marketing leaders were struggling.
Here’s why:
People: 40% said that they did not have the right people (or enough people) on their teams who could carry out the work necessary to succeed in website health and organic search.
Delays in implementing website changes: 39% said there were significant delays when it came to implementing changes on their sites that would benefit SEO.
Poor collaboration across teams: 23% said that there wasn’t the necessary level of collaboration happening across teams — and 23% also said that their tech/IT/development teams did not prioritize organic search — likely leading to the delays in implementation mentioned earlier!
A lack of inclusion in strategy: 29%, meanwhile, said that improving their websites’ health was not seen as part of their organizations’ strategic priorities — despite the fact they themselves understood the impact that website performance and organic search could have on larger goals such as revenue and awareness-building.
A lack of leadership buy-in: 23% also called out leadership specifically as creating blockers when it came to getting the resources they needed to implement website health
Change to slide 6 style
Change to slide 6 style
Change to slide 6 style
Animate these
We have access to Search Console to see what Google sees
We have log files, which is a huge haystack that can be full of needles
We have search analytics to show us what users are doing
We have backlink tools to show us the websites that link to us
We have site crawlers that find weird things we didn’t know were there all the time
But most importantly we have search results, which shows us exactly what other people see when they search for our businesses
We also often control what parts of a website Search engines (and users) can or can’t find
How to make this slide look nicer?
How to make this slide look nicer?
How to make this slide look nicer?
How to make this slide look nicer?
How to make this slide look nicer?
Worst case scenario: the user could initiate a meta refresh to an externally hosted robots.txt (google will follow redirects) which contains a Disallow: / rule, which stops google crawling ANYTHING
Your website or api endpoint etc
How to make this slide look nicer?
How to make this slide look nicer?
Remember your SERPs are a great example of how Googlebot sees your site
This is a graph showing Googlebot activity on a clients site
What has caused that big spike?Googlebot is the most used UA in DDOS attacks, because most sites will just let Googlebot straight in
Googlebot UA hitting possible locations of a file with known weaknesses - except the IP is not a googlebot IP and it is very weird that google would be hyper targeting possible locations of eval-stdin.php?Because if they then find one, they can fire a POST request at that url with custom php in it
Subdomains point to an IP
If your ownership of that IP expires, a third party can then buy usage of that IP and host dodgy stuff on there
Mention recent finding that the pirate update can tank a site by 90% - if someone can upload copyrighted material to your site, they can DMCA you
Set up a domain level property and look at googlebot activity across ALL subdomains! Pdf hack is very common