Infinite Loops Dirty Architecture And Too Many Indexed URLs

Dawn Anderson MSc DigM
Dawn Anderson MSc DigMManaging Director, Founder, Consultant & Lecturer at Bertey
INFINITE LOOPS
& crawl rank
DIRTY ARCHITECTURE
Dawn Anderson
CAME
INDUSTRY
VIA A DIFFERENT ROUTE
THIS
to
I decided to add an
additional dimension
to the site
TO ‘EXPLODE’ NATURAL
SEARCH TRAFFIC
1.5 Million
URLs
Crawl Rate
Going Down
Indexation
Levels
Going Up
GOOGLE
Only crawling
0.1%Of our pages per
day
Infinite Loops Dirty Architecture And Too Many Indexed URLs
Infinite Loop Definition:
An infinite loop is a sequence of
instructions in a computer program which
loops endlessly, either due to the loop
having no terminating condition, having
one that can never be met, or one that
causes the loop to start over. ..
PENGUIN & PANDA
updates came along
TOO MANY URLS
=SEO DEATH
‘WE’RE ALL ‘DOOMED’’
Budget
CRAWL
Roughly proportionate to PageRank
Pages with a lot of links get crawled more
Still applies in current search landscape
Rank
CRAWL
A ranking metric for ‘no’ to ‘low’ PageRank
pages??
Pages crawled more often rank higher
Get ‘low’ to ‘no’ PageRank pages crawled more
than competitors = YOU WIN
CRAWL
OPTIMISATION
Googlebot goes
AND KEEP
WATCHING
FIND OUT WHERE
CHECK & MONITOR
for over-indexation
500 Page
Website
50,00 URLs in
Google
YOU MAY HAVE DODGY CODE
Shoes.sitemap.xml
Dresses.sitemap.xml
tshirts.sitemap.xml
Check THOROUGHLY, Name
& Categorise XML Sitemaps
yoursite.sitemap.xml
DON’T BE AFRAID
of hard 404’s
Use 410’s where
you can
Giraffe
AVOID
soft 404’s
ENSURE THAT
Dynamic variables / parameters
are checked for validation
Don’t render to just any old
thing with a ‘200 OK’ response
code or return a soft 404
HOW WILL YOU KNOW IF
THERE’S A PROBLEM?
You won’t
AVOID A ‘JUMBLE SALE’
BUT
Use Robots.txt,
nofollows, sitemaps,
nav paths & cross
module
internal
linking
‘Herd’ Googlebot
Get Those Low Level
Pages Crawled - Often
Whichever way
you can
Pass equity to
Siblings as
Well as children
Visit the internal links section on GWT
Most Important Page 1
Most Important Page 2
Most Important Page 3
IS THIS YOUR
BLOG?? HOPE
NOT
CANONICALISATIONIn web search and search engine optimization (SEO), URL
canonicalization deals with web content that has more than one
possible URL. Having multiple URLs for the same web content
can cause problems for search engines - specifically in
determining which URL should be shown in search results.[2]
Example:
•http://wikipedia.com
•http://www.wikipedia.com
•http://www.wikipedia.com/
•http://www.wikipedia.com/?source=asdf
All of these URLs point to the homepage of Wikipedia,
but a search engine will only consider one of them to
be the canonical form of the URL.(source - Wikipedia)
Deal Well With
Near & near
duplicate content
Via
canonicalization,
301’s & Content
Build Out
STOP LYING & ‘GET
FRESH’
Genuine ‘last
modified dates’
are ALL important
- FORGET PRIORITY
"It's not that Google will
penalize you, it's the
opportunity cost for dirty
architecture based on a finite
crawl budget" (A.J.Kohn)
(BLIND FIVE YEAR OLD)
REMEMBER THIS
Me
@dawnieando
1 of 26

More Related Content

What's hot(20)

Crawl Budget - Some Insights & Ideas @ seokomm 2015Crawl Budget - Some Insights & Ideas @ seokomm 2015
Crawl Budget - Some Insights & Ideas @ seokomm 2015
Jan Hendrik Merlin Jacob20.1K views
Pubcon florida 2018 logs dont lie dawn andersonPubcon florida 2018 logs dont lie dawn anderson
Pubcon florida 2018 logs dont lie dawn anderson
Dawn Anderson MSc DigM2.6K views
SEO and The Mobile-First Paradigm ShiftSEO and The Mobile-First Paradigm Shift
SEO and The Mobile-First Paradigm Shift
Dawn Anderson MSc DigM763 views
Colloquim Report - Rotto Link Web CrawlerColloquim Report - Rotto Link Web Crawler
Colloquim Report - Rotto Link Web Crawler
Akshay Pratap Singh1.8K views

Similar to Infinite Loops Dirty Architecture And Too Many Indexed URLs(20)

More from Dawn Anderson MSc DigM(20)

Google BERT - SMX London 2020 Virtual ConferenceGoogle BERT - SMX London 2020 Virtual Conference
Google BERT - SMX London 2020 Virtual Conference
Dawn Anderson MSc DigM578 views
Google BERT - What SEOs and Marketers Need to KnowGoogle BERT - What SEOs and Marketers Need to Know
Google BERT - What SEOs and Marketers Need to Know
Dawn Anderson MSc DigM1.8K views
SEO in a Mobile First WorldSEO in a Mobile First World
SEO in a Mobile First World
Dawn Anderson MSc DigM749 views
Modern Ecommerce SEOModern Ecommerce SEO
Modern Ecommerce SEO
Dawn Anderson MSc DigM571 views

Recently uploaded(20)

Panel - Digital Marketing Trends - Martin Weinberg, MarketGenesisPanel - Digital Marketing Trends - Martin Weinberg, MarketGenesis
Panel - Digital Marketing Trends - Martin Weinberg, MarketGenesis
DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions33 views
Marketing Automation Master Class - Yemi Oluseun, The Change HiveMarketing Automation Master Class - Yemi Oluseun, The Change Hive
Marketing Automation Master Class - Yemi Oluseun, The Change Hive
DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions47 views
How To Build Digital Marketing Strategies - Kuralay Assainova, Liana Technolo...How To Build Digital Marketing Strategies - Kuralay Assainova, Liana Technolo...
How To Build Digital Marketing Strategies - Kuralay Assainova, Liana Technolo...
DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions52 views
B2B Lead Generation - Driving Sales from Marketing - Anet (de Kock) van Stade...B2B Lead Generation - Driving Sales from Marketing - Anet (de Kock) van Stade...
B2B Lead Generation - Driving Sales from Marketing - Anet (de Kock) van Stade...
DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions31 views
Digital Marketing Trends - Kuralay Assainova, Liana TechnologiesDigital Marketing Trends - Kuralay Assainova, Liana Technologies
Digital Marketing Trends - Kuralay Assainova, Liana Technologies
DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions48 views
What I learned from Online advertising - Michelle Geere, AdbotWhat I learned from Online advertising - Michelle Geere, Adbot
What I learned from Online advertising - Michelle Geere, Adbot
DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions31 views
Revolutionizing Marketing - Harnessing the Power of AI - Michael Letschin, BrevoRevolutionizing Marketing - Harnessing the Power of AI - Michael Letschin, Brevo
Revolutionizing Marketing - Harnessing the Power of AI - Michael Letschin, Brevo
DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions31 views
Panel - Digital Marketing Trends - Michelle Geere, AdbotPanel - Digital Marketing Trends - Michelle Geere, Adbot
Panel - Digital Marketing Trends - Michelle Geere, Adbot
DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions32 views
Panel - Digital Marketing in the New Era of Privacy & Data Governance - Ashle...Panel - Digital Marketing in the New Era of Privacy & Data Governance - Ashle...
Panel - Digital Marketing in the New Era of Privacy & Data Governance - Ashle...
DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions32 views
Growing Beyond Expectations, 10 Marketing Lessons From Hyper-growth Companies...Growing Beyond Expectations, 10 Marketing Lessons From Hyper-growth Companies...
Growing Beyond Expectations, 10 Marketing Lessons From Hyper-growth Companies...
DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions28 views
 social media marketing tips  social media marketing tips
social media marketing tips
aysh2afidha35 views
Driving value from first-party data in a privacy-centric world - Vimal Badian...Driving value from first-party data in a privacy-centric world - Vimal Badian...
Driving value from first-party data in a privacy-centric world - Vimal Badian...
DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions47 views
MultiChannel Marketing Strategy Master Class - Jeff Turnbow, WinningLocalMultiChannel Marketing Strategy Master Class - Jeff Turnbow, WinningLocal
MultiChannel Marketing Strategy Master Class - Jeff Turnbow, WinningLocal
DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions16 views
The Modern Content Challenge - Charlie Bell, ContentfulThe Modern Content Challenge - Charlie Bell, Contentful
The Modern Content Challenge - Charlie Bell, Contentful
DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions31 views
Panel - Digital Marketing Trends Experts Insights on How to Gain a Competitiv...Panel - Digital Marketing Trends Experts Insights on How to Gain a Competitiv...
Panel - Digital Marketing Trends Experts Insights on How to Gain a Competitiv...
DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions18 views
The Relationship Between Strategy, Marketing and Technology - Nikki Cockcroft...The Relationship Between Strategy, Marketing and Technology - Nikki Cockcroft...
The Relationship Between Strategy, Marketing and Technology - Nikki Cockcroft...
DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions34 views
Predictive Data Generation for New Agile Marketing Systems - Michael Cohen, P...Predictive Data Generation for New Agile Marketing Systems - Michael Cohen, P...
Predictive Data Generation for New Agile Marketing Systems - Michael Cohen, P...
DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions27 views
Human Connection in Modern Ecommerce Marketing - Kyle Allison, The Doctor of ...Human Connection in Modern Ecommerce Marketing - Kyle Allison, The Doctor of ...
Human Connection in Modern Ecommerce Marketing - Kyle Allison, The Doctor of ...
DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions20 views
Growing The Small To Medium E-commerce Enterprise Market - Chanel Hardman, Us...Growing The Small To Medium E-commerce Enterprise Market - Chanel Hardman, Us...
Growing The Small To Medium E-commerce Enterprise Market - Chanel Hardman, Us...
DigiMarCon - Digital Marketing, Media and Advertising Conferences & Exhibitions34 views

Infinite Loops Dirty Architecture And Too Many Indexed URLs