A search engine results page (SERP) is the list of results that a search
engine returns in response to a specific word or phrase query.
SPIDERS - CRAWLERS - BOTS
SPIDERS CRAWLERS BOTS
A browser like program that
downloads webpages and
A program and follows the
links on a webpage
An automated computer
program that visit websites
& perform predefined task.
• Spelling suggestions, definitions, related searches, related questions,
calculator, translator, answers, featured snippets, maps, images,
videos, movie showtimes, sports scores and fixtures, weather, flight
statuses, news results, tweets
• The Knowledge Graph
• QUERY UNDERSTANDING AND EXPANSION
• RETRIEVAL AND SCORING
• POST RETRIEVAL ADJUSTMENTS
• Crawls all available pages and URLs on the web
• Analysis of crawled pages (extract links, annotate semantics)
• Store this data and build an index
• Index is like the index of a book
• For each ‘keyword’, there is a list of pages it appears on
• Broken up into groups (known as shards)
UNDERSTANDING THE QUERY
• Does the query have any known entities?
• Beyonce, Cristiano Ronaldo, Sydney Opera House
• Are there useful synonyms?
• Are the synonyms ambiguous? (gm = general motors, genetically
• What’s the context?
RETRIEVAL AND SCORING
• Send the query to all the shards
• Each shard then:
• Finds matching pages
• Computes a score (S1) for query+page
• Sends back the top N pages (by S1)
• Combines the top pages and then sorts (ranks) by score
POST RETRIEVAL ADJUSTMENTS
• Too much duplication in results?
• Negative user interactions?
• Search abandonment
• Need for spam demotions/manual actions?
Depending on the brand and business model, these panels might include:
• Basic info and a brief description from Wikipedia
• Photos from Google Plus
• Stock prices from Google Finance, Yahoo Finance and MSN Money
• Reviews from Google
• Social profiles
• Related searches
HOW DOES GOOGLE VIEW YOUR SITE?
WHAT TO LOOK OUT FOR
• Can you see all of your menu links (drop downs too!)?
• Do all of the menu items and links appear as plain text?
• Are all the links clickable?
• Does this reveal any text that was previously hidden? (Hidden text can
send a red flag to Googlebot. It might not always be there maliciously,
but it shouldn't be there.)
• Is your sidebar or widgets content all the way at the top? Remember,
your most important links and content should be at the top of the
HTML. This is more important the bigger the site is.
BROKEN LINKS & DEAD ENDS
• Broken Link Checker
• Site Liner http://siteliner.com/
• Dead Link Checker http://www.deadlinkchecker.com/
SITE SPEED & PERFORMANCE
Website performance radically impacts our success, 64% of smartphone
users expect websites to load in under four seconds.
• Avoid plugins where possible. Features like carousels, tabbed
interfaces, lightboxes and scrolling are often implemented using
plugins, but these plugins are heavy, as they try to solve every
• Optimise Images
• Optimise fonts
SITEMAPS - XML
• Upload via Google Search Console
• Helps Google identify all the pages on your site
• Only include indexed pages on the sitemap
SITEMAPS - HTML
• Allow bots to navigate the site better
• Can help Google establish a page hierarchy
• Mobile friendly sites
• Optimising online slide decks
• Deeper social integration
• Image optimisation
MOBILE FRIENDLY SITES
• April 21, 2015 ‘Mobilegeddon’
• March 2016, Google announced it would update the algorithm in May
• April 2016, Google warns webmasters that their site’s aren’t mobile
• May 2016, Expected slow rollout of algorithm update
• Google Mobile Friendly Checker -