1. 10 — Search Engine Visibility
From Code to Product
gidgreen.com/course

2. The Google bomb
From Code to Product Lecture 10 — Search Engines — Slide 2 gidgreen.com/course

3. Feeling lucky?
From Code to Product Lecture 10 — Search Engines — Slide 3 gidgreen.com/course

4. Lecture 10
• Introduction
• How search works
• Anatomy of HTML
• On-page factors
• Off-page factors
• PageRank
From Code to Product Lecture 10 — Search Engines — Slide 4 gidgreen.com/course

5. History of web search
• 1994 — WebCrawler — first full text
• 1994 — Yahoo — directory then portal
• 1995 — AltaVista — first big index
• 1997 — Google — link citation analysis
• 2000 — 2004 — Yahoo uses Google
• 2000 — Baidu — now leader in China
• 2006 — Microsoft Live Search
• 2009 — Bing, switched to by Yahoo
From Code to Product Lecture 10 — Search Engines — Slide 5 gidgreen.com/course

6. Importance of search
http://www.searchenginejournal.com/24-eye-popping-seo-statistics/42665/
• US: 17.1B “core searches” in April 2012
Sources: comScore, Pew Internet Report, User Centric, PC Magazine,
– 55 per US citizen [comScore]
• 92% of online US adults use search
– 96% of college graduates
– 98% with income $75k+
• 70–80% ignore paid ads on right
– (but only 10% ignore ads on top)
• 80% of sessions begins with a search
From Code to Product Lecture 10 — Search Engines — Slide 6 gidgreen.com/course

7. Googling google
From Code to Product Lecture 10 — Search Engines — Slide 7 gidgreen.com/course

8. Search as traffic source
From Code to Product Lecture 10 — Search Engines — Slide 8 gidgreen.com/course

9. Global market share
Source: May 2012 figures from http://www.netmarketshare.com/
Bing, 4.15% Other,
2.05%
Baidu, 5.65%
Yahoo, 6.42%
Google,
81.73%
Global
From Code to Product Lecture 10 — Search Engines — Slide 9 gidgreen.com/course

10. USA market share
Source: May 2012 figures from http://www.netmarketshare.com/
AOL, Ask, 1.33%
Yahoo, 9.83% 1.47%
Bing, 10.46%
Google,
76.57%
USA
From Code to Product Lecture 10 — Search Engines — Slide 10 gidgreen.com/course

11. China market share
Source: http://chineseseoshifu.com/china-search-engine-market-share/
SoSo, 1.40%
Sougou, Others, 0.70%
Google, 2.80%
16.60%
Baidu,
78.50%
Also: Japan,
Czech Republic,
South Korea, China
Russia,
From Code to Product Lecture 10 — Search Engines — Slide 11 gidgreen.com/course

12. Search engine results page
From Code to Product Lecture 10 — Search Engines — Slide 12 gidgreen.com/course

13. Where do people look?
From Code to Product Lecture 10 — Search Engines — Slide 13 gidgreen.com/course

14. Where do people click?
http://www.seomoz.org/blog/mission-imposserpble-
establishing-clickthrough-rates
From Code to Product Lecture 10 — Search Engines — Slide 14 gidgreen.com/course

15. Black-hat vs white-hat
Black-hat SEO White-hat SEO
Tricking Google Working with Google
Hidden keywords Prominent keywords
Cloaking for search Structured for search
Content scraping Unique content
Link spam and farms Attracting links
Short-lived boost (maybe) Long-term results
From Code to Product Lecture 10 — Search Engines — Slide 15 gidgreen.com/course

16. Google’s recommendations
http://support.google.com/webmasters/bin/answer.py?hl=en&answer=35769
From Code to Product Lecture 10 — Search Engines — Slide 16 gidgreen.com/course

17. Lecture 10
• Introduction
• How search works
• Anatomy of HTML
• On-page factors
• Off-page factors
• PageRank
From Code to Product Lecture 10 — Search Engines — Slide 17 gidgreen.com/course

18. How search works
• Crawling
– Finding content to index
• Indexing
– Preparing content for search
• Searching
– Showing results to user
From Code to Product Lecture 10 — Search Engines — Slide 18 gidgreen.com/course

19. Basic crawling
• Create an empty URL queue (“frontier”)
• Add one good URL, e.g. wikipedia.org
• Repeat:
– Select random URL from queue
– Retrieve content for that URL
– Add links in content to queue
– Keep track to prevent repeat visits
• Is that it?
From Code to Product Lecture 10 — Search Engines — Slide 19 gidgreen.com/course

20. Crawling issues
• Link prioritization
• Duplicate content
– Print versions, sorting
• Infinite loops
– Database-driven sites
• Revisiting pages
• Site overloading
• Parallelization
From Code to Product Lecture 10 — Search Engines — Slide 20 gidgreen.com/course

21. Indexing
From Code to Product Lecture 10 — Search Engines — Slide 21 gidgreen.com/course

22. Indexing
From Code to Product Lecture 10 — Search Engines — Slide 22 gidgreen.com/course

23. From Code to Product
Lecture 10 — Search Engines — Slide 23
Inverted index
https://developer.apple.com/library/mac/#documentation/userexperience/
Conceptual/SearchKitConcepts/searchKit_basics/searchKit_basics.html
gidgreen.com/course

24. Other indexed information
• Page metadata
• More about words
– Prominence
– Position
– Frequency
• Links between pages
– Including anchor text
• Images, etc…
From Code to Product Lecture 10 — Search Engines — Slide 24 gidgreen.com/course

25. Other formats
http://support.google.com/webmasters/bin/
answer.py?hl=en&answer=35287
Forms?
Javascript?
From Code to Product Lecture 10 — Search Engines — Slide 25 gidgreen.com/course

26. Stemming, proximity, ANDs
From Code to Product Lecture 10 — Search Engines — Slide 26 gidgreen.com/course

27. Recent Google changes
• May 2012: knowledge graph
http://www.seomoz.org/google-algorithm-change
• Apr 2012: overoptimization penalty
• Jan 2012: top heavy ads penalty
• Nov 2011: rewarding freshness
• Feb 2011: hitting content farms
• Dec 2010: social media signals
• Dec 2009: real-time search
From Code to Product Lecture 10 — Search Engines — Slide 27 gidgreen.com/course

28. Google web history (2005–2009)
From Code to Product Lecture 10 — Search Engines — Slide 28 gidgreen.com/course

29. Search + your world (2012)
http://www.ubergizmo.com/2012/01/google-now-searches-your-world/
From Code to Product Lecture 10 — Search Engines — Slide 29 gidgreen.com/course

30. Keyword research
But: consider also long tail (referrer logs)
From Code to Product Lecture 10 — Search Engines — Slide 30 gidgreen.com/course

31. Keyword research
From Code to Product Lecture 10 — Search Engines — Slide 31 gidgreen.com/course

32. Lecture 10
• Introduction
• How search works
• Anatomy of HTML
• On-page factors
• Off-page factors
• PageRank
From Code to Product Lecture 10 — Search Engines — Slide 32 gidgreen.com/course

33. HTTP protocol
GET /wiki/Hypertext_Transfer_Protocol HTTP/1.1
Host: en.wikipedia.org
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_5_8) AppleWebKit/
534.50.2 (KHTML, like Gecko) Version/5.0.6 Safari/533.22.3
Referer: http://www.rexswain.com/httpview.html
Connection: close
HTTP/1.0 200 OK
Date: Sun, 17 Jun 2012 06:05:03 GMT
Server: Apache
Cache-Control: private, s-maxage=0, max-age=0, must-revalidate
Content-Language: en
Last-Modified: Sat, 16 Jun 2012 03:14:24 GMT
Content-Length: 164814
Content-Type: text/html; charset=UTF-8
Connection: close
From Code to Product Lecture 10 — Search Engines — Slide 33 gidgreen.com/course

34. Page structure
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01
Transitional//EN" "http://www.w3.org/TR/html4/
loose.dtd">
<html lang="en">
<head>
<title>Uber List Manager</title>
</head>
<body>
<h1>Uber List Manager</h1>
<p>The world's leading &amp; best priced list
management software.</p>
</body>
</html>
From Code to Product Lecture 10 — Search Engines — Slide 34 gidgreen.com/course

35. Key <HEAD> elements
<head>
<meta http-equiv="content-type" content="text/
html; charset=utf-8">
<link rel="stylesheet" type="text/css"
href="styles.css">
<script type="text/javascript" src=”script.js"></
script>
<title>Uber List Manager</title>
<meta name="description" content="An excellent
and well priced list management program.">
<meta name="keywords" content="lists, list
manager, uber, mailing software">
</head>
From Code to Product Lecture 10 — Search Engines — Slide 35 gidgreen.com/course

36. Key <BODY> elements
<body>
<h1>Uber List Manager</h1>
<p>The world's leading &amp; best priced list
management software.</p>
<h2>Features</h2>
<h2>Customer stories</h2>
<img src="images/ulm.jpeg" width="320" height="240"
alt="Screenshot" title="ULM in action">
<form action="form.php" method="post">
<input type="submit" value="Submit">
</form>
<iframe src="iframe.html" width="300"
height="300"></iframe>
</body>
From Code to Product Lecture 10 — Search Engines — Slide 36 gidgreen.com/course

37. External file summary
styles.css
body
{background-
color:yellow;} index.html
h1 {font-size:
24px;}
a:hover {text-
decoration:
underline;}
<html lang="en">
ulm.jpeg
<head>
<title>
Uber List...
</title>
</head>
script.js <body> Iframe.html
document.onkey ... <html>
down=okd; </body> <head>
window.onbefor
eunload=obl;
</html> </head>
<body>
..
.
if (t>=0)
</body>
d=new Date(); </html>
From Code to Product Lecture 10 — Search Engines — Slide 37 gidgreen.com/course

38. Links
Click <a href="more-information.html">here</a> for
ULM benefits and pricing.
Click for <a href="more-information.html">ULM
benefits and pricing</a>.
Click for <a href="more-information.html" title="ULM
benefits and pricing">more about ULM</A>.
Better than <a href="http://slowlists.com/"
rel="nofollow">our competitors</a>.
<a href="pricing.html"><img src="dollar-bill.jpeg"
alt="Pricing"></a>
From Code to Product Lecture 10 — Search Engines — Slide 38 gidgreen.com/course

39. Internal targets
<a href="#History">2 History</a>
.
.
.
<h2 id="history">History</h2>
<a name="history"></
a><h2>History</h2>
From Code to Product Lecture 10 — Search Engines — Slide 39 gidgreen.com/course

40. Rich snippets
<span itemprop="reviewCount">808</span>
<span itemprop="streetAddress">204 E 43rd St</
span>
<span id="bizPhone" itemprop="telephone">(212)
972-1001</span>
<span itemprop="priceRange">$$$$</span>
<meta itemprop="ratingValue" content="4.5">
From Code to Product Lecture 10 — Search Engines — Slide 40 gidgreen.com/course

41. HTML5
<article>
http://www.michaelcropper.co.uk/2012/03/html5-
<section>
<hgroup>
<aside>
seo-best-practices-1032.html
<header>
<footer>
<nav>
From Code to Product Lecture 10 — Search Engines — Slide 41 gidgreen.com/course

42. Lecture 10
• Introduction
• How search works
• Anatomy of HTML
• On-page factors
• Off-page factors
• PageRank
From Code to Product Lecture 10 — Search Engines — Slide 42 gidgreen.com/course

43. Page title
No more than ~70 characters / 10 words
From Code to Product Lecture 10 — Search Engines — Slide 43 gidgreen.com/course

44. URLs
SEO Cheat Sheet: Anatomy of A URL
http://www.seomoz.org/blog/seo-cheat-
1 2 3 4 5 5 6 7
1 http://store.example.com/topics/subtopic/descriptive-product-name#top
SEO-FRIENDLY URL
1 Protocol
Keyword Priority1 SEO Tips for URLs
2 Subdomain
Observed Google priority • Use subdomains carefully. They may be treated as
3 Domain of keyword placement: separate entities, splitting domain authority.
4 Top-Level Domain • Separate path & page keywords with hyphens (”-”).
(1) Domain
5 Folders / Paths (2) Subdomain • Anchors may help engines understand page structure.
6 Page (3) Folder • Keyword effectiveness in URLs decreases as URL
7 Named Anchor (4) Path/Page length and keyword position increases.1
sheet-anatomy-of-a-url
1 SEOmoz correlational data (2009)
1 2 3 4 5 6 7 7 7
2 http://www.example.com/index.php?product=1234&sort=price&print=1
OLD DYNAMIC URL
1 Protocol
Popular TLDs2 Popular ccTLDs* Popular Extensions
2 Subdomain
.com - commercial .cn - China .htm - Static HTML
3 Domain .net - infrastructure .de - Germany .html - Static HTML
4 Top-Level Domain .org - non-profit .uk - United Kingdom .php - PHP code
5 Page / File Name .edu - schools .nl - Netherlands .asp - ASP code
.info - informational .eu - European Union .aspx - ASP.NET
6 File Extension .biz - small business .ru - Russian Federation .cfm - ColdFusion
7 CGI Parameters .name - personal sites .ar - Argentina .jsp - Java Code
2 Verisign domain report (2009) * ccTLD = Country Code TLD
©2009 SEOmoz ∙ w w w.seomoz.org ∙ Read SEOmoz. Rank Be9er.
From Code to Product Lecture 10 — Search Engines — Slide 44 gidgreen.com/course

45. URLs: good vs bad
www.really-cheap-great-mailing-list-manager.info
www.mailingmanager.com
googleblog.blogspot.com/view?
post_id=3982098&section_id=231
googleblog.blogspot.com/2012/04/introducing-google-
drive.html
amazon.com/store/products/books/computing/internet/
seo/Eric+Edge/The%20Art%20Of%20SEO/details
amazon.com/The-Art-SEO-Eric-Edge
From Code to Product Lecture 10 — Search Engines — Slide 45 gidgreen.com/course

46. Meta descriptions
Used for display but not for ranking
Length: 150~160 characters
Avoid duplication across many pages
From Code to Product Lecture 10 — Search Engines — Slide 46 gidgreen.com/course

47. Keyword density
From Code to Product Lecture 10 — Search Engines — Slide 47 gidgreen.com/course

48. Formatting
and <b>good value</b>
and <span style="font-weight:bold;">good value</span>
and <span class="emboldened">good value</span>
and <em>good value</em>
and <strong>good value</strong>
<font size="+2>Features</font>
<big>Features</big>
<p style="font-size:24px;">Features</p>
<h2>Features<h2>
<h2 style="font-size:24px;">Features</h2>
From Code to Product Lecture 10 — Search Engines — Slide 48 gidgreen.com/course

49. Freshness and speed
• Freshness determined by:
– Date the page appeared
– Frequency of content change
– Amount of content change
– Rate of new incoming links
From Code to Product Lecture 10 — Search Engines — Slide 49 gidgreen.com/course

50. Javascript and Flash
From Code to Product Lecture 10 — Search Engines — Slide 50 gidgreen.com/course

51. Lecture 10
• Introduction
• How search works
• Anatomy of HTML
• On-page factors
• Off-page factors
• PageRank
From Code to Product Lecture 10 — Search Engines — Slide 51 gidgreen.com/course

52. Links from external sites
• From high ranking sites
– Hard to manipulate
• From .edu or .gov
– No commercial motivation
• From topic-related sites
• From many sites
– Diversity of subject
– Different ownership / IP block
From Code to Product Lecture 10 — Search Engines — Slide 52 gidgreen.com/course

53. Links on external pages
• All-important anchor text
– First appearance counts
– Diversity of anchors
• Higher on linking page
• From core text content
– Not navigation/footers
– Image ALT text weaker
• Page has other good links
From Code to Product Lecture 10 — Search Engines — Slide 53 gidgreen.com/course

54. Power of anchors
From Code to Product Lecture 10 — Search Engines — Slide 54 gidgreen.com/course

55. Titles and URLs in anchors
Wikipedia, the free encyclopedia — 450 saves
Visit Wikipedia for more information
Recent referrers: en.wikipedia.org
http://en.wikipedia.org/wiki/Main_Page
From Code to Product Lecture 10 — Search Engines — Slide 55 gidgreen.com/course

56. Attracting links
• (Directories e.g. dmoz)
• Inbound marketing
– Great on-site content
– Post articles elsewhere
– Request reviews
• Viral marketing
– Banners + widgets
– Social network sharing
From Code to Product Lecture 10 — Search Engines — Slide 56 gidgreen.com/course

57. Link bait
From Code to Product Lecture 10 — Search Engines — Slide 57 gidgreen.com/course

58. Domain information
From Code to Product Lecture 10 — Search Engines — Slide 58 gidgreen.com/course

59. User monitoring
From Code to Product Lecture 10 — Search Engines — Slide 59 gidgreen.com/course

60. Duplicate content
• Other sites stealing your content
• www.domain.com vs domain.com
• domain.com/ vs domain.com/index.html
• Printer-friendly versions
• URL parameters
From Code to Product Lecture 10 — Search Engines — Slide 60 gidgreen.com/course

61. robots.txt
User-agent: * Or in <HEAD> of page:
Disallow: /cgi-bin/
Disallow: /images/ <meta name="robots"
Disallow: /tmp/ content="noindex">
Disallow: /private/
User-agent: BadBot
Disallow: /
Sitemap: http://www.example.com/sitemap.xml
From Code to Product Lecture 10 — Search Engines — Slide 61 gidgreen.com/course

62. XML sitemaps
<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/
schemas/sitemap/0.9">
<url>
<loc>http://www.example.com/</loc>
<lastmod>2005-01-01</lastmod>
<changefreq>monthly</changefreq>
<priority>0.8</priority>
</url>
</urlset>
From Code to Product Lecture 10 — Search Engines — Slide 62 gidgreen.com/course

63. Redirects and rel=canonical
http://www.seomoz.org/blog/an-seos-guide-to-http-status-codes
From Code to Product Lecture 10 — Search Engines — Slide 63 gidgreen.com/course

64. Putting it all together
http://www.seomoz.org/article/search-ranking-factors#predictions
From Code to Product Lecture 10 — Search Engines — Slide 64 gidgreen.com/course

65. Lecture 10
• Introduction
• How search works
• Anatomy of HTML
• On-page factors
• Off-page factors
• PageRank
From Code to Product Lecture 10 — Search Engines — Slide 65 gidgreen.com/course

66. A random walk
A C
E
B D
From Code to Product Lecture 10 — Search Engines — Slide 66 gidgreen.com/course

67. Probability distribution
http://en.wikipedia.org/wiki/File:PageRanks-Example.svg
From Code to Product Lecture 10 — Search Engines — Slide 67 gidgreen.com/course

68. The maths
http://en.wikipedia.org/wiki/PageRank
From Code to Product Lecture 10 — Search Engines — Slide 68 gidgreen.com/course

69. From Code to Product
Lecture 10 — Search Engines — Slide 69
PageRank sculpting?
http://www.seomoz.org/blog/google-says-yes-you-can-still-
sculpt-pagerank-no-you-cant-do-it-with-nofollow
gidgreen.com/course

70. PageRank in reality
• Domain authority signals
• Nofollow links are clicked by people
• Interval vs external links
• Paid link and link farm detection
• Removed from toolbar in 2009
https://sites.google.com/site/webmasterhelpforum/en/faq--crawling--indexing---ranking#pagerank
From Code to Product Lecture 10 — Search Engines — Slide 70 gidgreen.com/course

71. Tools
From Code to Product Lecture 10 — Search Engines — Slide 71 gidgreen.com/course