1. Enterprise Link
Spam Analysis
Ian Lurie
@portentint
ian@portent.com

3. MACHINE LEARNING
How hard can it be?
Ian Lurie
Portent, Inc
@portentint

4. http://portent.co/machine-spam
yes, this is right, no ‘m’

5. NERDOSITY
TIME

6. 1: There’s gotta be a better way
2: How hard can it be?
3: Lessons learned

7. ONE: THESE ARE GOOD LINKS!!!!

8. try to relax...

9. Some of these are real

10. BUILD A REALLY BIG SPREADSHEET.

11. GET ALL LINKS FROM
GOOGLE WEBMASTER TOOLS
OPENSITE EXPLORER
MAJESTIC SEO

13. URL
TITLE
ANCHOR TEXT
MAJESTIC TRUSTFLOW
MOZ DA AND PA

14. EVALUATE URLS
FREELINKSHERE.COM
ARTICLEFUNHOSTING.COM
GETYERGREATARTICLESHERE.COM
NEWJERSEYPRESSRELEASES.COM
NO

15. EVALUATE TITLES
BAD GRAMMAR
JUST STUPID
MAKES YOU ITCH
NO

16. CONTACT
DISAVOW

17. So cute. so
harmless.

18. AFILTRATIONPROBLEM

19. 500 LINKS: THE LOSERS ARE OBVIOUS

20. 500,000 LINKS: YOU GET A MIGRAINE

22. A KNOWLEDGEPROBLEM

23. Anchor
Text
Distribu1on
Col
Von
Miseryburgen
Alp
d'
Huez
Sufferlandria
Grunter
von
Agony
Mithril
Shirt
The
Devil's
Den
John
Snow
Broad
Street
Pump
GalacDca
Bagshot
Row
Li<le
Round
Top
Pegasus
The
Lonely
Mountain
0
10000
20000
30000
40000
50000
60000
70000
80000
90000
100000

24. BUT THESE ARE
GOOD LINKS!!!!

25. WE CAN’T READ GOOGLE’S MIND

26. MACHINE LEARNING

27. TWO: HOW HARD CAN IT BE?

28. MACHINE LEARNING
IN 60 SECONDS

29. START WITH A
QUESTION

30. START WITH A IS THIS PAGE
QUESTION SPAM?

31. THE ANSWER IS A
CLASSIFICATION

32. + TRAINING SET
ALGORITHM
CLASSIFICATION

33. + TRAINING SET
ALGORITHM
CLASSIFICATION

34. + TRAINING SET
ALGORITHM
PREDICTION

35. + TRAINING SET TEXT-BASED?
ALGORITHM SUPERVISED?
UNSUPERVISED?
CLASSIFICATION

36. + TRAINING SET
ALGORITHM CORRECT?
CLASSIFICATION OR MORONIC?

37. CLASSIFICATION
= QUESTION

38. CLASSIFICATION
= SPAM? TRUE OR FALSE

39. TRAINING SET 1
JUST WORDS

40. A home cooking blog featuring healthy low-glycemic recipes with
step-by-step photos, as well as cooking tips, vegetable gardening,
and products Kalyn loves. Mouse Trap (originally titled Mouse
Trap Game) is a board game first published by Ideal in 1963 for
two or more players. Over the course of the game, players at first.
A delicious and refreshing cherry pie and a story about making
friends of enemies :). Brady Bunch Punch drink recipe made with
Amaretto,Cranberry juice,Orange juice,Triple Sec,Vodka,. How to
make a Brady Bunch Punch with all the instructions and. A blog
about a foreigner's life in Japan, on a mountainside above Lake
Biwa. SEE the world's greatest collection of tattoo designs!
Sample FREE Downloads! Cutting Edge Art by Famous Tattoo
Artists! YOUR TATTOO DESIGN IS HERE!.

41. + TRAINING SET
BAYESIAN
FAIL

42. +

43. TRAINING SET 2
WORDS INTO NUMBERS

44. who da
nerd?!!! WHO DA
NERD?!!!!!
+ TRAINING SET
LOGISTIC REGRESSION
WIN

46. logistic regression

47. python
nltk
scikit-learn
mongodb

48. Flesch-Kincaid (FK)
FK grade level
FK reading ease
word count
sentence count
syllable count

49. links/word
MajesticSEO
Page TrustFlow
Domain TrustFlow
Unique c-blocks

50. THE TRAINING SET
python
nltk
scikit-learn
mongodb

51. seogadget.co.uk

52. Is seogadget.co.uk spam?
true = 1.93%
false = 98.07%

57. THREE:
LESSONS LEARNED

58. ABOUT GOOGLE

59. IT’S ABOUT LINKS. NOT PAGES.
HREF=“HTTP://
GETYERLINKS
HEREFREE.COM

60. TRUST FLOW 71
DA 97
PA 47

66. LESSON 1: THERE IS NO SPAM

67. HOW LIKELY IS IT THAT THIS LINK,
FROM THIS PAGE, IN THE CONTEXT
OF ALL OTHER LINKS TO THIS SITE,
MIGHT SEEM SPAM-LIKE?

68. USEFUL
INTERFLORA CNN.COM
MANIPULATIVE TRUSTWORTHY
GODX.NET DAILY SQUEE
NOPE

69. LINKS FROM EDU SITES 45

70. LINKS FROM EDU SITES 45

72. 2: DECLINING SPAM TOLERANCE

73. Percent
spam
links
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%

74. THIS WAS
SPAM IN APRIL

75. THIS MAY BE
SPAM NOW
http://www.cs.hiram.edu/
~oliphantlt/cpsc171/links.html

76. GOOGLE’S GETTING
GRUMPIER

77. CLEAN UP.
NOW.

78. ABOUT MACHINE LEARNING

79. NEED A BIGGER TRAINING SET
TOO BIG.

80. USE WORDS, TOO

81. SPECIALIZE BY VERTICAL

82. ian wtf!!!!

83. MACHINE LEARNING
IS GROWING

84. MACHINE LEARNING
IS BECOMING EASIER

85. GOOGLE PREDICTION API
BIGML
EXCEL DATASCOPE
….?

86. How hard can it be?
EASY TO UNDERSTAND

87. How hard can it be?
SO UNDERSTAND IT

88. http://portent.co/machine-spam
yes, this is right, no ‘m’

89. THE END.
Ian Lurie
Portent, Inc
@portentint