Could you build your own, private view of the Internet? One that isn't reliant on Google or Bing? Majestic has done this and now has one of the largest web indexes on the planet. Whilst known and a backlink analysis engine, Majestic infact has its own, unique view of the Internet and is able to derive meaning, influence and context out of its dataset. Here's how they did it. (2018)

1. How To Evaluate The Whole Web
(Without being Google)

2. @Majestic
The BIG Specialist Search Engine
Twitter has 500,000,000
Tweets per day on average
In the same day, Majestic
crawls 7,000,000,000 URLs (and 3
billion of these are “new”)

3. @Majestic
How Do We Collect That Much Data?

4. @Majestic
How Do We Collect That Much Data?

5. @Majestic
How Do We Collect That Much Data?

6. @Majestic
How we ANALYZE links to get CF/TF
Topical Trust
Flow using
decay
Algorithm
???

7. @Majestic
Works best with Universal Data set
• Every signal is small
• Individually prone to
error or opinion
• At scale the error
decreases
• Confidence increases
http://info.majestic.com/universal

8. @Majestic
Jon M. Kleinberg, Cornell University 1999
Hub
Hub
Hub
Authority
Authority
Authority
Authority
Authority
Authority
Authority AuthorityAuthority
In Penguin, Google Muted
the Hubs!
Knowledge
Graph

9. @Majestic
The Next Hard Part of Search
Information Retrieval in
the “Zeta” age;
1.Data Collection
2.Data Grouping
3.Data Indexing
4.Data Matching

10. @Majestic
Groups Make Search Much Better
• Find a Fact
• Find a Friend
• Find a Customer
• Find Anything
LibraryofCongresscirca1940
Research At: info.majestic.com/groupresearch

11. @Majestic
Why Categorize the Web?
• Categorizing into topics prevents spam:
• http://maj.to/1yZOdW5 (TrustRank Paper)
http://maj.to/1Bb1lHu (Stanford Paper)

12. @Majestic
Categorisation by Data Type
• Country IP/TLD
• Image Search
• Video Search
• Local Listings
• Maps
• Travel

13. @Majestic
Approaches to Defining Topics

14. @Majestic
Approaches to Defining Topics

15. @Majestic
How We Categorize the Web
• Content connects in
Groups
• Keywords gravitate
into Groups

16. @Majestic
We See AND ANALYZE Links
Topical Trust
Flow using
decay
Algorithm
???

17. @Majestic
MIT=Reference
MIT.edu’s Link Network
Source: Labs.majestic.com
Reference
News Arts & Society
Spam?Reference

18. @Majestic
MIT=Reference/Education

19. @Majestic
800 Topics
In 15 Super
Categories

20. @Majestic
Topical Trust Flow is Very Powerful
• More granular than (say) PageRank
• Updates continually
• Provides measurable context and influence at scale
• Page level AND site level tracking
• 800 topics
• Can compare all types of online data (People, Pages,
Sites, Images, Plugins)

21. @Majestic
Use Cases
• Already used in Search
• Finding Influencers
• Credit checking
• Comparing different media channels
• Link building
• Company evaluations

22. @Majestic
Who is more popular on Twitter?
Use case: Finding influencers
Lady Gaga? Barack Obama?
Trust Flow
74
Trust Flow
70

23. @Majestic
Let’s go and see Topical
Pagerank in Action

24. @Majestic
Example: Credit Cards

25. @Majestic
Example: “Majestic” (Low correlation)

26. @Majestic
Example: “the whitehouse”

27. @Majestic
Takeaways
• Search Results are stronger if sites are categorized first
• Google has suggested their engineers have worked on
“Topical Page Rank”
• There are correlations between rankings and topics
Strategies:
• Build content around topical themes to create hubs of
authority or
• Change brand awareness around the keyword

28. @Majestic
Out of Trust Flows
understanding
Real insight into the world wide web from
Majestic,the specialist search engine