Understanding the working of search engines such as Google,yahoo, bing etc

1. PRESENTED BY
AISSHWARYA.T
III rd YEAR ECE
DEPARTMENT OF ECE JEPPIAAR ENGINEERING COLLEGE 1

2. CONTENT
 WHAT IS A SEARCH ENGINE
 SPIDERS
 BASIC TASKS
 DIRECTORIES Vs SEARCH ENGINES
 TYPES OF SEARCH ENGINES
 GOOGLE WORKING
 SEO
 THE RANKING FACTORS
 FUTURE
DEPARTMENT OF ECE JEPPIAAR ENGINEERING COLLEGE 2

3. WHAT IS A SEARCH ENGINE?
 Internet is a large source of information.
 The amount of data is increasing vigorously with
increasing number of users.
 It is a tedious job to search for a particular data from
the ocean of information.
 The need for simplifying the process is handled using
SEARCH ENGINES.
 Search engines can give any information on the
subject based on the keywords.
DEPARTMENT OF ECE JEPPIAAR ENGINEERING COLLEGE 3

4. SPIDERS
THE SEARCH ENGINE ROBOTS
 Spiders or robots are automated software programs to
survey the web and build a data base.
 Web documents are retrieved and analysed using the
spider.
 They provide a better personalised experience for the
user.
DEPARTMENT OF ECE JEPPIAAR ENGINEERING COLLEGE 4

5. Working of spiders or robots
DEPARTMENT OF ECE JEPPIAAR ENGINEERING COLLEGE 5

6. BASIC TASKS OF SEARCH ENGINES
 Selects pieces of the web content based on
keywords.
 They keep index of the words and where they
find.
 They allow users to look for words or
combinations of words found in that index.
DEPARTMENT OF ECE JEPPIAAR ENGINEERING COLLEGE 6

7. DIRECTORY Vs SEARCH ENGINE
DIRECTORY SEARCH ENGINE
 A directory stores the name of a
site, a relevant category and a
short description of what is
contained in the site.
 Search is based on the title and
description of the site.
 Example: World Wide Web
Virtual Library, Joe ant and
DMOZ.
 Search engine links all the URLs
on the web.
 It searches based on the
keywords using crawlers, which
return the linked pages with the
keywords as hints.
 Example: Google, Yahoo, Bing,
Yandex etc.
DEPARTMENT OF ECE JEPPIAAR
ENGINEERING COLLEGE 7

8. TYPES OF SEARCH ENGINES
1. Spider or crawler
based search
engines.
2. Directories powered
by humans.
3. Combination or
hybrids of spider
and directories.
DEPARTMENT OF ECE JEPPIAAR ENGINEERING COLLEGE 8

9. CRAWLER BASED SEARCH ENGINES
 Most popular type of
search engine.
 The crawler or spider or
robot crawls along the
data to provide the
relevant information.
 Constantly updating.
 Example: Google
DEPARTMENT OF ECE JEPPIAAR ENGINEERING COLLEGE 9

10. DIRECTORIES
 A directory uses human
editors to decide on
which category the site
belongs.
 The human editors
analyze the website and
rank them based on the
predefined set of rules.
 Example: Yahoo
DEPARTMENT OF ECE JEPPIAAR ENGINEERING COLLEGE 10

11. HYBRID SEARCH ENGINES
 Hybrid search engines uses combinations of
crawler and directories.
 Provides better results.
 Example: Google, Yahoo.
DEPARTMENT OF ECE JEPPIAAR ENGINEERING COLLEGE 11

12. THE CYCLE OF HYBRID SEARCH ENGINES
DEPARTMENT OF ECE JEPPIAAR ENGINEERING COLLEGE 12

13. META SEARCH ENGINES
 Meta search engines
take results from all
other search engines
results.
 Combines them into
one large listing.
 Example: Metacrawler,
Dogpile
DEPARTMENT OF ECE JEPPIAAR ENGINEERING COLLEGE 13

14. SO HOW DO THEY SEARCH?
DEPARTMENT OF ECE JEPPIAAR ENGINEERING COLLEGE 14

15. TERM FREQUENCY
 Term frequency is the
number of times a
given term or query
appears in a search
index.
 It is a key component
for determining the
relevance of a given
document for a
particular query.
DEPARTMENT OF ECE JEPPIAAR ENGINEERING COLLEGE 15

16. METHODS TO STORE DATA
 BIG FILES
 REPOSITORY
 DOCUMENT INDEX
 LEXICON
 HIT LISTS
 FORWARD INDEX
 INVERTED INDEX
DEPARTMENT OF ECE JEPPIAAR ENGINEERING COLLEGE 16

17. HOW DOES GOOGLE SEARCH ENGINE
WORK?
• QUERY GIVEN IS
TO THE SEARCH
ENGINE.
GOOGLE
WEB SERVER
• QUERY TAKEN TO
DOC SERVERS
• RETRIEVES
STORED
DOCUMENT
INDEX
SERVERS • SEARCH RESULTS
ARE RETURNED
TO USER IN A
FRACTION OF A
SECOND.
GOOGLE
USER
DEPARTMENT OF ECE JEPPIAAR ENGINEERING COLLEGE 17

18. SEO:
SEARCH ENGINE OPTIMIZATION
SEO stands for Search
Engine
Optimization, which
is the practice of
increasing
the quantity and qualit
y of traffic to your
website
through organic search
engine results.
DEPARTMENT OF ECE JEPPIAAR ENGINEERING COLLEGE 18

19. THE RANKING FACTORS
1. Content
2. Back links
3. Mobile-first user
experience
4. Other technical
factors
DEPARTMENT OF ECE JEPPIAAR ENGINEERING COLLEGE 19

20. DEPARTMENT OF ECE JEPPIAAR ENGINEERING COLLEGE 20

21. NEURAL TECHNOLOGY
 Neural click models is an predictive approach to
the user’s query.
 There are two methods or approaches:
DEPARTMENT OF ECE JEPPIAAR ENGINEERING COLLEGE
Probabilistic Graphical Model (PGM)
Distributive Representation Approach (DRA)
21

22. PGM
 User behavior is
represented as
sequence of observable
and hidden evens such
as clicks, skips and
document
examinations
DEPARTMENT OF ECE JEPPIAAR ENGINEERING COLLEGE 22

23. DISTRIBUTED REPRESENTATION APPROACH
 Here user behavior is represented as vector states
that capture user’s behavior in the form of his/her
information need and information consumed
during the session
DEPARTMENT OF ECE JEPPIAAR ENGINEERING COLLEGE 23

24. DEPARTMENT OF ECE JEPPIAAR ENGINEERING COLLEGE 24

25. THE FUTURE
And yes the answer to
the future is
 DATA SCIENCE
 MACHINE
LEARNING
 ARTIFICIAL
INTELLIGENCE
DEPARTMENT OF ECE JEPPIAAR ENGINEERING COLLEGE 25

26. CONCLUSION
Search engine has become unavoidable in our life.
This means that the information rate is increasing
abundantly and proportionally there must be
increase in the speed of acquiring those
information. So modern search engines has begun
to seek help of the future, ARTIFICIAL
INTELLIGENCE.
DEPARTMENT OF ECE JEPPIAAR ENGINEERING COLLEGE 26

27. DEPARTMENT OF ECE JEPPIAAR ENGINEERING COLLEGE 27