THIS PPT IS BASED ON THE PAPER BY SERGEY BRIN ON EXTRACTING PATTERNS AND RELATIONS FROM THE WORLD WIDE WEB

1. EXTRACTING PATTERNS AND RELATIONS
FROM THE WORLD WIDE WEB
BY,
SUJITHA R
S7 CSE - A
AM.EN.U4CSE10055

2. INTRODUCTION
•The World Wide Web as an information resource :
• Widely distributed
• Huge
• Complex, various styles and formats
• Scattered information
 If we could integrate the chunks of information they
would form an valuble source of information.

3. MOTIVATION
Discover information sources
Extract information of a particular type automatically/with
minimal human intervention
Integrate into a relational form
The largest and most diverse source of information

4. APPLICATIONS
To extract relational data from the entire World Wide
Web
Types of data that can be extracted: books,movies,
music, restaurants, etc.

5. Problem:
To extract a relation of books ( author, title)
pairs from the World Wide Web.

6. DIPRE:DUAL ITERATIVE PATTERN RELATION
EXTRACTION
 A technique called DIPRE is used to extract relations
from the sources.
 It relies on duality of patterns and relations.
DIPRE:
 Initial seed set:
Author Book title
Isaac Asimov The Robots of Dawn
David Brin Startide Rising
James Gleick Chaos:Making a New Science
Charles Dickens Greations Expectat
William Shakespeare The Comedy of Errors.
Initial seed
tuples.

7.  OCCURENCES OF SEED TUPLES:
The famous book ,The robots of Dawn is
authored by Isaac Asimov
David Brin’s Startide rising is a good book
which…….
James Gleick wrote Chaos:Making a New
Science
Charles Dickens wrote Great Expectations, book
The comedy of errors by william shakespeare is
one among the….
Author Book title
Isaac
Asimov
The Robots
of Dawn
David Brin Startide
Rising
James
Gleick
Chaos:Makin
g a New
Science
Charles
Dickens
Great
Expectations
William
Shakespear
e
The Comedy
of Errors.

8. DIPRE PATTERNS:
<STRING 2> is authored by <STRING1>
<STRING1> ‘s <STRING2>
<STRING1> wrote <STRING2>
<STRING 2> by <STRING 1>
DIPRE pattern is 5 Tuple <order, urlprefix, left, middle, right>
• Here the order is boolean value and other attributes are strings
•Verify order and middle of all occurrences is the same.
• If order is true, an(author,title) pair matches the pattern
•If the order is false which means the title and author are switched

9. GENERATING NEW SEED TUPLES
 After initial pattern generation DIPRE scans
the text for segments of text that match the
pattern.
 New tuples generated used as new seed.
 Process done all over again to identify new
promising pattern.

10. PROBLEMS:PATTERNS
 Pattern :<string1> ’s <string 2>
 J.K.Rowling ’s Harrypotter Series is the one of
the best selling………
 Sheena ‘s purse is with……
 Invalid tuples generated
 Degrade quality of tuples on subsequent iterations
 Pattern representation
 Patterns must be specific . It must not be too generalize

11. EXPERIMENTS
 For data here used a repository of 24 million web
pages. This data is part of the stanford webBase and
is used for the Google Search Engine.
 5 books and their information is given as the initial
seed set.
Author Book title
Isaac Asimov The Robots of Dawn
David Brin Startide Rising
James Gleick Chaos:Making a New
Science
Charles Dickens Great Expectations
William Shakespeare The Comedy of Errors.

12.  These produced 199 occurrences and generated 3
patterns
Patterns found in the first iteration
URL Pattern Text Pattern
www.sff.net/locus/c.* <title> by <author>
Dns.city-
net.com/imann/awards/hugos.html
<title> by author <author>
Dolphin.upenn,edu/dcummins/texts.ht
m
<author> , <title>
o A run of these patterns over matching URL’s produced 4047
unique(author,title) pairs.

13. Author Title
H.D.Everett The Death mask and other ghosts
H.G.Wells First men in moon
H.G.Wells The invisible man
H.G.Wells The island of Dr.Moreau
H.G.Wells The time machine
H.P.Lovecraft The case of Charles Dexter Ward
H.M.Hoover Journey through the empty.
Sample of books found in the first iteration
•These occurrences produced 105 patterns,24 of which had
url prefixes

14. RESULTS:
 A pass over couple million urls produced 969
unique(author,title) pairs
 There were some Bogus Books among these.
 Some of which had url prefixes which were not
complete urls.

15. QUALITY OF RESULTS
 Select 20 books to analyse the quality of the results
and searched online.
 Out of 20, nineteen books were bona fied.
 Some of the books were online books.
 Some were obscure or out of print.
 Some books are mentioned several times due to
the small difference in spacing,capitalization,how
the author was listed(eg: E.R.Burroughs versus
Edgar Rice Burroughs)

16. Conclusions
• DIPRE --- a remarkable tool to extract relational data
from the Web
• Minimum human intervention
• Application in different domains other than books
• Finding books not listed in major online sources

17. REFERENCE
 Sergey Brin and Larry Page. http://google
.stanford.edu
 Sergey Brin List of Books. http://www-
db.stanford.edu/~sergey/booklist.html
 Douglas Clark. Disbanded Benjamin press.
http://www.batch.ac.uk/~exxdgdc/poetry/library/di1.html
 http://www.research.att.com/~suciu/workshop-
papers.html,May 1997