Freebase
A socially managed semantic database



Jamie Taylor
SemTech 2010 Data Camp
Freebase has Many Types of Things
12 Million Topics
A Multiplicity Strong Identifiers

            http://rdf.freebase.com/ns/en.berlin_wall




            http://www.ellerdale.com/topics/view/0080-6ba0




            http://www.bbc.co.uk/music/artists/7f347782-eb14-40c3-98e2-17b6e1bfe56c

                   http://musicbrainz.org/artist/7f347782-eb14-40c3-98e2-17b6e1bfe56c

http://rdf.freebase.com/ns/authority.musicbrainz.7f347782-eb14-40c3-98e2-17b6e1bfe56c
Relations
contains
                          400 Million
           contained-by

                                  event               label
                                          albums

                            member-of
                                          member-of

           nationality

                           education
                                          education

                          contained-by
What’s in Freebase?
http://www.bestbuy.com/site/She+Wolf…

              http://www.daylife.com/topic/Shakira

                         http://twitter.com/shakira

                  http://www.facebook.com/shakira

                  http://www.myspace.com/shakira

                  http://www.last.fm/music/Shakira

http://www.netflix.com/RoleDisplay/Shakira/20046629

          http://www.guardian.co.uk/music/shakira
99% pure

All data undergoes rigorous QA before load
Major focus is reconciliation
Use sampling to assure 99% accuracy
Data that does not meet 99% accuracy is not loaded
What's been built on Freebase?
Up to 100,000 Queries a Day




 Quarterly dumps of graph
    http://download.freebase.com
Users contribute data




Users extend the data model
The Freebase Commons
                      Top-level domains
                      ·American football       ·Internet
                      ·Anime/Manga             ·Language
                      ·Architecture            ·Law
                      ·Astronomy               ·Library
                      ·Automotive              ·Location
                      ·Aviation                ·Martial Arts
                      ·Awards                  ·Measurement Unit
                      ·Baseball                ·Media Common
                      ·Basketball              ·Medicine
                      ·Bicycles                ·Metaweb Types
                      ·Biology                 ·Meteorology
                      ·Boats                   ·Military
                      ·Broadcast               ·Music
                      ·Business                ·Olympics
                      ·Celebrities             ·Opera
                      ·Chemistry               ·Organization
                      ·Comics                  ·People
                      ·Common                  ·Geography
                      ·Computers               ·Projects
                      ·Conferences             ·Protected Places
                      ·Cricket                 ·Publishing
                      ·Data World              ·Radio
                      ·Digicams                ·Rail
                      ·Education               ·Religion
                      ·Engineering             ·Royalty
                      ·Event                   ·Soccer
                      ·Clothing and Textiles   ·Spaceflight
                      ·Fictional Universes     ·Sports
                      ·Film                    ·Symbols
                      ·Food & Drink            ·Tennis
                      ·Freebase                ·Theater
                      ·Games                   ·Time
                      ·Geology                 ·Transportation




schema = vocabulary
                      ·Government              ·Travel
                      ·Hobbies and Interests   ·TV
                      ·Ice Hockey              ·Video Games
                      ·Influence               ·Visual Art
The Scope of Schema
   10,448 Properties
      describing
     4,936 Types*
     organized into
     641 Domains
     (77 Commons)
            *types with 10 or more instances
Strength through Exemplars
                                                   Type Instances


            100,000,000


             10,000,000



                                                              >10 instances,
              1,000,000


               100,000
                                                              4936 types
Instances




                10,000


                  1,000
                                                              1424 Commons
                   100


                    10


                     1
                          0   1000   2000   3000   4000   5000    6000   7000   8000   9000   10000 11000
                                                                 Rank
Metaweb Query Language
      [{
           "name" : null,
           "type" : "/film/film"
      }]




               MQL
[{
     "name" : null,
     "type" : "/film/film",
     "directed_by":{"id":"/en/george_lucas"},
     "starring":[{
            "actor":{"id":"/en/harrison_ford"}
         }]
}]




                      MQL
[{
      "name" : null,
      "type" : "/film/film",
      "directed_by":{"id":"/en/george_lucas"},
      "starring": [{
          "actor": {
             "name": null,
             "film": [{
                 "film": {"id": "/en/the_great_escape"}
             }]
          }
     }]
}]


                     Donald Pleasence
                        THX 1138
Freebase Suggest
Reconciliation
        {
             "/type/object/name":"Blade Runner",
             "/type/object/type":"/film/film",
             "/film/film/starring/actor":["Harrison Ford", "Rutger Hauer"],
             "/film/film/director":"Ridley Scott",
             "/film/film/release_date_s":"1981"
         }
[{
     "id":"/guid/9202a8c04000641f8000000000009e89",
     "name":["Blade Runner", "Bladerunner"],
     "score":1.4320519,
     "match":true,
     "type":["/common/topic", "/film/film","/media_common/adapted_work", "/award/award_winning_work",
     ]},
 {
     "id":"/guid/9202a8c04000641f80000000002643d0",
     "name":["Blade"],
     "score":0.48852453,
     "match":false,
     "type":["/common/topic", "/film/film", "/award/award_winning_work", "/award/award_nominated_work",
     ]}

               http://data.labs.freebase.com/recon/
Topic Blocks
Topic API
         Shortcut to building Topic displays
         Two forms:
             basic (names, types, description)
             standard (basic + keys, properties)




http://www.freebase.com/experimental/topic/standard?id=/en/ncis
Geo Search API



Semantic              Spatial              Semantic




      http://www.freebase.com/docs/geosearch
Gridworks
Acre Development Environment
Getting Started++
•   Freebase Documentation Hub
    •   http://www.freebase.com/docs
•   Developer Mailing List
    •   http://lists.freebase.com/mailman/listinfo/freebase-discuss
    •   http://freebase.markmail.org
•   Real Time help on IRC
    •   Freenode #freebase
•   Freebase Happenings
    •   http://blog.freebase.com
•   About the Graph Store
    •   Google: "ACM SIGMOD schema last tuple store"

Freebase - Semantic Technologies 2010 Code Camp

  • 1.
    Freebase A socially managedsemantic database Jamie Taylor SemTech 2010 Data Camp
  • 3.
    Freebase has ManyTypes of Things
  • 4.
  • 6.
    A Multiplicity StrongIdentifiers http://rdf.freebase.com/ns/en.berlin_wall http://www.ellerdale.com/topics/view/0080-6ba0 http://www.bbc.co.uk/music/artists/7f347782-eb14-40c3-98e2-17b6e1bfe56c http://musicbrainz.org/artist/7f347782-eb14-40c3-98e2-17b6e1bfe56c http://rdf.freebase.com/ns/authority.musicbrainz.7f347782-eb14-40c3-98e2-17b6e1bfe56c
  • 7.
    Relations contains 400 Million contained-by event label albums member-of member-of nationality education education contained-by
  • 8.
  • 10.
    http://www.bestbuy.com/site/She+Wolf… http://www.daylife.com/topic/Shakira http://twitter.com/shakira http://www.facebook.com/shakira http://www.myspace.com/shakira http://www.last.fm/music/Shakira http://www.netflix.com/RoleDisplay/Shakira/20046629 http://www.guardian.co.uk/music/shakira
  • 11.
    99% pure All dataundergoes rigorous QA before load Major focus is reconciliation Use sampling to assure 99% accuracy Data that does not meet 99% accuracy is not loaded
  • 12.
    What's been builton Freebase?
  • 13.
    Up to 100,000Queries a Day Quarterly dumps of graph http://download.freebase.com
  • 16.
    Users contribute data Usersextend the data model
  • 17.
    The Freebase Commons Top-level domains ·American football ·Internet ·Anime/Manga ·Language ·Architecture ·Law ·Astronomy ·Library ·Automotive ·Location ·Aviation ·Martial Arts ·Awards ·Measurement Unit ·Baseball ·Media Common ·Basketball ·Medicine ·Bicycles ·Metaweb Types ·Biology ·Meteorology ·Boats ·Military ·Broadcast ·Music ·Business ·Olympics ·Celebrities ·Opera ·Chemistry ·Organization ·Comics ·People ·Common ·Geography ·Computers ·Projects ·Conferences ·Protected Places ·Cricket ·Publishing ·Data World ·Radio ·Digicams ·Rail ·Education ·Religion ·Engineering ·Royalty ·Event ·Soccer ·Clothing and Textiles ·Spaceflight ·Fictional Universes ·Sports ·Film ·Symbols ·Food & Drink ·Tennis ·Freebase ·Theater ·Games ·Time ·Geology ·Transportation schema = vocabulary ·Government ·Travel ·Hobbies and Interests ·TV ·Ice Hockey ·Video Games ·Influence ·Visual Art
  • 18.
    The Scope ofSchema 10,448 Properties describing 4,936 Types* organized into 641 Domains (77 Commons) *types with 10 or more instances
  • 19.
    Strength through Exemplars Type Instances 100,000,000 10,000,000 >10 instances, 1,000,000 100,000 4936 types Instances 10,000 1,000 1424 Commons 100 10 1 0 1000 2000 3000 4000 5000 6000 7000 8000 9000 10000 11000 Rank
  • 20.
    Metaweb Query Language [{ "name" : null, "type" : "/film/film" }] MQL
  • 21.
    [{ "name" : null, "type" : "/film/film", "directed_by":{"id":"/en/george_lucas"}, "starring":[{ "actor":{"id":"/en/harrison_ford"} }] }] MQL
  • 22.
    [{ "name" : null, "type" : "/film/film", "directed_by":{"id":"/en/george_lucas"}, "starring": [{ "actor": { "name": null, "film": [{ "film": {"id": "/en/the_great_escape"} }] } }] }] Donald Pleasence THX 1138
  • 23.
  • 24.
    Reconciliation { "/type/object/name":"Blade Runner", "/type/object/type":"/film/film", "/film/film/starring/actor":["Harrison Ford", "Rutger Hauer"], "/film/film/director":"Ridley Scott", "/film/film/release_date_s":"1981" } [{ "id":"/guid/9202a8c04000641f8000000000009e89", "name":["Blade Runner", "Bladerunner"], "score":1.4320519, "match":true, "type":["/common/topic", "/film/film","/media_common/adapted_work", "/award/award_winning_work", ]}, { "id":"/guid/9202a8c04000641f80000000002643d0", "name":["Blade"], "score":0.48852453, "match":false, "type":["/common/topic", "/film/film", "/award/award_winning_work", "/award/award_nominated_work", ]} http://data.labs.freebase.com/recon/
  • 25.
  • 26.
    Topic API Shortcut to building Topic displays Two forms: basic (names, types, description) standard (basic + keys, properties) http://www.freebase.com/experimental/topic/standard?id=/en/ncis
  • 27.
    Geo Search API Semantic Spatial Semantic http://www.freebase.com/docs/geosearch
  • 28.
  • 29.
  • 30.
    Getting Started++ • Freebase Documentation Hub • http://www.freebase.com/docs • Developer Mailing List • http://lists.freebase.com/mailman/listinfo/freebase-discuss • http://freebase.markmail.org • Real Time help on IRC • Freenode #freebase • Freebase Happenings • http://blog.freebase.com • About the Graph Store • Google: "ACM SIGMOD schema last tuple store"