Rabj freebase all
Upcoming SlideShare
Loading in...5
×
 

Rabj freebase all

on

  • 1,286 views

 

Statistics

Views

Total Views
1,286
Views on SlideShare
1,286
Embed Views
0

Actions

Likes
1
Downloads
12
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Rabj freebase all Rabj freebase all Presentation Transcript

  • The Anatomy of a Large-Scale Human-Computation Engine Shailesh Kochhar, Stefano Mazzocchi, Praveen Paritosh Freebase August Meetup
  • 1: Freebase & Human Computation 2: Example – Stanford Library 3: RABJ 4: Consensus Aug 18, 2010 Freebase Meetup
  • Freebase Structured database 12 MM entites, 300 MM triples/facts Aug 18, 2010 Freebase Meetup
  • Where does the data come from? Aug 18, 2010 Freebase Meetup
  • Community contributions Mass Data Loads Aug 18, 2010 Freebase Meetup
  • Human Judgments Improve Both Aug 18, 2010 Freebase Meetup
  • Community Simplify contribution through games Aug 18, 2010 Freebase Meetup
  • http://typewriter.freebaseapps.com/ Aug 18, 2010 Freebase Meetup
  • Community Simplify contribution through games Enable QA for Gridworks loads Aug 18, 2010 Freebase Meetup
  • Aug 18, 2010 Freebase Meetup
  • Mass Data Loads Precision: QA for >99% accuracy Aug 18, 2010 Freebase Meetup
  • Book Edition QA Aug 18, 2010 Freebase Meetup
  • Mass Data Loads Precision: QA for >99% accuracy Coverage: Manual reconciliation Aug 18, 2010 Freebase Meetup
  • matchmaker http://matchmaker2.freebaseapps.com/ Aug 18, 2010 Freebase Meetup
  • 1: Freebase & Human Computation 2: Example – Stanford Library 3: RABJ 4: Consensus Aug 18, 2010 Freebase Meetup
  • Reconcile Stanford Library Catalog with freebase.com Aug 18, 2010 Freebase Meetup
  • Stanford Library Catalog 4.4MM book editions 1.3MM English book editions 1.2MM English books 600K authors Aug 18, 2010 Freebase Meetup
  • For freebase, identity is key match books, match authors Aug 18, 2010 Freebase Meetup
  • Automatic matching insufficient Trained judges needed to decide hard cases Aug 18, 2010 Freebase Meetup
  • How to get this? Aug 18, 2010 Freebase Meetup
  • RABJ Redundant Array of Brains in a Jar Aug 18, 2010 Freebase Meetup
  • What? Abstraction Powers human judgment (HJ) applications 3.1MM judgments in 16 months Aug 18, 2010 Freebase Meetup
  • Provides primitive elements for more sophisticated applications Aug 18, 2010 Freebase Meetup
  • Questions Judgments Queues Agents Aug 18, 2010 Freebase Meetup
  • Design Constraints Aug 18, 2010 Freebase Meetup
  • Content-agnostic Dynamic data Low latency Aug 18, 2010 Freebase Meetup
  • Architecture Aug 18, 2010 Freebase Meetup
  • Questions contain metadata, pointers to dynamic content Questions added to queues Metadata allows slicing and dicing Aug 18, 2010 Freebase Meetup
  • Acre applications pull questions from RABJ RABJ matches judge to available tasks Acre renders question, sends judgment back Aug 18, 2010 Freebase Meetup
  • Declarative consensus Yes: 3, No: 3, Skip: 4, Invalid: 3, Max: 6 RABJ notifies agents when consensus is reached Aug 18, 2010 Freebase Meetup
  • Scale Aug 18, 2010 Freebase Meetup
  • 2.3 MM questions 3.1 MM judgments 500+ queues 20+ applications Aug 18, 2010 Freebase Meetup
  • 1: Freebase & Human Computation 2: Example – Stanford Library 3: RABJ 4: Consensus Aug 18, 2010 Freebase Meetup
  • Always have leftovers Aug 18, 2010 Freebase Meetup
  • Perfect Consensus? Not! Aug 18, 2010 Freebase Meetup
  • Evaluating QAers Aug 18, 2010 Freebase Meetup
  • Explore http://rabj.freebaseapps.com/explorer Create http://wiki.freebase.com/wiki/RABJ_Tutorial Reference http://wiki.freebase.com/wiki/RABJ_API/ Aug 18, 2010 Freebase Meetup
  • Questions? Aug 18, 2010 Freebase Meetup