Glue Conference

662 views

Published on

Published in: Technology, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
662
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
2
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide

Glue Conference

  1. 1. Integrating Disparate Data May 27, 2010<br />Steve Newman – CTO/Gist.com<br />
  2. 2. the WHY? What we believe in…<br />All your important people already reside in email, calendar, contact lists, social sites<br />The web is a rich source of information about the people you care about<br />One tool should exist that can pull all this together in a single, rich, integrated experience<br />
  3. 3. Pain Points (External)<br />Disparate Data/API sources and protocols<br />e.g. GNIP<br />Change notification (when/what)<br />e.g. Linked Open Data Dataset Dynamics, pubsubhub<br />Standard entity data structures<br />e.g. Portable Contacts, vcard, hcard<br />3<br />
  4. 4. The Problem (Internal)<br />Need a single, disambiguated set of entities where an entity itself contains accurate/disambiguated attributes<br />Entity attributes can be sourced from one or more endpoints<br />Email<br />Twitter/Facebook <br />Calendar<br />Google Contacts, Outlook Contacts, Plaxo<br />Google Social Graph API<br />Rapleaf API<br />
  5. 5. The Problem (Internal)<br />Now that we have this data, we need to process and make sense of it<br />Need to support reoccurring updates<br />Merge and unmerge support<br />Recursive derivation is a huge win if done correctly<br />Historical Tracking is necessary both to drive operations but also for debugging (and it’s a cool user feature)<br />
  6. 6. How we did it<br />Enhancers<br />Execute the request and creation of attribute data<br />Can be called synch or asynch<br />Cached, Logged, Rate Limited<br />Meta data about attributes<br />Source, Source Type, When created, Derived?, Derived Source, Score<br />Rules for ‘enhancement’<br />Rules for recursion<br />Scoring methodology (accuracy and relative prioritization)<br />6<br />
  7. 7. Example – Email Enhancer<br />“Brad Feld” vs “Brad”<br />Data/Time<br />Score<br />State<br />Value<br />
  8. 8. Key Takeaways<br />Worry about integration both external and internal to your application<br />Lots of good work on the external issues…take advantage of it!<br />Create a strong object model for internal data representation (workers, meta data, engines) so you can perform concise/discrete operations<br />
  9. 9. Additional Info<br />GIST API coming out this Summer<br />Direct interface to Fragments <br />Standard and Third party Enhancer support<br />@stevepnewman, @gist <br />
  10. 10. « We know now that the source of wealth is something specificallyhuman : knowledge. Applied to tasksthatwealready know how to do, itbecomes'productivity'. Applied to tasksthat are new and differentwe call it'innovation'. Onlyknowledgeallows us to achievethesetwo goals. »<br />Peter Drucker<br />Management challenges of the XXIst Century-1999<br />

×