Developing Data Products
Upcoming SlideShare
Loading in...5
×
 

Developing Data Products

on

  • 6,852 views

Examples, techniques, and lessons learned building data products over the last 3 years at LinkedIn. ...

Examples, techniques, and lessons learned building data products over the last 3 years at LinkedIn.

Pete Skomoroch is a Principal Data Scientist at LinkedIn where he leads a team focused on building data products leveraging LinkedIn's powerful identity and reputation data.

The talk describes some techniques and best practices applied to develop products like LinkedIn Skills & Endorsements.

This was the inaugural UberData Tech Talk, held in SF at Uber HQ.

Statistics

Views

Total Views
6,852
Views on SlideShare
6,561
Embed Views
291

Actions

Likes
17
Downloads
100
Comments
1

13 Embeds 291

https://twitter.com 123
http://www.linkedin.com 109
https://www.linkedin.com 26
http://tweetedtimes.com 11
http://twitter.com 7
http://www.twylah.com 7
http://www.datawrangling.com 2
http://getpocket.com 1
https://twimg0-a.akamaihd.net 1
http://www.diffbot.com&_=1361650883176 HTTP 1
http://localhost 1
http://moderation.local 1
http://datawrangling.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

11 of 1

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Mission: For us, fundamentally changing the way the world works begins with our mission statement: To connect the world’s professionals to make them more productive and successful. This means not only helping people to find their dream jobs, but also enabling them to be great at the jobs they’re already in. Vision: But, we’re just getting started. By our measure,there are more than 640 million professionals in the world. And roughly 3.3 billion people in the global workforce. Ultimately, our vision is to create economic opportunity for every professional, which we believe is an especially crucial objective in light of current macroeconomic trends.Our most important core value is that members come first.

Developing Data Products Developing Data Products Presentation Transcript

  • Developing Data ProductsUber Tech TalkPete Skomoroch @peteskomorochDecember 5 2012©2012 LinkedIn Corporation. All Rights Reserved.
  • Examples, Techniques, & Lessons LearnedDeveloping Data Products
  • Our Mission Connect the world’s professionals to make them more productive and successful.Our VisionCreate economic opportunity for every professional in the world.Members First!
  • LinkedIn is the leading professional network site 187M+ 1 LinkedIn Members 2 640M+ Worldwide Professionals 2 3,300M+ Worldwide Workforce©2012 LinkedIn Corporation. All Rights Reserved. 4
  • LinkedIn profiles represent our professional identity 1 2 187M Members 187M Member Profiles©2012 LinkedIn Corporation. All Rights Reserved. 5
  • We have a lot of data.©2012 LinkedIn Corporation. All Rights Reserved.
  • We have a lot of data. And (like everyone else), we store it in Hadoop.©2012 LinkedIn Corporation. All Rights Reserved.
  • We have a lot of data. And (like everyone else), we store it in Hadoop. And people build awesome things with that data.©2012 LinkedIn Corporation. All Rights Reserved.
  • What do we mean by dataproducts?
  • Building products from data at LinkedInA few examples: People You May Know Skills and Endorsements Year in Review Network Updates Digest InMaps Who’s viewed my profile Collaborative Filtering Groups You May Like and more…©2012 LinkedIn Corporation. All Rights Reserved.
  • Collaborative Filtering: LinkedIn Skill Pages©2012 LinkedIn Corporation. All Rights Reserved.
  • Classification: giving structure to unstructured data Extract©2012 LinkedIn Corporation. All Rights Reserved.
  • Clustering & Disambiguation©2012 LinkedIn Corporation. All Rights Reserved.
  • De-duplication and Normalization©2012 LinkedIn Corporation. All Rights Reserved.
  • Network Algorithms: Relevance & Ranking©2012 LinkedIn Corporation. All Rights Reserved. 15
  • Prediction: Personalized Skill Recommendations©2012 LinkedIn Corporation. All Rights Reserved.
  • Skill Endorsements©2012 LinkedIn Corporation. All Rights Reserved.
  • Social Proof and the Skill Endorsement Graph©2012 LinkedIn Corporation. All Rights Reserved. 20
  • The Economic Graph: Skills, Jobs, People, Locations… Location©2012 LinkedIn Corporation. All Rights Reserved. 21
  • Lessons learned developing dataproducts
  • Collect the right data at the right time
  • Large amounts of data can reveal new patterns Probability of Job Title Months since graduation©2012 LinkedIn Corporation. All Rights Reserved. 24
  • Be wary of “black-box” approaches©2012 LinkedIn Corporation. All Rights Reserved. 25
  • Look at your data©2012 LinkedIn Corporation. All Rights Reserved. 26
  • Aggregate statistics can be misleading 12 10 8 6 4 2 0 1 2 3 4 5 6 7 8 9 10©2012 LinkedIn Corporation. All Rights Reserved. 27
  • Build a viewer app, “micro-listen”©2012 LinkedIn Corporation. All Rights Reserved. 28
  • Algorithmic intuition: include data geeks in design©2012 LinkedIn Corporation. All Rights Reserved. 29
  • OODA: Think like a jet fighter©2012 LinkedIn Corporation. All Rights Reserved. 30
  • OODA: Observe, Orient, Decide, Act©2012 LinkedIn Corporation. All Rights Reserved. 31
  • OODA: The speed you can move determines victory©2012 LinkedIn Corporation. All Rights Reserved. 32
  • Red teaming: what can go wrong likely will©2012 LinkedIn Corporation. All Rights Reserved. 33
  • Error data is super valuable, analyze it and adapt©2012 LinkedIn Corporation. All Rights Reserved. 34
  • Conclusion: tips for developing data products Collect the right data at the right time Large amounts of data can reveal new patterns Be wary of “black box” approaches Look at your raw data Aggregate statistics can be misleading Build and use viewer apps Include data geeks in design process OODA: Think like a jet fighter Red-teaming: anticipate edge cases Find opportunity in your error data©2012 LinkedIn Corporation. All Rights Reserved.
  • Questions?More info: data.linkedin.com@peteskomoroch©2012 LinkedIn Corporation. All Rights Reserved. 36