• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
1M.10M.100M. Data! - @mrogati's talk at Strata 2011
 

1M.10M.100M. Data! - @mrogati's talk at Strata 2011

on

  • 3,603 views

Follow along w/ the video: http://www.youtube.com/watch?v=2SQ0O_oPpe4 ...

Follow along w/ the video: http://www.youtube.com/watch?v=2SQ0O_oPpe4

How do data infrastructure, insights and products change when your user base grows by orders of magnitude? When should you move your user-facing data product off your laptop? (hint: now!) Does your data offer insights about the world at large, or is it just mirroring your early adopters? In this talk, I will share some of the data scaling lessons we've learned at LinkedIn, recount war stories (and close calls!) and document the evolution of the data scientist.

Statistics

Views

Total Views
3,603
Views on SlideShare
3,201
Embed Views
402

Actions

Likes
3
Downloads
1
Comments
1

8 Embeds 402

https://twitter.com 253
http://seenthis.net 94
http://www.linkedin.com 27
http://a0.twimg.com 11
http://us-w1.rockmelt.com 10
https://www.linkedin.com 4
http://tweetedtimes.com 2
http://tweets.kissmetrics.com 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

11 of 1 previous next

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • 1M.10M.100M. Data! : How do data infrastructure, insights and products change when your user base grows by orders of magnitude.
  • Scale changes your problems, your solutions and your mindset.
  • Scale changes your problems, your solutions and your mindset.
  • Hardware, software/technology, people and everywhere
  • You can truly chase the long tail.

1M.10M.100M. Data! - @mrogati's talk at Strata 2011 1M.10M.100M. Data! - @mrogati's talk at Strata 2011 Presentation Transcript

  • @mrogati
  • Scale changes what’s possible.
    Scale changes what’s possible.
    Scal
  • 2004-2006
  • Possible :
    High risk,
    rapid innovation
    Chasing the long tail
    (by hand)
    Not possible:
    Long tail recommendations
    Network effects
    Insights into the world at large
  • The
    Data
    Scientist
    is born
    -- LinkedIn job ad, April 2008
  • Data products
    infrastructure innovation
    and adoption
  • Data products
    infrastructure innovation
    and adoption
    Voldemort
    Azkaban
  • Data products
    infrastructure innovation
    and adoption
  • 1999
    software engineer
    web developer
    2001
    research assistant
    PhD student
    2008
    game artists
    Insights: beyond the early adopters
  • Scale changes what’s possible.
    Scale changes what’s possible.
    Sca
  • Possible :
    Insights into the
    world at large
    Network effects
    Infrastructure innovation
    Not possible:
    Long tail recommendations
    Segmented insights and products
  • Data infrastructure team!
    ~1900
    machines
    Kafka
    real time data streams
    Reporting
    there’s a (mobile) app for that!
    … and servers, and dedicated teams
    Infrastructure – evolved.
  • Insights: The world – sliced & diced
  • Insights: The world – sliced & diced
  • Insights:
    The world
    – sliced & diced
  • Data Products: Chasing the long tail
  • Data Products: Personalized insights
  • Data Products:
    Crowdsourced Insights
  • Scale changes what’s possible.
    Scale changes what’s possible.
    Scal
  • Possible :
    Sliced-and-diced
    insights and
    products
    Network effects
    Economies of scale
    Fast A/B tests
    Not possible:
    Casual, hour-long outages
    Testing in production on 100% of users
  • The
    Data
    Scientist
    is born
    -- LinkedIn job ad, April 2008
  • The
    Data
    Scientist
    & the teenage years
    +
    -- LinkedIn job ad,
    September 2011
  • Scale changes what’s possible.
    Scale changes what’s possible.
    Scal
    @mrogati