• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Big Data, We Have a Communication Problem
 

Big Data, We Have a Communication Problem

on

  • 25,505 views

Big Data, We Have a Communication Problem ...

Big Data, We Have a Communication Problem

by Daniel Tunkelang

Presented on April 30, 2013 at the TTI/Vanguard Conference on Ginormous Systems
http://www.ttivanguard.com/conference/2013/ginormous.html

It's a cliché that we live in a world of Big Data. But the bottleneck in understanding data is not computational. Rather, the biggest challenge is designing technical solutions that effectively leverage human cognitive ability. Data analysis systems should augment people's capabilities rather than replace them. This argument is as old as computer science itself: in 1962, Doug Engelbart said that the goal of technology is “the enhancement of human intellect by increasing the capability of a human to approach a complex problem situation.” Algorithms extract signal from raw data, but people fill in the gaps, creating models and evaluating analyses.

Empowering people to understand data is not just a surface problem of building better interfaces and visualizations. We need to interact with data not only after performing computational analysis, but throughout the analysis process in order to improve our models and algorithms. In order to do so, we need tools and processes specifically designed to offer people transparency, guidance, and control.

Human-computer information retrieval has been revolutionizing our approach to information seeking -- no modern search engine limits users to black-box relevance ranking and ten blue links. We need to take similar steps in our analysis of big data, making people the center of the analysis process and developing the technical innovations that enable people to fulfill this role.

Statistics

Views

Total Views
25,505
Views on SlideShare
7,763
Embed Views
17,742

Actions

Likes
14
Downloads
79
Comments
1

24 Embeds 17,742

http://booleanblackbelt.com 12727
http://www.sourcecon.com 2816
http://www.linkedin.com 1462
http://feeds.feedburner.com 454
https://www.linkedin.com 88
http://specialedition.linkedin.com 87
http://www.google.com 40
http://newsblur.com 14
https://www.rebelmouse.com 10
http://www.newsblur.com 9
http://www.pcrecruiter.net 8
https://twitter.com 8
http://translate.googleusercontent.com 4
https://www.google.com 2
http://feedly.com 2
http://webcache.googleusercontent.com 2
http://www.mediaredefined.com 2
http://booleanblackbelt.com. 1
http://feedproxy.google.com 1
http://pulse.me&_=1367589651443 HTTP 1
http://www.feedspot.com 1
https://www.google.co.jp 1
http://pulse.me&_=1367547363202 HTTP 1
http://www.google.de 1
More...

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel

11 of 1 previous next

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Big Data, We Have a Communication Problem Big Data, We Have a Communication Problem Presentation Transcript

    • BIG DATA,We have a communication problem.GINORMOUS SYSTEMSApril 30–May 1, 2013Washington, D.C.Daniel TunkelangHead of Query Understanding, LinkedIn
    • BIG DATA IS EVERYWHERE
    • BIG DATA POWERS EVERYTHING
    • DATA SCIENTISTS WORRY ABOUTVOLUME, VELOCITY, VARIETY, …
    • BUT THE BOTTLENECKISN’T COMPUTATIONALIT’S COGNITIVE
    • TOOLS AUGMENTHUMAN INTELLECTBIG DATA IS A TOOLDoug Engelbart, inventor ofthe mouse, hypertext, etc.
    • NOT EVERYONE SUBSCRIBESTO THIS POINT OF VIEW…Claudia Perlich, Chief Scientist of media6degrees, speaking atTTI/Vanguard 2012 Conference on Understanding Understanding:
    • SHE HAS A POINT
    • BUT PREDICTIVE MODELINGIS NOT ENOUGH
    • TRAININGDATA?OBJECTIVEFUNCTION?
    • WE NEED APEOPLE-CENTRICAPPROACH TOBIG DATAINTERPRETABILITYINTERACTIONINSIGHT
    • LET’S START WITHINTERPRETABILITY
    • EXAMPLE:SVMvs.DECISION TREE
    • DECISION TREES HAVE FLAWS…DISCRETE
    • BUT THEYCOMMUNICATE(if they’re shallow)early splits providebig picture…fat leaves guidefeature engineering…or reveal trainingdata problems
    • ITERATION
    • INTERPRETABILITY DELIVERSKey search leader favors rule-basedapproach for key scoring algorithms.Replaced regression with decisiontree in local search model: gainedaccuracy and insight.Using trees to recognize spam,analyze search abandonment,model / quantify social proof.
    • GO DEEP vs INTERPRETABILITYA KEY DATA SCIENCE TRADE-OFF
    • ON TOINTERACTION
    • DON’T OVERPAY FOR PRECISION
    • BE FAST, CHEAP, AND 98% RIGHThttp://metamarkets.com/2012/fast-cheap-and-98-right-cardinality-estimation-for-big-data/
    • ARE PEOPLE THAT IMPATIENT?tolerable wait time for web users0.1s increase in latency significantlyreduces # of searches, ad revenuetl;dr: YES
    • IMPATIENCE IS GOODSPEED MATTERS
    • INSIGHT
    • http://blog.takejune.com/archives/52334044.html
    • BE TRENDY AND NORMALIZEvs
    • Sept. 11thAbu GhraibWeapons InspectorsSOLVE FOR INTERESTINGNESS
    • COMPUTE POTENTIAL INSIGHTSAPPLY HUMAN INTUITION
    • SUMMARY: Let’s have aconversation with Big Data.INTERPRETABILITYINTERACTIONINSIGHT