Web science - How is it different?
Upcoming SlideShare
Loading in...5
×
 

Web science - How is it different?

on

  • 4,011 views

Web Science: How is it different? ...

Web Science: How is it different?

Daniel Tunkelang, LinkedIn
Keynote Address at ACM Web Science 2014 Conference

The scientific method of observation, measurement, and experiment may be our greatest achievement as a species. The technological innovation we enjoy today is the product of a culture of systematized scientific experimentation.

But historically scientific experimentation has been expensive. Experiments consumed natural resources, took a long time to conduct, and required even more time and labor to analyze. In order to be productive, scientists have had to factor these costs into their work and to optimize accordingly.

Web science is different. Not, as some have speciously argued, because big data has made the scientific method obsolete. The key difference is that web science has changed the economics of scientific experimentation. Thus, even as web scientists apply the traditional scientific method, they optimize based on very different economics.
In this talk, I'll survey how web science has changed our approach to experimentation, for better and for worse. Specifically, I'll talk about differences in hypothesis generation, offline analysis, and online testing.


Bio

Daniel Tunkelang is Head of Query Understanding at LinkedIn, where he previously formed and led the product data science team. LinkedIn search allows members to find people, companies, jobs, groups and other content. His team aims to provide users with the best possible results that satisfy their information needs and help to get insights from professional data. Tunkelang has BS and MS degrees in computer science and math from MIT, and a PhD in computer science from CMU. He co-founded the annual symposium on human-computer interaction and information retrieval (HCIR) and wrote the first book on Faceted Search (Morgan and Claypool 2009). Prior to joining LinkedIn, Tunkelang was Chief Scientist of Endeca (acquired by Oracle in 2011 for $1.1B) and leader of the local search quality team at Google, mapping local businesses to their home pages. He is the co-inventor of 20 patents.

Statistics

Views

Total Views
4,011
Views on SlideShare
3,191
Embed Views
820

Actions

Likes
16
Downloads
25
Comments
6

7 Embeds 820

https://www.linkedin.com 735
http://www.linkedin.com 48
https://twitter.com 25
http://feedly.com 5
http://www.google.com 4
http://www.inoreader.com 2
http://www.slideee.com 1
More...

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • Susu Kambing Etawa http://www.susukambings.com
    Susu Kambing Etawa http://www.susukambing.info
    Susu Kambing Etawa http://www.grosirsusukambing.com
    Are you sure you want to
    Your message goes here
    Processing…
  • 1, Main St.miLIXx:\xss''/s/-->%22%27"&apos
    Are you sure you want to
    Your message goes here
    Processing…
  • 1, Main St.
    Are you sure you want to
    Your message goes here
    Processing…
  • FREE to Join: earn Upto $2500/Referral

    Extremely Urgent : I got free in this new MLM prelaunch which pays upto $2500/referral, 10% Binary and launches on 7th August. Check it out : It’s very impressive ..

    http://vidcommx.com/@richguy

    OR
    bit.ly/1m6reRY
    Are you sure you want to
    Your message goes here
    Processing…
  • very nice
    http://pixgallery.org عکس
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • James Lind thought that scurvy was due to putrefaction of the body which could be helped by acids, and thus included a dietary supplement of an acidic quality in the experiment. This began after two months at sea when the ship was afflicted with scurvy. He divided twelve scorbutic sailors into six groups of two. They all received the same diet but, in addition, group one was given a quart of cider daily, group two twenty-five drops of elixir of vitriol (sulfuric acid), group three six spoonfuls of vinegar, group four half a pint of seawater, group five received two oranges and one lemon, and the last group a spicy paste plus a drink of barley water. The treatment of group five stopped after six days when they ran out of fruit, but by that time one sailor was fit for duty while the other had almost recovered. Apart from that, only group one also showed some effect of its treatment. <br />

Web science - How is it different? Web science - How is it different? Presentation Transcript

  • Daniel Web Science: How is it different? Daniel Tunkelang Head of Query Understanding
  • tl;dr: The scientific method is alive and well. Big data has just changed the economics.
  • How have the web and big data changed science? Let’s ask some of the experts.
  • “You have to kiss a lot of frogs to find one prince. So how can you find your prince faster? By finding more frogs and kissing them faster and faster.” Mike Moran Do It Wrong Quickly: How the Web Changes the Old Marketing Rules, 2007 Cited by Kohavi in Online Controlled Experiments at Large Scale, 2013
  • Web Science = faster, cheaper experiments.
  • “The cost of experimentation is now the same or less than the cost of analysis. You can get more value…by doing a quick experiment than from doing a sophisticated analysis.” Michael Schrage Value-Creation, Experiments, and Why IT Does Matter, 2010
  • Web Science = more experiments, less analysis?
  • “with massive data, this approach to science — hypothesize, model, test — is becoming obsolete… Petabytes allow us to say: "Correlation is enough." We can stop looking for models…analyze the data without hypotheses…throw the numbers into the biggest computing clusters the world…and let… algorithms find patterns where science cannot.” Chris Anderson The End of Theory, 2008
  • RIP Scientific Method 1600 BCE – late 20th century Killed by Big Data ?
  • No.
  • Let’s rewind.
  • What makes it science?
  • Hypothesis
  • Model
  • Test
  • The scientific method still works today. What’s changed is the economics.
  • Scientific Method 1747
  • Scientific Method Today
  • It’s the economy, science. Yesterday Experiments are expensive, choose hypotheses wisely. Today Experiments are cheap, do as many as you can!
  • What about Web Science?
  • A/B testing: everybody’s doing it.
  • Google: 20k search experiments per year
  • hypotheses
  • The Myth of Insight
  • Scientists gain insight by staring at data.
  • Big data tools improve data exploration.
  • In hypothesis generation, quantity trumps quality.
  • Except when it doesn’t.
  • Easier to analyze data than research humans.
  • But we pay the price. Example: search engine improvements in batch evaluations don’t always predict real user benefits. [Hersh et al, 2000] Do Batch and User Evaluations Give the Same Results? [Turpin & Hersh, 2001] Why Batch and User Evaluations do not Give the Same Results [Turpin, Scholer, 2006] User Performance versus Precision Measures for Simple Search Tasks But also see… [Smucker & Jethani, 2010] Human Performance and Retrieval Precision Revisited
  • When local optimization is cheap, you neglect the rest.
  • To summarize: how is web science different? • Online testing is cheaper and scalable. • Data exploration tools make hypothesis generation cheaper and easier. • But the experiments that are easy and cheap aren’t always the most valuable. • Easy to forget our biases as scientists.
  • Take-Aways • The scientific method is alive and well. Big data has just changes the economics. • Cheaper hypothesis testing and generation has already been transformative.That’s why big data matters. • But we neglect the human side of scientific experimentation at our peril.
  • Daniel Tunkelang dtunkelang@linkedin.com https://linkedin.com/in/dtunkelan g