Your SlideShare is downloading. ×
Lightning Talk, Weber: Seeing is believing, but measuring is knowing: Quantifying the impact o…
Upcoming SlideShare
Loading in...5

Thanks for flagging this SlideShare!

Oops! An error has occurred.

Saving this for later? Get the SlideShare app to save on your phone or tablet. Read anywhere, anytime – even offline.
Text the download link to your phone
Standard text messaging rates apply

Lightning Talk, Weber: Seeing is believing, but measuring is knowing: Quantifying the impact o…


Published on

Nic Weber

Nic Weber

  • Be the first to comment

  • Be the first to like this

No Downloads
Total Views
On Slideshare
From Embeds
Number of Embeds
Embeds 0
No embeds

Report content
Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

No notes for slide


  • 1. <TongueInCheek>Seeing is believing, but measuring is knowing</TongueInCheek> Nic WeberThursday, April 4, 13
  • 2. 2 thingsThursday, April 4, 13
  • 3. First thingThursday, April 4, 13
  • 4. There is nothing “soft” about metric incentivesThursday, April 4, 13
  • 5. History of Political Economy 20:4 Science is driven by curiosity, 0 1988 by Duke University Press CCC 00 18-2702/88/$1.50 The market for fame and fortune AND fame and fortune David M.Levy The Problem The modem discussion of the economics of fame treats fame as an instru- ment to material income. This view I should like to contrast with the classical Greek view of the maker. While the classical Greek discussions sometimes do look upon fame as a means to material income, fame itself Science as a gift is seen as a good thing; indeed, it is such a good thing that it can allow mortal man to rival even the undying gods themselves. Although fame cannot compensate for death, fame is the closest mortal man can come to real immortality.’ If fame and material income are both ultimate ends, giving economy might not there be a tradeoff between the two? The thrust of the article is to explore the possibility that there is a com- plicated tradeoff between scientific fame and material income. The specific problem addressed is whether scientists in an amateur period of their dis- cipline will behave differently than scientists in a professional period. The Model Modem economic research on the incentives to publish economic ar- ticles has established an important point which adds considerably to the Greek discussion. Scientific fame can be viewed as an investment: one Correspondence may be addressed to the author, Center for Study of Public Choice, George Mason University, Fairfax VA 22030. 1. Vermeule (1979, p 23): “Unfortunately, the psyche usually lost creative powers after death, and so was helpless in comparison with divinity and nature. This was the source of the familiar old quarrel between Simonides and Kleoboulos of Lindos. Kleoboulos thought men could make things-a bronze statue, a carved stele, a poem-which would not waste away or be extinguished ‘so long as water flows or the tall trees bloom, so long as the sun goes up to shine, and the brilliant moon, so long as rivers flow and the sea-waves surge on the shore.’ That was immortal memory, fame, kleos uphthiton, the oldest ambition in Greek poetry. Creativity was a magic drug, a pharmukon against death and against being forgot- ten. Simonides though he saw a tougher truth, that rivers, flowers, sun, moon and waves were immeasurably stronger than human creation because they were divine. It was idiocy to ‘set the strength of a stele against the flame of the sun and the gold moon and the whirling sea’ because our work is inescapably mortal. Still, in the short time between the Greeks and ourselves, rivers have dried up and oceans shifted while many Greek pharmaka against death are still potent.” 2. My interest in this problem was stimulated by Stigler 1949. Stigler rightly observed that the classical economists focused on a different sort of problem than did those econo- mists who followed. 615 Google “Not Hagstrom”Thursday, April 4, 13
  • 6. Vanity (fame) metrics are powerfulThursday, April 4, 13
  • 7. Don’t we already do this?Thursday, April 4, 13
  • 8. Sort of...Thursday, April 4, 13
  • 9. We don’t do it that wellThursday, April 4, 13
  • 10. We certainly don’t do thisThursday, April 4, 13
  • 11. For the most part, we treat repository metrics like this not like thisThursday, April 4, 13
  • 12. Second thingThursday, April 4, 13
  • 13. To do this well we should focus on... Usage Metrics != Data Citation MetricsThursday, April 4, 13
  • 14. Hold up...Thursday, April 4, 13
  • 15. “We’ve been promoting data citations for this!!!!!!!!!!!!”Thursday, April 4, 13
  • 16. GoodThursday, April 4, 13
  • 17. other people are on that...Thursday, April 4, 13
  • 18. but citations (alone) won’t do that... (no no, no they won’t do that)Thursday, April 4, 13
  • 19. Thursday, April 4, 13
  • 20. The Data Usage IndexThursday, April 4, 13
  • 21. Indicator Explanation Coincident Unique Users (UU) Unique users that downloaded data during a time window Unique Users - Advanced UUs that accessed data programmatically Unique Users - Assisted UUs that accessed data via GUI or Service Number of Datasets Number of Datasets assigned DS number Files DS Number of files in Dataset per time window Download Frequency Total number of files downloaded per time window Download Frequency - Advanced Files downloaded by Advanced users Leading Download Frequency - Assisted Files downloaded by Assisted users Homepage Hits Dataset Homepage Hits per time window Homepage Hits - Direct Access Dataset Homepage Hits per time window by users with direct access (link not indexed or retrieved by search) Homepage Hits - With Link Dataset Homepage Hits per time window by users with link (from indexed list or retrieved by search) Subset Requests Subsets Requests per time window Download Density Average number of files downloaded per UU Usage Impact Total number of downloaded files over total files in dataset Usage Impact - Advanced  “ Lagging Usage Impact - Assisted  “ Interest Impact Total homepage hits per number of files in dataset Usage Balance Files downloaded by number of homepage hits per time window Subset Ratio Number of subset requests over total number files downloaded per time window Secondary Interest Impact Homepage over UUThursday, April 4, 13
  • 22. And the thing about usage metrics are...Thursday, April 4, 13
  • 23. ... (alone) they can measure curation impact 18" 300" Changeindownload 16" frequency(3monthskip) Usage&Impact&(3&Month&Skip)&& 250" 14" Usage&Impact&Score&& 12"Numberoffilesdownloaded(n=433) 200" Files"downloaded"by"Advanced"users" 10" ds083.2" Files"downloaded"by"Assisted"users" ds083.2","advanced" 150" 8" ds083.2","assisted" Subsets"Requests"of"Dataset"Per"?me" window" 6" 100" 4" 50" 2" 0" 0"Thursday, April 4, 13
  • 24. Thanks! @nniiiccThursday, April 4, 13