ITEC, Klagenfurt University, Austria

What is a Hype and Where Can I Get One? Mathias Lux [email_address] Department for Information Technology, Klagenfurt University, Austria

What is this about … Power Laws & Pareto Distributions Just a Theory? Conclusions ITEC, Klagenfurt University, Austria by betta_design http://www.flickr.com/photos/betta_design/2200198472/

Power Laws & Pareto Distributions

Just a Theory?

Conclusions

The Long Tail Common for certain distributions Zipf‘s Law Power Law Pareto Distribution In Web 2 Context Chris Anderson … ITEC, Klagenfurt University, Austria maitland 82 - http://www.flickr.com/photos/maitland82/346065497/

Common for certain distributions

Zipf‘s Law

Power Law

Pareto Distribution

In Web 2 Context

Chris Anderson …

Zipf‘s Law Few events occur often, many occur rarely P n ~ 1/n a ... Frequency of the n th ranked item, a close to 1. Prominent examples Ranking of words in documents Ranking of cities and their size Ranking of movies and sold cinema tickets … and many more ITEC, Klagenfurt University, Austria

Few events occur often, many occur rarely

P n ~ 1/n a ... Frequency of the n th ranked item, a close to 1.

Prominent examples

Ranking of words in documents

Ranking of cities and their size

Ranking of movies and sold cinema tickets

… and many more

Zipf‘s Law Plot of the word frequency in Wikipedia Most popular: the, of, and from http://en.wikipedia.org/wiki/Zipf's_law ITEC, Klagenfurt University, Austria

Plot of the word frequency in Wikipedia

Most popular: the, of, and

from http://en.wikipedia.org/wiki/Zipf's_law

Pareto Distribution 80:20 Rule Economics Continous (Zipf is discrete) Practical issues Time Management, … ITEC, Klagenfurt University, Austria

80:20 Rule

Economics

Continous (Zipf is discrete)

Practical issues

Time Management, …

Power Law Made famous by Albert Barabasi Scale free networks (web, power supply, …) In-degree of web sites, etc. Defines actually a class of distributions f(x)=a*x^b + e Pareto and Zipf are part of the group ITEC, Klagenfurt University, Austria

Made famous by Albert Barabasi

Scale free networks (web, power supply, …)

In-degree of web sites, etc.

Defines actually a class of distributions

f(x)=a*x^b + e

Pareto and Zipf are part of the group

How to detect a power law? Simple empirical tests Draw points on a log-log plot Is it a „straight line“? ITEC, Klagenfurt University, Austria

Simple empirical tests

Draw points on a log-log plot

Is it a „straight line“?

How to detect a power law? Statistical Means E.g. KS-Test, Chi-Square Test Open research issue … See e.g. Clauset, A., Shalizi, C.R., Newman, M.E.J.: Power-law distributions in empirical data. arXiv:0706.1062v1 (2007) ITEC, Klagenfurt University, Austria

Statistical Means

E.g. KS-Test, Chi-Square Test

Open research issue …

See e.g. Clauset, A., Shalizi, C.R., Newman, M.E.J.: Power-law distributions in empirical data. arXiv:0706.1062v1 (2007)

A note on plots … ITEC, Klagenfurt University, Austria Taken from phun.org, tnx to enzo nadrag

A note on statistical means … http://www.phun.org/newspics/funny_friday/2538.jpg tnx to Enzo Nadrag ITEC, Klagenfurt University, Austria

http://www.phun.org/newspics/funny_friday/2538.jpg tnx to Enzo Nadrag

Zipf, Pareto & Power Law: Conclusions They emerge when people are involved They have interesting characteristics Mean has virtually no information Area under the curve (cp. amazon’s long tail strategy) Power laws emerge somehow … Multiple generative models (preferntial attachement, memory kernels, etc.) No one knows for sure ITEC, Klagenfurt University, Austria

They emerge when people are involved

They have interesting characteristics

Mean has virtually no information

Area under the curve (cp. amazon’s long tail strategy)

Power laws emerge somehow …

Multiple generative models (preferntial attachement, memory kernels, etc.)

No one knows for sure

Is this just theory? Basically: YES! But there are related practical questions Are you using Flickr? How many “interesting” photos did you publish? How many views do your photos have? Imagine you publish a video on YouTube What are the chances that your video is a big hit? How to “help out” the process of getting a big hit? Can one distinguish between hit or flop? ITEC, Klagenfurt University, Austria

Basically: YES!

But there are related practical questions

Are you using Flickr?

How many “interesting” photos did you publish?

How many views do your photos have?

Imagine you publish a video on YouTube

What are the chances that your video is a big hit?

How to “help out” the process of getting a big hit?

Can one distinguish between hit or flop?

Is this just theory? (2) More related practical questions Do you have a website? How to “flat out” resource popularity? How select popular resources (e.g. for caching, adaptation, preprocessing)? ITEC, Klagenfurt University, Austria

More related practical questions

Do you have a website?

How to “flat out” resource popularity?

How select popular resources (e.g. for caching, adaptation, preprocessing)?

Big hits on YouTube ITEC, Klagenfurt University, Austria © 2007 by Aigner Thomas and Oraze Manuel

Getting popular … Starting with the first view (user) Some other users find the same resource They point other to it Blogging, Digging, word of mouth Multiplicator of information – cp. Metcalfe’s law Number of views (users) “explodes” ITEC, Klagenfurt University, Austria

Starting with the first view (user)

Some other users find the same resource

They point other to it

Blogging, Digging, word of mouth

Multiplicator of information – cp. Metcalfe’s law

Number of views (users) “explodes”

Some graphs … ITEC, Klagenfurt University, Austria Data from del.icio.us Shows bookmarks / day relative user count

Data from del.icio.us

Shows

bookmarks / day

relative user count

Observations There is an initial bend in the curve The mean user # at the bend is rather small Around 50 There are outliers Google Video was doomed to be a success ITEC, Klagenfurt University, Austria

There is an initial bend in the curve

The mean user # at the bend is rather small

Around 50

There are outliers

Google Video was doomed to be a success

Conclusions If there is a bend … Chances are better for a big hit. Time is still an issue Slow start, long vs. short hype, etc. Resources without this bend: Better Chances that they are shelf warmers Decision support for portfolio adaptation ITEC, Klagenfurt University, Austria

If there is a bend …

Chances are better for a big hit.

Time is still an issue

Slow start, long vs. short hype, etc.

Resources without this bend:

Better Chances that they are shelf warmers

Decision support for portfolio adaptation

The Flickr way Flickr defined “Interestingness” Patented combining views, comments, age, etc. Interesting photos are presented Users see new photos Not all photos (2.000 new / minute, checked Feb. 1 2008, ~ 11.oo UTC) They have no “big hit” ITEC, Klagenfurt University, Austria Kudos given to Horst Gutmann and Marian Kogler

Flickr defined “Interestingness”

Patented

combining views, comments, age, etc.

Interesting photos are presented

Users see new photos

Not all photos (2.000 new / minute, checked Feb. 1 2008, ~ 11.oo UTC)

They have no “big hit”

The YouTube way Smaller resource data base than Flickr Around 45 videos a day (65.000 a day) But a lot more views (data Feb. 1st, 08) 73.245.607 for „Evolution of Dance“ 20 most viewed have > 30M views Not obvious counter strategy Might not (yet) be necessary ITEC, Klagenfurt University, Austria

Smaller resource data base than Flickr

Around 45 videos a day (65.000 a day)

But a lot more views (data Feb. 1st, 08)

73.245.607 for „Evolution of Dance“

20 most viewed have > 30M views

Not obvious counter strategy

Might not (yet) be necessary

Digg Assumption: Diggs also follow a power law Quite reasonable … How to avoid the Digg- effect? Digg has a mirror … ITEC, Klagenfurt University, Austria

Assumption: Diggs also follow a power law

Quite reasonable …

How to avoid the Digg- effect?

Digg has a mirror …

Thanks ... ... for your attention You are interested? Then talk to me … ITEC, Klagenfurt University, Austria by Gexydaf http://www.flickr.com/photos/gexydaf/2208215419/

... for your

attention

You are interested?

Then talk to me …

Mathias Lux Affiliation Klagenfurt University, ITEC Contact mathias @ juggle.at http://www.semanticmetadata.net ITEC, Klagenfurt University, Austria

Affiliation

Klagenfurt University, ITEC

Contact

mathias @ juggle.at

http://www.semanticmetadata.net