1. Aristotle University, Department of Mathematics Master in Web Science supported by Municipality of Veria Stochastic Modeling of Web Evolution S. Amarantidis, I. Antoniou, M. Vafopoulos Mathematics Department Aristotle University of Thessaloniki SMTDA 2010 Chania, Crete, Greece

2. Contents What is the Web? What are the issues and problems? The Web as a Complex System Query-Web Models Stochastic Models and the Web 2

3. What is the Web? Internet ≠ Web Web: a system of interlinked hypertext documents (html) with unique addresses (URI) accessed via the Internet (http) 3

4. Web milestones 4 1992: Tim Berners-Lee presents the idea in CERN 1993: Dertouzos (MIT) and Metakides (EU) create W3C appointing TBL as director

5. Why the Web is so successful? Is based on architecture (HTTP, URI, HTML) which is: simple, free or cheap, open source, extensible tolerant networked fun & powerful universal 5

6. Why is so successful? New experience of exploring & editing huge amount of information, people, abilities anytime, from anywhere The biggest human system with no central authority and control but with log data (Yotta* Bytes/sec) Has not yet revealed its full potential… *10248 6

7. We knew the Web was big... 1 trillion unique URIs(Google blog 7/25/2008) 2 billion users Google: 300 million searches/day US: 15 billion searches/month 72% of the Web population are active on at least 1 social network … 7 Source blog.usaseopros.com/2009/04/15/google-searches-per-day-reaches-293-million-in-march-2009/

8. Web: the new continent Facebook: 400 million active users 50% of our active users log on to Facebook in any given day 35 million users update their status each day 60 million status updates posted each day 3 billion photos uploaded to the site each month Twitter: 75 million active users 141 employees Youtube: 350 million daily visitors Flickr: 35 million daily visitors 8

9. Web: the new continent Online advertising spending in the UK has overtaken television expenditure for the first time [4 billion Euros/year] (30/9/2009, BBC) In US, spending on digital marketing will overtake that of print for the first time in 2010 Amazon.com: 50 million daily visitors 60 billion dollars market capitalization 24.000 employess 9

10. Web generations 10

11. Web: What are the issues and related problems? Safe surfing (navigating) Find relevant and credible information (example: research) Create successful e-business Reduce tax evasion Enable local economic development Communicate with friends, colleagues, customers, citizens, voters,… 11

12. Need to study the Web The Web is the largest human information construct in history. The Web is transforming society… It is time to study it systematically as stand- alone socio-technical artifact 12

13. How to study the Web? Analyze the interplay among the: Structure Function Evolution the Web as a highly inter-connected large complex system 13

14. Web Modeling understand measure and model its evolution in order to optimize its social benefit through effective policies 14

15. What is the Structure of the Web The Web as a Graph: Nodes: the websites (URI) more than 1 trillion Links: the hyperlinks 5 links per page (average) Weights: link assessment The WWW graph is a Directed Evolving Graph 15 2 0.5 1.2 3 0.2 1 2.1 4

16. Statistical Analysis of Graphs:The degree distribution P(k) = P(d ≤ k) is the distribution function of the random variable d that counts the degree of a randomly chosen node.

17. Statistical Analysis of the Web Graph four major findings: power law degree distribution (self-similarity) internet traffic: Faloutsos Web links: Barabasi small world property (the diameter is much smaller than the order of the graph) easy communication many dense bipartite subgraphs on-line property (the number of nodes and edges changes with time)

18. Distribution of links on the World-Wide Web P(k)∼ k−γ power law a, Outgoing links (URLs found on an HTML document); b, Incoming links Web.c, Average of the shortest path between two documents as a function of system size [Barabasi,ea 1999]

19. Small World Property Social Communication Networks Watts-Strogatz(1998) Short average path lengths and high clustering. WWW Average Distance (Shortest Path) between 2 Documents: <ℓ> = 0.35 + 2.06 log(n) <ℓ> = 18.6, n = 8 x 108 (1999) <ℓ> = 18.9, n = 109 (2009) two randomly chosen documents on the web are on average 19 clicks away from each other. (Small World)

20. Web dynamics Search (PageRank, HITS, Markov matrices) Traffic Evolution graph generators Games mechanism design (auctions) Queries-search engine-Web 20

21. Search The Hyperlink Matrix The page rank vector π, is an eigenvector of the Hyperlink Markov matrix M, For the eigenvalue 1. π is a stationary distribution Mπ = π π = (π(κ)), π(κ) = the pagerank of the web page κ dimΜ = the number of the web pages that can be crawled by search engines.

22. Basis of Google’s Algorithm If the Μarkovmatrix M is ergodic, the stationary distribution vector ρ is unique. If the Μarkovmatrix Μ is mixing, then π is calculated as the limit for every initial probability distribution ρ. The 2ndeigenvalueοf M estimates the speed of Convergence

23. Internet Traffic Prigogine and Herman 1971 Stochastic model of vehicular traffic dynamics based on statistical physics between the macroscopic “fluid dynamics” model and the individual vehicle model (1st order SDE) f0 is the "desired" velocity distribution function x and v are the position and velocity of the "vehicles“ is the average velocity c is the concentration of the "vehicles“ P is the probability of "passing" in the sense of increase of flow, T is the relaxation time.

24. Adaptation of the Prigogine - Hermann Model for the Internet Traffic [Antoniou, Ivanov 2002,2003] Vehicles = the Information Packages Statistics of Information Packages: Log-Normal Distribution

25. The Origin of Power Law in Network Structure and Network Traffic Kolmogorov 1941, The local structure of the turbulence in incompressible viscous fluid for very large Reynolds numbers, Dokl. Akad. Nauk SSSR 30, 301. The origin of Self-Similar Stochastic Processes Model of the homogeneous fragmentation Applying a variant the central limit theorem, Kolmogorov found that the logarithms of the grain sizes are normally distributed Before Fractals and Modern scale-free models, Wavelet Analysis of data [Antoniou, Ivanov 2002]

26. Evolution: Graph Generators Erdős-Rényi (ER)model [Erdős, Rényi ‘60] Small-world model [Watts, Strogatz ‘98] Preferential Attachment [Barabási, Albert ‘99] EdgeCopying models [Kumar et al.’99], [Kleinberg et al.’99], Forest Fire model [Leskovec, Faloutsos ‘05] Kroneckergraphs [Leskovec, Chakrabarti, Kleinberg, Faloutsos ‘07] Optimization-based models [Carlson,Doyle,’00] [Fabrikant et al. ’02] 26

27. Evolution: Game theoretic models Stageman (2004) Information Goods and Advertising: An Economic Model of the Internet ZsoltKatona and MiklosSarvary (2007) Network Formation and the Structure of the Commercial World Wide Web Kumar (2009), Why do Consumers Contribute to Connected Goods 27

28. Evolution: Queries- Search Engine -Web Kouroupas, Koutsoupias, Papadimitriou, SideriKKPS 2005 Economic-inspired model (utility) Explains scale-free behavior In the Web three types of entities exist: Documents-i.e. web pages, created by authors [n] Users [m] Topics [k] k≤m≤n 28

29. the KKPS model The Search Engine recommends Documents to the Users A User obtains satisfaction (Utility) after presented with some Documents by a Search Engine Users choose and endorse those that have the highest Utility for them, and then Search Engines make better recommendations based on these endorsements 29

30. Documents For each topic t ≤ kthere is a Document vector Dt of length n(relevance of Document d for Topic t) For Dtthe value 0 is very probable so that about k - 1 of every k entries are 0 30

31. User-Query There are Users that can be thought as simple Queries asked by individuals. For each topic t there is a User vector Rtof length m, (relevance of User-Query i for Topic t) with about m/k non-zero entries 31

32. User-Query the number of Documents proposed by the Search Engine is fixed and denoted by α the number of endorsements per User-Query is also fixed and denoted by b b ≤ α ≤ n 32

33. the algorithm Step 1: A User-Query,for a specific Topic, isentered in the Search Engine Step 2: The Search Engine recommends αrelevant Documents. The listing order is defined by a rule. In the very first operation of the Search Engine the Documents the rule is random listing according to some probability distribution Step 3: Among the α recommended Documents, bareendorsed on the basis of highest Utility. In this way, the bipartite graph S= ([m], [n], L) of Document endorsements is formed. Compute the in-degree of the Documents from the endorsements 33

34. the algorithm Step 4: Repeat Step 1 for another Topic. Step 5: Repeat Step 2. The rule for Documents listing is the decreasing in-degree for the specific User-Query computed in Step 3. Step 6: Repeat Step 3. Step 7: Repeat Steps 4, 5, 6 for a number of iterations necessary for statistical convergence (“that is, until very few changes are observed in the www state” ) 34

35. utility 35

36. 36

37. results of statistical experimentsKKPS for a wide range of values of the parameters m, n, k, a, b, the in-degree of the documents is power-law distributed the price of anarchy (efficiency of algorithm) improved radically during the first 2-3 iterations and later the improvement had a slower rate 37

38. results of statistical experimentsKKPS When the number of topics k increases the efficiency of the algorithm increases When a increases (the number of recommended documents by the search engine) the efficiency of the algorithm also increases Increasing b (number of endorsed documents per user) causes the efficiency of the algorithm to decrease 38

39. results of statistical experimentsAmarantidis-Antoniou-Vafopoulos we extend the investigation in two directions: for Uniform, Poisson and Normal initial random distribution of Documents in-degree (Step 2) and for different values of α, b and k 39

40. results of statistical experimentsAmarantidis-Antoniou-Vafopoulos in the case α=b the validity of the power law becomes less significant as bincreases b: number of endorsed documents per user a: the number of recommended documents by the search engine 40

41. results of statistical experimentsAmarantidis-Antoniou-Vafopoulos an increase in the number of Topics k, results faster decay of the power law exponent 41

42. Power law for the case b≤α

43. efficiency of the search algorithm α=b efficiency of the search algorithm increases when the number of topics k increases [confirmation of KKPS results]

44. efficiency of the search algorithm in the case b≤α efficiency of the search algorithm increases when the number of recommended Documents by the Search Engine α increases [confirmation of KKPS results]

45. efficiency of the search algorithm b≤α efficiency of the search algorithm increases when the number of bof endorsed Documents per User-Query increases [KKPS results not confirmed]

46. Discussion of statistical experimentsAmarantidis-Antoniou-Vafopoulos α=b: all recommended Documents are endorsed according to the highest in-degree criterion Utility is useful only in terms of establishing compatibility between Utility Matrix and the Users-Queries and Documents bipartite graph 46

47. Discussion of statistical experimentsAmarantidis-Antoniou-Vafopoulos origin of the power law distribution of the in-degree of Documents, two mechanisms are identified in the KKPS model: Users-Queries endorse a small fraction of Documents presented (b) Assuming a small fraction of poly-topic Documents, the algorithm creates a high number of endorsements for them The above mechanisms are not exhaustive for the real Web graph. Indexing algorithms, crawler’s design, Documents structure and evolution should be examined as possible additional mechanisms contributing to the manifestation of the power law distribution 47

48. Discussion on the Endorsement Mechanism “The endorsement mechanism does not need to be specified, as soon as it is observable by the Search Engine. For example, endorsing a Document may entail clicking it, or pointing a hyperlink to it.” This KKPS hypothesis does not take into account the fundamental difference between clicking a link (browsing) and creating a hyperlink. 48

49. discussion Web traffic is observable by the website owner or administrator through the corresponding log file and by third parties authorized (like search engine cookies which can trace clicking behavior or malicious 49

50. discussion On the contrary, creating a hyperlink results in a more “permanent” link between two Documents which is observable by all Users-Queries and Search Engines. Therefore, the KKPS algorithm actually examines the Web traffic and not the hyperlink structure of Documents which is the basis of the in-degree Search engine’s algorithm 50

51. discussion Web traffic as well as Web content editing, are not taken into account in the algorithms of Search engines based on the in-degree (i.e. Pagerank). These algorithms were built for Web 1.0 where Web content update and traffic monetization were not so significant 51

52. discussion In the present Web 2.0 era with rapid change, the Web graph, content and traffic should be taken into account in efficient search algorithms. Therefore, birth-death processes for Documents and links and Web traffic should be introduced in Web models, combined with content update (Web 2.0) and semantic markup (Web 3.0) for Documents. 52

53. discussion The discrimination between Users and Queries could facilitate extensions of the KKPS model: teleportation (a direct visit to a Document which avoids Search Engines) different types of Users and relevance feedback between Documents and Queries 53

54. discussion KKPS: Utility is defined to be time invariant linear function of R and D which by construction is not affecting the www state when α=b not take into account the dynamic interdependence of the Utility on the www state. In reality, the evolution of the www state will change both R and D A future extension of KKPS model should account for user behavior by incorporating Web browsing and editing preferences 54

55. discussion useful to offer deeper insight in the Web’s economic aspects in the KKPS model: valuation mechanisms for Web traffic and link structures and monetizing the search procedure (sponsored search, digital goods, excludable, anti-rival goods etc) 55

56. Stochastic Models and the Web Webmetrics: statistical models for the Web function, structure & evolution in order to evaluate individual, business and public policies 56

57. Master in web science is based on Web assessment, mathematical modeling and operation combined with business applications and societal transformations in the knowledge society. Web science studies, apart from Academic, Research and Training careers offer remarkable opportunities in Business. 57 WS.01 lecture 1: Web history Michalis Vafopoulos

58. Master in web science 58 Michalis Vafopoulos