A Survey of Peer-to-Peer Content Distribution Technologies
STEPHANOS ANDROUTSELLIS-THEOTOKIS AND DIOMIDIS SPINELLIS
Athens University of Economics and Business
Distributed computer architectures labeled “peer-to-peer” are designed for the sharing
of computer resources (content, storage, CPU cycles) by direct exchange, rather than
requiring the intermediation or support of a centralized server or authority.
Peer-to-peer architectures are characterized by their ability to adapt to failures and
accommodate transient populations of nodes while maintaining acceptable connectivity
and performance.
Content distribution is an important peer-to-peer application on the Internet that
has received considerable research attention. Content distribution applications
typically allow personal computers to function in a coordinated manner as a distributed
storage medium by contributing, searching, and obtaining digital content.
In this survey, we propose a framework for analyzing peer-to-peer content
distribution technologies. Our approach focuses on nonfunctional characteristics such
as security, scalability, performance, fairness, and resource management potential, and
examines the way in which these characteristics are reflected in—and affected by—the
architectural design decisions adopted by current peer-to-peer systems.
We study current peer-to-peer systems and infrastructure technologies in terms of
their distributed object location and routing mechanisms, their approach to content
replication, caching and migration, their support for encryption, access control,
authentication and identity, anonymity, deniability, accountability and reputation, and
their use of resource trading and management schemes.
Categories and Subject Descriptors: C.2.1 [Computer-Communication Networks]:
Network Architecture and Design—Network topology; C.2.2 [Computer-
Communication Networks]: Network Protocols—Routing protocols; C.2.4
[Computer-Communication Networks]: Distributed Systems—Distributed
databases; H.2.4 [Database Management]: Systems—Distributed databases; H.3.4
[Information Storage and Retrieval]: Systems and Software—Distributed systems
General Terms: Algorithms, Design, Performance, Reliability, Security
Additional Key Words and Phrases: Content distribution, DOLR, DHT, grid computing,
p2p, peer-to-peer
1. INTRODUCTION systems such as [Gnutella 2003], Seti@
Home [SetiAtHome 2003], OceanStore
A new wave of network architectures [Kubiatowicz et al. 2000], and many
labeled peer-to-peer is the basis of others. Such architectures are gener-
operation of distributed computing ally characterized by the direct sharing
Authors’ address: Athens University of Economics and Business, 76 Patission St., GR-104 34, Athens, Greece;
email: stheotok@aueb.gr.
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted
without fee provided that copies are not made or distributed for profit or direct commercial advantage and
that copies show this notice on the first page or initial screen of a display along with the full citation.
Copyrights for components of this work owned by others than ACM must be honored. Abstracting with
credit is permitted. To copy otherwise, to republish, to post on servers, to redistribute to lists, or to use any
component of this work in other works requires prior specific permission and/or a fee. Permissions may be
requested from Publications Dept., ACM, Inc., 1515 Broadway, New York, NY 10036 USA, fax: +1 (212)
869-0481, or permissions@acm.org.
c 2004 ACM 0360-0300/04/1200-0335 $5.00
ACM Computing Surveys, Vol. 36, No. 4, December 2004, pp. 335–371.

336 S. Androutsellis-Theotokis and D. Spinellis
of computer resources (CPU cycles, stor- We then present the main attributes of
age, content) rather than requiring the in- peer-to-peer content distribution systems,
termediation of a centralized server. and the aspects of their architectural de-
The motivation behind basing appli- sign which affect these attributes are an-
cations on peer-to-peer architectures alyzed with reference to specific existing
derives to a large extent from their ability peer-to-peer content distribution systems
to function, scale, and self-organize in and technologies.
the presence of a highly transient popu- Throughout this report the terms
lation of nodes, network, and computer “node”, “peer” and “user” are used inter-
failures, without the need of a central changeably, according to the context, to re-
server and the overhead of its adminis- fer to the entities that are connected in a
tration. Such architectures typically have peer-to-peer network.
as inherent characteristics scalability,
resistance to censorship and centralized
1.1. Defining Peer-to-Peer Computing
control, and increased access to resources.
Administration, maintenance, respon- A quick look at the literature reveals a con-
sibility for the operation, and even the siderable number of different definitions
notion of “ownership” of peer-to-peer sys- of “peer-to-peer”, mainly distinguished
tems are also distributed among the users, by the “broadness” they attach to the
instead of being handled by a single com- term.
pany, institution or person (see also Agre The strictest definitions of “pure” peer-
[2003] for an interesting discussion of in- to-peer refer to totally distributed sys-
stitutional change through decentralized tems, in which all nodes are completely
architectures). Finally, peer-to-peer archi- equivalent in terms of functionality and
tectures have the potential to accelerate tasks they perform. These definitions fail
communication processes and reduce to encompass, for example, systems that
collaboration costs through the ad hoc ad- employ the notion of “supernodes” (nodes
ministration of working groups [Schoder that function as dynamically assigned
and Fischbach 2003]. localized mini-servers) such as Kazaa
This report surveys peer-to-peer con- [2003], which are, however, widely ac-
tent distribution technologies, aiming to cepted as peer-to-peer, or systems that
provide a comprehensive account of ap- rely on some centralized server infras-
plications, features, and implementation tructure for a subset of noncore tasks
techniques. As this is a new and (thank- (e.g. bootstrapping, maintaining reputa-
fully) rapidly evolving field, and advances tion ratings, etc).
and new capabilities are constantly being According to a broader and widely ac-
introduced, this article will be present- cepted definition in Shirky [2000], “peer-
ing what essentially constitutes a “snap- to-peer is a class of applications that take
shot” of the state of the art around the advantage of resources—storage, cycles,
time of its writing—as is unavoidably the content, human presence—available at
case for any survey of a thriving research the edges of the internet”. This defini-
field. We do, however, believe that the tion, however, encompasses systems that
core information and principles presented completely rely upon centralized servers
will remain relevant and useful for the for their operation (such as seti@home,
reader. various instant messaging systems, or
In the next section, we define the basic even the notorious Napster), as well as
concepts of peer-to-peer computing. We various applications from the field of Grid
classify peer-to-peer systems into three computing.
categories (communication and collab- Overall, it is fair to say that there is
oration, distributed computation, and no general agreement about what “is” and
content distribution). Content distribu- what “is not” peer-to-peer.
tion systems are further discussed and We feel that this lack of agreement on
categorized. a definition—or rather the acceptance of
ACM Computing Surveys, Vol. 36, No. 4, December 2004.

A Survey of Content Distribution Technologies 337
various different definitions—is, to a large nections fail or recover, in order to main-
extent, due to the fact that systems or tain its connectivity and performance.
applications are labeled “peer-to-peer” not
because of their internal operation or ar- We therefore propose the following defi-
chitecture, but rather as a result of how nition:
they are perceived “externally”, that is, Peer-to-peer systems are distributed systems
whether they give the impression of pro- consisting of interconnected nodes able to self-
viding direct interaction between comput- organize into network topologies with the purpose
ers. As a result, different definitions of of sharing resources such as content, CPU cycles,
“peer-to-peer” are applied to accommodate storage and bandwidth, capable of adapting to
the various different cases of such systems failures and accommodating transient popula-
or applications. tions of nodes while maintaining acceptable con-
From our perspective, we believe that nectivity and performance, without requiring the
the two defining characteristics of peer-to- intermediation or support of a global centralized
server or authority.
peer architectures are the following:
—The sharing of computer resources by This definition is meant to encompass
direct exchange, rather than requir- “degrees of centralization” ranging from
ing the intermediation of a centralized the pure, completely decentralized sys-
server. Centralized servers can some- tems such as Gnutella, to “partially cen-
times be used for specific tasks (sys- tralized” systems1 such as Kazaa. How-
tem bootstrapping, adding new nodes ever, for the purposes of this survey, we
to the network, obtain global keys for shall not restrict our presentation and dis-
data encryption), however, systems that cussion of architectures and systems to
rely on one or more global centralized our own proposed definition, and we will
servers for their basic operation (e.g. for take into account systems that are consid-
maintaining a global index and search- ered peer-to-peer by other definitions as
ing through it—Napster, Publius) are well, including systems that employ a cen-
clearly stretching the definition of peer- tralized server (such as Napster, instant
to-peer. messaging applications, and others).
As the nodes of a peer-to-peer network The focus of our study is content distri-
cannot rely on a central server coordi- bution, a significant area of peer-to-peer
nating the exchange of content and the systems that has received considerable re-
operation of the entire network, they are search attention.
required to actively participate by inde-
pendently and unilaterally performing 1.2. Peer-to-Peer and Grid Computing
tasks such as searching for other nodes, Peer-to-peer and Grid computing are two
locating or caching content, routing in- approaches to distributed computing, both
formation and messages, connecting to concerned with the organization of re-
or disconnecting from other neighboring source sharing in large-scale computa-
nodes, encrypting, introducing, retriev- tional societies.
ing, decrypting and verifying content, as Grids are distributed systems that en-
well as others. able the large-scale coordinated use and
—Their ability to treat instability and sharing of geographically distributed re-
variable connectivity as the norm, au- sources, based on persistent, standards-
tomatically adapting to failures in both based service infrastructures, often with
network connections and computers, as a high-performance orientation [Foster
well as to a transient population of et al. 2001].
nodes.
This fault-tolerant, self-organizing ca- 1 In our definition, we refer to “global” servers, to
pacity suggests the need for an adap- make a distinction from the dynamically assigned
tive network topology that will change “supernodes” of partially centralized systems (see
as nodes enter or leave and network con- also Section 3.3.3)
ACM Computing Surveys, Vol. 36, No. 4, December 2004.

338 S. Androutsellis-Theotokis and D. Spinellis
As Grid systems increase in scale, they Chat/Irc, Instant Messaging (Aol, Icq,
begin to require solutions to issues of self- Yahoo, Msn), and Jabber [Jabber 2003].
configuration, fault tolerance, and scala- Distributed Computation. This category
bility, for which peer-to-peer research has includes systems whose aim is to take ad-
much to offer. vantage of the available peer computer
Peer-to-peer systems, on the other hand, processing power (CPU cycles). This is
focus on dealing with instability, tran- achieved by breaking down a computer-
sient populations, fault tolerance, and self- intensive task into small work units and
adaptation. To date, however, peer-to-peer distributing them to different peer com-
developers have worked mainly on verti- puters, that execute their corresponding
cally integrated applications, rather than work unit and return the results. Cen-
seeking to define common protocols and tral coordination is invariably required,
standardized infrastructures for interop- mainly for breaking up and distributing
erability. the tasks and collecting the results. Exam-
In summary, one can say that “Grid com- ples of such systems include projects such
puting addresses infrastructure, but not as Seti@home [Sullivan III et al. 1997;
yet failure, while peer-to-peer addresses SetiAtHome 2003], genome@home [Lar-
failure, but not yet infrastructure” [Foster son et al. 2003; GenomeAtHome 2003],
and Iamnitchi 2003]. and others.
In addition to this, the form of sharing Internet Service Support. A number of
initially targeted by peer-to-peer has been different applications based on peer-to-
of limited functionality, providing a global peer infrastructures have emerged for
content distribution and filesharing space supporting a variety of Internet ser-
lacking any form of access control. vices. Examples of such applications
As peer-to-peer technologies move into include peer-to-peer multicast systems
more sophisticated and complex applica- [VanRenesse et al. 2003; Castro et al.
tions, such as structured content distri- 2002], Internet indirection infrastruc-
bution, desktop collaboration, and net- tures [Stoica et al. 2002], and secu-
work computation, it is expected that rity applications, providing protection
there will be a strong convergence be- against denial of service or virus attacks
tween peer-to-peer and Grid computing [Keromytis et al. 2002; Janakiraman et al.
[Foster 2000]. The result will be a new 2003; Vlachos et al. 2004].
class of technologies combining elements Database Systems. Considerable work
of both peer-to-peer and Grid comput- has been done on designing distributed
ing, which will address scalability, self- database systems based on peer-to-peer
adaptation, and failure recovery, while, infrastructures. Bernstein et al. [2002]
at the same time, providing a persis- propose the Local Relational Model
tent and standardized infrastructure for (LRM), in which the set of all data stored
interoperability. in a peer-to-peer network is assumed to
be comprised of inconsistent local rela-
tional databases interconnected by sets
1.3. Classification of Peer-to-Peer
of “acquaintances” that define translation
Applications
rules and semantic dependencies between
Peer-to-peer architectures have been em- them. PIER [Huebsch et al. 2003] is a
ployed for a variety of different application scalable distributed query engine built
categories, which include the following. on top of a peer-to-peer overlay network
Communication and Collaboration. topology that allows relational queries
This category includes systems that to run across thousands of computers.
provide the infrastructure for facilitating The Piazza system [Halevy et al. 2003]
direct, usually real-time, communica- provides an infrastructure for building
tion and collaboration between peer semantic Web [Berners-Lee et al. 2001]
computers. Examples include chat and applications, consisting of nodes that can
instant messaging applications, such as supply either source data (e.g. from a
ACM Computing Surveys, Vol. 36, No. 4, December 2004.

A Survey of Content Distribution Technologies 339
relational database), schemas (or ontolo- An examination of current peer-to-peer
gies) or both. Piazza nodes are transitively technologies in this context suggests that
connected by chains of mappings between they can be grouped as follows (with ex-
pairs of nodes, allowing queries to be amples in Tables I and II).
distributed across the Piazza network.
Finally, Edutella [Nejdl et al. 2003] is an —Peer-to-Peer Applications. This category
open-source project that builds on the includes content distribution systems
W3C metadata standard RDF, to provide that are based on peer-to-peer tech-
a metadata infrastructure and querying nology. We attempt to further subdi-
capability for peer-to-peer applications. vide them into the following two groups,
Content Distribution. Most of the cur- based on their application goals and per-
rent peer-to-peer systems fall within the ceived complexity:
category of content distribution, which in- Peer-to-peer “file exchange” systems.
cludes systems and infrastructures de- These systems are targeted towards
signed for the sharing of digital media simple, one-off file exchanges between
and other data between users. Peer-to- peers. They are used for setting up
peer content distribution systems range a network of peers and providing fa-
from relatively simple direct fileshar- cilities for searching and transferring
ing applications, to more sophisticated files between them. These are typically
systems that create a distributed stor- light-weight applications that adopt a
age medium for securely and efficiently best-effort approach without addressing
publishing, organizing, indexing, search- security, availability, and persistence. It
ing, updating, and retrieving data. There is mainly systems in this category that
are numerous such systems and in- are responsible for spawning the repu-
frastructures. Some examples are: the tation (and in some cases notoriety) of
late Napster, Publius [Waldman et al. peer-to-peer technologies.
2000], Gnutella [Gnutella 2003], Kazaa Peer-to-peer content publishing and stor-
[Kazaa 2003], Freenet [Clarke et al. age systems. These systems are targeted
2000], MojoNation [MojoNation 2003], towards creating a distributed storage
Oceanstore [Kubiatowicz et al. 2000], medium in—and through—which users
PAST [Druschel and Rowstron 2001], will be able to publish, store, and dis-
Chord [Stoica et al. 2001], Scan [Chen tribute content in a secure and persis-
et al. 2000], FreeHaven [Dingledine tent manner. Such content is meant to
et al. 2000], Groove [Groove 2003], and be accessible in a controlled manner by
Mnemosyne [Hand and Roscoe 2002]. peers with appropriate privileges. The
This survey will focus on content distri- main focus of such systems is security
bution, one of the most prominent appli- and persistence, and often the aim is
cation areas of peer-to-peer systems. to incorporate provisions for account-
ability, anonymity and censorship resis-
tance, as well as persistent content man-
1.4. Peer-to-Peer Content Distribution agement (updating, removing, version
In its most basic form, a peer-to-peer con- control) facilities.
tent distribution system creates a dis- —Peer-to-Peer Infrastructures. This cate-
tributed storage medium that allows for gory includes peer-to-peer based infras-
the publishing, searching, and retrieval tructures that do not constitute working
of files by members of its network. As applications, but provide peer-to-peer
systems become more sophisticated, non- based services and application frame-
functional features may be provided, in- works. The following infrastructure ser-
cluding provisions for security, anonymity, vices are identified:
fairness, increased scalability and perfor- Routing and location. Any peer-to-peer
mance, as well as resource management content distribution system relies on
and organization capabilities. All of these a network of peers within which re-
will be discussed in the following sections. quests and messages must be routed
ACM Computing Surveys, Vol. 36, No. 4, December 2004.

340 S. Androutsellis-Theotokis and D. Spinellis
Table I. A Classification of Current Peer-to-Peer Systems
(RM: Resource Management; CR: Censorship Resistance; PS: Performance and Scalability; SPE: Security,
Privacy and Encryption; A: Anonymity; RA: Reputation and Accountability; RT: Resource Trading.)
Peer-to-Peer File Exchange Systems
System Brief Description
Napster Distributed file sharing—hybrid decentralized.
Kazaa [2003] Distributed file sharing—partially centralized.
Gnutella [2003] Distributed file sharing—purely decentralized.
Peer-to-Peer Content Publishing and Storage Systems
System Brief Description Main Focus
Scan [Chen et al. 2000] A dynamic, scalable, efficient content distribution PS
network. Provides dynamic content replication.
Publius [Waldman et al. 2000] A censorship-resistant system for publishing RM
content. Static list of servers. Enhanced content
management (update and delete).
Groove [Groove 2003] Internet communications software for direct RM,PS,SPE
real-time peer-to-peer interaction.
FreeHaven [Dingledine et al. 2000] A flexible system for anonymous storage. A,RA
Freenet [Clarke et al. 2000] Distributed anonymous information storage and A,RA
retrieval system.
MojoNation [MojoNation 2003] Distributed file storage. Fairness through the use SPE,RT
of currency mojo.
Oceanstore [Kubiatowicz et al. 2000] An architecture for global scale persistent storage. RM,PS,SPE
Scalable, provides security and access control.
Intermemory [Chen et al. 1999] System of networked computers. Donate storage RT
in exchange for the right to publish data.
Mnemosyne [Hand and Roscoe 2002] Peer-to-peer steganographic storage system. SPE
Provides privacy and plausible deniability.
PAST [Druschel and Rowstron 2001] Large scale persistent peer-to-peer storage utility. PS,SPE
Dagster [Stubblefield and Wallach 2001] A censorship-resistant document publishing CR,SPE
system.
Tangler [Waldman and Mazi 2001] A content publishing system based on document CR,SPE
entanglements.
with efficiency and fault tolerance, and 2. ANALYSIS FRAMEWORK
through which peers and content can
In this survey, we present a description
be efficiently located. Different infras-
and analysis of applications, systems, and
tructures and algorithms have been
infrastructures that are based on peer-
developed to provide such services.
to-peer architectures and aim at either
Anonymity. Peer-to-peer based infras-
offering content distribution solutions, or
tructure systems have been designed
supporting content distribution related
with the explicit aim of providing user
activities.
anonymity.
Our approach is based on:
Reputation Management. In a peer-
to-peer network, there is no central —identifying the feature space of nonfunc-
organization to maintain reputation tional properties and characteristics of
information for users and their behavior. content distribution systems,
Reputation information is, therefore, —determining the way in which the non-
hosted in the various network nodes. In functional properties depend on, and
order for such reputation information to can be affected by, various design fea-
be kept secure, up-to-date, and avail- tures, and
able throughout the network, complex —providing an account, analysis, and
reputation management infrastructures evaluation of the design features and
need to be employed. solutions that have been adopted by
ACM Computing Surveys, Vol. 36, No. 4, December 2004.

A Survey of Content Distribution Technologies 341
Table II. A Classification of Current Peer-to-Peer Infrastructures
Infrastructures for routing and location
Chord [Stoica et al. 2001] A scalable peer-to-peer lookup service. Given a key, it
maps the key to a node.
CAN [Ratnasamy et al. 2001] Scalable content addressable network. A distributed
infrastructure that provides hash-table functionality
for mapping file names to their locations.
Pastry [Rowstron and Druschel 2001] Infrastructure for fault-tolerant wide-area location and
routing.
Tapestry [Zhao et al. 2001] Infrastructure for fault-tolerant wide-area location and
routing.
Kademlia [Mayamounkov and Mazieres 2002] A scalable peer-to-peer lookup service based on the
XOR metric.
Infrastructures for anonymity
Anonymous remailer mixnet [Berthold et al. 1998] Infrastructure for anonymous connection.
Onion Routing [Goldschlag et al. 1999] Infrastructure for anonymous connection.
ZeroKnowledge Freedom [Freedom 2003] Infrastructure for anonymous connection.
Tarzan [Freedman et al. 2002] A peer-to-peer decentralized anonymous network layer.
Infrastructures for reputation management
Eigentrust [Kamvar et al. 2003] A Distributed reputation management algorithm.
A Partially distributed reputation management A partially distributed approach based on a
system [Gupta et al. 2003] debit/credit or debit-only scheme.
PeerTrust [Xiong and Liu 2002] A decentralized, feedback-based reputation
management system using satisfaction and number of
interaction metrics.
current peer-to-peer systems, as well as data and processing methods. Unautho-
their shortcomings, potential improve- rized entities cannot change data; ad-
ments, and proposed alternatives. versaries cannot substitute a forged doc-
ument for a requested one.
For the identification of the nonfunc- Privacy and confidentiality. Ensuring
tional characteristics on which our study that data is accessible only to those
is based, we used the work of Shaw and authorized to have access, and that
Garlan [1995], which we adapted to the there is control over what data is col-
field of peer-to-peer content distribution. lected, how it is used, and how it is
The various design features and solu- maintained.
tions that we examine, as well as their
Availability and persistence. Ensuring
relationship to the relevant nonfunctional
that authorized users have access to
characteristics, were assembled through
data and associated assets when re-
a detailed analysis and study of current
quired. For a peer-to-peer content dis-
peer-to-peer content distribution systems
tribution system this often means al-
that are either deployed, researched, or
ways. This property entails stability in
proposed.
the presence of failure, or changing node
The resulting analysis framework is il-
populations.
lustrated in Figure 1. The boxes in the
periphery show the most important at- Scalability. Maintaining the system’s per-
tributes of peer-to-peer content distribu- formance attributes independent of the
tion systems, described as follows. number of nodes or documents in its
network. A dramatic increase in the
Security. Further analyzed in terms of: number of nodes or documents will
Integrity and authenticity. Safeguard- have minimal effect on performance and
ing the accuracy and completeness of availability.
ACM Computing Surveys, Vol. 36, No. 4, December 2004.

342 S. Androutsellis-Theotokis and D. Spinellis
Fig. 1. Illustration of the way in which various design features affect the main characteristics of peer-
to-peer content distribution systems.
Performance. The time required for per- grouping, based on the content itself;
forming the operations allowed by the grouping, based on locality or network
system, typically publication, searching, distance; grouping, based on organiza-
and retrieval of documents. tion ties, as well as others.
Fairness. Ensuring that users offer and The relationships between nonfunc-
consume resources in a fair and bal- tional characteristics are depicted as a
anced manner. May rely on account- UML diagram in Figure 1. The Figure
ability, reputation, and resource trading schematically illustrates the relationship
mechanisms. between the various design features and
Resource Management Capabilities. In the main characteristics of peer-to-peer
their most basic form, peer-to-peer con- content distribution systems.
tent distribution systems allow the pub- The boxes in the center show the design
lishing, searching, and retrieval of docu- decisions that affect these attributes. We
ments. More sophisticated systems may note that these design decisions are mostly
afford more advanced resource manage- independent and orthogonal.
ment capabilities, such as editing or We see, for example, that the perfor-
removal of documents, management of mance of a peer-to-peer content distribu-
storage space, and operations on meta- tion system is affected by the distributed
data. object location and routing mechanisms,
Semantic Grouping of Information. An as well as by the data replication,
area of research that has attracted con- caching, and migration algorithms. Fair-
siderable attention recently is the se- ness, on the other hand, depends on
mantic grouping and organization of the system’s provisions for accountabil-
content and information in peer-to-peer ity and reputation, as well as on any re-
networks. Various grouping schemes source trading mechanisms that may be
are encountered, such as semantic implemented.
ACM Computing Surveys, Vol. 36, No. 4, December 2004.

A Survey of Content Distribution Technologies 343
In the following sections of this survey, such networks are often termed “servents”
the different design decisions and features (SERVers+clieENTS).
will be presented and discussed, based on Partially Centralized Architectures. The
an examination of existing peer-to-peer basis is the same as with purely decentral-
content distribution systems, and with ized systems. Some of the nodes, however,
reference to the way in which they affect assume a more important role, acting as
the main attributes of these systems, local central indexes for files shared by lo-
as presented. A relevant presentation cal peers. The way in which these supern-
and discussion of various desired prop- odes are assigned their role by the network
erties of peer-to-peer systems can also varies between different systems. It is im-
be found in Kubiatowics [2003], while portant, however, to note that these su-
an analysis of peer-to-peer systems from pernodes do not constitute single points
the end-user perspective can be found in of failure for a peer-to-peer network, since
Lee [2003]. they are dynamically assigned and, if they
fail, the network will automatically take
3. PEER-TO-PEER DISTRIBUTED OBJECT action to replace them with others.
LOCATION AND ROUTING Hybrid Decentralized Architectures. In
these systems, there is a central server fa-
The operation of any peer-to-peer content cilitating the interaction between peers by
distribution system relies on a network maintaining directories of metadata, de-
of peer computers (nodes), and connec- scribing the shared files stored by the peer
tions (edges) between them. This network nodes. Although the end-to-end interac-
is formed on top of—and independently tion and file exchanges may take place di-
from—the underlying physical computer rectly between two peer nodes, the central
(typically IP) network, and is thus referred servers facilitate this interaction by per-
to as an “overlay” network. The topology, forming the lookups and identifying the
structure, and degree of centralization of nodes storing the files. The terms “peer-
the overlay network, and the routing and through-peer” or “broker mediated” are
location mechanisms it employs for mes- sometimes used for such systems [Kim
sages and content are crucial to the oper- 2001].
ation of the system, as they affect its fault Obviously, in these architectures, there
tolerance, self-maintainability, adaptabil- is a single point of failure (the central
ity to failures, performance, scalability, server). This typically renders them inher-
and security; in short almost all of the sys- ently unscalable and vulnerable to censor-
tem’s attributes as laid out in Section 2 ship, technical failure, or malicious attack.
(see also Figure 1).
Overlay networks can be distinguished
in terms of their centralization and 3.2. Network Structure
structure. By structure, we refer to whether the over-
lay network is created nondeterministi-
3.1. Overlay Network Centralization
cally (ad hoc) as nodes and content are
Although in their purest form peer-to-peer added, or whether its creation is based
overlay networks are supposed to be to- on specific rules. We categorize peer-to-
tally decentralized, in practice this is not peer networks as follows, in terms of their
always true, and systems with various de- structure:
grees of centralization are encountered. Unstructured. The placement of content
Specifically, the following three categories (files) is completely unrelated to the over-
are identified. lay topology.
Purely Decentralized Architectures. All In an unstructured network, content
nodes in the network perform exactly the typically needs to be located. Searching
same tasks, acting both as servers and mechanisms range from brute force meth-
clients, and there is no central coordi- ods, such as flooding the network with
nation of their activities. The nodes of propagating queries in a breadth-first
ACM Computing Surveys, Vol. 36, No. 4, December 2004.

344 S. Androutsellis-Theotokis and D. Spinellis
or depth-first manner until the desired Table III. A classification of Peer-to-Peer Content
content is located, to more sophisticated Distribution Systems and Location and Routing
Infrastructures in Terms of Their Network Structure,
and resource-preserving strategies that With Some Typical Examples
include the use of random walks and rout-
ing indices (discussed in more detail in Centralization
Section 3.3.4). The searching mechanisms
employed in unstructured networks have Hybrid Partial None
obvious implications, particularly in re- Unstructured Napster, Kazaa, Gnutella,
gards to matters of availability, scalability, Publius Mor- FreeHaven
and persistence. pheus,
Unstructured systems are generally Gnutella,
Edutella
more appropriate for accommodating
highly-transient node populations. Some Structured Chord,
representative examples of unstructured Infrastructures CAN,
systems are Napster, Publius [Waldman Tapestry,
Pastry
et al. 2000], Gnutella [Gnutella 2003],
Kazaa [Kazaa 2003], Edutella [Nejdl et al. Structured OceanStore,
2003], FreeHaven [Dingledine et al. 2000], Systems Mnemosyne,
Scan, PAST,
as well as others. Kademlia,
Structured. These have emerged mainly Tarzan
in an attempt to address the scalability
issues that unstructured systems were A category of networks that are in be-
originally faced with. In structured net- tween structured and unstructured are re-
works, the overlay topology is tightly ferred to as loosely structured networks.
controlled and files (or pointers to them) Although the location of content is not
are placed at precisely specified locations. completely specified, it is affected by rout-
These systems essentially provide a map- ing hints. A typical example is Freenet
ping between content (e.g. file identifier) [Clarke et al. 2000; Clake et al. 2002].
and location (e.g. node address), in the Table III summarizes the categories we
form of a distributed routing table, so that outlined, with examples of peer-to-peer
queries can be efficiently routed to the content distribution systems and architec-
node with the desired content [Lv et al. tures. Note that all structured and loosely
2002]. structured systems are inherently purely
Structured systems offer a scalable so- decentralized; form follows function.
lution for exact-match queries, that is, In the following sections, the overlay
queries where the exact identifier of the network topology and operation of differ-
requested data object is known (as com- ent peer-to-peer systems is discussed, fol-
pared to keyword queries). Using exact- lowing the above classification, according
match queries as a substrate for keyword to degree of centralization and network
queries remains an open research problem structure.
for distributed environments [Witten et al.
1999]. 3.3. Unstructured Architectures
A disadvantage of structured systems is 3.3.1. Hybrid Decentralized. Figure 2 il-
that it is hard to maintain the structure lustrates the architecture of a typical hy-
required for efficiently routing messages brid decentralized peer-to-peer system.
in the face of a very transient node popu- Each client computer stores content
lation, in which nodes are joining and leav- (files) shared with the rest of the network.
ing at a high rate [Lv et al. 2002]. All clients connect to a central directory
Typical examples of structured systems server that maintains:
include Chord [Stoica et al. 2001], CAN
[Ratnasamy et al. 2001], PAST [Druschel —A table of registered user connection in-
and Rowstron 2001], Tapestry [Zhao et al. formation (IP address, connection band-
2001] among others. width etc.)
ACM Computing Surveys, Vol. 36, No. 4, December 2004.

A Survey of Content Distribution Technologies 345
torious Napster and Publius systems
[Waldman et al. 2000] that rely on a static,
system-wide list of servers. Their architec-
ture does not provide any smooth, decen-
tralized support for adding a new server,
or removing dead or malicious servers. A
comprehensive study of the behavior of
hybrid peer-to-peer systems and a com-
parison of their query characteristics and
their performance in terms of bandwidth
and CPU cycle consumption is presented
in Yang and Garcia-Molina [2001].
It should be noted that systems that
do not fall under the hybrid decentral-
ized category may still use some central
administration server to a limited extent,
Fig. 2. Typical hybrid decentralized peer-to-peer for example, for initial system boot-
architecture. A central directory server maintains strapping (e.g. Mojonation [MojoNation
an index of the metadata for all files in the network.
2003]), or for allowing new users to
—A table listing the files that each user join the network by providing them
holds and shares in the network, along with access to a list of current users
with metadata descriptions of the files (e.g. gnutellahosts.com for the gnutella
(e.g. filename, time of creation, etc.) network).
A computer that wishes to join the 3.3.2. Purely Decentralized. In this sec-
network contacts the central server and tion, we examine the Gnutella network
reports the files it maintains. Client com- [Gnutella 2003], an interesting and rep-
puters send requests for files to the server. resentative member of purely decentral-
The server searches for matches in its in- ized peer-to-peer architectures, due to its
dex, returning a list of users that hold the open architecture, achieved scale, and self-
matching file. The user then opens direct organizing structure. FreeHaven [Dingle-
connections with one or more of the peers dine et al. 2000] is another system using
that hold the requested file, and down- routing and search mechanisms similar to
loads it (see Figure 2). those of Gnutella.
The advantage of hybrid decentralized Like most peer-to-peer systems,
systems is that they are simple to imple- Gnutella builds a virtual overlay net-
ment, and they locate files quickly and ef- work with its own routing mechanisms
ficiently. Their main disadvantage is that [Ripeanu and Foster 2002], allowing its
they are vulnerable to censorship, legal ac- users to share files with other peers.
tion, surveillance, malicious attack, and There is no central coordination of the
technical failure, since the content shared, activities in the network and users connect
or at least descriptions of it, and the abil- to each other directly through a software
ity to access it are controlled by the single application that functions both as a client
institution, company, or user maintain- and a server (users are referred to as a
ing the central server. Furthermore, these servents).
systems are considered inherently unscal- Gnutella uses IP as its underlying net-
able, as there are bound to be limitations work service, while the communication be-
to the size of the server database and its tween servents is specified in a form of
capacity to respond to queries. Large Web application level protocol supporting four
search engines have, however, repeatedly types of messages [Jovanovich et al. 2001]:
provided counterexamples to this notion.
Examples of hybrid decentralized con- Ping. A request for a certain host to an-
tent distribution systems include the no- nounce itself.
ACM Computing Surveys, Vol. 36, No. 4, December 2004.

346 S. Androutsellis-Theotokis and D. Spinellis
Pong. Reply to a Ping message. It con- Once a node receives a QueryHit mes-
tains the IP and port of the responding sage, indicating that the target file has
host and number and size of the files being been identified at a certain node, it ini-
shared. tiates a download by establishing a di-
Query. A search request. It contains a rect connection between the two nodes.
search string and the minimum speed re- Figure 3 illustrates an example of the
quirements of the responding host. Gnutella search mechanism.
Query Hits. Reply to a Query message. Scalability issues in the original purely
It contains the IP, port, and speed of the decentralized systems arose from the fact
responding host, the number of match- that the use of the TTL effectively seg-
ing files found, and their indexed result mented the network into “sub-networks”,
set. imposing on each user a virtual “hori-
zon” beyond which their messages could
After joining the Gnutella network (by not reach [Jovanovic 2000]. Removing the
connecting to nodes found in databases TTL limit, on the other hand, would re-
such as gnutellahosts.com), a node sends sult in the network being swamped with
out a Ping message to any node it is messages.
connected to. The nodes send back a Significant research has been carried
Pong message identifying themselves, and out to address the above issues, and
also propagate the Ping message to their various solutions have been proposed.
neighbors. These will be discussed in more detail in
In order to locate a file in unstructured Section 3.3.4.
systems such as gnutella, nondeterminis-
tic searches are the only option since the 3.3.3. Partially Centralized. Partially cen-
nodes have no way of guessing where (at tralized systems are similar to purely
which nodes) the files may lie. decentralized, but they use the con-
The original Gnutella architecture uses cept of supernodes: nodes that are dy-
a flooding (or broadcast) mechanism to dis- namically assigned the task of servic-
tribute Ping and Query messages: each ing a small subpart of the peer network
Gnutella node forwards the received mes- by indexing and caching files contained
sages to all of its neighbors. The response therein.
messages received are routed back along Peers are automatically elected to be-
the opposite path through which the origi- come supernodes if they have sufficient
nal request arrived. To limit the spread of bandwidth and processing power (al-
messages through the network, each mes- though a configuration parameter may
sage header contains a time-to-live (TTL) allow users to disable this feature)
field. At each hop, the value of this field [FastTrack 2003].
is decremented, and when it reaches zero, Supernodes index the files shared by
the message is dropped. peers connected to them, and proxy search
The above mechanism is implemented requests on behalf of these peers. All
by assigning each message a unique iden- queries are therefore initially directed to
tifier and equipping each host with a dy- supernodes.
namic routing table of message identifiers Two major advantages of partially cen-
and node addresses. Since the response tralized systems are that:
messages contain the same ID as the orig-
inal messages, the host checks its routing —Discovery time is reduced in compari-
table to determine along which link the re- son with purely decentralized systems,
sponse message should be forwarded. In while there still is no unique point of
order to avoid loops, the nodes use the failure. If one or more supernodes go
unique message identifiers to detect and down, the nodes connected to them can
drop duplicate messages, to improve effi- open new connections with other su-
ciency, and preserve network bandwidth pernodes, and the network will continue
[Jovanovic 2000]. to operate.
ACM Computing Surveys, Vol. 36, No. 4, December 2004.

A Survey of Content Distribution Technologies 347
Fig. 3. An example of the Gnutella search mechanism. Solid lines between
the nodes represent connections of the Gnutella network. The search orig-
inates at the “requestor” node, for a file maintained by another node. Re-
quest messages are dispatched to all neighboring nodes, and propagated
from node-to-node as shown in the four consecutive steps (a) to (d). When
a node receives the same request message (based on its unique identifier)
multiple times, it replies with a failure message to avoid loops and minimize
traffic. When the file is identified, a success message is returned.
—The inherent heterogeneity of peer-to- mentation (as it is a proprietary system,
peer networks is taken advantage of, there is no detailed documentation on its
and exploited. In a purely decentralized structure and operation). Edutella [Nejdl
network, all of the nodes will be equally et al. 2003] is another partially centralized
(and usually heavily) loaded, regard- architecture.
less of their CPU power, bandwidth, or Yang and Garcia-Molina [2002a, 2002b]
storage capabilities. In partially central- present research addressing the design
ized systems, however, the supernodes of, and searching techniques for, partially
will undertake a large portion of the centralized peer-to-peer networks.
entire network load, while most of the The concept of supernodes has also been
other (so called “normal”) nodes will be proposed in a more recent version of the
very lightly loaded, in comparison (see Gnutella protocol. A mechanism for dy-
also [Lv et al. 2002; Zhichen et al. 2002]). namically selecting supernodes organizes
the Gnutella network into an interconnec-
Kazaa is a typical, widely used instance tion of superpeers (as they are referred to)
of a partially centralized system imple- and client nodes.
ACM Computing Surveys, Vol. 36, No. 4, December 2004.

348 S. Androutsellis-Theotokis and D. Spinellis
When a node with enough CPU power sues by means of:
joins the network, it immediately be-
comes a superpeer and establishes con- —A dynamic topology adaptation protocol
nections with other superpeers, forming a that ensures that most nodes will be at a
flat unstructured network of superpeers. short distance from high capacity nodes.
If it establishes a minimum required num- This protocol, coupled with replicating
ber of connections to client nodes within a pointers to the content of neighbors, en-
specified time, it remains a superpeer. Oth- sures that high capacity nodes will be
erwise, it turns into a regular client node. able to provide answers to a very large
number of queries.
3.3.4. Shortcomings and Evolution of Unstruc- —An active flow control scheme that ex-
tured Architectures. A number of methods ploits network heterogeneity to avoid
have been proposed to overcome the origi- hotspots, and
nal unscalability of unstructured peer-to- —A search protocol that is based on ran-
peer systems. These have been shown to dom walks directed towards high capac-
drastically improve their performance. ity nodes.
Lv et al. [2002] proposed to replace
the original flooding approach by multiple Simulations of Gia were found to in-
parallel random walks, where each node crease overall system capacity by three
chooses a neighbor at random, and propa- to five orders of magnitude. Similar ap-
gates the request only to it. The use of ran- proaches, based on taking advantage of
dom walks, combined with proactive object the underlying network heterogeneity, are
replication (discussed in Section 4), was described in Lv et al. [2002] and Zhichen
found to significantly improve the perfor- et al. [2002].
mance of the system as measured by query In another approach, Crespo and
resolution time (in terms of numbers of Garcia-Molina [2002] use routing indices
hops), per-node query load, and message to address the searching and scalability
traffic generated. Proactive data replica- issues. Routing indices are tables of infor-
tion schemes are further examined in Co- mation about other nodes, stored within
hen and Shenker [2001]. each node. They provide a list of neighbors
Yang and Garcia-Molina [2002b] sug- that are most likely to be “in the direction”
gested the use of more sophisticated of the content corresponding to the query.
broadcast policies, selecting which neigh- These tables contain information about
bors to forward search queries to based on the total number of files maintained by
their past history, as well as the use of local nearby nodes, as well as the number
indices: data structures where each node of files corresponding to various topics.
maintains an index of the data stored at Three different variations of the approach
nodes located within a radius from itself. are presented, and simulations are shown
A similar solution to the information re- to improve performance by 1-2 orders
trieval problem is proposed by Kalogeraki of magnitude with respect to flooding
et al. [2002] in the form of an Intelli- techniques.
gent Search Mechanism built on top of Finally, the connectivity properties and
a Modified Random BFS Search Mecha- reliability of unstructured peer-to-peer
nism. Each peer forwards queries to a sub- networks such as Gnutella have been
set of its neighbors, selecting them based studied in Ripeanu and Foster [2002]. In
on a profile mechanism that maintains in- particular, emphasis was placed on the
formation about their performance in re- fact that peer-to-peer networks exhibit
cent queries. Neighbours are ranked ac- the properties of power-law networks,
cording to their profiles and queries are in which the number of nodes with L
forwarded selectively to the most appro- links is proportional to L−k , where k is
priate ones only. a network dependent constant. In other
Chawathe et al. [2003] presented the words, most nodes have few links (thus, a
Gia System, which addresses the above is- large fraction of them can be taken away
ACM Computing Surveys, Vol. 36, No. 4, December 2004.

A Survey of Content Distribution Technologies 349
without seriously damaging the network —CAN is a system using an n-dimensional
connectivity), while there are a few highly- Cartesian coordinate space to imple-
connected nodes which, if taken away, are ment the distributed location and rout-
likely to cause the whole network to be ing table, whereby each node is respon-
broken down in disconnected pieces. One sible for a zone in the coordinate space.
implication of this is that such networks —Tapestry (and the similar Pastry and
are robust when facing random node Kademlia) are based on the plaxton
attacks, yet vulnerable to well-planned mesh data structure, which maintains
attacks. The topology mismatch between pointers to nodes in the network whose
the Gnutella network and the underlying IDs match the elements of a tree-like
physical network infrastructure was also structure of ID prefixes up to a digit
documented in Ripeanu and Foster [2002]. position.
Tsoumakos and Roussopoulos [2003]
present a more comprehensive analysis 3.4.1. Freenet—A Loosely Structured Sys-
and comparison of the above methods. tem. The defining characteristic of loosely
Overall, unstructured peer-to-peer con- structured systems is that the nodes of the
tent distribution systems might be the peer-to-peer network can produce an esti-
preferred choice for applications where the mate (not with certainty) of which node is
following assumptions hold: most likely to store certain content. This
affords them the possibility of avoiding
—keyword searching is the common oper- blindly broadcasting request messages to
ation, all (or a random subset) of their neighbors.
—most content is typically replicated at a Instead, they use a chain mode propaga-
fair fraction of participating sites, tion approach, where each node makes a
—the node population is highly transient, local decision about which node to send the
—users will accept a best-effort content re- request message to next.
trieval approach, and Freenet [Clarke et al. 2000] is a typical,
purely decentralized loosely-structured
—the network size is not so large as to in-
content distribution system. It operates
cur scalability problems [Lv et al. 2002]
as a self-organizing peer-to-peer network,
(The last issue is alleviated through
pooling unused disk space in peer com-
the various methods described in this
puters to create a collaborative virtual
Section).
file system. Important features of Freenet
include its focus on security, publisher
3.4. Structured Architectures anonymity, deniability, and data replica-
The various structured content distribu- tion for availability and performance.
tion systems and infrastructures employ Files in Freenet are identified by unique
different mechanisms for routing mes- binary keys. Three types of keys are sup-
sages and locating data. Four of the most ported, the simplest is based on applying
interesting and representative mecha- a hash function on a short descriptive text
nisms and their corresponding systems string that accompanies each file as it is
are examined in the following sections. stored in the network by its original owner.
Each Freenet node maintains its own lo-
—Freenet is a loosely structured system cal data store, that it makes available to
that uses file and node identifier similar- the network for reading and writing, as
ity to produce an estimate of where a file well as a dynamic routing table contain-
may be located, and a chain mode propa- ing the addresses of other nodes and the
gation approach to forward queries from files they are thought to hold. To search
node-to-node. for a file, the user sends a request message
—Chord is a system whose nodes maintain specifying the key and a timeout (hops-to-
a distributed routing table in the form of live) value.
an identifier circle on which all nodes are Freenet uses the following types of mes-
mapped, and an associated finger table. sages, which all include the node identifier
ACM Computing Surveys, Vol. 36, No. 4, December 2004.

350 S. Androutsellis-Theotokis and D. Spinellis
(for loop detection), a hops-to-live value If a node receives a request for a locally-
(similar to the Gnutella TTL, see Sec- stored file, the search stops and the data
tion 3.3.2), and the source and destination is forwarded back to the requestor.
node identifiers: If the node does not store the file that the
requestor is looking for, it forwards the re-
Data insert. A node inserts new data
quest to its neighbor that is most likely to
in the network. A key and the actual data
have the file, by searching for the file key
(file) are included.
in its local routing table that is closest to
Data request. A request for a certain
the one requested. The messages, there-
file. The key of the file requested is also
fore, form a chain, as they propagate from
included.
node-to-node. To avoid huge chains, mes-
Data reply. A reply initiated when the
sages are deleted after passing through
requested file is located. The actual file is
a certain number of nodes, based on the
also included in the reply message.
hops-to-live value they carry. Nodes also
Data failed. A failure to locate a file.
store the ID and other information of the
The location (node) of the failure and the
requests they have seen, in order to handle
reason are also included.
“data reply” and “data failed” messages.
New nodes join the Freenet network If a node receives a backtracking “data
by first discovering the address of one failed” message from a downstream node,
or more existing nodes, and then start- it selects the next best node from its rout-
ing to send Data Insert messages. To in- ing stack and forwards the request to it.
sert a new file in the network, the node If all nodes in the routing table have been
first calculates a binary key for the file, explored in this way and failed, it sends
and then sends a data insert message to back a “data failed” message to the node
itself. Any node that receives the insert from which it originally received the data
message, first checks to see if the key is request message.
already taken. If the key is not found, the If the requested file is eventually found
node looks up the closest key (in terms at a certain node, a reply is passed back
of lexicographic distance) in its routing through each node that forwarded the re-
table, and forwards the insert message quest to the original node that started
to the corresponding node. By this mech- the chain. This data reply message will
anism, newly inserted files are placed include the actual data, which is cached
at nodes possessing files with similar in all intermediate nodes for future re-
keys. quests. A subsequent request with the
This continues as long as the hops-to- same key will be served immediately with
live limit is not reached. In this way, more the cached data. A request for a similar
than one node will store the new file. At key will be forwarded to the node that pre-
the same time, all the participating nodes viously provided the data.
will update their routing tables with the To address the problem of obtaining
new information (this is the mechanism the key that corresponds to a specific file,
through which the new nodes announce Freenet recommends the use of a special
their presence to the rest of the network). class of lightweight files called “indirect
If the hops-to-live limit is reached without files”. When a real file is inserted, the au-
any key collision, an “all clear” result thor also inserts a number of indirect files
will be propagated back to the original that are named according to search key-
inserter, informing that the insert was words and contain pointers to the real file.
successful. These indirect files differ from normal files
If the key is found to be taken, the node in that multiple files with the same key
returns the preexisting file as if a request (i.e. search keyword) are permitted to ex-
were made for it. In this way, malicious ist, and requests for such keys keep going
attempts to supplant existing files by in- until a specified number of results is ac-
serting junk will result in the existing files cumulated, instead of stopping at the first
being spread further. file found. The problem of managing the
ACM Computing Surveys, Vol. 36, No. 4, December 2004.

A Survey of Content Distribution Technologies 351
Fig. 4. The use of indirect files in Freenet.
The diagram illustrates a regular file with key
“A652D17D88”, which is supposed to contain a tu-
torial about photography. The author chose to in-
sert the file itself, and then a set of indirect files
(marked with an i), named according to search key- Fig. 5. A Chord identifier circle consisting of the
words she considered relevant. The indirect files three nodes 0,1 and 3. In this example, key 1 is lo-
are distributed among the nodes of the network; cated at node 1, key 2 at node 3, and key 6 at node 0.
they do not contain the actual document, simply a
“pointer” to the location of the regular file contain-
ing the document. [Karger et al. 1997]. All node identifiers
are ordered in an “identifier circle” modulo
large volume of such indirect files remains 2m (Figure 5 shows an identifier circle with
open. Figure 4 illustrates the use of indi- m = 3). Key k is assigned to the first node
rect files. whose identifier is equal to, or follows k, in
The following properties of Freenet are the identifier space. This node is called the
a result of its routing and location algo- successor node of key k. The use of consis-
rithms: tent hashing tends to balance load, as each
node receives roughly the same number of
—Nodes tend to specialize in searching keys.
for similar keys over time, as they get The only routing information required
queries from other nodes for similar is for each node to be aware of its succes-
keys. sor node on the circle. Queries for a given
—Nodes store similar keys over time, due key are passed around the circle via these
to the caching of files as a result of suc- successor pointers until a node that con-
cessful queries. tains the key is encountered. This is the
—Similarity of keys does not reflect simi- node the query maps to.
larity of files. When a new node n joins the network,
—Routing does not reflect the underlying certain keys previously assigned to n’s suc-
network topology. cessor will become assigned to n. When
node n leaves the network, all keys as-
3.4.2. Chord. Chord [Stoica et al. 2001] signed to it will be reassigned to its suc-
is a peer-to-peer routing and location in- cessor. These are the only changes in key
frastructure that performs a mapping of assignments that need to take place in or-
file identifiers onto node identifiers. Data der to maintain load balance.
location can be implemented on top of As discussed, only one data element
Chord by identifying data items (files) per node needs to be correct for Chord to
with keys and storing the (key, data item) guarantee correct (though slow) routing of
pairs at the node that the keys map to. queries. Performance degrades gracefully
In Chord, nodes are also identified by when routing information becomes out of
keys. The keys are assigned both to files date due to nodes joining and leaving the
and nodes by means of a deterministic system, and availability remains high only
function, a variant of consistent hashing as long as nodes fail independently. Since
ACM Computing Surveys, Vol. 36, No. 4, December 2004.

352 S. Androutsellis-Theotokis and D. Spinellis
Fig. 6. CAN: (a) Example 2-d [0, 1] × [0, 1] coordinate space partitioned
between 5 CAN nodes; (b) Example 2-d space after node F joins.
the overlay topology is not based on the by supporting the insertion, lookup, and
underlying physical IP network topology, a deletion of (key,value) pairs in the table.
single failure in the IP network may man- Each individual node of the CAN net-
ifest itself as multiple, scattered link fail- work stores a part (referred to as a “zone”)
ures in the overlay [Saroiu et al. 2002]. of the hash table, as well as information
To increase the efficiency of the loca- about a small number of adjacent zones
tion mechanism described previously, that in the table. Requests to insert, lookup, or
may, in the worst case, require traversing delete a particular key are routed via in-
all N nodes to find a certain key, Chord termediate zones to the node that main-
maintains additional routing information, tains the zone containing the key.
in the form of a “finger table”. In this table, CAN uses a virtual d -dimensional
each entry i points to the successor of node Cartesian coordinate space (see Figure 6)
n + 2i. For a node n to perform a lookup to store (key K ,value V ) pairs. The zone
for key k, the finger table is consulted to of the hash table that a node is respon-
identify the highest node n whose ID is sible for corresponds to a segment of this
between n and k. If such a node exists, the coordinate space. Any key K is, therefore,
lookup is repeated starting from n . Oth- deterministically mapped onto a point P
erwise, the successor of n is returned. Us- in the coordinate space. The (K , V ) pair is
ing the finger table, both the amount of then stored at the node that is responsible
routing information maintained by each for the zone within which point P lies. For
node and the time required for resolving example, in the case of Figure 6(a), a key
lookups are O(logN ) for an N -node sys- that maps to coordinate (0.1,0.2) would be
tem in the steady state. stored at the node responsible for zone B.
Achord [Hazel and Wiley 2002] is pro- To retrieve the entry corresponding to
posed as a variant of Chord that pro- K , any node can apply the same determin-
vides censorship resistance by limiting istic function to map K to P , and then re-
each node’s knowledge of the network in trieve the corresponding value V from the
ways similar to Freenet (see Section 3.4.1). node covering P . Unless P happens to lie
in the requesting node’s zone, the request
3.4.3. CAN. The CAN (“Content must be routed from node-to-node until it
Addressable Network”) [Ratnasamy reaches the node covering P .
et al. 2001] is essentially a distributed, CAN nodes maintain a routing table
Internet-scale hash table that maps file containing the IP addresses of nodes that
names to their location in the network, hold zones adjoining their own, to enable
ACM Computing Surveys, Vol. 36, No. 4, December 2004.

A Survey of Content Distribution Technologies 353
routing between arbitrary points in space. advanced routing metrics, such as connec-
Intuitively, routing in CAN works by fol- tion latency and underlying IP topology,
lowing the straight line path through the alongside the Cartesian distance between
Cartesian space from source to destination source and destination; allowing multiple
coordinates. For example, in Figure 6(a), a nodes to share the same zone, mapping the
request from node A for a key mapping to same key onto different points; and the ap-
point p would be routed though nodes A, plication of caching and replication tech-
B, E, along the straight line represented niques [Ratnasamy et al. 2001].
by the arrow.
A new node that joins the CAN system is
allocated its own portion of the coordinate 3.4.4. Tapestry. Tapestry [Zhao et al.
space by splitting the allocated zone of an 2001] supports the location of objects and
existing node in half, as follows: the routing of messages to objects (or the
closest copy of them, if more than one copy
(1) The new node identifies a node al- exist in the network) in a distributed, self-
ready in CAN network, using a boot- administering, and fault-tolerant manner,
strap mechanism as described in Fran- offering system-wide stability by bypass-
cis [2000]. ing failed routes and nodes, and rapidly
(2) Using the CAN routing mechanism, adapting communication topologies to cir-
the node randomly chooses a point P cumstances.
in the coordinate space and sends a The topology of the network is self-
JOIN request to the node covering P . organizing as nodes come and go, and net-
The zone is then split, and half of it is work latencies vary. The routing and loca-
assigned to the new node. tion information is distributed among the
(3) The new node builds its routing table network nodes; the topology’s consistency
with the IP addresses of its new neigh- is checked on-the-fly, and if it is lost due to
bors, and the neighbors of the split failures or destroyed, it is easily rebuilt or
zone are also notified to update their refreshed.
routing tables to include the new node. Tapestry is based on the location and
routing mechanisms introduced by Plax-
When nodes gracefully leave CAN, the ton, Rajamaran and Richa [1997], in which
zones they occupy and the associated they present the Plaxton mesh, a dis-
hash table entries are explicitly handed tributed data structure that allows nodes
over to one of their neighbors. Under to locate objects and route messages to
normal conditions, a node sends periodic them across an arbitrarily-sized overlay
update messages to each of its neighbors network, while using routing maps of
reporting its zone coordinates, its list of small and constant size. In the original
neighbors, and their zone coordinates. If Plaxton mesh, the nodes can take on the
there is prolonged absence of such an up- role of servers (where objects are stored),
date message, the neighbor nodes realize routers (which forward messages), and
there has been a failure, and initiate a con- clients (origins of requests).
trolled takeover mechanism. If many of Each node maintains a neighbor map,
the neighbors of a failed node also fail, an as shown in the example in Table IV. The
expanding ring search mechanism is ini- neighbor map has multiple levels, each
tiated by one of the neighboring nodes, to level l containing pointers to nodes whose
identify any functioning nodes outside the ID must be matched with l digits (the x’s
failure region. represent wildcards). Each entry in the
A list of design improvements is pro- neighbor map corresponds to a pointer to
posed over the basic CAN design described the closest node in the network whose ID
including the use of multi-dimensional matches the number in the neighbor map,
coordinate space, or multiple coordinate up to a digit position.
spaces, for improving network latency and For example, the 5th entry for the 3rd
fault tolerance; the employment of more level for node 67493 points to the node
ACM Computing Surveys, Vol. 36, No. 4, December 2004.

354 S. Androutsellis-Theotokis and D. Spinellis
Table IV. The Neighbor Map Held by Tapestry Node
With ID 67493
Level 5 Level 4 Level 3 Level 2 Level 1
Entry 0 07493 x0493 xx093 xxx03 xxxx0
Entry 1 17493 x1493 xx193 xxx13 xxxx1
Entry 2 27493 x2493 xx293 xxx23 xxxx2
Entry 3 37493 x3493 xx393 xxx33 xxxx3
Entry 4 47493 x4493 xx493 xxx43 xxxx4
Entry 5 57493 x5493 xx593 xxx53 xxxx5
Entry 6 67493 x6493 xx693 xxx63 xxxx6
Entry 7 77493 x7493 xx793 xxx73 xxxx7
Entry 8 87493 x8493 xx893 xxx83 xxxx8
Entry 9 97493 x9493 xx993 xxx93 xxxx9
Each entry in this table corresponds to a pointer to
another node.
closest to 67493 in network distance whose
ID ends in ..593. Table IV shows an exam-
Fig. 7. Tapestry: Plaxton mesh routing ex-
ple neighbor map maintained by a node ample, showing the path taken by a message
with ID 67493. originating from node 67493, and destined for
Messages are, therefore, incrementally node 34567 in a Plaxton mesh, using decimal
routed to the destination node digit-by- digits of length 5.
digit, from the right to the left. Figure 7
shows an example path taken by a mes- required for assigning and identifying root
sage from node with I D = 67493, to node nodes, and (2) the vulnerability of the root
I D = 34567. The digits are resolved right nodes.
to left as follows: The Plaxton mesh assumes a static node
population. Tapestry extends its design to
xxxx7 → xxx67 → xx567 → x4567 adapt it to the transient populations of
peer-to-peer networks and provide adapt-
→ 34567 ability, fault tolerance, as well as vari-
ous optimizations described in Zhao et al.
The Plaxton mesh uses a root node for [2001] and outlined below:
each object, that serves to provide a guar-
anteed node from which the object can —Each node additionally maintains a list
be located. When an object o is inserted of back-pointers, which point to nodes
in the network and stored at node ns , where it is referred to as a neighbor.
a root node nr is assigned to it by us- These are used in dynamic node inser-
ing a globally consistent deterministic al- tion algorithms to generate the appro-
gorithm. A message is then routed from priate neighbor maps for new nodes. Dy-
ns to nr , storing data in the form of a namic algorithms are employed for node
mapping (object id o, storer id ns ) at all insertion, populating neighbor maps,
nodes along the way. During a location and notifying neighbors of new node in-
query, messages destined for o are ini- sertions.
tially routed towards nr , until a node is —The concept of distance between nodes
encountered containing the (o, ns ) location becomes semantically more flexible, and
mapping. locations of more than one replica of
The Plaxton mesh offers: (1) simple an object are stored, allowing the ap-
fault-handling by its potential to route plication architecture to define how the
around a single link or node by choos- “closest” node will be interpreted. For
ing a node with a similar suffix, and (2) example, in the Oceanstore architecture
scalability (with the only bottleneck exist- [Kubiatowicz et al. 2000], a “freshness”
ing at the root nodes). Its limitations in- metric is incorporated in the concept
clude: (1) the need for global knowledge of distance, that is taken into account
ACM Computing Surveys, Vol. 36, No. 4, December 2004.

A Survey of Content Distribution Technologies 355
when finding the closest replica of a doc- availability to adapt to regional outages
ument. and denial of service attacks.
—The use of soft-state to maintain cached Pastry [Rowstron and Druschel 2001]
content, based on the announce/listen is a scheme very similar to Tapestry,
approach [Deering 1998], is adopted by differing mainly in its approach to
Tapestry to detect, circumvent, and re- achieving network locality and object
cover from failures in routing or ob- replication. It is employed by the PAST
ject location. Caches are periodically [Druschel and Rowstron 2001] large-
updated by refreshment messages, or scale persistent peer-to-peer storage
purged if no such messages are re- utility.
ceived. Additionally, the neighbor map Finally Kademlia [Mayamounkov and
is extended to maintain two backup Mazieres 2002] is proposed as an improved
neighbors, in addition to the closest (pri- XOR-topology-based routing algorithm
mary) neighbor. Furthermore, to avoid similar to Tapestry and Pastry, focusing
costly reinsertions of nodes after fail- on consistency and performance. It intro-
ures, when a node realizes that a neigh- duces the use of a concurrence parameter,
bor is unreachable, instead of removing that lets users trade bandwidth for better
its pointer, it temporarily marks it as in- latency and fault recovery.
valid in the hope that the failure will be
repaired, and, in the meantime, routes 3.4.5. Comparison, Shortcomings and Evo-
messages through alternative paths. lution of Structured Architectures. In the
—To avoid the single point of failure that previous sections, we have presented
root nodes constitute, Tapestry assigns characteristic examples of structured
multiple roots to each object. This en- peer-to-peer object location and routing
ables a trade-off between reliability and systems, and their main characteristics.
redundancy. A distributed algorithm A comparison of these (and similar) sys-
called surrogate routing is employed to tems can be based on the size of the
compute a unique root node for an object routing information maintained by each
in a globally consistent fashion, given node (the routing table), their search
the nonstatic set of nodes in the network and retrieval performance (measured in
—A set of optimizations improve per- number of hops), and the flexibility with
formance by adapting to environment which they can adapt to changing network
changes. Tapestry nodes tune their topologies.
neighbor pointers by running refresher It turns out that in their basic de-
threads that update network latency sign, most of these systems are equiva-
values. Algorithms are implemented to lent in terms of routing table space cost,
detect query hotspots and offer sugges- which is O(log N ), where N is the size of
tions as to where additional copies of network. Similarly, their performance is
objects can be placed to significantly im- again mostly O(log N ), with the exception
prove query response time. A “hotspot of CAN, where the performance is given by
1
cache” is also maintained at each node. O( d N d ), d being the number of employed
4
dimensions.
Tapestry is used by several systems Maintaining the routing table in the
such as Oceanstore [Kubiatowicz et al. face of transient node populations is rela-
2000; Rhea et al. 2001], Mnemosyne tively costly for all systems, perhaps more
[Hand and Roscoe 2002], and Scan [Chen for Chord, as nodes joining or leaving in-
et al. 2000]. duce changes to all other nodes, and less
Oceanstore further enhances the per- so in Kademlia which maintains a more
formance and fault tolerance of Tapestry flexible routing table. Liben-Nowell et al.
by applying an additional “introspec- [2002a] introduce the notion of the “half-
tion layer”, where nodes monitor usage life” of a system as an approach to this
patterns, network activity, and resource issue.
ACM Computing Surveys, Vol. 36, No. 4, December 2004.

356 S. Androutsellis-Theotokis and D. Spinellis
Overall, these systems present rela- The ability of structured peer-to-peer
tively equivalent solutions to routing and systems to maintain the structure re-
location in distributed peer-to-peer en- quired for routing in order to function effi-
vironments, focusing, however, on dif- ciently when nodes are joining and leaving
ferent aspects, and prioritizing differ- at a high rate is an open research issue.
ently the various design issues (see also The resilience of structured peer-to-peer
Balakrishnan et al. [2003]) that have been systems in the face of a very transient user
adopted by various systems. population is considered in Lv et al. [2002]
One of the major limitations of struc- and Liben-Nowell et al. [2002b].
tured peer-to-peer systems is that they are Finally, Saroiu et al. [2002] argue that,
designed to support exact match lookups. due to the deterministic nature of the
One needs to know the exact identifier topology of structured systems, it is pos-
(key) of a data item in order to be able to sible for an attacker to construct a set
locate the node that stores it. To address of node identifiers that, if inserted in the
this issue, keyword query techniques can network, could intercept all lookup re-
be designed on top of exact-match queries, quests originating from a particular node.
although it is arguable whether such There are mechanisms for preventing a
techniques can be efficiently implemented peer from deliberately selecting arbitrary
[Lv et al. 2002]. In Garces-Erice et al. identifiers, however, they can probabilisti-
[2004] an approach is proposed for aug- cally be circumvented, if enough addresses
menting peer-to-peer systems based on are selected.
distributed hash tables with a mecha-
nism for locating data using incomplete
information. The mechanism relies on 4. CONTENT CACHING, REPLICATION AND
distributing hierarchically-organized in- MIGRATION
dexes that maintain information about the
Peer-to-peer content distribution systems
data stored in nodes. A system to support
rely on the replication of content on more
complex queries over structured peer-to-
than one node for improving the avail-
peer systems is proposed in Harren et al.
ability of content, enhancing performance,
[2002]. This approach relies on the un-
and resisting censorship attempts. Con-
derlying peer-to-peer system for both in-
tent replication can be categorized as fol-
dexing and routing, and implements a
lows:
parallel query processing layer on top of it.
Heleher et al. [2002], furthermore, ar- Passive Replication. Content replication
gue that by creating keys for accessing occurs naturally in peer-to-peer systems
data items (i.e. by virtualizing the names as nodes request and copy content from
of the data items) two main problems one another. This is referred to as passive
arise: replication.
Cache-Based Replication. A form of
—Locality is Destroyed. Files originating replication employed in several systems,
from a single site are not usually colo- cache-based replication (e.g. OceanStore,
cated, meaning that opportunities for Mojonation, Freenet) is the result of
enhanced browsing, prefetching, and ef- caching copies of content as it passes
ficient searching are lost. They propose through nodes in the network. In Freenet
an approach allowing systems to exploit [Clarke et al. 2000] for instance, when a
object locality. search request succeeds in locating a file,
—Useful Application Level Information is the file is transferred through the network
Lost. Data used by applications is often node-by-node back to the originator of the
naturally organized in a hierarchical- request (see also Section 3.4.1). In the pro-
fashion. The virtualization of the file cess, copies of the file are cached on all
namespace that takes place as files are intermediated nodes, increasing its avail-
represented by keys discards this infor- ability and location characteristics in fu-
mation. ture requests.
ACM Computing Surveys, Vol. 36, No. 4, December 2004.

A Survey of Content Distribution Technologies 357
Active Replication. Active (or proactive) extensive data replication and higher
content replication and migration meth- availability.
ods are often employed to improve locality
of data, as well as availability and perfor- Replication is of increased significance
mance. to systems employing steganographic file
Instrospective replica management tech- storage techniques for encryption, such as
niques are employed by OceanStore, on Mnemosyne [Hand and Roscoe 2002], as
top of extensive caching, whereby traffic these systems must ensure that enough
is observed, and replicas of content ob- copies of the files are available to avoid
jects are created to accommodate demand collisions. Such encryption techniques are
[Rhea et al. 2001]. OceanStore thus adapts further discussed in Section 5.
to regional outages and denial of service Replication is more of a challenge for
attacks by migrating data to areas of use, structured systems such as Chord, where
and maintaining sufficiently high levels of the location of objects is directly bound to
data redundancy. the object identifiers. In some cases, the
A similar approach is seen in MojoNa- use of aliases is employed to allow for repli-
tion, where additional copies of popular cation [Stoica et al. 2001; Saroiu et al.
data are made by a central services bro- 2002]. In Tapestry, a replication function
ker that monitors traffic and requests, and produces a set of random keys, yielding a
are distributed to peer nodes for storage set of replica roots at random points in the
[MojoNation 2003]. ID space; a replica function in Chord maps
Dynamic replica management algo- an object’s key to the node IDs of the neigh-
rithms are used in Scan to dynami- bor set of the key’s root; in CAN a replica
cally place a minimum number of repli- function produces random keys for stor-
cas, while meeting quality of service and ing replicas at diverse locations [Wallach
server capacity constraints [Chen et al. 2002]
2000]. Such algorithms are designed to Finally mechanisms for trading content,
satisfy both client latency and server loads discussed in Section 8, essentially result in
by first searching for existing replicas an additional form of data replication. In
that meet the client latency constraint FreeHaven, for example, peers frequently
without being overloaded, and, if unsuc- trade and exchange parts of documents
cessful, proceeding to place new replicas. (shares) with each other, with the sole aim
Different versions of these algorithms dif- of increasing availability and censorship
fer in the degree to which replicas are en- resistance [Dingledine et al. 2001a].
forced to lie close to the client requesting
them.
5. SECURITY
In proactive replication studies for the
Gnutella network [Cohen and Shenker Peer-to-peer content distribution architec-
2001], it was found that the optimal repli- tures present a particular challenge for
cation scheme for nondeterministic search providing the levels of availability, privacy,
is to replicate objects such that confidentiality, integrity, and authentic-
√ ity often required, due to their open and
p∝ qr , autonomous nature. The network nodes
must be considered untrusted parties, and
where p is the number of object replicas, no assumptions can be made regarding
and qr is the object query rate. their behavior.
By replicating content, data consistency This Section discusses various ap-
and synchronization issues come up that proaches that address a number of secu-
need to be addressed, especially in sys- rity issues characteristic of peer-to-peer
tems that allow the deletion and update of content distribution systems. We particu-
content (see also Section 9). Some appli- larly focus on secure storage, secure rout-
cations effectively decide to weaken their ing, access control, authentication, and
consistency restrictions in favor of more identity management.
ACM Computing Surveys, Vol. 36, No. 4, December 2004.

358 S. Androutsellis-Theotokis and D. Spinellis
5.1. Secure Storage Anonymous Cryptographic Relays. A
mechanism based on publisher, forwarder,
Various cryptographic algorithms and pro-
storer, and client peers communicating
tocols are employed to provide security for
through anonymous connection layers is
content published and stored in peer-to-
employed by PAST [Serjantov 2002]. A
peer networks.
publisher selects several forwarders and
Self-Certifying Data. Self-certifying sends to them, via an anonymous con-
data is data whose integrity can be nection, encrypted shares of a file. The
verified by the node retrieving it [Castro forwarders, in turn, select other nodes
et al. 2002]. A node inserting a file in the to act as storers for the shares, and for-
network calculates a cryptographic hash ward the shares to them (again via an
of the file’s content, based on a known anonymous connection). Once all shares
hashing function, to produce the file key. are stored, the publisher destroys them,
When a node retrieves the file using its and announces the file name, together
key, it calculates the same hash function with the list of forwarders that were
to verify the integrity of the data. Note the used.
requirement for common knowledge of the In order to retrieve the content, a client
hashing function by all nodes in the net- will contact the forwarders; the forwarders
work. The above approach is adopted by will, in turn, contact random servers to
the CFS system [Dabek et al. 2001], while act as decryptors for the addresses of the
PAST uses a similar approach, based storers holding the shares. The forwarders
on signed files [Druschel and Rowstron will then contact the storers requesting the
2001]. Note that the method depends on shares. The storers will decrypt the shares
using the hash-derived signature as a and send them back to the client. The pro-
file’s unique access key. cess repeats until enough shares are col-
Information Dispersal. The information lected to reconstruct the document.
dispersal algorithm by Rabin [1989] is Forwarders are visible to a potential at-
widely used. Publius [Waldman et al. tacker, but they are less likely to be at-
2000], Mnemosyne [Hand and Roscoe tacked since they don’t store the content,
2002], and FreeHaven [Dingledine et al. nor the addresses of the nodes storing the
2000] employ the algorithm for encoding content.
information to be published in the net- Smartcards. The use of smartcards for
work. Files are encoded into m blocks, tracking each node’s use of remote re-
so that any n is sufficient to reassem- sources and issuing digitally signed tick-
ble the original data (m < n). This gives ets was also suggested in the PAST system
resilience “proportional” to a redundancy [Druschel and Rowstron 2001]. This would
factor equal to m .
n
allow nodes to prove to other nodes that
Shamir’s Secret Sharing Scheme. they are operating within their quota. It
Shamir’s [1979] secret sharing scheme is argued, however, that it may be imprac-
(SHA) is also used in several systems tical to issue smartcards to a very large
(Publius [Waldman et al. 2000], Mo- number of nodes, and that the danger of
joNation [MojoNation 2003], and PAST the smartcard keys being compromised by
[Druschel and Rowstron 2001]). The pub- nodes of the network is considerable [Wal-
lisher of the content encrypts a file with lach 2002].
a key K , then splits K into l shares, so Distributed Steganographic File Sys-
that any k of them can reproduce K , but tems. A distributed steganographic file sys-
k − 1 give no hints about K . Each server tem, based on Anderson et al. [1998] is
then encrypts one of the key shares, along implemented by Mnemosyne [Hand and
with the file block. In order for the file to Roscoe 2002]. The main property of a
become inaccessible, at least (l − k − 1) steganographic file system is that en-
servers containing the key must be shut crypted blocks are indistinguishable from
down. a random substrate, so that their presence
ACM Computing Surveys, Vol. 36, No. 4, December 2004.

A Survey of Content Distribution Technologies 359
cannot even be detected. The system is 5.2. Secure Routing
prepared by first writing random data to
The goal of secure routing is to resolve
all blocks, and then files are stored by
the problem of malicious nodes attempt-
encrypting their blocks and placing them
ing to corrupt, delete, deny access to, or
at pseudo-randomly chosen locations (e.g.
supply stale copies of object replicas that
by hashing the block number with a ran-
are transferred between nodes.
domly chosen key). To avoid collisions, a
The secure routing primitives proposed
considerable amount of replication is re-
in Castro et al. [2002] cope with the prob-
quired.
lems of:
Erasure Coding. A mechanism based on
erasure coding for data durability and a
(1) secure assignment of IDs to nodes.
Byzantine agreement protocol [Lamport
et al. 1982] for consistency and update se- (2) secure maintenance of routing tables.
rialization is implemented by OceanStore (3) secure forwarding of messages.
[Rhea et al. 2001].
With erasure coding, the data is broken These primitives can be combined with
in to blocks and spread over many servers. other existing security techniques to yield
Only a fraction is needed to reconstruct more robust applications.
the original block (similar to the infor-
mation dispersal algorithm). The objects 5.3. Access Control, Authentication and
and their associated fragments are then Identity Management
named using a secure hash over the ob-
ject contents, giving them globally unique The issues of access control, authentica-
identifiers. This provides data integrity, by tion, and identity management are of-
ensuring that a recovered file has not been ten ignored in peer-to-peer content distri-
corrupted, since a corrupted file would pro- bution systems. When an access control
duce a different identifier. Blocks are dis- mechanism is used, it follows the discre-
persed with care to avoid possible corre- tionary access control paradigm [Ander-
lated failures, picking nodes in different son 2001], clearly an insecure approach in
geographic locations or administrative do- the face of untrusted clients.
mains, or based on models of historical Within a distributed environment, it is
measurements. possible for the same physical entity to
An “inner ring” of servers is set up for appear under different identities, partic-
each object to provide versioning capa- ularly in systems with highly transient
bilities. The role of the ring is to main- populations of nodes. This poses a secu-
tain a mapping from the original identi- rity threat, especially in peer-to-peer sys-
fier of the object, to the identifier of the tems that employ content replication, or
most recent version of the object. The fragmentation schemes over many peers
ring is kept consistent through a Byzan- for security and availability (see also Sec-
tine agreement protocol that lets 3 f + tion 5.1), and, therefore, rely on the exis-
1 servers reach an agreement when no tence of independent peers with different
more than f are faulty. So the map- identities.
ping from an active (original) identifier, This problem is labeled a “Sybil Attack”
to the most recent identifier, is fault and analyzed in Douceur [2002], conclud-
tolerant. ing that unless a central certification or
The inner ring is also responsible for identification authority is employed, peer-
verifying the object’s legitimate writers to-peer systems will be susceptible to this
and maintaining a history of the object’s kind of attack. The only proposed alter-
updates, thus providing a universal undo native is the use of (typically imprac-
mechanism, as well as true referential tical) resource-demanding identification
integrity by storing previous versions of challenges, that assume that the resources
objects. of the attacker are limited.
ACM Computing Surveys, Vol. 36, No. 4, December 2004.

360 S. Androutsellis-Theotokis and D. Spinellis
In several systems, the lack of authen- ing anonymity to its users by making it
tication is overcome by the distribution of infeasible to discover the true origin or
the keys necessary for accessing content destination of a file passing through its
to a subset of privileged users. network, and difficult for a node opera-
Access control lists can also be as- tor to determine or be held responsible for
signed to objects by their original authors the actual physical content of their own
through the use of signed certificates (e.g. node. It employs a mechanism whereby,
in Oceanstore [Kubiatowicz et al. 2000]). when a search for a file succeeds, the file
All content modifications relating to an ob- is passed from the node that holds it to
ject are then verified against the access the originator of the request through ev-
control list that is assigned to it, and unau- ery node that forwarded the search re-
thorized updates are ignored. quest. In order to achieve anonymity, any
The issue of peers that often appear node along this path can unilaterally de-
with different identities in order to escape cide to claim itself or another arbitrarily
the consequences of past actions, and, in chosen node as the data source. In this
particular, the effects of this on the per- way, there is no connection between the
formance of reputation or incentive mech- requestor of the file and the actual source
anisms is discussed in Lai et al. [2003]. node.
Finally, it is argued that access control As a further measure, the initial value
is closely related to intellectual property of the hops-to-live counter associated with
management and digital rights manage- each search message is randomly chosen
ment issues. Whether—or which—peer-to- in order to obscure the distance of the
peer systems should enforce access control message from the originator node. More
and to what extent, and whether this en- details about the location and routing
forcement should take place at the end- mechanism employed by Freenet are dis-
points of the network, remains an open cussed in Section 3.4.1.
issue [Daswani et al. 2003]. Anonymous Connection Layers. Con-
tent distribution systems that seek to
6. PROVISIONS FOR ANONYMITY provide anonymity often employ infras-
tructures for providing anonymous con-
Anonymity is the main focus of sev- nection layers, such as onion routing
eral peer-to-peer based infrastructures [Goldschlag et al. 1999], mix networks
and content distribution systems target- [Chaum 1981; Berthold et al. 1998] or
ing privacy, confidentiality, and censor- the Freedom anonymity system [Freedom
ship resistance. In content distribution 2003]. For instance, the anonymizing,
systems anonymity can refer to [Dingle- censorship-resistant system proposed in
dine et al. 2001a]: Serjantov [2002] splits documents to be
—the author (or publisher) of the content, published into encrypted shares, and uses
an anonymizing layer of nodes (which it
—the identity of a node storing the
refers to as “forwarders”) to pick nodes
content,
on which to store the shares, construct
—the identity and details of the content “onions” around them (as in onion rout-
itself, and ing), and anonymously forward them,
—the details of a query for retrieval of the including their anonymous return ad-
content. dresses, while the original documents
are destroyed. FreeHaven [Dingledine
This section describes the main ap-
et al. 2000] is a similar system built
proaches currently adopted for providing
on top of anonymous remailers for pro-
anonymity.
viding pseudonyms and communication
Disassociation of Content Source and channels.
Requestor. Freenet [Clarke et al. 2000] The Tarzan system [Freedman et al.
is a peer-to-peer content distribution sys- 2002] is a completely decentralized
tem specifically targeted towards provid- anonymizing network layer infrastructure
ACM Computing Surveys, Vol. 36, No. 4, December 2004.

A Survey of Content Distribution Technologies 361
that builds anonymous IP tunnels be- their node. As a consequence, users cannot
tween an open-ended set of peers. By be held responsible for the content stored
using the Tarzan infrastructure, a client in their node, or for actions that have been
can communicate with a server without carried out by their node as part of it’s op-
anybody being able to determine their eration in the peer-to-peer network.
identity. A deniable peer-to-peer system can be
The Tarzan anonymizing layer is com- used without fear of censorship or reper-
pletely decentralized and transparent cussions for dealing with particular con-
both to clients and servers, and it involves tent, increasing the degree of freedom it
sequences of mix relays chosen from a provides to its users. On the other hand,
large pool of volunteer participant nodes. it makes the exchange of illegal content
Packets are routed through a tunnel of less risky and, therefore easier, as there is
randomly chosen Tarzan peers using mix- less control over the operation of content
style layered encryption, similar to onion stored at each node of the network.
routing. The two ends of the tunnel are a Deniability can apply both to content be-
Tarzan node, running a client application, ing stored and content being transferred.
and a Tarzan node, running a network ad-
dress translator. The latter forwards the Deniability of Stored Content. This fea-
traffic to the ultimate destination, an or- ture is offered by systems that store en-
dinary Internet server. A suggested policy crypted shares of files and no keys for
to reduce the risk of attack to the system is them, and, therefore, cannot know the con-
for tunnels to contain peers from different tent of the files whose shares they are
jurisdictions or organizations, even if per- storing (examples are Publius [Waldman
formance is sacrificed. Crowds [Reiter and et al. 2000], Oceanstore [Kubiatowicz
Rubin 1999] is a system similar to Tarzan, et al. 2000], and PAST [Druschel and
its main difference being that, in Tarzan, Rowstron 2001; Serjantov 2002]). Simi-
the data is encrypted in the tunnels, while larly, systems that implement distributed
in Crowds it is not. steganographic storage systems (such as
Censorship Resistant Lookup. A censor- Mnemosyne [Hand and Roscoe 2002]) of-
ship resistant design based on the Chord fer deniability through the fact that blocks
lookup service is proposed in the Achord of files that are written on a peer node’s
system [Hazel and Wiley 2002]. Achord file system are undetectable (cannot be di-
provides censorship resistance by focus- rectly searched).
ing on publisher, storer, and retriever Deniability of Content in Transit. This
anonymity, and by making it difficult for feature can be offered through the use
a node to voluntarily assume responsi- of anonymous connection layers incorpo-
bility for a certain document. The de- rated in systems such as PAST (see also
sign of Chord is carefully varied so that Section 6). It makes it possible to deny
these anonymity and censorship resis- that a request from a client node nc had
tance properties are achieved without anything to do with a share arriving at nc
altering its main operation. In particular, some time later.
the Chord algorithm is modified so that In a different approach with the same
the node identification information is sup- goal, Freenet [Clarke et al. 2000] offers
pressed as the successor nodes are located. deniability by making it infeasible to dis-
More details about the Chord Distributed cover the true origin of a file passing
Object Location and Routing system can through the network (see Section 3.4.1
be found in Section 3.4.2. for a description of the Freenet location
algorithm). Furthermore, as files passing
through the network are transparently
7. PROVISIONS FOR DENIABILITY
cached at all nodes encountered, it is dif-
Deniability in a peer-to-peer content dis- ficult for a node operator to determine or
tribution system refers to each user’s abil- be held responsible for the actual physical
ity to deny knowledge of content stored in content of their own node.
ACM Computing Surveys, Vol. 36, No. 4, December 2004.

362 S. Androutsellis-Theotokis and D. Spinellis
Additional measures for deniability can ticularly challenging task, especially due
be incorporated by inhibiting the use to the absence of a ubiquitous, effective,
of traffic analysis for concluding where robust, and secure system for making and
a file was read from. For example, in accepting anonymous micropayments.
Mnemosyne [Hand and Roscoe 2002], dur- Two general categories of solutions are
ing the retrieval of a file, more nodes than proposed:
those needed will be contacted and more
files will be retrieved so that the actual —Trust-based Incentive Mechanisms.
file targeted will be disguised. Trust is a straightforward incentive for
It should be noted that structured sys- cooperation, in which one engages in
tems are apparently nondeniable, as the a transaction based on whether he/she
identifiers of the files stored at the nodes trusts the other party. Reputation
are bound to the node addresses; if a file mechanisms belong in this category.
is known to exist in the network, its lo- —Trade-based Incentive Mechanisms. In
cation, and, therefore, the identity of the trade-based incentive mechanisms, one
node that stores it, is also known. On the party offering some service to another
other hand, the owner of the node has not is explicitly remunerated, either di-
necessarily requested the file, and, in any rectly or indirectly. This category is
event, has no control over whether the file mainly represented by various mi-
will be stored in their node, and, in this cropayment mechanisms and resource
sense, cannot be held responsible for it. trading schemes.
8. INCENTIVE MECHANISMS AND 8.1. Reputation Mechanisms
ACCOUNTABILITY
Online reputation management systems
The operation, performance and availabil- can be described as large-scale “online
ity of an uncontrolled decentralized peer- word-of-mouth communities” in which in-
to-peer system relies to a large extent on dividuals share opinions about other indi-
the voluntary participation of its users. It viduals.
is, therefore, necessary to employ mech- Centralized reputation systems (such
anisms that provide incentives and stim- as the one found in eBay) are success-
ulate cooperative behavior between the ful to a large extent because people trust
users, as well as some notion of account- the reputation information presented by
ability for actions performed. them [Dellarocas 2001]. In a peer-to-
In the absence of such provisions, the re- peer network, however, there is no sin-
sults can range from significant degrada- gle, recognizable organization or entity to
tion of performance to variable and unpre- maintain and distribute reputation infor-
dictable availability of resources, or even mation. As a result, reputation informa-
to complete collapse. tion must be distributed throughout the
An example of uncooperative behavior network, and hosted on many different
is the so-called “free-rider” effect, where nodes.
users only consume resources without con- The main goal of a peer-to-peer rep-
tributing any. This can be interpreted as a utation mechanism is to take the repu-
manifestation of the “Tragedy of the Com- tation information that is locally gener-
mons” [Harding 1968], which argues that ated as a result of an interaction between
people tend to abuse shared resources that peers, and spread it throughout the net-
they do not have to pay for in some way. work to produce a global reputation rat-
Providing incentives and accountabil- ing for the network nodes. In the pro-
ity in peer-to-peer networks with tran- cess, such reputation information must be
sient populations of users, where it is hard kept secure and available. Various com-
to identify peers and obtain information plex reputation management mechanisms
about their past behavior in order to pre- have been developed to address these chal-
dict their future performance, can be a par- lenging tasks.
ACM Computing Surveys, Vol. 36, No. 4, December 2004.

A Survey of Content Distribution Technologies 363
Kamvar et al. [2003] propose the Eigen- At the development level, peer-to-peer
Trust algorithm, which produces global frameworks such as Sun’s JXTA [Jxta
reputation ratings for users based on their 2003] include provisions for incorporat-
history of uploads. These ratings can then ing peer-monitoring hooks in a distributed
be used by other peers to decide where to content distribution environment, that
download files from. The global reputation could be used for collecting transaction in-
values are computed from the local repu- formation and building centralized or dis-
tation values assigned to a peer by other tributed reputation databases.
peers, weighted by the global reputation Some potential problems with dis-
of the assigning peers. This approach was tributed reputation management need to
found to reduce the number of rogue files be addressed. For example, an entity ac-
on the network. quiring reputation in one context where it
Gupta et al. [2003] present a partially has functioned well, may spend it in an-
centralized mechanism using reputation other context for malicious purposes. An
computation agents and data encryption, immediate extension of the above scenario
where the reputation values are calcu- is the possibility of using reputation as
lated, encrypted, and stored locally using currency, exchanging it for content or ser-
a reputation computation agent. They pro- vices.
pose two different schemes for calculating A discussion of accountability and rep-
reputation values, a credit/debit scheme, utation can also be found in Dingledine
and a credit only scheme. et al. [2001b].
Xiong and Liu [2002] present a
feedback-based reputation mechanism
8.2. Micropayments Mechanisms
where a scalar trust value for a peer is
computed based on a figure of the satis- A variety of both centralized [MojoNation
faction received by other peers, the total 2003; Ioannidis et al. 2002] and decen-
number of interactions, and a balancing tralized [Vishnimurthy et al. 2003; Yu
factor to counteract reports by malicious and Singh 2003] currency-based micro-
peers. The reputation information is dis- payment and credit systems have been de-
tributed in the network so that each peer ployed in peer-to-peer networks.
maintains a portion of the total trust data. A typical example is the MojoNation
Various simple (and to some extent system, where a currency (the “Mojo”) is
naive) approaches calculate a user par- gained by offering disk space, bandwidth,
ticipation level [Kazaa 2003] and reward or CPU cycles, and used to obtain access
peers with higher participation by giving to distributed storage space. This system
higher priority to their requests, or by re- employs a scheme based on trusted third
ducing their waiting times in file trans- parties to ensure honest transactions be-
fer queues. In a more abstract approach tween peers.
[Ramaswamy and Liu 2003], utility More complex systems offer more ad-
functions, based on the amount and pop- vanced capabilities to users, for example,
ularity of content stored by the peer, are allowing them to negotiate prices and per-
used to estimate a figure for their useful- form auctions for services rendered (e.g.
ness. Such mechanisms are generally easy Vishnimurthy et al. [2003])
to subvert by malicious users. It is, however, claimed in Buragohain
Finally, distributed auditing mecha- et al. [2003] that monetary schemes, al-
nisms have also been devised to apply though providing clear economic models,
peer-pressure to misbehaving nodes [Wal- may be impractical for most applications.
lach 2002]. Nodes can consume resources
to compare the logs of another node, cho-
8.3. Resource Trading Schemes
sen at random, with the logs of every node
with which it is sharing information. The Various decentralized resource trading
system appears to provide strong disin- and exchange schemes have been pro-
centives to cheaters at a reasonable cost. posed [Anagnostakis and Greenwald
ACM Computing Surveys, Vol. 36, No. 4, December 2004.

364 S. Androutsellis-Theotokis and D. Spinellis
2004; Cohen 2003; Cooper and Garcia- and less likely to be lost due to failures.
Molina 2002; Dingledine et al. 2000]. Digital libraries may collaborate with one
Anagnostakis and Greenwald [2004] another to provide content preservation
proposed an extension to accommodate by storing each other’s material. Systems
transitive chains of peer transactions, in- such as OceanStore [Kubiatowicz et al.
stead of simple one-to-one exchanges. 2000] and Intermemory [Chen et al. 1999]
A system in which each node publishes employ this idea.
and digitally signs logs containing lists of Finally, it is argued in Chun et al. [2003]
files it stores on behalf of remote nodes and that such resource trading models are par-
lists of files that other nodes are storing on ticularly suitable for bootstrapping peer-
its behalf is proposed in Wallach [2002]. to-peer economies in the first period of
Through the use of these lists it can be en- their operation.
sured that no node is using more resources
than it is providing, and a balanced system
is achieved. 9. RESOURCE MANAGEMENT
A flexible algorithm for trading content CAPABILITIES
is proposed by Cooper and Garcia-Molina
The resources that peer-to-peer content
[2002]. Peers make local decisions about
distribution systems typically deal with
how much data to trade, what resources to
are content (files), storage (disk space),
contribute and how many trades to make.
and transmission capacity (bandwidth).
This algorithm ensures fairness while try-
Obviously inserting, locating (searching),
ing to maximize content preservation by
and retrieving content are the minimum
choosing the most appropriate policies for
operations that any system is required
deed trading, advertising, remote site se-
to perform. Additional resource manage-
lection, and bidding.
ment facilities may include removing or
Another example of a purely decentral-
updating content, maintaining previous
ized reputation system based on resource
versions of updated content, managing
trading exist seen in FreeHaven [Dingle-
storage space, and setting bandwidth lim-
dine et al. 2000]. The FreeHaven nodes
its. Different peer-to-peer content distri-
form contracts to store each other’s ma-
bution system allow different levels of re-
terial for a certain period of time. By
source management, as now described.
successfully fulfilling a contract, a node
increases its reputation. Contracts are Content Deletion and Update. Deleting
formed whenever new data is inserted in and updating content are not straight-
the network, or swapped between peers. A forward operations in a peer-to-peer envi-
trading handshake protocol is employed. ronment if correct synchronization is to be
It includes “receipts” for resources traded, maintained. Systems such as MojoNation
and the use of “buddy” peer nodes that [MojoNation 2003] use immutable files,
record the transactions carried out by which cannot be updated. The only way
other nodes. Each node, therefore, main- to update the content is to post a new
tains a database containing the perceived version of the document under a differ-
reputation of others. Nodes periodically ent name. PAST [Druschel and Rowstron
check whether other nodes have dropped 2001] is similar in this respect. It does of-
data earlier than they were supposed to, fer a delete functionality for reclaiming
and decrease the level of trust of these the disc space occupied by a file, but it
peers. In order for nodes to leave the net- does not guarantee that the file will no
work without decreasing their reputation longer be available anywhere in the net-
value, they must first swap their docu- work. FreeHaven [Dingledine et al. 2000]
ments for shorter-lived ones, and wait un- also forbids unpublishing, or revocation of
til they expire. documents, but mainly for reasons of se-
An additional benefit of trading content curity and robustness to attacks.
is that it offers a way of actively replicat- Both deletion and update of content are
ing it, and hence, making it more available offered by Publius [Waldman et al. 2000],
ACM Computing Surveys, Vol. 36, No. 4, December 2004.

A Survey of Content Distribution Technologies 365
which is, however, based on a static set of contributed can be specified by each peer
servers that store the content. Files are independently. In systems such as Mo-
published along with a password file that joNation users contribute storage in ex-
ensures that only the publisher will be change for economic or other compensa-
able to delete or modify the content. tion. PAST [Druschel and Rowstron 2001]
Content Expiration. Document expi- uses a secure quota system, where users
ration dates are effectively introduced are either assigned a fixed quota of storage
in FreeHaven [Dingledine et al. 2000] they can use, or they can use as much as
through the use of contracts with different they contribute on their node. This system
durations, as described in Section 8. can be optionally extended to incorporate
Content Versioning. A more sophisti- the use of smartcards.
cated approach is employed by OceanStore In order to protect from denial of ser-
[Rhea et al. 2001], which offers a version- vice attacks, apart from enforcing a pub-
based archival storage system. Versions of lished size quota to users, the idea of
documents are stored in permanent read- “hash cash” is also employed (e.g. in Pub-
only form, encoded with erasure codes, lius [Waldman et al. 2000]). A publisher
and spread over hundreds of servers. is requested to carry out computational
OceanStore maintains active maps of doc- work to solve a mathematical problem be-
ument IDs, which redirect to the most fore being allowed to publish a certain
recent version of a document, provid- amount of content. Alternatively, a per-
ing both an update history and an undo node rate limiter can be used, as is the case
mechanism. for example in Mnemosyne [Hand and
Directory Structure. An entire dis- Roscoe 2002]. Bandwidth management is
tributed directory structure system in- also available in systems such as Kadem-
cluding directories and Unix-like inodes lia [Mayamounkov and Mazieres 2002],
[Bach 1986] is built on top of files by which allows the users to trade bandwidth
Mnemosyne [Hand and Roscoe 2002]. Di- for latency and fault recovery.
rectories are optionally used to aggregate
files sharing a common key, serving as a
10. SEMANTIC GROUPING OF
mnemonic service for a set of file names.
INFORMATION
Content Searching. Searching facilities
may also vary in their degree of function- The topic of semantic grouping and orga-
ality and performance. Unstructured sys- nization of content and information within
tems such as Gnutella, Kazaa, and Free- peer-to-peer networks has attracted con-
Haven offer keyword-search mechanisms siderable research attention lately [Zhou
that are convenient, but do not scale well et al. 2003; Ayyasamy et al. 2003; Castano
(see also our discussion in 3.3.4). In struc- et al. 2003; Broekstra et al. 2003], driven
tured systems, searches are very efficient, to a large extent by the recent advances in
but can only be based on file identifiers. the field of knowledge management.
The problem of providing keyword search Khambatti et al. [2003] introduce the
facilities on top of exact-match queries is notion of interest-based “peer communi-
open. One solution proposed in Freenet is ties”, similar to real human communities,
the use of indirect files, published along in which membership depends on the re-
with regular files and pointing to them, lationship between peers that share com-
which are named according to relevant mon interests. The authors argue—and
search keywords. The problem of manag- show through their experiments—that
ing the large volume of such files, however, such communities can be used to make
also remains unresolved [Clarke et al. searching more efficient and produce bet-
2000]. ter quality of results. The grouping is im-
Storage and Bandwidth Management. plemented through sets of attributes that
Managing the storage space available the peers choose to claim, and communi-
to the peers also varies between sys- ties are formed between peers that share
tems. Usually the amount of disk space similar attributes. As an extension of this
ACM Computing Surveys, Vol. 36, No. 4, December 2004.

366 S. Androutsellis-Theotokis and D. Spinellis
notion, a peer-to-peer community-based able to self-organize into network topolo-
trust model is introduced in Khambatti gies with the purpose of sharing resources
et al. [2004], where links between peers such as content, CPU cycles, storage, and
that trust each other are used to form dy- bandwidth. Content distribution, a promi-
namic coalitions between different com- nent application area of peer-to-peer sys-
munities. tems, is based on systems and infrastruc-
The semantic overlay clustering ap- tures designed for sharing digital media
proach, based on partially-centralized and other data between users.
(super-peer) networks [Loeser et al. 2003] Peer-to-peer content distribution sys-
aims at creating logical layers above the tems range from relatively simple direct
physical network topology, by matching file sharing applications, to more sophis-
semantic information provided by peers ticated systems that create a distributed
to clusters of nodes based on super-peers. storage infrastructure for securely and ef-
Clusters of nodes with similar semantic ficiently publishing, organizing, indexing,
profiles are, therefore, formed based on searching, updating, and retrieving data.
some matching engine and clustering pol- We performed this study of peer-to-
icy. Each cluster contains a set of peers, peer content distribution systems and
together with their corresponding super- infrastructures by identifying the fea-
peer. ture space of their nonfunction proper-
Bonifacio et al. [2002] propose a dis- ties, and determining the way in which
tributed knowledge management archi- these nonfunctional properties depend on,
tecture consisting of knowledge nodes and are affected by various design fea-
(peers that own local knowledge), that co- tures. The main nonfunctional character-
operate through a set of services. Each istics we identified include provisions for
node can act as a seeker (allowing the user security, anonymity, fairness, increased
to search for information and forwarding scalability, and performance, as well as
queries), as a provider (accepting and re- resource management, and organization
solving queries and returning results), or capabilities.
both. Searching can be either semantic One of the central characteristics of
(based on a context matching algorithm), peer-to-peer systems—which reflects upon
or textual (based on keywords). Groups of issues of performance, availability, and
nodes have the ability to form federations, scalability—is their ability to function,
similar to social aggregations, by agreeing scale, and self-organize in the presence
to be considered as a sole entity by other of a highly transient population of nodes
peers when they request certain semantic and network and computer failures, with-
search services. out the need for a central server admin-
A peer-to-peer architecture based on se- istration. This characteristic is mainly at-
mantic grouping, context, and peer pro- tributed to the network organization and
files applied in the area of eLearning is location and routing algorithms. Two gen-
proposed in Hummel et al. [2003]. eral categories of systems can be iden-
From a different perspective, a num- tified in this respect: the unstructured
ber of different mechanisms are proposed systems and the structured (or DHT-based)
for constructing efficient overlay network systems. The rationale behind structured
topologies by taking into account the lo- systems often was to conquer the se-
cality and network proximity of peer com- vere scalability and performance prob-
puters to decrease the communication cost lems of unstructured ones. However, re-
[Zhang et al. 2004; Zhao et al. 2002; Castro cent research developments in both struc-
et al. 2002]. tured and unstructured systems, ren-
ders them both as complementary and
acceptable solutions.
11. CONCLUSIONS
Research on other aspects and prop-
Peer-to-peer systems are distributed sys- erties of peer-to-peer content distri-
tems consisting of interconnected nodes, bution systems has taken advantage
ACM Computing Surveys, Vol. 36, No. 4, December 2004.

A Survey of Content Distribution Technologies 367
of knowledge and solutions from nu- ANAGNOSTAKIS, K. AND GREENWALD, M. 2004.
merous scientific fields. Characteristic Exchange-based incentive mechanisms for
peer-to-peer file sharing. To Appear in the Pro-
examples include research on the applica- ceedings of the 24th International Conference on
tion of reputation systems in peer-to-peer Distributed Computing (ICDCS04).
environments, the semantic organization ANDERSON, R. 2001. Security Engineering: A Guide
of information, as well as the use of cryp- to Building Dependable Distributed Systems.
tographic techniques for the protection of John Wiley & Sons, New York.
data, both stored and in transit through ANDERSON, R., NEEDHAM, R., AND SHAMIR, A. 1998.
The steganographic file system. In Proceedings
the network. of International Workshop on Information Hid-
Finally, as peer-to-peer technologies are ing (IWIH) .
still evolving, there are a multitude of open AYYASAMY, S., PATEL, C., AND LEE, Y. 2003. Seman-
research problems, directions, and oppor- tic web services and dht-based peer-to-peer net-
tunities, including, but not limited to: works: A new symbiotic relationship. In Proceed-
ings of the 1st Workshop on Semantics in Peer-
—The design of new distributed object lo- to-Peer and Grid Computing at the 12th Inter-
cation, routing and distributed hash ta- national World Wide Web Conference. Budapest,
Hungary.
ble data structures and algorithms for
BACH, M. 1986. The Design of the UNIX Operating
maximizing performance, security and System. Prentice-Hall.
scalability, both in structured and un- BALAKRISHNAN, H., KAASHOEK, M., KARGER, D., R, M.,
structured network architectures. AND STOICA, I. 2003. Looking up data in p2p
—The study of more efficient security, systems. Comm. ACM 46, 2 (Feb.), 43–48.
anonymity, and censorship resistance BERNERS-LEE, T., HENDLER, J., AND LASSILA, O. 2001.
The semantic web. Scientific American.
schemes. These features will be critical
BERNSTEIN, P., GIUNCHIGLIA, F., KEMENTSIETSIDIS, A., MY-
to the future of peer-to-peer systems and LOPOULOS, J., SERAFINI, L., AND ZAIHRAYEU, I. 2002.
their adoption for increasingly more sen- Data management for peer-to-peer computing: A
sitive applications. vision. In Proceedings of the Workshop on the Web
and Databases (WebDB’02).
—The semantic grouping of information
BERTHOLD, O., FEDERRATH, H., AND KOPSELL, S. 1998.
in peer-to-peer networks. This direction Web mixes: A system for anonymous and unob-
that has a lot in common with efforts in servable internet access. In Proceedings of the
the semantic Web domain. Workshop on Design Issues in Anonymity and
Unobservability. Berkeley, CA.
—The design of incentive mechanisms and
BONIFACIO, M., CUEL, R., MAMELI, G., AND NORI,
reputation systems that will stimulate M. 2002. A peer-to-peer architecture for dis-
the cooperative behavior between the tributed knowledge management. In Proceed-
users, and make the overall operation of ings of the 3rd International Symposium on
peer-to-peer networks more fair. Multi-Agent Systems, Large Complex Systems,
and E-Businesses (MALCEB’02).
—The convergence of Grid and Peer-to- BROEKSTRA, J., EHRIG, M., HAASE, P., VAN HARME-
Peer systems. Research in this direction LEN, F., KAMPMAN, A., SABOU, M., SIEBES, R.,
aims to combine the benefits of the es- STAAB, S., STUCKENSCHMIDT, H., AND TEMPICH,
tablished field of distributed computing C. 2003. A metadata model for semantics-
(including interoperability standards) based peer-to-peer systems. In Proceedings of
the 1st Workshop on Semantics in Peer-to-
with the merits of new peer-to-peer ar- Peer and Grid Computing at the 12th Interna-
chitectures. tional World Wide Web Conference. Budapest,
Hungary.
BURAGOHAIN, C., AGRAWAL, D., AND SURI, S. 2003. A
ACKNOWLEDGMENTS
game theoretic framework for incentives in p2p
The authors would like to thank the anonymous re- systems. In Proceedings of the 3rd International
Conference on Peer-to-Peer Computing.
viewers for their insightful comments and sugges-
tions. CASTANO, S., FERRARA, A., MONTANELLI, S., PAGANI,
E., AND ROSSI, G. 2003. Ontology-addressable
contents in p2p networks. In Proceedings of
REFERENCES the 1st Workshop on Semantics in Peer-to-
Peer and Grid Computing at the 12th Interna-
AGRE, P. 2003. P2p and the promise of internet tional World Wide Web Conference. Budapest,
equality. Comm. ACM 46, 2 (Feb.), 39–42. Hungary.
ACM Computing Surveys, Vol. 36, No. 4, December 2004.

368 S. Androutsellis-Theotokis and D. Spinellis
CASTRO, M., DRUSCHEL, P., GANESH, A., A, R., AND WAL- Distributed Computing Systems (ICDCS’02). Vi-
LACH, D. 2002. Secure routing for structured enna, Autria.
peer-to-peer overlay networks. In Proceedings of DABEK, F., KAASHOEK, M., KARGER, D., MORRIS, R., AND
the 5th Usenix Symposium on Operating Sys- STOICA, I. 2001. Wide-area cooperative stor-
tems. Boston, MA. age with CFS. In Proceedings of the ACM
CASTRO, M., DRUSCHEL, P., KERMARREE, A.-M., AND ROW- SOSP’01 Conference. Banff, Canada.
STRON, A. 2002. Scribe: A large-scale and de- DASWANI, N., GARCIA-MOLINA, H., AND YANG, B. 2003.
centralized application-level multicast infras- Open problems in data-sharing peer-to-peer sys-
tructure. IEEE J. Select. Areas Comm. 20, 8 tems. In Proceedings of the 9th International
(Oct.). Conference on Database Theory. Siena, Italy.
CASTRO, M., DRUSCHEL, P., YC, H., AND ROWSTRON, A. DEERING, S. 1998. Host extensions for IP multicas-
2002. Exploiting network proximity in peer-to- ting. Tech. Rep. RFC-1112, IETF, (Aug.). SRI In-
peer overlay networks. In Proceedings of the In- ternational, Menlo Park, CA.
ternational Workshop on Future Directions in DELLAROCAS, C. 2001. Analyzing the economic ef-
Distributed Computing (FuDiCo’02). ficiency of ebay-like online reputation mecha-
CHAUM, D. 1981. Untraceable electronic mail, re- nisms. In Proceedings of the 3rd ACM Conference
turn addresses and digital pseudonyms. Comm. on Electronic Commerce. Tampa, FL.
ACM 24, 2, 84–88. DINGLEDINE, R., FREEDMAN, M., AND MOLNAR, D. 2000.
CHAWATHE, Y., RATNASAMY, S., BRESLAU, L., LANHAM, N., The FreeHaven project: Distributed anonymous
AND SHENKER, S. 2003. Making Gnutella-like storage service. In Workshop on Design Issues in
p2p systems scalable. In Proceedings of the 2003 Anonymity and Unobservability. 67–95.
Conference on Applications, Technologies, Archi- DINGLEDINE, R., FREEDMAN, M., AND MOLNAR, D.
tectures, and Protocols for Computer Communi- 2001a. Peer-to-peer: Harnessing the Power of
cations. Karlsruhe, Germany, 407–418. Disruptive Technology, 1st Ed. O’Reilly (Chap-
CHEN, Y., EDLER, J., GOLDBERG, A., GOTTLIEB, A., SOBTI, ter 1. A network of peers: Peer-to-peer models
S., AND YANILOS, P. 1999. A prototype imple- through the history of the Internet, 3–20)
mentation of archival intermemory. In Proceed- DINGLEDINE, R., FREEDMAN, M., AND MOLNAR, D.
ings of the 4th ACM Conference on Digital Li- 2001b. Peer-to-peer: Harnessing the power of
braries. Berkeley, CA. disruptive technology, 1st Ed. O’Reilly (Chapter
CHEN, Y., KATZ, R., AND KUBIATOWICZ, J. 2000. Scan: 16. Accountability, 271–340)
A dynamic, scalable and efficient content distri- DOUCEUR, J. 2002. The Sybill attack. In Proceed-
bution network. In Proceedings of International ings of the 1st International Workshop on Peer-
Conference on Pervasive Computing. to-Peer Systems (IPTPS’02). MIT Faculty Club,
CHUN, B., FU, Y., AND VAHDAT, A. 2003. Boot- Cambridge, MA.
strapping a distributed computational economy DRUSCHEL, P. AND ROWSTRON, A. 2001. Past: A large-
with peer-to-peer bartering. In Proceedings of scale, persistent peer-to-peer storage utility. In
the 1st Workshop on Economics of Peer-to-Peer Proceedings of the Eighth Workshop on Hot Top-
Systems. ics in Operating Systems.
CLAKE, I., HONG, T., SANBERG, O., AND WILEY, B. FastTrack Accessed on-line 2003. The FastTrack web
2002. Protecting free expression online with site. http://www.fasttrack.nu.
Freenet. IEEE Internet Comput. 6, 1 (Jan.-Feb.), FOSTER, I. 2000. Internet computing and the
40–49. emerging grid. Nature Web Matters.
CLARKE, I., SANDBERG, O., AND WILEY, B. 2000. FOSTER, I. AND IAMNITCHI, A. 2003. On death,
Freenet: A distributed anonymous information taxes, and the convergence of peer-to-peer and
storage and /etrieval system. In Proceedings of grid computing. In Proceedings of the 2nd In-
the Workshop on Design Issues in Anonymity and ternational Workshop on Peer-to-Peer Systems
Unobservability. Berkeley, CA. (IPTPS’03). Berkley, CA.
COHEN, B. 2003. Incentives build robustness in FOSTER, I., KESSELMAN, C., AND TUECKE, S. 2001. The
bitorrent. In Proceedings of the 1st Workshop on anatomy of the grid. Intl. J. Supercomput. Appl.
Economics of Peer-to-Peer Systems.
FRANCIS, P. 2000. Yoid: Extending the inter-
COHEN, E. AND SHENKER, S. 2001. Optimal replica- net multicast architecture. Unpublished
tion in random search networks. Preprint. Paper, available on-line at http://www.aciri.
COOPER, B. AND GARCIA-MOLINA, H. 2002. Peer- org/yoid/docs/index.html.
to-peer resource trading in a reliable dis- FREEDMAN, M., SIT, E., CATES, J., AND MORRIS, R. 2002.
tributed system. In Proceedings of the 1st In- Introducing tarzan, a peer-to-peer anonymiz-
ternational Workshop on Peer-to-Peer Systems ing network layer. In Proceedings of the 1st
(IPTPS ’02). MIT Faculty Club, Cambridge, International Workshop on Peer-to-Peer Sys-
MA. tems (IPTPS’02). MIT Faculty Club, Cambridge,
CRESPO, A. AND GARCIA-MOLINA, H. 2002. Routing MA.
indices for peer-to-peer systems. In Proceed- Freedom 2003. The Freedom anonymity system
ings of the 22nd International Conference on web site. http://www.freedom.net.
ACM Computing Surveys, Vol. 36, No. 4, December 2004.

A Survey of Content Distribution Technologies 369
GARCES-ERICE, L., FELBER, P., BIERSACK, E., URVOY- International Conference on Financial Cryptog-
KELLER, G., AND ROSS, K. 2004. Data indexing raphy.
in peer-to-peer dht networks. In Proceedings of Jabber 2003. The Jabber web site. http://www.
the 24th IEEE International Conference on Dis- jabber.org/.
tributed Computing Systems (ICDCS). Tokyo,
JANAKIRAMAN, R., WALDVOGEL, M., AND ZHANG, Q.
Japan, 200–208. 2003. Indra: A peer-to-peer approach to net-
GenomeAtHome 2003. The genome@home work intrusion detection and prevention. In Pro-
project web site. http://genomeathome.stanford. ceedgings of 2003 IEEE WET ICE Workshop on
edu/. Enterprize Security. Linz, Austria.
Gnutella 2003. The Gnutella web site: http:// JOVANOVIC, M. 2000. Modelling large-scale peer-to-
gnutella.wego.com. peer networks and a case study of Gnutella. M.S.
GOLDSCHLAG, D., REED, M., AND SYVERSON, P. 1999. thesis, Department of Electrical and Computer
Onion routing for anonymous and private inter- Engineering and Computer Science, University
net connections. Comm. ACM 42, 2, 39–41. of Cincinnati, Cincinnati, OH 45221.
Groove 2003. The Groove web site. http://www. JOVANOVICH, M., ANNEXSTEIN, F., AND BERMAN, K.
groove.net. 2001. Scalability issues in large peer-to-peer
GUPTA, M., JUDGE, P., AND AMMAR, M. 2003. A repu- networks—a case study of Gnutella. Tech. rep.,
tation system for peer-to-peer networks. In Pro- ECECS Department, University of Cincinnati,
ceedings of the NOSSDAV’03 Conference. Mon- Cincinnati, OH 45221.
terey, CA. Jxta 2003. The project JXTA web site.
HALEVY, A., IVES, Z., MORK, P., AND TATARINOV, I. 2003. http://www.jxta.org.
Piazza: Data management infrastructure for se- KALOGERAKI, V., GUNOPOULOS, D., AND ZEINALIPOUR-
mantic web applications. In Proceedings of the YAZTI, D. 2002. A local search mechanism for
12th International Conference on World Wide peer-to-peer networks. In Proceedings of the 11th
Web. Budapest, Hungary, 556–567. International Conference on Information and
HAND, S. AND ROSCOE, T. 2002. Mnemosyne: Peer- Knowledge Management (CIKM’02). McLean,
to-peer steganographic storage. In Proceedings VA.
of the 1st International Workshop on Peer-to-Peer KAMVAR, S. D., SCHLOSSER, M. T., AND GARCIA-MOLINA,
Systems (IPTPS’02). MIT Faculty Club, Cam- H. 2003. The Eigentrust algorithm for repu-
bridge, MA. tation management in p2p networks. In Pro-
HARDING, G. 1968. The tragedy of the commons. ceedings of the 12th International Conference on
Science 162, 1243–1248. World Wide Web. ACM Press, 640–651.
HARREN, M., HELLERSTEIN, J., HUEBSCH, R., LOO, B., KARGER, D., LEHMAN, E., LEIGHTON, F., LEVINE, M.,
SHENKER, S., AND STOICA, I. 2002. Complex LEWIN, D., AND PANIGRAHY, R. 1997. Consistent
queries in dht-based peer-to-peer networks. In hashing and random trees: Distributed caching
Proceedings of the 1st International Workshop on protocols for relieving hot spots on the world
Peer-to-Peer Systems (IPTPS ’02). MIT Faculty wide web. In Proceedings of the 29th Annual
Club, Cambridge, MA. ACM Symposium on Theory of Computing. El
HAZEL, S. AND WILEY, B. 2002. Achord: A variant of Paso, TX, 654–663.
the Chord lookup service for use in censorship re- Kazaa 2003. The Kazaa web site. http://www.
sistant peer-to-peerpublishing systems. In Pro- kazaa.com.
ceedings of the 1st International Workshop on KEROMYTIS, A., V, M., AND RUBENSTEIN, D. 2002.
Peer-to-Peer Systems (IPTPS ’02). MIT Faculty SOS: Secure overlay services. In Proceedings of
Club, Cambridge, MA. the ACM SIGCOMM’02 Conference. Pittsburgh,
HELEHER, P., BHATTACHARJEE, B., AND SILAGHI, B. PA.
2002. Are vitrualized overlay networks too KHAMBATTI, M., DASGUPTA, P., AND RYU, K. 2004. A
much of a good thing? In Proceedings of the role-based trust model for peer-to-peer com-
1st International Workshop on Peer-to-Peer Sys- munities and dynamic coalitions. In Pro-
tems (IPTPS ’02). MIT Faculty Club, Cambridge, ceedings of the Second IEEE International
MA. Information Assurance Workshop. Charlotte,
HUEBSCH, R., HELLERSTEIN, J., LANHAM, N., AND NC.
THAU LOO, B. 2003. Querying the internet KHAMBATTI, M., RYU, K., AND DASGUPTA, P. 2003.
with pier. In Proceedings of the 29th VLDB Con- Structuring peer-to-peer networks using
ference. Berlin, Germany. interest-based communities. In Proceedings of
HUMMEL, K., KOTSIS, G., AND KOPECNY, R. 2003. Peer the International Workshop On Databases, In-
profile driven group support for mobile learn- formation Systems and Peer-to-Peer Computing
ing teams. In Proceedings of the CATE/IASTED (P2PDBIS). Berlin, Germany.
Conference. Rhodes, Greece. KIM, H. 2001. P2p overview. Tech. rep., Korea Ad-
IOANNIDIS, J., IOANNIDIS, S., KEROMYTIS, A., AND PREVE- vanced Institute of Technology. (Aug.)
LAKIS, V. 2002. Fileteller: Paying and getting KUBIATOWICS, J. 2003. Extracting guarantees from
paid for file storage. In Proceedings of the Sixth chaos. Comm. ACM 46, 2 (Feb.), 33–38.
ACM Computing Surveys, Vol. 36, No. 4, December 2004.

370 S. Androutsellis-Theotokis and D. Spinellis
KUBIATOWICZ, J., BINDEL, D., CHEN, Y., EATON, P., GEELS, PLAXTON, C., RAJARAMAN, R., AND RICHA, A. 1997. Ac-
D., GUMMADI, S., WEATHERSPOON, H., WEIMER, W., cessing nearby copies of replicated objects in a
WELLS, C., AND ZHAO, B. 2000. Oceanstore: An distributed environment. In Proceedings of ACM
architecture for global-scale persistent storage. SPAA. ACM.
In Proceedings of ACM ASPLOS. RABIN, M. 1989. Efficient dispersal of information
LAI, K., FELDMAN, M., STOICA, I., AND CHUANG, J. 2003. for security, load balancing and fault tolerance.
Incentives for cooperation in peer-to-peer net- J. ACM 36, 2 (April), 335–348.
works. In Proceedings of the Workshop on Eco- RAMASWAMY, L. AND LIU, L. 2003. Free riding: A new
nomics of Peer-to-Peer Systems. Berkeley, CA. challenge for peer-to-peer file sharing systems.
LAMPORT, L., SHOSTAK, R., AND PEASE, M. 1982. The In Proceedings of the Hawaii International Con-
Byzantine generals problem. ACM Trans. Pro- ference on Systems Science.
gram. Lang. Syst. 4, 3 (July), 382–401. RATNASAMY, S., FRANCIS, P., HANDLEY, M., AND KARP, R.
LARSON, S., SNOW, C., AND PANDE, V. 2003. Mod- 2001. A scalable content-addressable network.
ern Methods in Computational Biology,. (Chap- In Proceedings of SIGCOMM 2001.
ter Folding@Home and Genome@Home: Using REITER, M. AND RUBIN, A. 1999. Anonymous web
distributed computing to tackle previously in- transactions with Crowds. Comm. ACM 42, 2,
tractable problems in computational biology) 32–38.
Horizon Press.
RHEA, S., WELLS, C., ET AL. 2001. Maintenance-free
LEE, J. 2003. An end-user perspective on file- global storage. IEEE Internet Compu., 40–49.
sharing systems. Comm. ACM 46, 2 (Feb.), 49–
53. RIPEANU, M. AND FOSTER, I. 2002. Mapping the
Gnutella network: Macroscopic properties of
LIBEN-NOWELL, D., BALAKRISHNAN, H., AND KARGER, D. large-scale peer-to-peer systems. In Proceedings
2002a. Analysis of the evolution of peer-to-peer of the 1st International Workshop on Peer-to-Peer
systems. In Proceedings of the ACM Sympo- Systems (IPTPS’02). MIT Faculty Club, Cam-
sium on the Principles of Distributed Computing bridge, MA.
(PODC). Monterey, CA.
ROWSTRON, A. AND DRUSCHEL, P. 2001. Pastry: Scal-
LIBEN-NOWELL, D., BALAKRISHNAN, H., AND KARGER, D.
able, distributed object location and routing
2002b. Observations on the dynamic evolution
for large-scale peer-to-peer systems. In Proceed-
of peer-to-peer networks. In Proceedings of the
ings of IFIP/ACM Middleware. Heidelberg, Ger-
1st International Workshop on Peer-to-Peer Sys-
many.
tems (IPTPS’02). MIT Faculty Club, Cambridge,
MA. SAROIU, S., GUMMADI, P., AND GRIBBLE, S. 2002. Ex-
ploring the design space of distributed peer-to-
LOESER, A., WOLPERS, M., SIBERSKI, W., AND NEJDL, W.
peer systems: Comparing the web, TRIAD and
2003. Semantic overlay clusters within super-
Chord/CFS. In Proceedings of the 1st Interna-
peer networks. In Proceedings of the Interna-
tional Workshop on Peer-to-Peer Systems (IPTPS
tional Workshop On Databases, Information Sys-
’02). MIT Faculty Club, Cambridge, MA.
tems and Peer-to-Peer Computing (P2PDBIS).
Berlin, Germany. SCHODER, D. AND FISCHBACH, K. 2003. Peer-to-peer
prospects. Comm. ACM 46, 2 (Feb.), 27–29.
LV, Q., CAO, P., COHEN, E., LI, K., AND SHENKER, S.
2002. Search and replication in unstructured SERJANTOV, A. 2002. Anonymizing censorship re-
peer-to-peer networks. In Proceedings of the 16th sistant systems. In Proceedings of the 1st In-
ACM International Conference on Supercomput- ternational Workshop on Peer-to-Peer Systems
ing (ICS’02). New York, NY. (IPTPS’02). MIT Faculty Club, Cambridge, MA.
LV, Q., RATNASAMY, S., AND SHENKER, S. 2002. Can SetiAtHome 2003. The seti@home project web
heterogeneity make Gnutella scalable? In Pro- site. http://setiathome.ssl.berkeley.edu.
ceedings of the 1st International Workshop on SHAMIR, A. 1979. How to share a secret. Comm.
Peer-to-Peer Systems (IPTPS’02). MIT Faculty ACM 22, 612–613.
Club, Cambridge, MA. SHAW, M. AND GARLAN, D. 1995. Formulations and
MAYAMOUNKOV, P. AND MAZIERES, D. 2002. Kadem- formalisms in software architecture. In Com-
lia: A peer-to-peer information system based on puter Science Today: Recent Trends and Devel-
the xor metric. In Proceedings of the 1st In- opments, Lecture Notes in Computer Science,
ternational Workshop on Peer-to-Peer Systems 1000. J. van Leeuwen, Ed. Springer Verlag, 307–
(IPTPS’02). MIT Faculty Club, Cambridge, MA. 323.
MojoNation 2003. The MojoNation web site. SHIRKY, C. 2000. What is p2p... and what isnt’t.
http://www.mojonation.net. Network, available online at http://www. or-
NEJDL, W., WOLF, B., QU, C., DECKER, S., SINTEK, M., eillynet.com/pub/a/p2p/2000/11/24/shirky1
NAEVE, A., NILSSON, M., PALMER, M., AND RISCH, T. -whatisp2p.html. O’Reilly
2003. Edutella: A p2p networking infrastruc- STOICA, I., ADKINS, D., ZHUANG, S., SHENKER, S.,
ture based on rdf. In Proceedings of the 12th In- AND SURANA, S. 2002. Internet indirection in-
ternational Conference on World Wide Web. Bu- frastructure. In Proceedings of the ACM SIG-
dapest, Hungary. COMM’02 Conference. Pittsburgh, PA.
ACM Computing Surveys, Vol. 36, No. 4, December 2004.

A Survey of Content Distribution Technologies 371
STOICA, I., MORRIS, R., KARGER, D., KAASHOEK, M., XIONG, L. AND LIU, L. 2002. Building trust in de-
AND BALAKRISHNAN, H. 2001. Chord: A scalable centralized peer-to-peer communities. In Pro-
peer-to-peer lookup service for internet applica- ceedings of the International Conference on Elec-
tions. In Proceedings of SIGCOMM 2001. tronic Commerce Research.
STUBBLEFIELD, A. AND WALLACH, D. 2001. YANG, B. AND GARCIA-MOLINA, H. 2001. Comparing
Dagster:censorship-resistant publishing with- hybrid peer-to-peer systems. In Proceedings of
out replication. Tech. Rep. Technical Report the 27th VLDB Conference. Rome, Italy, 561–
TR01-380, Rice University, Dept. of Computer 570.
Science. (July). YANG, B. AND GARCIA-MOLINA, H. 2002a. De-
SULLIVAN III, W., WERTHIMER, D., BOWYER, S., COBB, J., signing a super-peer network. Tech. rep.,
GEDYE, D., AND ANDERSON, D. 1997. A new ma- Stanford University. (Feb.). Available online:
jor seti project based on project serendip data http://dbpubs.stanford.edu/pub/2002-13.
and 100,000 personal computers. In Proceedings YANG, B. AND GARCIA-MOLINA, H. 2002b. Improv-
of the 5th International Conference on Bioastron- ing search in peer-to-peer networks. In Proceed-
omy. ings of the 22nd International Conference on
TSOUMAKOS, D. AND ROUSSOPOULOS, N. 2003. A com- Distributed Computing Systems (ICDCS’02). Vi-
parison of peer-to-peer search methods. In Pro- enna, Autria.
ceedings of the Sixth International Workshop on YU, B. AND SINGH, M. 2003. Incentive mechanisms
the Web and Databases. San Diego, CA. for peer-to-peer systems. In Proceedings of the
VANRENESSE, R., BIRMAN, K., BOZDOG, A., DIM- 2nd International Workshop on Agents and Peer-
ITRIU, D., SINGH, M., AND VOGELS, W. 2003. to-Peer Computing.
Heterogeneity-aware peer-to-peer multicast. In ZHANG, X., ZHANG, Q., ZHANG, Z., SONG, G., AND ZHU, W.
Proceedings of the 17th International Sympo- 2004. A construction of locality-aware overlay
sium on Distributed Computing (DISC2003). network: moverlay and its performance. IEEE
Sorrento, Italy. JSAC Special Issue on Recent Advances on Ser-
VISHNIMURTHY, V., CHANDRAKUMAR, S., AND GUN SIRER, vice Overlay Networks.
E. 2003. Karma: A secure economic frame- ZHAO, B., JOSEPH, A., AND KUBIATOWICZ, J. 2002. Lo-
work for p2p resource sharing. In Proceedings cality aware mechanisms for large-scale net-
of the 1st Workshop on Economics of Peer-to-Peer works. In Proceedings of the International Work-
Systems. shop on Future Directions in Distributed Com-
VLACHOS, V., ANDROUTSELLIS-THEOTOKIS, S., AND SPINEL- puting (FuDiCo2002).
LIS, D. 2004. Security applications of peer-to- ZHAO, B., KUBIATOWICZ, J., AND JOSEPH, A. 2001.
peer networks. Comput. Netw. J. 45, 2, 195–205. Tapestry: An infrastructure for fault-tolerant
WALDMAN, M., AD, R., AND LF, C. 2000. Publius: wide-area location and routing. Tech. Rep.
A robust, tamper-evident, censorship-resistant UCB/CSD-01-1141, Computer Science Divi-
web publishing system. In Proceedings of the 9th sion, University of California, Berkeley, 94720.
USENIX Security Symposium. (April)
WALDMAN, M. AND MAZI, D. 2001. Tangler: a ZHICHEN, X., MAHALINGAM, M., AND KARLSSON, M.
censorship-resistant publishing system based on 2002. Turning heterogeneity to an advantage
document entanglements. In Proceedings of the in overlay routing. Tech. Rep. HPL-2002-126, HP
ACM Conference on Computer and Communica- Labs.
tions Security. 126–131. ZHOU, J., DIALANI, V., DE ROURE, D., AND HALL, W.
WALLACH, D. 2002. A survey of peer-to-peer secu- 2003. A semantic search algorithm for peer-
rity issues. In International Symposium on Soft- to-peer open hypermedia systems. In Proceed-
ware Security. Tokyo, Japan. ings of the 1st Workshop on Semantics in Peer-
WITTEN, I., MOFFAT, A., AND BELL, T. 1999. Manag- to-Peer and Grid Computing at the 12th Inter-
ing Gigabytes: Compressing and Indexing Docu- national World Wide Web Conference. Budapest,
ments and Images, 2nd ed. Morgan Kauffman. Hungary.
Received May 2003; revised May 2004; accepted September 2004
ACM Computing Surveys, Vol. 36, No. 4, December 2004.