Peer to Peer Networks including JXTAPresentation Transcript
Peer to Peer Networks including JXTA Euresco Conference at Castelvecchio Pascoli Italy June 16-21 2001 EuroConference on Problem-Solving Environments for Numerical Mathematics, Science and Engineering Applications Geoffrey Fox IPCRES Laboratory for Grid Technology Computer Science, Informatics, Physics Indiana University Bloomington IN email@example.com
P2P: Peer-to-Peer Networks
Peer-to-Peer Networks are considered to be the next " killer application " for the Internet.
There are several important technology challenges and applications that vary from the sublime to the ridiculous.
Like most such over hyped concepts, P2P is rather loosely defined and covers a set of rather disparate ideas.
Perhaps the only common theme is a client oriented view of the world and P2P can be thought of as Power to the People .
Clients do most of the work, communicate with each other and really are the most important computers;
Servers may be around and even essential but remain subservient to the clients.
We will first discuss the most well-known and popular P2P system, Napster .
General discussions on P2P technology can be found at two good web sites;
http://www.openp2p.com from the O'Reilly group and
http://www.peer-to-peerwg.org/ as an industry working group originally initiated by Intel.
There is a remarkable book Peer-to-Peer: Harnessing the Power of Disruptive Technologies by Andrew Oram, Nelson Minar, Clay Shirky, Tim O'Reilly (March 15, 2001, O'Reilly & Associates; ISBN: 059600110X) ;
This has several articles describing either key systems or key services
Disruptive Technology (DT): One that does something (perhaps a new capability) far better than previous best practice; it does old things less well at the start but gradually catches up.
The Web itself is a DT: hyperlinks empower new ways of doing things. Browsers are (initially) clumsier user interface than traditional GUI’s.
Shawn Fanning developed the original Napster application and service in January 1999 while a freshman at Northeastern University.
Napster allowed any client to advertise any MP3 files stored on its disk and choose to download MP3 files from other clients connected to the Napster server network.
It is said that Shawn was taking a computer-programming course at Northeastern , but had to buy a programming book to build Napster ……..
Like most good ideas, Napster was designed to solve a real need - in this case to allow Shawn, a musician himself, to share his music with his friends on campus.
A PSE for getting music
The system has become staggeringly popular. Quoting a legal opinion from last summer, approximately 10,000 music files are shared per second using Napster , and every second more than 100 users attempt to connect to the system and there will be 75 million Napster users by the end of 2000. http://news.cnet.com/News/Pages/Special/Napster/napster_patel.html
Current legal debate has increased instantaneous use
Napster has some other typical P2P services; Instant Messenger, chat rooms, "Buddy lists" and information about " hot music " (music portal) but the key feature is the ability to share files between any Internet connected consenting clients.
This is roughly the Web version of NFS (Network File System) familiar from traditional computing environments.
Napster III: Key Features
MP3 files are important as a popular digital encoding for audio
it is straightforward to "rip" files off an audio CD and look up key meta-data (Artist, Title etc.) in a CDDB database on the Web (http://www.gracenote.com/).
Length of songs uniquely identifies CD’s
The audio and meta-data can be stored and accessed as a single unit.
Although a server is used to establish the initial connection, the file transfer is done efficiently - directly from client to client ; a P2P service.
This is an improvement over most NFS systems where use of distributed files is not easy (except possibly for the originator who named file) as usually all you have is a cryptic filename.
The added value of meta-data for files lies at the heart of the Semantic Web - a vision from the W3C Web Consortium related to P2P. http://www.w3.org/2001/sw/
Napster Like Services I
There some 200 available Napster clones to support this area http://www.ultimateresourcesite.com/napster/main.htm
Currently the most popular is Imesh [http://www.imesh.com], which has some 2 million users and can share any type of file.
Some of the best known file sharing systems are
Freenet [http://freenet.sourceforge.net/] ,
These three are not server based like Napster but rather support waves of software agents expressing resource availability and interest propagating among an informal dynamic networks of peers.
Napster Like Services II
There are many interesting ideas being explored;
Breaking shared files into many parts to both increase bandwidth (parallel I/O) and increase security of content as no one site can access files without cooperation from its peers. This type of technology is controversial as it makes censorship very hard.
MojoNation has a load balancing and scheduling algorithm in the form of micro payments to reward those who contribute most to the community of peers.
Gnutella - which is a family of related products -- is usually described as a P2P search engine as its interface is nearer that of a search engine than a Web file system.
So far we have identified some basic P2P services ; file registration, access and search.
We can categorize other P2P systems in various ways; we will choose distributed computing , collaboration , and core technologies .
P2P for Distributed Computing or Web Computing I
The distributed computing P2P applications are highlighted by the use of millions of Internet clients to analyze data looking for extraterrestrial life ( SETI@home http://setiathome.ssl.berkeley.edu/ ) and the
Newer project examining the folding of proteins ( Folding@home http://www.stanford.edu/group/pandegroup/Cosm/ ).
These are building distributed computing solutions for a special class of applications:
Those that can be divided into a huge number of essentially independent computations, and a central server system doles out separate work chunks to each participating client.
In the parallel computing community, these problems are called " pleasingly or embarrassingly parallel ".
This approach is included in the P2P category because the computing is Peer based even though it does not have the "Peer only communication" characteristic of all aspects of Gnutella and Napster for information transfer.
SETI@home and Folding@home are elegantly implemented as screen savers that you download.
Pure Java model
Entropia Financial Modeling I
Entropia Financial Modeling II
Each basic financial instrument can be calculated independently
Central Server interprets the total simulation
Make Money or Learn what causes market swings or ….
Drug Structure Simulations
United Devices also does Drug Simulation
Parameter Study : do billions of simulations – each with different parameters
Search Engine like interface to simulation
Works as each calculation fits in a PC – a detailed molecular model would usually not do this
Performance of Entropia Network
P2P for Distributed Computing or Web Computing II
Other projects of this type include:
United Devices (http://www.ud.com/home.htm based on SETI@home),
AppliedMeta (http://www.appliedmeta.com based on well known Legion project from the University of Virginia),
Parabon computation (http://www.parabon.com),
Condor (from Wisconsin http://www.cs.wisc.edu/condor/) and
Other applications for this type of system include financial modeling, bio-informatics, measurement of web server performance and the scheduling of different jobs to use idle time on a network of workstations.
Ian Foster has given a more detailed review of these activities at http://www.nature.com/nature/webmatters/grid/grid.html and related them to computational grids ( http://www.gridforum.org ).
Collaboration with P2P Systems
Collaborative systems form a rather different type of P2P network.
We have a community of clients working together and sharing different Internet resources.
Probably the Instant Messenger (IM) or various forms of chat room are the most used capability in this arena.
We have a set of clients exchanging messages with each other.
Unlike the file-sharing case, one typically needs to multi-cast the same message to multiple clients at the same time and the best architecture is still active research.
Groove Networks ( http://www.groove.net/ ) founded by the creator of Lotus Notes is the best-known P2P collaboration project and uses relay servers to implement the P2P multi-cast. Has underlying XML Object database .
Peer to Peer P2P “Illusion” among collaborating clients For Napster like Services or Collaboration Server Server Server Server Server Server
Core P2P technologies or services I
These include P2P management, messaging, security , client grouping as well as the file or more generally object registration, discovery and access capabilities already discussed for Napster.
Sun Microsystems has two important technology projects.
Jini ( http://www.sun.com/jini/ ) has a simple model for dynamic self defining objects which act like Napster peers and register with distributed servers allowing other peers to discover and access them.
JXTA (from juxtaposition http://www.jxta.org ) is a new project from Bill Joy aiming at core P2P capabilities including grouping (of the peers) and security.
Core P2P technologies or services II
Digital Cash or “ Digital Reputation ” implemented with Digital Cash is an important area as even in “free systems” need mechanism to limit abuse of resources
“Tragedy of the Commons”
Digital Cash has many very clever implementations
Public Key based “coins” issued by digital banks – issues of safety (no overdrafts or forgery) and anonymity (cash is anonymous, credit cards are not)
POW (Proofs of Work) to stop DOS (Denial of Service)
Create the Small World effect where only a few hops needed to get from any A to any B
Intelligent (bad worse case) versus random (bad average) strategies
Core P2P technologies or services III
I expect research and commercial experience to identify more base services, as we understand better the common needs of P2P systems
Management of resources in such a network must be an important challenge; it is our Nirvana - the Web Operating System .
Maybe society can live in a Gallimaufry of unstructured knowledge swept back and forth by armies of Gnutella agents.
However this will not do for what the Gartner Group ( http://www.oreillynet.com/pub/d/547 ) and O'Reilly term Enterprise P2P needed by a Fortune 500 organization.
Here we will need to manage structured information within a dynamic P2P grouping.
http://www.jxta.org April 25 2001
Network computer maker Sun Microsystems Inc. on Wednesday opened a toolbox for programmers to build a decentralized, Napster - like Web that it hopes will speed up and deepen the Internet.
Rather than anarchy , Sun expects pockets of programmers to build a more secure and reliable network that will include every device imaginable, from computers to kitchen appliances.
``Our goal at the end of this is to build a completely reliable system from unreliable parts, '' John Gage, chief researcher at Palo Alto, California-based Sun, said in an interview.
Sun's formula for peer-to-peer computing, which links computers directly to one another without relying on a server to coordinate communication, rests on a small program that sets out how machines will talk to each other.
JXTA for the Press II
``The idea is that entities on the Net can find each other and then send information back and forth,'' said Sun's chief scientist, Bill Joy, who heads Project JXTA, pronounced ''juxta,'' which released the protocol, or rules, for peer-to-peer networks.
Sun bets that it will be able to sell more of its Internet-building computers as the Web grows on its software, similar to the strategy it has taken with its Java programming language.
Peer-to-peer computing has achieved a kind of cult celebrity because of the notoriety of Napster and a number of companies are looking for commercial applications for the way that it allows individuals to work directly with each other, avoiding mediation --and snoops.
JXTA for the Press III
Microchip maker Intel Corp. for instance, has used peer-to-peer technology to link employees' PCs, using the collective power of the herd to design new chips faster.
Song-swapping service Napster holds a central index which points members to others who have the songs they want, but members swap directly. Other services avoid a central index, instead passing lists from member to member in a type of computer gossip.
But distributed computing, as peer-to-peer is more formally known, can also ease network bottlenecks by making it possible to break down big programs into lots of little ones which can communicate with each other.
``The role of jxta will be as a solvent to dissolve the applications running on the back end into small pieces to run more quickly and more spread out,'' Gage said.
JXTA for the Press IV
Sun said that manufacturers could start building jxta into devices from mobile handsets to refrigerators to get them on the web in a year or so.
The tiny layer of jxta layer released on Wednesday includes protocols for computers to talk to each other, a way to organize groups within the broader environment, and ways to follow transactions and keep them secure.
``The monitoring and security will ultimately determine the success of the thing,'' Joy told a Webcast.
Sun's archrival Microsoft Corp. also is focusing its energy on making transactions secure with its own plans to build the infrastructure of an always-on, always-connected Web, although as usual Sun dismissed Microsoft's efforts.
Keep it familiar: where practical, use stuff that's standard and has worked before
Leverage experts: engage a variety of experts early and often
Encourage open development: the design, the specs, the code, and whatever the result is.
Vision for the software
promote communication (not isolation) among applications
develop administrative commands for peers, peer groups, and groups of peers in the spirit of UNIX pipes and shells
keep the core small and elegant: make an architectural distinction between core mechanisms and optional policies
support multiple platforms and languages, micro devices to servers
address security from the beginning.
Key JXTA Concepts
Peers, Messages, Groups, Pipes
JXTA Service Advertisements – XML metadata
PDA (Java KVM) Implementation
Li Gong: JXTA: A Network Programming Environment
IEEE Internet Computing May June 2001
JXTA P2P Architecture
JXTA Projects in Categories
Configurator : A GUI configuration tool for the JXTA platform
Instantp2p :JXTA Demonstration GUI
Shell : JXTA Command Line Shell for interactive access to the JXTAP2P platform
Platform : JXTA P2P platform infrastructure building blocks and protocols.
This project defines the JXTA core P2P building blocks: peers, peer groups, pipes, codats and peer group policies. Includes specifications and reference implementations for core JXTA protocols (peer and peer group discovery, peer group membership, peer group pipes and codat sharing).
Security : JXTA P2P Security Project
Cms : JXTA Content Management System
Monitoring : Monitoring and Metering
Rvmanager : RendezVous Manager
JXTA Protocols -- Discovery
Project JXTA has defined six protocols so far.The developer community
may define more over time.
Peer Discovery Protocol — enables a peer to find advertisements on other peers, and can be used to find any of the peer, peer group, or advertisements.
This protocol is the default discovery protocol for all peer groups, including the World Peer Group.
It is conceivable that someone may want to develop a premium discovery mechanism that may or may not choose to leverage this default protocol, but the inclusion of this default protocol means that all JXTA peers can understand each other at the very basic level.
Peer discovery can be done with or without specifying a name for either the peer to be located or the group to which peers belong.When no name is specified, all advertisements are returned.
Peer Resolver Protocol — enables a peer to send and receive generic queries to search for peers, peer groups, pipes, and other information.
Typically, this protocol is implemented only by those peers that have access to data repositories and offer advanced search capabilities.
Peer Information Protocol — allows a peer to learn about the capabilities and status of other peers.
For example, a ping message can be sent to see if a peer is alive.
A query can also be sent regarding a peer’s properties where each property has a name and a value string.
Peer Membership Protocol — allows a peer to obtain group membership requirements, to apply for membership and receive a membership credential along with a full group advertisement, to update an existing membership or application credential, and to cancel a membership or an application credential.
Authenticators and security credentials are used to provide the desired level of protection.
JXTA Protocols – Binding, Endpoints
Pipe Binding Protocol — allows a peer to bind a pipe advertisement to a pipe endpoint, thus indicating where messages actually go over the pipe.
In some sense, a pipe can be viewed as an abstract, named message queue that supports a number of abstract operations such as create, open, close, delete, send, and receive.
Bind occurs during the open operation, whereas unbind occurs during the close operation.
Peer Endpoint Protocol — allows a peer to ask a peer router for available routes for sending a message to a destination peer.
For example, when two communicating peers are not directly connected to each other, such as they are not using the same network transport protocol or when they are separated by firewalls or NATs, peer routers respond to queries with available route information — that is, a list of gateways along the route.
Any peer can decide to become a peer router by implementing the Peer Endpoint Protocol