Collaborative Peer-to-Peer
Information Sharing
Content Query System
Tom Lendacky
IBM Corporation

Overview
Distributed Peer-to-Peer Information Sharing System
Communities
Collection of related data (i.e. defect data, music, sales)
Optional password authentication to control data provider(s)
Content Sources
Collection of specific data (i.e. Linux defect data, rock, regional sales)
XML Messages
Client Request Interface and Control Interface
\"Pluggable\" Query Engines
Allows for any type of information to be returned
Defined API's and Input/Output formats (XML)
Browser User Interface
Apache/PHP based UI
XML results translated to HTML
XML allows for any type of UI

Background
Initially developed to:
Provide easier way to access Linux defect data
Currently must go to each web site to query defect data
Improve service and support
Quicker response
Reduce duplicate effort
Problem could already be known with a patch in process
Problem could already be fixed with patch available
But it has many more uses than that...

Community
Community Community \"A\"
Collection of related information Host A
Password authentication (optional) Host B
Controls who can provide information Host C
Content Sources (specific information)
Multiple Content Sources supported Host D
Can specify the same Content Source multiple
times (i.e.. different information sources)
Can specify multiple Content Sources
Host E
Host G
Host F
Community \"B\"

Community...
Community...
Uses TCP/IP Sockets for connections between Hosts
Configurable maximum connections limit
Configurable AutoConnect capability
Examines (Announce) messages to find new Hosts to connect to
Configurable Retry capability
Repeated attempt to connect to \"Startup\" connections that are not connected
\"Startup\" connections
Attempt to connect to specific Hosts during CQS startup
Connection \"Listener\"
Listens for requests from Hosts to connect to the Community
Dedicated connections
A connection is only used for the Community to which it connected
Access control
Based upon IP address

Community...
Community...
Handshake used to join Community
Community Name: Verify Community to Join
Host UUID: Verify connection to/from a unique Host
Only 1 Connection per Host allowed
Uses Host UUID (as opposed to IP address) to allow connections from multiple Hosts on a
single machine
Authentication: SHA-1 message digest value for newly generated UUID
and password
Password does not flow in the clear
XML messages over connections
4 byte header (length of message that follows)
XML message

Content Source
Content Source
Community \"A\"
Part of a Community
Collection of specific information
Query input is standardized
Host A
Uses a Query Engine to interface ContentSources:
Host B
\"A\"
with the information ContentSources:
\"B\" and \"C\"
Host announces Content Host C
Sources ContentSources:
\"A\" and \"B\"
ANNOUNCE message Host D
ContentSources:
Name and URI of input form \"B\"
Informs other Hosts of Content Sources
in the Community

Content Source...
Content Source
Supports multiple Content Sources
Both identically or uniquely named
<CONTENT-SOURCE name=\"A\" module=\"libA1.so\" data=\"db1\"... />
<CONTENT-SOURCE name=\"A\" module=\"libA2.so\" data=\"db2\"... />
<CONTENT-SOURCE name=\"A\" module=\"libA2.so\" data=\"db3\"... />
<CONTENT-SOURCE name=\"B\" module=\"libB1.so\" data=\"db4\"... />
Identically named Content Sources are processed sequentially

XML Messages
XML Messages
Defined format
<MESSAGE uuid= ttl= hops= reply=... >
<MESSAGE-HOST... />
<message-name... >
</message-name>
</MESSAGE>
Attributes on <MESSAGE> tag identify message and purpose
UUID: Identifies the message, used to avoid processing duplicate requests
TTL/HOPS: Indicates how many more times the message can be forwarded and how many times it already has
REPLY: Indicates whether this message is a reply (no=forward to all connected Hosts, yes=return to sender)
Allows new messages to be created without having to upgrade the Host
Host processes message that contain a message-name that is recognized
Host simply forwards/returns messages that are not recognized
Three defined messages currently
ANNOUNCE, CONTENT-SOURCES, QUERY-CONTENT
Reply defined as message-name-REPLY
Defined XML query language for QUERY-CONTENT
Both input and output

Request Interface
Request Interface
Used by a Client User Interface
Obtain information from a Host and request a Host to perform an action
Uses TCP/IP Sockets for request initiation
One request per socket connection
Access control
Based upon IP address
Four defined requests currently
Get Community names
Get Content Source name within a Community
Get Content Source URI for Content Source within a Community
Start a query for a Content Source within a Community
Query results returned in XML format

Control Interface
Control Interface
Provides some administration information and control
Uses TCP/IP Sockets for control initiation
One control request per socket connection
Access control
Based upon IP address
Six defined controls currently
Get Community names
Get Community information
Get Community connections (active connections)
Get ContentSource names within a Community (locally defined only)
Get ContentSource configuration information within a Community
Shutdown CQS

Query Engines
Query Engines
\"Pluggable\" interface between Host and information
Specified on the Content Source configuration statement
<CONTENT-SOURCE name= uri= module= data= map= />
MODULE: the path/name of the module to load for this Content Source
DATA: Query Engine defined value - supplied to the Query Engine during initialization
Possible use as configuration input
MAP: Query Engine defined value - supplied to the Query Engine during initialization
Possible use as configuration input
Must support define API set
Five API's
Startup / Termination related (invoked only once):
CQELoad: Invoked during CQS startup just after the Query Engine has been loaded
CQEUnload: Invoked during CQS termination just before the Query Engine is unloaded
Query related (invoked as a series of calls each time a query request is received):
CQECreate: Invoked to create a new instance of an object that will be responsible for performing the query
CQEQuery: Invoked to perform the query
CQEDestroy: Invoked to cleanup / destroy the instance of the object that performed the query
Defined XML format for query language
Input and output

Browser User Interface
Browser UI
Apache/PHP based
Defined set of URIs to access the CQS system
http://hostname/.../CQS
URI to display the startup webpage
http://hostname/.../CQS/Communities
URI to display a webpage of a list of Communities that the Host has joined
http://hostname/.../CQS/ContentSources/Community
URI to display a webpage of a list of Content Sources within the specified Community
http://hostname/.../CQS/ContentSource/Community/ContentSource/Input
URI to display a webpage of the input form page to submit the query (Refer to documentation on how the form must be
designed
http://hostname/.../CQS/ContentSource/Community/ContentSource/Query
URI to initiate the Query request.
Results are displayed in table format as they are returned by the participating CQS systems
Translates XML query results into HTML for presentation on browsers

Project Information
Open Source
Hosted on DeveloperWorks
http://oss.software.ibm.com/developerworks/projects/cqs
Project HomePage
http://oss.software.ibm.com/developerworks/opensource/cqs/index.html
Documentation
Developer Information
Build Information