CA*net 4 Research Program Update – UCLP Roadmap for creating ...Document Transcript
CA*net 4 Research Program Update – UCLP Roadmap for
creating User Controlled and Architected Networks using Service
You are free to distribute this paper as long as the following the disclaimer is included:
“This is a discussion paper intended to provoke further thought and debate on the implications of web
services and web services workflow for allowing users to architect their own Articulated Private Networks
on a common network substrate connecting and managing geographically distributed router, switches
research instruments and sensors to networks based on the original User Controlled LightPath (UCLP)
software developed for CANARIE to manage geographically distributed optical and SONET/SDH cross
connects and switches. Please send all comments, suggestions and criticisms to the author”
Last revised draft: January 3, 2006
Around the world there are several initiatives to develop the next generation canonical Internet
network architecture . This discussion paper provides an alternative approach largely based on
two significant development, first: the trend for more users to own, control and manage their own
network resources and secondly: the demand by big science and large enterprises to have
dedicated network resources for the data flows generated by their high end applications. As
opposed to other approaches a key assumption in this discussion paper in regards to network
architecture is that there is NO canonical network architecture. Instead this document describes
new features and enhancements to the current implementations of the User Controlled LightPath
(UCLP) software which will enable users to define their own packet or switched based network
architecture including topology, routing, virtual routers, switches virtual machines and protocols
based on the concept of many separate and independently managed Articulated Private Networks
(APN) operating on top of one or more network substrates across different ownership domains.
APNs can be considered as a next generation Virtual Private Network where, rather than
signaling for a single end to end connection, a user can create a complex network structure, in
any way they wish, by binding together layer 1 through 3 network links, instruments, computers,
time slices and virtual or real routing and/or switching nodes. This capability is enabled
through UCLP by representing all such network element, devices and links as web services, and
by using web services workflow as the tool to allow the user to bind together their various web
services to create a long lived APN instantiation. With web services workflow the user also has
the ability to offer all, or portions of their APN as a web service (or set of services) in is own
right to other downstream users.
On August 28, 2005 the National Science Foundation (NSF) in the United States
announced the Global Environment for Networking Investigations (GENI) [NSF-GENI].
Research that falls under the broad conceptual umbrella of this initiative will focus on
designing new network architectures and services that range from new wireless and
sensor devices to customized routers and optical switches to control and management
software. The objective of the program is an attempt to “de-ossify” the Internet and
develop and architecture that will allow an easy migration strategy such that users and
applications can migrate effortlessly from the current Internet to a future more robust and
secure environment. As such it is felt that a national testbed is necessary to carry out
research in defining new network architectures.
One proposed architecture under the GENI framework is “Network Virtualization”
[VIRTUALIZATION] which provides for multiple virtual networks to co-exist on top of
a shared “substrate”. Different virtual networks provide alternate end-to-end packet
delivery systems and may use different protocols and packet formats. Virtual networks
may also be interconnected with virtual routers and interconnect wireless, instrument and
sensor networks as illustrated in Figure 1.0
Figure 1.0 GENI Network Virtualization
The original concept for User Controlled LightPaths (UCLP)[UCLP-DESIGN] as
subsequently updated with the UCLPv2 program [UCLPv2] has many similar objectives
as that with the proposed GENI program in particular relation to the concept of network
virtualization. The major difference between these concepts is that UCLP assumes there
is no predefined canonical network architecture. Rather the user is given the tools to build
any variation of virtual (and real) network architecture that so desire on top of one or
more substrates across different ownership domains. This user defined network can be
packet or switched and the user decides what type of topology, restoral, protection and
routing that may be necessary for their particular requirements. In essence the future
network architecture has no predefined network architecture. Instead a substrate is
provided which is capable of supporting many separate user defined networks which can
be assembled through a series of lego block type web services as illustrated in Figure
Child Lightpath WS
(may run over IP
Virtual Ethernet, MPLS, etc
WS Wireless Sensor
Figure 1.1 UCLP Network Virtualization
The major differences between UCLP and GENI is that in UCLPv1 the network was
assumed to be made up only of wavelengths and optical switches which could be
partitioned into smaller lightpaths and where the lightpath owner could be given control
of the optical or SONET cross connects that touched that particular lightpath. UCLPv2
allows for the user control of both wavelengths, MPLS tunnels, IP packet networks and
UCLP allows for a creation of a set of VPNs which can be configured into a topology by
the user decided upon by the user including meshes, rings, etc. As well UCLP allows the
user to reconfigure the topology of this mesh of VPNs and change their shape, size and
path. Hence the moniker of “articulated” Private Networks.
Articulated Private Networks (APNs), in many ways are similar to the GENI concept of
many virtual networks sharing a common network substrate linking together virtual
instruments, router and devices. With APNs all the virtual links and devices are
represented as web services and are linked together via web service workflow tools as
described in more detail in the following section. In GENI there is yet no defined process
for creating virtual links and connecting them to virtual devices and/or time slices.
By representing the substrate, child lightpaths, switches, routers and time slices or
processes the user with UCLPv2 has complete flexibility and control to architect the
resulting network in way they wish using web service workflow tools.
1.1.1 Market drivers for UCLP
One of the drivers for UCLP is the growing trend for organizations such as universities,
hospitals, schools and businesses to acquire their own fiber and point to point
wavelengths from different suppliers. These organizations would like to manage these
facilities as part of their own network domain, and, in turn offer a variety of downstream
services to their users such as APNs, VPNs, etc. These needs are very similar to the
“carrier’s carrier” market where carriers want to service customers outside of their
facilities based footprint by acquiring leased layer 1 or layer 3 services from some other
carrier and interconnect these facilities with virtual routers and switches. The driver for
“carrier’s carrier” solutions is increasingly be driven by the large enterprise as well as
other competitive local exchange carriers.
Another important driver for UCLP is the growing recognition around the world that
there are many large remote sensor and instrumentation projects whose large data flows
will need to be delivered to computing facilities many thousands of kilometers away. As
well, researchers will want to remotely control these instruments and, from time to time,
re-direct the flow of data to different computing destinations. Examples of such facilities
include the European eVLBI project- JIVE [E-VLBI], the US-Canada undersea network
Neptune [NEPTUNE], the Canadian Light Source Synchrotron project (CLS) [CLS], the
Ottawa University-NRC Nuclear Magnetic Resonance Instrument Facility [NRC-NMR] ,
and of course, the granddaddy of them all the CERN Large Hadron Collider (LHC)
Up to now the interconnection of these instruments to networks and remote computers
was static and pre-configured by network or software engineers using VPNs or
lightpaths. Increasingly researchers need to do their own traffic engineering and control
how these instruments are connected to the network and redirect or share the data flow
amongst different geographically distributed researchers. As well they may want to
control the transport mechanisms on how the data is delivered as for example through
their own IP routed network with guaranteed QoS parameters.
In some cases the expected data flow demands of these projects, in terms of throughput
and duration will easily exceed by order of magnitude, that of conventional Internet
traffic flows. In anticipation of these requirements CANARIE has been leading the
development of User Controlled LightPaths [UCLP] in order to permit researchers to
create “discipline” or “application” specific Layer 1 VPN or IP networks which would
create separate high speed networks for these specific applications. As opposed to
traditional VPNs the UCLP allows end users to do their own traffic engineering, change
the topology of their collection of VPNs, and cross connect their collection of VPNs over
multiple domains to VPNs operated by other users as shown in Figure 1.2.
In this example we see an APN created from three separate independent managed
domains and in turn a “child” APN has been created out of the mult-domain APN. The
nodes can be either virtual routers or switches. The nodes that indicate that they have
been partitioned indicates that control of the virtual router or switch has also been handed
over to the owner of the APN or child APN, which ever the case may be.
Pass Through Node
Domain A Domain B Domain C
Figure 1.2 Creation of APN from multiple independent managed domains
1.1 Web Services Workflow
With UCLPv2 it was proposed that Service Oriented Architecture (SOA) [SOA] new
technologies of web services workflow and/or orchestration, that were originally
conceived for eBusiness applications, would be the underpinning architectural framework
for extending UCLP to allow the interconnection of instruments, time slices and sensors
and for incorporating virtual routers and switches.. Specifically Web Services Description
Language (WSDL) [WSDL] and Work flow languages such as Business Process
Engineering Language (BPEL) [BPEL-INFO] were proposed as the implementation tools
for such a reference framework.
One of the major attractions of web service and work flow tools is that they provide a
consistent and standardized set of interfaces to all sorts of process including instruments
and networks and allow users to bind processes together as required for particular
applications, unconstrained by the original design requirements.
Another attraction of many work flow tools is their service oriented nature as opposed to
the document oriented architecture of most web service standards. For example every
BPEL orchestration is represented as new WSDL web service and so it is possible to
encapsulate web service orchestrations within other web service orchestrations allowing
for a truly recursive object oriented architecture. BPEL as well supports life time
management tools for the leasing and consumption of resources.
There are many other possible tool service and object oriented sets including some of the
original distributed object oriented middleware such as DCOM and CORBA.
Alternatively Jini/Javaspaces as also been used in such applications. There are many
approaches with various pros and cons that would be an entire paper in itself. But for a
variety of reasons there seems to be an industry trend to web services and web services
work flow language tools as a common platform for such applications. Various standards
bodies are migrating their protocols to web services including the Object Management
Group Unified Modeling Language (UMLv2), UPnP Forum for connecting devices in the
home, The Instrumentation, Systems, and Automation Society (ISA) and many others.
1.2 e-Business tools and instrument/sensor networks
The application of e-Business tools to the control of networks, instrumentation and
sensors may also have large impact on many industrial and business process control
applications. To date the e-Business revolution has had the biggest impact on
“information" industries and processes such as financial trading, business transactions,
on-line shopping and information services. But with UCLP and web services for
instruments the same tools can be used to interconnect "physical" processes and process
control systems in manufacturing, agriculture, pulp and paper, oil and gas, power
distribution, marine, transport, environmental etc.
To date many of these process control systems and Supervisory Control And Data
Acquisition (SCADA) systems use proprietary or industry unique protocols and network
architectures. The use of web services and web service workflow technologies promises
to provide a standardized and open interface to these systems and allow them to be
seamlessly integrated into existing e-Business applications. Just as importantly the web
service and grid security standards can then also be adopted to these systems, many of
which have only rudimentary security.
Although many of these industrial applications will only encapsulate instruments and
sensors connected to local area networks, there will be requirements for wide area
network applications as well. A simple illustrative example is to use these tools to
interconnect oil and gas sensors on the ocean floor across a network to computer systems
in oil refineries to accurately control the flow of raw crude to the refinery with less
inventory in ships and storage tanks [IBM-MQ]. Another example is to connect
agriculture sensors networks in wheat fields over the network with remote sensing
databases to provide future traders accurate and instant estimates of crop production. Or
weather and environmental sensors that can simultaneously feed data to remote
supercomputers for weather forecasts as well as to local users who may need more
immediate and local information.
1.3 Definition of Layer 1 VPNs, LightPaths and Articulated Private Networks
The terms “LightPaths”, “Layer 1 Virtual Private Networks (VPNs)”[L1VPN] and
“Articulated Private Networks (APNs)” are used extensively in this document. VPNs
have normally been associated with statistical multiplexed networks while layer 1
facilities are usually circuits like STS channels or optical wavelengths. Layer 1 VPNs are
generic to describe a number of layer “1” real or virtual network connections such as
optical VPNs, concatenation of STS circuits etc. This is also consistent with the ITU
definition of a Layer 1 VPN [ITU_Y.1312].
In the original UCLP design document [UCLP-DESIGN] a lightpath was defined as any
communications link that could be treated as a circuit with fixed and guaranteed
bandwidth. In addition some types of lightpaths can be subtended into smaller lightpaths
while retaining the original attributes. Optical wavelengths, STS channels, MPLS with
QoS, ATM CBR PVCs and 802.1 p/q links were therefore considered as lightpaths under
that original definition.
For the purposes of the UCLP roadmap it is proposed that the fixed bandwidth restriction
be removed in defining a lightpath, and instead the definition be expanded to include any
channel or link where an user can control the end points and topology.
To distinguish between UCLP services and traditional end-to-end VPNs, whether at
layer 3 or layer 1 we propose to use the term “Articulated Private Networks (APNs).
Because the term lightpath is used with UCLP, many people assume it is a protocol to
provide end-toend bandwidth on demand or allow for scheduled end-to-end bandwidth.
Although UCLP can be used for these types of solutions, this was not its primary
intended application. It’s primary purpose is to allow the partitioning of lightpaths and
switches to be managed by third parties for the creation of multiple virtual VPNs sharing
a common substrate or parent VPN. The outcome of this partitioning may coincidentally
result in the creation of an end-to-end lightpath and/or inter-domain optical VPN, but not
necessarily so. The defining characteristic of UCLP is that the user can reconfigure the
partitioned lightpaths and cross connects at each node resulting in an “Articulated”
private network. The articulated private network concept can be extended beyond the
physical network links to devices outside of the network such as instruments, sensors,
computers and so forth. As well the APN can incorporate internal virtual routers,
switches and other devices to make up an entire network solution. Again this does not
necessarily mean the APN must be “end-to-end”. A single APN can be thought of as a
complex “object” linking together both virtual and real network elements, instruments
and so forth, which when linked to other APNs may result in a complete end-to-end
1.4 Definition of a “User”
The term “user” is used frequently in this document. It causes considerable confusion
amongst readers as they assume it mean an “end-user”. However as explained later in
this document, because UCLP exhibits inheritance the term user is generic reference to
anyone who consumes the web services offered by an entity. The user could be a
physical network administrator, a campus network administrator, a departmental network
administrator or ultimately ( but probably least likely) a member of the general public. In
each case the parent may aggregate and abstract a collection of web services and
advertise them as single instantiated web service to their child users.
1.5 Security, Authorization and Authentication
With more and more devices and software processes linked to local and wide areas
networks, security, authentication and authorization becomes a major concern. It has
becoming increasingly recognized that centralized single system firewalls and
authentication/authorization system are inadequate. Many enterprise and process control
networks are extremely complex systems and have numerous backdoors, multiple
network connections and other weaknesses.
The alternate approach is to compartmentalize security by building a complete security
suite around each instrument and software web service. Each service or aggregation of
services is then responsible for its own internal authorization, authentication and
accounting (AAA) each of which also can be exposed as web services and orchestrated
into a new service offering. This approach therefore necessitate some sort of delegated
certificate security architecture.
The whole of security for distributed resources in multiple independent management
domains with computer to computer interaction is an entire separate discussion topic in
its own right and so will only be briefly touched upon in this document. However, it is
believed that by complying to some of the grid standard for security which employ the
concepts of delegated certificates and virtual organizations may be the best approach for
the management and control of APNs
1.6 Important concepts and assumption presented in this document
There are several important inter-related concepts and assumptions presented in this
document that will require further analysis and discussion. These are highlighted here to
guide the reader through some of the basic design parameters and objectives. It is
possible that these design parameters or objectives may not be fully realized in any actual
working implementation, but they are presented here to set a framework and a “mind set”
of the underlying architectural philosophy, namely:
(a) With UCLPv2 the first rule of network architecture is that there is no network
(b) Exposing all software (and/or time slices), hardware processes, network elements,
individual lightpath and cross connect as web services;
(c) Extension of the “network” into the application and/or computer;
(d) Using web services and web services workflow to build a universal control plane
capability across instruments, lightpaths, cross connects, networks and software
with connections and work flow state maintained by the end user orchestration;
(e) Eliminating the concept of network layers which only communicate to the layer
above and below such as the OSI 7 layer stack and replacing them with a stack of
services which can communicate on a peer to peer basis with any other web
services, anywhere in the stack or even outside the network stack; and
(f) Using the semantic web with specially developed ontologies or RDF/OWL
vocabularies [RDF] for UCLP and instrument web service discovery.
These concepts and assumptions are explained in more detail as follows:
1.6.1 No pre-defined network architecture
One of the fundamental concepts with UCLP is that there is no need to define, a priori, a
network architecture. Instead the user (which may be a network administrator or in some
cases an actual end user) may create a network of their choice including topology,
routing, network protocols and so forth. One user may wish to create an IP over optical
switched network, while another may wish to build an IPv6 or for that matter an ATM
It is important to note that the resulting network is not necessarily a fixed static
architecture. The user can change the network architecture at any time they so wish to
adopt to new applications or requirements.
The first question that springs to mind with such an approach is how to assure end-to-end
universality, which is one of the defining hallmarks of the Internet today. But since one
of the fundamental drivers of UCLP is the assumption that the customer will control
and/or possibly own the network, and therefore it is in the customer’s own interest to
deploy an architecture that provides the greatest end-to-end universality and peers with
similar customer owned networks which are similarly motivated. This is in stark contrast
with today’s network environment where service providers are trying to build walled
gardens, and want to limit universality as it is seen as a competitive threat.
Peering between customers may exist at several different levels, and again the
assumption here is that is the user or customer who chooses with whom and in what way,
they would like to peer with like minded networks. In some case user controlled
networks may peer at the optical plane, in other cases they may peer at the IP layer, and
in other cases they may peer only at the application layer as for example with VoIP or
port 80 peering. Again the important point to stress that is the user or owner of the
network, not a nameless, faceless network engineer who makes the decision of how the
networks should interconnect and peer.
Of course, in many cases, there will be many users who will want to keep their network
closed and private and have no interest in peering with others at any plane. Again this
should be a user’s choice.
Similar to GENI, this adaptability of having no predefined network architecture allows
users and researchers to explore new network architectures and concepts. For example
one area of interesting research would be to investigate other possible addressing and
naming mechanisms as for example using URIs (Uniform Resources Indicators) for end
host names and addresses and more traditional IPv4/IPv6 for network addressing. In
essence this would eliminate the need for DNS as the actual host name is in the edge
forwarding table. Of course there are all sorts of issues with regard to performance and
speed as well as memory with effectively unbounded size limits on addresses. This type
of research would not be possible on a conventional R&E network.
1.6.2 Exposing web services
In terms of the first concept of “exposing” or “atomizing” web services is an assumption
that web services will increasingly be less of a “shell” or “wrapper” around a legacy
application and be replaced by small native web service applications running on a mini-
web server or chip set equivalent in functionality to an Axis/Apache/Linux kernel proxy.
Many manufacturers of instruments, software processes, network equipment are already
adding these capabilities with the new single chip or single card processors.
Exposing services is also one of the key underlying principles of UCLP. UCLP pioneered
the concept of rather than building one common generic IP network service for many
users, one instead could deploy many parallel networks for individual groups of users.
Every lightpath and every cross connect is a service and the concatenation or subtending
of these services is also represented as a service. Each of these individual networks on a
common substrate has their own separate routing topology and discovery mechanisms.
This architecture then makes it possible to extend some of these networks into new areas,
which would be impractical with a general purpose IP network, such as deep into the
application because every routing node may be in fact a software abstraction rather than a
1.6.3 Extending the network into the application
”Networks” today are often thought of as physical entities that route or forward packets
between “physical” interfaces on computers, routers or switches. By exposing individual
application and software processes there is no reason why the network itself can be
extended beyond the physical realm deep into the computer or application itself and route
packets not only between hardware interfaces, but software interfaces as well.
In fact, this was how the original routers in the Internet were originally constructed. The
move to specialized high performance routers was required over time because of the
sustained high volume packets these devices were required to forward and the increased
complexity of routing over the global general purpose Internet network. However by
moving from a common IP network with high performance routers to many smaller
parallel customer controlled (or owned) or application specific networks or APNs (i.e.
routable and reconfigurable VPNs), may enable some these networks to be extended past
the physical interface of a computer and be connected directly to the application.
Currently routing of packets between software processes is largely done through overlay
networks or through virtual interfaces e.g. XML routing, peer to peer networking etc.
However by extending the physical network into the software might provide for a more
global, seamless and portable applications. URI (Uniform Resources Indicators)
addressing could be used rather than IPv4 or IPv6 with ports and sockets. This is
described in more detail in Section 4.3
This concept again is similar to the proposed GENI Virtualization architecture to have
different time slices on a host linked to different virtual networks.
1.6.4 Web services workflow as a universal control plane
This leads us to the third concept of using web services and web services work flow as a
common open standard for the control plane, not only of the network but the equipment,
routers, switches, time slices and sensors as well. The attraction of using web services as
opposed to a myriad number of other control plane mechanisms like GMPLS, ASON, O-
UNI, etc is that web services can not only interact with each of these control planes
through a wrapper, but also communicate directly with other control plane web services
that may or may not be network related.
1.6.5 Network service stack instead of network layers
The above concepts can be further extended by thinking of the network itself as a set of
exposed web services rather than layers, in such a way that any network service can
communicate with any other service, network or not. This has the potential to provide a
lot more flexibility into the network layer and also allow the many software concepts of
object oriented design, inheritance, etc to be applied to the network in what is sometimes
referred to Object Oriented Networks (OON).
1.6.6 Using the Semantic web as a discovery tool
To date most network architectures and instrument systems have had a very fixed and
defined device and topology discovery protocol generally using a very simple syntactic
structure for query and expression. The semantic web with carefully crafted vocabularies
and/or ontologies may provide a greater degree of flexibility and allow the intersection of
different ontologies for different instrument types and network connectivity.
For example an end user is not going to know, or care, about a syntactic structure of end
to end network connection made up of AS numbers, IP address and port/slot/channel
assignments. Rather they will want to say (most likely use a graphical orchestration tool)
"I want a lightpath from my bio-informatics application on my computer at University X
to your visualization engine on a computer University Y" The end user does not want to
be bothered by complex hierarchical naming structures.
In this roadmap we proposed that he UCLP or instrument WSDL query a well known
service for a UCLP semantic web server. The semantic web server would then do a
distributed search with an ontology cross reference that matched these input
requirements. Some third party such as graduate student or another researcher may have
already created such a specific end to end lightpath with that name and has offered it up
for lease to other parties which would be discovered by the semantic web.
If there is no existing lightpath exists then the UCLP semantic web might resolve
"University X” to "AS 506" and "University Y" to "AS 403". In turn the semantic web
would continue to resolve these highest level hierarchical names to lower names until it
can establish a sequence of connection interfaces and cross connects that would form an
end to end lightpath.
1.7 Layout of this document
This document in Section 2.0 first provides an update and background on UCLP in its
current configuration. Understanding the drivers for the development of UCLP will
hopefully provide the context as to its adaptability to instrument and sensor networks.
Section 3.0 describes new features for UCLP that are required to complete its
functionality for use in networks, but more importantly will have applicability in the new
emerging sensor and instrumentation grids.
Section 4.0 lays out the architecture vision for the UCLP road map followed by specific
example scenario in Section 5.0 on how UCLP would work for interconnecting sensor
and instrument to a network.
2.0 UCLP status to date
In September 2003 CANARIE issued a call for proposals under the Directed Research
program to invite university researchers and business to develop a proof of concept for
User Control of Lightpaths over the CA*net 4 network. Three teams subsequently
undertook to develop the first implementations of the UCLP. Details on these
developments can be found at the CA*net 4 UCLP web [UCLP].
Each team undertook an entirely different design approach to the problem. This has
proved quite useful in identifying and comparing strengths and weaknesses in the
different implementations as well as providing useful directions for this roadmap
CANARIE staff has deployed and are testing the various UCLP implementations. This
UCLP roadmap document is an outcome of these ongoing tests and end user evaluations.
As well new developments in Web services and grid technology, particularly the new
evolving Web Service Work Flow and Orchestration have provided insight to possible
new features for the next version of the UCLP software.
2.1 UCLP description
It is important to review the original objectives of the UCLP program and as they are still
critical to the future evolution of the concept of user control not only of networks, but
also of instruments and software process. This will be fully elaborated further in this
There still remains considerable misunderstanding about the purpose of UCLP. Some
people think it is an alternative optical routing/switching protocol to GMPLS or ASON,
while others think that it is an “on-demand” inter-domain or end-to-end VPN protocol.
Some people also think it is a bandwidth scheduling and reservation mechanism. It is
none of these. UCLP can very simply be thought of as a configuration and partition
manager that exposes each lightpath in a physical network and each network
element associated with a lightpath as an “object” or “service” that can be put
under the control of different network users to create their own logical IP network
topologies. Related concepts have been developed elsewhere such as the ITU standards
Y.1312 [ITU_Y.1312] and Y.1313 [ITU_Y.1313], and the IETF MEGACO [MEGACO]
and General Switch Management Protocols (GSMP) [GSMP].
The primary purpose of UCLP is not yet another optical switching-routing protocol, or to
create on-demand, inter-domain and/or end-to-end VPNs. Nor is it intended for the
scheduling and reservation of bandwidth. However, as a secondary benefit, the many
implementations of UCLP can be used for all these purposes.
The major difference between other proposed partition manager protocols and UCLP is
the object oriented recursive architecture of UCLP. Rather than one layer of partitioning
UCLP assumes that owners of switches and lightpaths will create “root” lightpath (or
APN) objects. Once a root lightpath (or APN) object is created, the root can
independently create “child” lightpath objects. Each one of these child lightpath objects
can have its own independent AAA, routing, topology and end to end lightpath services.
In fact it is quite conceivable for the child APNs to use GMPS or other routing protocols
across their constituent components.
A key assumption when using UCLP is that all the resultant APNs will mostly be
used to support the interconnection of “long lived” IP networks. This greatly
simplifies the need for inter layer communication and puts the onus of link failure
detection and routing on the owner of the APN as opposed to the partition manager.
Given that no commercial switch or router supports partition management across multiple
domains or this recursive feature of lightpath objects, UCLP was developed as a “proxy”
server in front of existing switches and provide this virtual capability of lightpath objects
in terms of partitioning both the switches and the associated lightpaths.
Grid technology using web services (Globus Toolkit 3.0) as well as Jini/Javaspaces were
used as implementation strategies to provide this proxy service for the current UCLP
proxy solutions. The advantage of using such tools is that they incorporate proven
solutions for security and authentication as well as advertising and discovery of services
across independent managed domains.
2.2 UCLP objectives
UCLP was intended to address a number of unique and novel business and network
models that are only now starting to appear in the marketplace. These new models, for
the most part are starting in the same community where the Internet was first developed –
universities and research centers. Briefly they are:
(a) Customer owned and managed IP networks;
(b) Discipline or application specific fine grained IP networks; and
(c) User controlled traffic engineering of IP networks
These objectives are described in more detail as follows.
2.2.1 Customer owned and managed networks
Many large enterprises such as universities, regional networks, government departments
and commercial organizations are acquiring their own dark fiber and unprotected, point-
to-point wavelengths for their wide area networks. Ideally these institutions want to
integrate these wavelengths and dark fiber from different suppliers within their own
network management domain. They want be able to do their own topology and discovery
and protection and restoral as well as offer value added services such as VPNs (layer 1
through layer 3) as well as APNs to their downstream users. This is not possible if the
organization purchases current VPN services, optical or otherwise, from today’s carriers.
Current VPN architectures are only single domain, do not allow cross connection
between VPNs within a domain, and, as yet do not allow VPNs to cross connected to
VPNs in other domains.
The UCLP architecture allows a carrier and/or condominium wavelength provider to
partition their optical switches and wavelengths to enable different organizations to
manage their own partitions independently and link those partitions to partitions in other
independently managed networks so that the a comprehensive wide area network solution
can be operated by the enterprise. The enterprise can then run its own routing and
topology discovery protocols in these partitions and incorporate them within their
existing LAN or WAN network.
CAVEwave acquires a separate wavelength between
Seattle and Chicago and wants to manage it as part of
its network including add/drop, routing, partition etc
Figure 2.0 National LambdaRail Condominium Wavelength Network
A good example of such a model is the National LambdaRail [NLR] network in the
United States. This is a classic condominium wavelength network where different
organizations own and operate different wavelengths for different purposes. Since each
wavelength owner may operate a separate network, and may wish to integrate that
lambda with external network services it is not practical to have a central management
plane for topology discovery and routing that is typical of most traditional DWDM
carrier networks. This is illustrated in Figure 2.0 where one of the condominium
partners CAVEwave [CAVEwave], which has acquired one of the lambdas that is part of
NLR and wants to do its own add/drop, wavelength routing and partitioning on that
specific wavelength independent of the other wavelength owners. It also at some point in
the future wish to acquire an additional wavelength outside of the NLR and yet
interconnect that new wavelength and use it as an integral part of network with its
existing NLR wavelength. UCLP was developed specifically to allow for this
integration of wavelengths from other suppliers in different management domains.
2.2.2 Specialized Internet networks for high end applications and/or disciplines
Today’s Internet networks (both research and commercial) are generic in design and
optimized to serve the average traffic flows and requirements of many thousands of users
with relatively small flows. Increasingly however there are small communities of users
who have extreme traffic volume per flow requirements, that in many cases dwarf the
traffic volumes of regular IP networks. These traffic flows may originate from the newly
emerging grids of high performance computers, but are also likely to originate with
specialized instruments and sensors. A good example is the collection of data from radio
dishes involved in Very Long Baseline Interferometers (VBLI). Another example is the
data streams that originate at high energy physics detectors located in various facilities
around the world including TRIUMF, Fermilab, CERN etc. Additional examples include
acquiring the data streams directly from the charge coupled devices used in astronomy
and synchrotron detectors.
See next slide for more details
University Common DWDM substrate
Dept High Energy
Figure 2.1 Traditional Hierarchical Internet networks
Figure 2.x Discipline Specific Internet networks
Because of the size these data flow volumes it makes sense to enable the physical
network topology to be more dynamic and responsive to their specific application needs.
However, rather than having a network engineer undertaking traffic engineering
decisions, as is done today, these can be done by the end users or by their community
with UCLP, in effect, creating discipline specific IP networks as illustrated in Figure 2.x
Another advantage of such specialized IP networks is that they can be designed to bypass
campus firewalls and LAN networks through direct fiber or campus wavelength
connections as illustrated in Figure 2.y.
Universit DWDM Global
Figure 2.y Direct Campus Connections
This obviates the need for the campus network manager to make a large investment in
high end routers, switches and firewall to provide a solution which meets the needs of
only a very small subset of their client base who need such capabilities. Campus or
corporate network security policies can still be applied to discipline or application
specific networks especially with the move away from centralized security systems using
firewalls to more compartmentalized security systems based on web services standards.
2.2.3 User Controlled Traffic Engineering
A related concept is the enabling of network managers to dynamically undertake their
own traffic engineering, based on source-destination IP address block pairs as well as AS
paths. Already there are a number of commercial products that look at BGP routing and
allow the network manager at the edge, to “pref” the best BGP path based on throughput
and other metrics, but this approach can affect the routing of all traffic going through a
particular BGP peering. The UCLP approach allows an end user to “create” a new BGP
path rather than selecting the best BGP path that may be advertised by setting up direct
optical links between managed partitions on UCLP enabled network.
In addition with a plethora of lightpaths such as with the National Lambda LambdaRail,
it is possible for regional networks and institutes to establish direct peering with each
other rather than exchanging traffic through a hierarchy of IP networks as illustrated in
Other national networks
National or Pan-Nationl IP Network
NREN A NREN B NREN C NREN D
Figure 2.1 Traditional Hierarchical Internet networks
With UCLP and a national DWDM network like CA*net 4 or NLR a much richer peering
mesh is possible between regional networks, but also between institutions, and in some
cases individual departments as shown in Figure 2.2. In this example a number of UCLP
lightpaths (or APNs) have been created over a common DWDM network to enable the
direct peering. Some of the lightpaths are across the national DWDM network, but others
may be implemented through direct physical connection of the regional networks
The concept of inheritance in the creation of child lightpaths out of parent lightpaths is
also illustrated. In this example, the four optical regional advanced networks (ORANs)
could each own a wavelength across the national DWDM network such as the NLR. The
ORANs could use these wavelengths to create their own national IP networks, directly
peer with each other or international partners, or interconnect to national IP network for
international peering. Some of the ORANs may partition their wavelength into smaller
lightpaths and through UCLP assign control and management of that lightpaths to a
downstream client such as university. In turn the university may also further partition a
lightpath such that an individual department or server may have a direct lightpath
connection. Alternatively a researcher or department could acquire from the regional
network or the national DWDM network a direct wavelength or lightpath. Any number
of interconnections and peerings are possible.
2.3 UCLP discovery and advertisement
In order for end users to discover networks and or devices it was proposed that UCLP use
the BGP AS path information as a Uniform Resource Indicator (URI) to a well known
UCLP (or AAA) server. Upon finding an end to end path the BGP routing information at
each end of the link would be updated to indicate the new path. This process is referred to
as Optical BGP (OBGP). As well it was proposed that an existing end to end lightpath
which been previously established across multiple partitions and was subsequently not
being used by the partition owner could be advertised through well known resource
advertisement and discovery tools such as UDDI or WSIL.
ORAN A ORAN B
ORAN C ORAN D
Figure 2.2 Richer mesh of direct peering enabled with UCLP
In the case of OBGP, the end user’s application would invoke an agent that would query
the local border router to get the BGP AS path information to the destination IP address.
The agent would then query well known URIs, as for example http://ASxxx.net/UCLP-
server to see if a partition could be created to provide a lightpath across that domain to
the next associated AS in the BGP path.
If a path could be successfully set up then the OBGP agent at each end of the connection
would insert a new static route in the forwarding table, so that data packets with the
associated destination address would now be redirected along this new route. Because
the connection is layer 1 physical connection between interfaces in different domains
ARP proxy services may be required, or prior agreement on sharing the same address
An Internet RFC draft [OPTICAL_BGP] was also written which outlined a process of
how lightpath information could be carried in BGP updates. This would allow as user to
immediately know which AS system provided lightpath capability. The user would still
have to query an OBGP server associated with a given AS as to the availability and
policy of the lightpaths. However to date this version of OBGP has not been deployed as
it would require router manufacturers to implement it in their routing code.
2.4 UCLP design assumptions and criteria
There is often considerable confusion as to whether UCLP is a competing optical
network protocol to GMPLS, ASON or other inter-domain optical switching reservation
bandwidth protocols. UCLP can be thought of a technology that operates on the layer
below these protocols or any other protocol that operate on the network topology
discovery and routing layer.
Although UCLP can be used for reservation and scheduling of bandwidth and/or
lightpaths, that is not its primary intended purpose. In fact, it is still an open question as
to whether reservation and scheduling of bandwidth or lightpaths will ever be required in
networks. Any application that is “bursty” and requires a lot of bandwidth for a short
period of time is probably best served with a high bandwidth routed network. High
bandwidth IP networks are designed for statistical multiplexing such types of flows.
UCLP, on the other hand is intended to be a user controlled traffic engineering protocol
that allows users such as ISPs, enterprises and, in some cases, individual users to setup
direct high throughput, or high QoS, long duration IP connections.
As mentioned previously UCLP is akin to a partition manager, but with much more
functionality and extensibility. Once network elements and links have been partitioned,
the respective owners of those partitions may elect to implement their choice of network
topology discovery and routing protocol. So rather than one topology discovery and
routing plane there may be several parallel topology discovery and routing planes each
with independent topologies and routing protocols.
As such, with UCLP there is no primary requirement in terms of traditional optical/IP
routing to support routing, topology discovery, reservation, scheduling, authorization,
accounting, or other related services. However, variations of these services are largely
implemented in the current versions of UCLP not so much to support traditional
optical/IP routing but to enable partition management applications. It is expected that
most of the optical/IP network functionality will be provided by higher layer services
operating above the partition layer. Although the services are similar it is important to
distinguish between the respective objectives of these services in terms of an optical/IP
routing network using GMPLS or ASON versus a partition management service:
(a) UCLP does not require a central control plane for the establishment of end to end
lightpaths (although a discovery service is required to find the switches that offer
partition services and creation of “root” objects);
(b) UCLP primary purpose is NOT to offer reservation or scheduling mechanisms of
lightpaths, but owners of partitions or root lightpath object may offer this service
to users of any child lightpaths that are inherited from the parent or root lightpath;
(c) UCLP does not require millisecond provisioning as lightpath root objects or
partitions are assumed to be provisioned on a long term basis
(d) UCLP does not necessarily require node or link failure error messages to be
redirected to partition owner or the root lightpath object owner (as that owner
may in fact reside in another independent network domain and like the Internet
itself restoral and protection is done at layer 3 independent of failure on the
underlying transport); and
(e) UCLP does not require crank-back or fail over mechanisms to be provided in the
event of a link or node failure as this is a service that can be implemented above
the partition or root lightpath object.
It should be noted that some lightpath owners may want to operate a routable VPN
(APN) where the lightpaths, rather than the packets are re-routed in the event of a link
failure. In that case link status information needs to be delivered to the UCLP partition
owner in order that the cross connects in that partition can re-route the lightpath.
However in most cases it is assumed that a partition owner (as opposed to the switch and
facilities operator) who will run and maintain a set of layer 3 routing, topology and fail-
over daemons or protocols to provide such services between the partitions on different
switches which are under the control or ownership of that specific partition owner. These
routing daemons may be a logical router or a blade router that is integrated with the
switch. It is the subsequent partition owner’s layer 3 routing protocol that determines link
failures, rather than depending on the underlying switch or optical fabric to report
It is important to note that a user may have ownership of partitions on switches that are in
different physical network domains. As such the user must run their own keep alive or
hello messages between each link to detect link or node failure, as well as provide
restoral and protection services if required. The underlying physical network providers
will have no visibility or awareness of the other networks and cannot provide such a
Of course, in an optical/SONET network the routing information can not be done in band
and as such the partition owner must use an out of band channel, such as the Internet
itself or a common channel provided by the switch owners for this purpose. The later
arrangement is the most common in optical/SONET networks and so it is incumbent on
the switch owner to provide accessibility to the switch partition through the out of band
control channel to the partition owner. This can be easily achieved through industry
standard security techniques.
UCLP is designed with the assumption that the creation and assignment of partitions of
both network elements and lightpaths will be for long duration, in the order of hours and
days if not weeks. UCLP was never intended to be a fast switching algorithm for optical
circuits. Fast optical switching is required if maximum utilization of limited network
resources is required. It is expected that the demand for fast switching, or scheduling
and reservation, of optical circuits will become obsolete as the cost of wavelengths
continues to drop. Once wavelengths start following the same price curve as semi-
conductors users will waste bandwidth rather than incurring higher operational costs on
optimizing their utilization through fast switching, or scheduling and reservation
Finally it should be pointed out that UCLP can operate on a physical switch or a virtual
abstraction of a physical switch. It is not necessary that a network operator expose and
partition all their network switches and links within their network. Instead they can offer
a “virtual” UCLP service through a well known interface such as O-UNI. To the end
user the network cloud appears as single switch entity, but in fact, the “cross connects”
across the virtual switch may in fact be facilitated through the creation of edge to edge
VPNs across a network cloud. In reality, this is likely to be the most common way for
network operators to provide “intra-domain” UCLP services.
3.0 UCLPv2 additional features
As mentioned previously not all the features described in the original UCLP architecture
have been implemented in the current versions largely for cost and complexity reasons.
The first versions of UCLP were developed as a proof of concept to provide the minimal
functionality required in a research network like CA*net 4. These additional features
representing new services that need to be added to the UCLP software to make it more
scaleable and extensible, particularly in new application areas beyond networks. These
new features are described briefly as follows:
(a) Support for providing layer 3 and/or layer 2 routing and links , topology
discovery, and link status daemons in each partition through the use of virtual
routers, link failure detection, etc;
(b) Graphical tools to allow partition owners to link, create end to end lightpaths and
manage their partitions;
(c) Resolvers to map IP addresses to port/slot/channel numbers on the switches such
as Semantic Web tools, as most applications use IP addresses for end points and
(d) UCLP extensions for LAN campus networks;
(e) UCLP extensions to support interconnection of virtual and/or logical routers and
(f) Exposure of all lightpath, cross connect and lightpath creation services as WSDL
web services; and
(g) Semantic web ontology for describing UCLP services to be used by advertising
and discovery service and for describing WSDL web service port types across
heterogeneous network types from IP to optical networks
As described later in this document it is believed that web services using work flow tools
such as BPEL can enable most of these enhancements without major modification to the
existing UCLP software.
3.1 Adaptation of UCLP to sensor and instrumentation facilities
Early on in the UCLP design process it was recognized that the challenges and problems
that UCLP was supposed to address in allowing partitioning and providing user control
of an optical network could equally apply in providing users the ability in the
management and in the connection of data instruments to the network.
One of the primary applications for UCLP was to allow the deployment of application or
discipline specific IP networks for specialized high volume uses where instruments that
produce large volumes of data need to be connected to computers, grids and/or databases
in far away locations. It would make sense therefore not to limit user control to just the
network elements but also extend this capability to specific network instruments at one
end, and software process at the other end to allow researcher and other users to
completely control the entire research equipment connectivity.
The grid community is actively addressing the ability for remote user control of software
processes through the application of web services to control software process on
distributed independent managed on high performance computing facilities. But little has
been done to extend this same concept to the other end of the network where the
instruments and facilities would be located.
This document proposes to lay out an architecture that will allow UCLP to be adapted to
instrument and sensor networks as well as addressing some of the newly required features
that have been identified for UCLP to support greater functionality for use in CA*net 4
and similar infrastructures.
3.2 Adaptation of UCLP to IP Routed Networks
Many modern day routers support sophisticated IP flow structures with advanced queue
management. It is proposed in this UCLP Roadmap that UCLP be modified to provide
end users to modify certain IP flow management properties through a web services
interface and in some cases IP flows be mapped to optical lightpaths and vice versa.
The GENI Virtualization program has largely focused on this aspect of the problem.
Researchers like Jon Turner et al have done considerable in depth work in this area in the
architecture of special routers that can handle the forwarding of packets for the various
virtual networks. [TURNER]
In UCLPv2 we do not propose to duplicate this effort in creating the special router
technology to allow the identification and separation of flows through a router. Instead it
is proposed that web services be wrapped around existing tools for creating logical or
virtual routers on a larger parent router. The web service tools will allow the owner of
the substrate or parent router to create a virtual or logical router and assign one or more
interfaces to that router. The interfaces may be SONET STS channels, MPLS tunnels, or
GMPLS tunnels. The owner can also assign to a given APN manager and create proper
access control tools to the virtual or logical router.
The designated APN manager then has access to a limited set of router commands
applicable to that virtual or logical router. The range of commands and tools will be
largely dependent on the specific of the manufacture and type of router.
Once the virtual router is represented as a web service the APN manager has some
limited to do configuration and topology management – but this will be far more limited
than what is possible on a switch which allows for full inheritance and many sub child
To address the requirements for these new features in the UCLP software as described in
Section 3.0, including extensibility for the integration of sensor and instrumentation grids
a new high level functionality for UCLP will be described. This will serve as framework
for other investigation into detailed implementation issues and identification of
appropriate architectures and tools to support the implementation.
The driving technology that will enable this new functionality is the emerging grid/web
services technology called Service Oriented Architectures (SOA) built on integrating web
services using the emerging web services work flow and/or orchestration tools and
platforms for the creation of end to end solutions and the use of the Semantic Web and
ontologies for describing and finding resources.
4.1 Definition of a user
To fully describe the proposed architecture for the UCLP roadmap a common
understanding and meaning must be given to who and what is expected of a “user”. An
important philosophical underpinning to the UCLP roadmap is that, as much as possible,
all services should be exposed to an end-user and that the end-user has direct control of
the various web services and work flow orchestrations. Clearly only some users will be
interested in this type of flexibility and control, while to others it represents unnecessary
complexity. So to accommodate both requirements we must define different types of
users as follows:
(a) An “End-user” in many cases will be a researcher or a non-technical user who
will have little interest in the concepts of web services and web services work
flow and simply wants a working solution or a single abstraction of all the
underlying services. An “End-user” does not necessarily need to be a human
being. It can also be another web services orchestration which incorporates the
current orchestration into another web services abstraction.
(b) A “Super-user” will generally manipulate or aggregate published services to
produce partial or complete end-to- end solutions for some end-user. Some of
these services may in fact be offered as new web services or orchestrations to
other super users who will incorporate them within their own work flows or
orchestration. Super users may work closely with end users as technical support
staff or they may also be third parties such as Virtual Organizations (VOs) who
specialize in aggregating complex service orchestrations across multiple
administrative domains into new single web service entity.
(c) An “Administrative-User” is a special type of “super-user” who has ownership
or management control of a given domain and its resources. An administrative-
user is a super user within an administrative domain and is responsible for
aggregating or orchestrating internal services and/or creating “child” services or
work flows that will be advertised as a web service to an external user. An
Administrative-user can be an optical network and/or equipment facilities
manager or operator. For example an administrative user may aggregate a AAA
service with an instrument device and/or internal optical network link to produce
a single service instance for secure access to a specific instrument or network
In this document the generic form of “user” applies to all 3 definitions and does not
necessarily imply specifically an end-user or super-user.
4.2 Service Granularity and exposing of services
Although all the original UCLP implementations support the use of Open Grid Services
Architecture (web services) to varying degrees there was no consistent implementation of
such services. As well most of the UCLP implementations developed their own internal
database architecture for keeping track of the lightpath objects and their state. A large
part of this was driven by the technical nature of SONET/SDH multiplex hierarchy where
centralized records had to be maintained in order keep track of the creation of child
lightpath objects and their connection status.
In this UCLP roadmap it is proposed that a new architecture framework be established
that as much as practically possible will push web service constructs down to the smallest
network or instrument element and use Web services workflow or service orchestration to
bind or concatenate these “services’ together to build a complete network-instrument
In the current UCLP implementations only the lightpath creation process is exposed as a
web service and each lightpath segment is a simple database entry describing its end
points, lightpath ID, owner ID, bandwidth and other relevant parameters.
In the UCLP roadmap It is proposed that each lightpath and cross connect be
represented as a WSDL web service and that concatenation of lightpaths be done
with a web services orchestration tool which creates a new lightpath WSDL web
service. Conversely each partitioning of a lightpath and/or cross connect will result
in the spawning of a new lightpath service representing the child lightpath and/or
cross connect. Probably the easiest way of achieving this is to allow many instances of
the current implementations of UCLP (wrapped in web services) to operate on the same
More importantly rather than building a priori a hierarchical set of services or
abstractions that are permanently linked together, this new framework approach permits a
hierarchical set of services to be defined by super-users as required. A simple example is
to provide secure access to a UCLP web service for either a single lightpath or a network
cloud. Rather than definitely binding a UCLP web service to a security interface such as a
AAA server, they can be linked together a service orchestration by a super-user and
exposed a single web service to other users – end-users or other super-users. The
advantage of this approach is that the AAA service can be orchestrated as single service
entry point to a domain of switches or devices (i.e. network cloud) or linked to individual
lightpaths and devices as required. Or, if the switch or device can be partitioned into child
services it may be desirable to bind the AAA service to only specific child services or
A good model of this more generic solution is to look at WSFL and work flow tools for a
variety of grid eScience applications such as the Taverna open source workflow project
Taverna [TAVERNA], the Feefluo [FREEFLUO] project, Keppler [KEPPLER] or Active
End-Points [ACTIVE]. The Taverna and Active-Endpoint systems are graphic tools that
allow users to create web service work flows which constituent parts are various web
Another good model of a workflow Graphics User Interface (GUI) is LabView
[LABVIEW] which is a popular tool for virtualzing instruments and creating workflows
by connecting instruments together through a GUI.
Graphical workflow tools are analogous to “real time” Unified Modeling Language
(UML) tools where software architects can create complex linkages and inter-
relationships between many different software sub-routines and modules.
Web services are much more loosely coupled than traditional software modules, and
through SOAP and HTTP, they now have a common interface language. Both LabView
and the new UMLv2 [UMLv2] standard support web service interfaces.
This abstraction of a set of services into a workflow web service can allow “super-users”
to create innovative, custom, end-to-end workflow solutions that in turn can be
advertised to a user community, who may or may not be interested in creating these
workflow solutions themselves.
4.3 Moving from network “layers” to network “services”
Current network architecture perspectives are largely proscribed by the network layer
stack model where each network layer only communicates with layer above and below it
and is largely hidden from the end user.
However another view is that network is less of an layered set of services, but more akin
to a software service, conceptually like the Unix “pipe (|) ” command. The Unix pipe
command is a utility that links the output of one software program to the input of another.
This concept can now be applied to network services where the network is a software
utility that allows data to piped from one application to another.
Exposing all network links as individual web services would enable this concept of Unix
piping in the wider area not only between end points but between traditional network
layers. By viewing everything as a service from the traditional application layer to the
physical layer new relationships and linkages can be established between applications,
users and networks.
For example, traditional routing protocols were designed to forward packets between
physical network connected devices. But by abstracting the network into services, the
“network” can now be extended past the physical interface into the middle of a computer
to a “soft” interface on a software process or web service as shown in Figure 4.0. GENI
has a similar concept where individual time slices on a host are connected to different
Daemons for layer 3 routing protocols like OSPF, BGP, etc can also be exposed as
services and be linked to the software process with web service workflow orchestration
tools. So routing of packets can now occur, not only between devices but between
processes (time slices) and/or between processes and network devices. Within a single
“physical” computer software processes or time slices could be forwarding packets to
different interfaces and/or other internal processes at the same time. Moreover different
processes or services within a single machine could be integrated into different network
VPN extends into computer
to specific processes
Time slice or
Layer 3 WS User A
http://10.0. Virtual Routers
Time slice or http://10.0. DWDM
Interface Card or port
With URI addressing
Figure 4.0 Extending the network into the application
Our whole concept of a network now completely changes. Rather than being a physical
infrastructure layer that only connects physical devices and lies below the application
layer, it can now be a seamless part of the entire distributed application. The “network”
can now be deeply embedded into the application and provide a more open and
transparent method for forwarding packets from one application to another.
Figure 4.0 illustrates to separate APNs or virtual networks with their own separate virtual
routers interconnecting to separate processes or time slices on the host computer. In
addition on the host computer, in this example, there is a common routing daemon which
allows packet forwarding between the two processes. Again all the elements of these
APNs are exposed as web services and therefore the APN manager can manipulate or
change the architecture and topology any time they wish.
Of course, if software processes and/or embedded web services can communicate via a
routing protocol with other devices and/or processes the demand for address space will
increase significantly. Rather than using URIs for handles and ports, they can also be
used to identify interfaces and neighbors for these embedded software networked devices.
A physical interface on a computer may advertise itself not only to physical neighbors
such as routers and switches but to internal software processes and/or interfaces.
Similarly these devices would advertise their URI addresses to the physical interface for
In Figure 4.0 there are two separate routable APNs. The APNs can be at layer 1 or
higher layers. But in this example we assume they are operating at layer 1 with UCLP
routing daemons located at each optical/SONET cross connect switch. As noted
previously each VLAN can be operated independently by the APN owner with their own
separate routing protocols and services. More importantly because of the granularity of
the APNs it is possible to build one APN with software routing daemons that are
embedded within an application as show in the diagram. This would be impractical on a
normal routed IP network because of the size of the routing tables and the need for a
comprehensive set of protocols for router management, etc.
Another simple example illustrates the concept of exposing each layer of the OSI
network stack as a network service which can communicate with any other layer or with
services outside of the stack. Today many applications send and receive data through a
well known port and TCP/IP stack on their computer. The TCP/IP stack assembles the
data into packets and manages the actual TCP session with a remote host. Generally there
is little flexibility in this architecture. If the application needs to do a large file transfer
over long distances this type of “default” arrangement may not be adequate. The current
solution today in these situations is to custom assemble special applications protocols for
the file transfer. In fact there is a community of researchers who are actively involved in
this type of work [PFLDnet]. However, if the application was exposed as a web service
then it might be able orchestrate different file transfer techniques based on the size of the
file, distance, time to deliver, etc. Each file transfer technique would also be exposed as
web services – one might be for best efforts transmission while another may be integrated
directly with a optical/SONET channel to provide guaranteed throughput over longer
4.4 Web services for control plane versus data plane
To date the primary use of web services has been to encapsulate transaction data for
financial and commercial transactions. Consequently the biggest application for web
services and related work flow and orchestration technologies has been for e-business
Individual financial transactions generally have small data volumes and so the overhead
cost of encapsulating transactions within a verbose web services framework has not been
that onerous. However the data flows for instruments, sensors and networks can be quite
significant, sometimes on occasion in the order of Gigabits per second. Clearly
encapsulating this data would be impractical due to the high overhead costs. Therefore in
this document it is proposed that the web services constructs and workflows or
orchestration be used with the control plane to setup “pipes” and or linkages to enable the
rapid flow of un-encapsulated data between services.
Traditional web services are designed for transaction based interactions, but since the
establishment of pipes or linkages requires maintaining state, “state full” web services are
required such as WSRF. However, another approach is to use orchestration and workflow
tools such as BPEL or Keppler to maintain state. There is still an open debate as to which
is the best approach. But increasingly it is seen that tools like BPEL offer true object
oriented characteristics with inheritance and long lived state and as such are more suited
to instrument grids, as opposed to WSRF.
Over the past decade there have been several message passing and object oriented
protocols have been developed such as CORBA, DCOM, Jini/Javaspaces etc. Each
protocol has had a difficult time dealing with the issues of dealing with generic failures,
healing of state-full partitions, graceful degradation of guarantees in the presence of
partial failures, etc.
The attraction of WSDL and workflow tools such as BPEL and Keppler is that they are
not designed to replace existing protocols that handle these issues extremely well. What
WSDL and web service workflow tools enable is a common interface between existing
and new processes. WSDL and BPEL are more akin to provisioning and configuration
tools to link together existing state-full processes. The underlying processes and
protocols maintain actual state of the process and handle disruptions, outages, etc.
Although BPEL specifically supports state through correlation sets it is not the intent in
this framework document to use such tools to maintain state in the BPEL linked
processes. Hence there is no intent to replace existing routing protocols like OSPF or
BGP, but instead give the user greater flexibility of WSDL and BPEL in configuring
networks and software processes.
The other aspect of WSDL and BPEL that is appealing is that they are built on a much
simpler paradigm of the web itself. They allow for the very simple concept of linking
text and images through embedded http pointers to be extended to processes. And
because WSDL and BPEL (which really is a specialized version of WSDL) use URI
structures there is no predefined data types and constructs that must be adhered to by
developers in order for their applications to communicate with each other. The data types
and the message passing is defined through http itself using SOAP bindings.
4.5 UCLP (WSDL) Proxy for Instruments and Sensors
Most instruments and sensors used in research applications and process control have
some characteristics similar to that of optical cross connect switches. They assume a
single management entity and are intended to connected to a communication medium
through a fixed long term configuration, usually through a single LAN interface or serial
However increasingly researchers are recognizing it would be useful to be able to
independently link or “bind” these devices to different systems such as networks, APNs
and software services, particularly when these instruments may be shared by different
teams of scientists at multiple institutions. Connecting the instrument to a lightpath may
be one of many types of “bindings” that could be required. Other possible bindings
would be to connect the instrument or sensor to an IPsec tunnel for secure transmission of
data, or to another instrument in order to correlate or calibrate data flows.
Of course, today this can all be done by network managers just as easily as they can setup
lightpaths across an optical network without using UCLP. But the purpose of UCLP is to
allow end users to do their own configuration management and bindings without the need
to signal or request permission from a network operator to provide such a service. The
same principle could be applied to connection of remote instrumentation to computers
and servers located at a home institution.
As part of the UCLP roadmap it is hoped that a couple of sensor or instrumentation
facilities can be identified that would be willing to implement a UCLP proxy server to
allow user control of the instrument as well as the ability to create daughter services and
other features that are currently available with UCLP for optical switches. An open
question remains is whether child services should be described as workflows
(orchestrations) or state-full web services.
In order to do this, the current versions of UCLP must become more generic and not
assume that the only things they are concatenating or binding are lightpaths. More
importantly it would be useful to describe lightpath objects as a web service - WSDL (or
perhaps a web service work flow which can also be expressed as web service) in their
own right. This allow a user to link together an instrument and a software process through
a UCLP “service” as is currently available today, but also to discover existing pre-
configured lightpaths that may have been created by a previous invocation of UCLP.
Many of these other services already exist in one form or another and are used by specific
science teams around the world. But few of them have been adapted to serve a number of
different science disciplines or allow seamless integration with other services. A good
example of a prototypical web service based sensor network is ROADnet [ROADNET] -.
And a good example of a prototypical set of common database and network services for
multiple science disciplines is the US Integrated Ocean Observing System Plan [ORION]
using the Open Source Project for Network Data Access Protocol (OPeNDAP)
4.6 Semantic web, WSIL and UDDI for path discovery and resolution
As in the original UCLP design one of the key “services” that must be first invoked is a
path discovery and harvesting service. The path discovery and harvesting service can be
a web service in its own right or be a “harvesting” tool to find and locate appropriate web
services that describe all the possible segments between a source and destination. Path
information can consist of AS numbers, IP addresses of routing/switching devices along
the path, and/or slot port/channel interface identifiers of devices along the path.
Third Party Lightpath
Bidirectional -1 Gbps
Vancouver: Port x/Slot y/Channel z
Montreal: Port x/Slot y/Channel z
Available until 2006 to all Vancouver
CA*net 4 peers
Instrument WS UCLP
Chicago New York
Pwave MAN LAN WS
Figure 4.2 Possible graphical representation of UCLP web services
The path service can query a border router for the AS inter-domain path and/or an
internal routing metric such as OSPF or ISIS to get the intra-domain path information.
But the path discovery service must be also capable of discovering pre-configured partial
or complete end to end lightpaths that bypass existing hop by hop connections. So a
combination of UDDI, WSIL and topology discovery tools is needed. It is expected that
most APNs, and/or lightpaths will be expressed as existing WSDL services (or
factories). They will not created on demand.
The pre-established web services most likely will be “permission sets” to an existing
UCLP service where there those who meet the required criteria will be to create a
lightpath within a limited domain, or actual nailed up lightpaths between specific end-
Path connections may also use different sets of descriptors for bandwidth (e.g. OC3
versus 155 Mbps), connection types and channels. Intersection of ontologies or RDF
vocabularies may also be needed as different disciplines may describe connection port
types differently or with different relationships.
Ultimately it is essential that this information harvested from topology discovery
services, UDDI and WSIL resource brokers be presented graphically with each UCLP
web service graphically illustrating the various connections available to other web
services. This is illustrated in Figure 4.2
A graph connections tool can be built on top of any Keppler, Taverna or one of the
commercial BPEL visual scripting engines. One possible scenario would be for the user
to click and drag the required connections to create an end to end lightpath from the
Neptune instrument to the visualization engine. With each click and drag operation the
underlying web service for either a lightpath or cross connect would be queried for the
availability of a service that meets the specified requirements of the lightpath.
4.7 Extending UCLP to the LAN
There are already several tools that allow end users to create their own VALNs within a
campus network environment [VLAN-UBC], [VLAN-McGILL]. Some of these tools
also allow users to create nested VLANs similar in concept to creating child lightpaths. It
therefore is not a great leap in complexity to extend the UCLP concept to user controlled
campus VLANs. All that is required is to wrap these existing services in web services and
allowing multiple instantiations of such services.
Network administrators or “orchestration engineers” could provide orchestration services
to users, or create orchestration on behalf of users linking a VLAN web service to an
external lightpath web service. This is illustrated in Figure 4.3
Campus Border Router
Standard Ethernet Links
802.1 p/q VLAN
Lightpath Creation Web Service
Workflow Service VLAN to LightPath
W eb Service
Figure 4.3 Exposing Nested 802.1 p/q VLANs as web services
5.0 UCLP Next Generation Scenarios using workflow
The following scenarios will attempt to illustrate the use of web services work flow or
orchestration as a tool for linking remote instruments to users or computers over a user
controlled optical network and create a private application specific IP network. The
details presented are examples of how the underlying web services may be scripted, but
to the actual user it is expected that the primary interface will be through a specialized
GUI modeled on Taverna or Keppler such that the user can click and drag to connect the
various services together.
5.1 Creating a simple APN linking instrument to a visualization engine
This scenario is made up of several components to illustrate aspects of the proposed
roadmap for UCLP as illustrated in Figure 5.0. The scenario will demonstrate the
connection of a remote undersea instrument such like that would be used in the Neptune
project to a visualization engine located many thousands of kilometers away across four
independently managed networks – BCnet, CA*net 4, CAVEwave ( using a partitioned
wavelengths on NLR) and OMNInet. It is assumed in this scenario that the instrument
will produce extremely large data volumes and therefore will require dedicated
lightpaths. However, the same scenario could be used for small data flows where some
other end to end network service could be required such as IPsec.
Optiputer CAVEwave Engine
Figure 5.0 Orchestration scenario to connect instrument over network
There will be 4 different types of networks used in this scenario. It is assumed the
Neptune Instrument Web service (which in fact is a proxy running on a land based
server) will be connected via an IP flow QoS service or MPLS tunnel to a BCnet router.
The router interface supporting that flow is then connected to lightpath on the CA*net 4
network, which is then cross connected to a partitioned lightpath provide by the
CAVEwave at the Pacific Wave exchange point in Seattle. The CAVEwave lightpath
(which is a partitioned lambda on the NLR) is then connected to the OMNInet O-UNI
network in Chicago. From there the lightpath finally terminates on a visualization engine
at the Electronic Visualization Laboratory at the University of Illinois.
Each network will have its own independent authorization and policy management
mechanism. In this scenario the CA*net 4 lightpath for Neptune will also orchestrate a
separate policy and access mechanism that is different than that for the underlying
CA*net 4 network itself.
As a first step CAVEwave and Neptune must acquire lightpaths from CA*net 4 and NLR
respectively. It is assumed that NLR and CA*net 4 have UCLP implemented on their
respective networks. Both NLR and CA*net 4 engineering staff would create APNs on
their respective networks for the use of Neptune/BCnet and CAVEwave respectively. The
researcher or Neptune “super-user” would then see a number of web services for
lightpaths and cross connects that have been “harvested” and/or discovered as illustrated
in Figure 4.2. The super-user would then create a complete end to end orchestration as
detailed further in this document. The Neptune “super-user” described in Section 5.2.2
has the most important job of “harvesting” or “discovering” the various web services
representing APNs or lightpaths in each domain and then linking these together to create
a complete end-to-end solution from instrument to visualization engine.
In this scenario there will be at least four “Administrative-users” who will be have
control of networks and facilities in their respective independent managed domains. The
Administrative-users performing the critical role of defining which web services will be
exposed and how they will be advertised to third parties. The Neptune super-user will
then harvest these advertised web services to create their own orchestration.
5.2 Different users and different orchestration views
Probably the best way to understand service orchestration is to think of it being analogous
to of the familiar concept of specialized web pages where web-masters or “super-users”
create web pages on specific topics that link to other web pages. This makes it easy for
“end-users” who have a particular interest on a given topic to find one canonical source
to all related web sites for that specific topic. It is expected that similar “orchestrations”
of web services will be done by super-users rather than end users by themselves. As well
users of all kinds will be able to use search and discovery tools to find appropriate
services that are of specific interest to them.
It is assumed that every user, whether they are an end-user or super-user has a specialized
graphics orchestration work flow similar to Keppler, except that it can display
geographical relationships as well as workflows. The orchestration graphics tool actually
creates the orchestration XML scripts for the binding together of various selected web
services. Each icon on the graphics interface would represent a web service and be of
sufficient detail to view its portTypes, handles and other interfaces. To create an
orchestration of web services a user would highlight and drag a connection between each
portType and/or handle to a matching portType and handle on a corresponding service.
Figure 4.2 is an example of what a graphics representation of such an orchestration
where the links show the possible UCLP connections.
WSDL web services representing lightpaths and cross connects would have some unique
port types conditions that reflected the fact that they can only connect to other services
where a physical connection can exist. So, in Figure 4.2 if user tries to connect a
Chicago cross connect web service directly to a Vancouver lightpath an error condition
would result. But a Chicago to Montreal, New York, Winnipeg or Toronto would work.
Once the user has completed an orchestration a compilation engine would generate the
necessary WS scripts which would then expose the orchestration as a new web service
complete with its own WDSL description of portTypes and Handles.
5.2.1 End-user orchestration view
In the scenario of an ocean observatory, an end-user such as an oceanographer may have
a “Taverna” or “Keppler” like service work flow or orchestration interface where they
can link together services that are of particular interest to them. It is important to note that
none of these services may be related to the specific task of linking instruments to optical
networks. An oceanographer may be more interested in services and service
orchestration related to various oceanographic datasets and data manipulation tools such
as OPeNDAP. So on the oceanographer’s orchestration web page there may be icons for
various OPeNDAP web services for databases and data manipulation tools, of which one
single icon may be for a web service for a Neptune undersea instrument that is providing
a continuous live data feed. In fact this instrument web service may actually be a
complex orchestration exposed as single abstracted web service created by some third
party ”super-user” as described in Section 5.2.2.The complexity of connecting the
instrument to the network to the “end-user” is hidden by the orchestration contained in
the single exposed web service.
The oceanographer end user may use work flow orchestration tools to graphically paste
and link the Neptune instrument end-to-end visualization web service created in this
scenario to a calibration and “de-instrumentation” web service that may be available
elsewhere on the global Internet. Or they may graphically connect (or spawn) the
Neptune instrument end-to-end visualization web service to some other web service such
as OPeNDAP. But in the scenario further described in this document we will not look at
the oceanographic examples cited here, but instead explore in depth the creation of the
Neptune instrument end-to-end visualization web service on the end user’s orchestration
graphical interface and the connection to an optical lightpath across an optical network
like CA*net 4.
5.2.2 Super-user orchestration view
It is assumed that some “super-user” will create this abstracted service of linking the
instrument to the visualization engine as illustrated in Figure 5.1. The super user in this
case could be local technical staff such as a network technician or graduate student.
Alternatively it may be created by some third party which specializes in building web
service orchestrations across networks or even someone at the Neptune or ORION project
The focus of this scenario will largely be on how the super-user creates end to end
orchestrations by combining lightpath web services, cross connect web services,
instrument web services, possible campus VLAN services into a new web service that is
then exposed on the “end users” desktop as a single web service instance for the end-to-
end instrument visualization web service, as described in the previous section.
It is assumed that the super–user also has a special graphics interface built on tools like
Taverna or Keppler in order to graphically create orchestrations of various lightpath web
services as described in Section 5.0 previously. On the super-user’s graphic interface
there would be icons representing web services for a variety of tasks such as harvesting,
discovery and geographic displaying of lightpath web services as in Figure 4.2. In
addition the graphics tool may provide for discovery and harvesting of instrument
services with connection port types, notification services for fault and error handling, and
mid span routing services to create routable VPNs (APNs), etc
Lightpath Xconnect Lightpath Xconnect
IP Flow Bandwidth
WS WS WS WS
WS 2 3 4
Neptune admin orchestration Super user orchestration End user orchestration
Figure 5.1 Orchestration of orchestrations for example scenario
Discovery and harvesting mechanisms could be simple tools like Google that search on
key words or more specialized discovery and advertisement tools like the semantic web
with a lightpath ontology or web service directories like UDDI and/or WSIL. In either
case it is assumed that the super-user can quickly retrieve a URI (or an icon representing
a URI) for a specific instrument WSDL web service directly from the Neptune
web/orchestration site or by using a variety of search tools. Through a pre-arranged
certificate proxy service the user can reserve the resource by clicking on the icon.
Alternatively the icon may pop up with a separate web service in order for the user to
log-on and authenticate themselves with the specific resource.
Each of the administrative users in BCnet, CA*net 4, CAVEwave and OMNInet would
authorize specific lightpath resources to be made available to the Neptune super-user.
These resources can be allocated wavelengths or APNs with fixed end points and optical
or STS channels, or they could be a group of resources which are then fixed to specifioc
end points or channels upon request.
Alternatively a more hierarchical structure could be created where, for example, where a
CAVEwave super-user sets up an end-to-end lightpath or APN from the Seattle Pwave
across the OMNInet network to the visualization engine. It is assumed that OMNInet may
have given CAVEwave privileges to setup a lightpath across OMNInet, and such a
privilege may not be available to other third party users. The CAVEwave super-user
would represent this APN orchestration as a single WSDL web services a shown in
Figure 5.0. Similarly BCnet could arrange an end-to-end lightpath or APN between the
Neptune instrument and the Seattle Pwave using pre-allocated lightpath services from
CANARIE’s CA*net 4 and linking that to an “on-demand” IP flow based lightpath across
the BCnet network. Again the assumption in this approach is that BCnet super-user
might have authorization to CA*net 4 lightpaths or APNs which might not be available to
a Neptune super-user.
Any number of permutations and combinations are possible in how lightpaths and APNs
might be created. The examples given here are only a small sample of what is possible.
Finally, as illustrated in Figure 5.1 the Neptune super-user would graphically connect the
various advertised web services in sequence to produce an end to end solution all the way
from the network instrument web service to the destination end user’s desktop or
5.2.3 Administrative-user orchestration view
The Neptune as well as the intervening network web services administrators would also
have a geographic based GUI similar to Taverna or Keppler in order to create a host of
various web service orchestrations that would be exposed as WSDL web services to
external users, either other super users and/or and users.
Many of their clients may not have the authorization to access internal or external
network facilities or instrumentation. So the administrative user, in some cases may also
be a super-user in order to create complete, or partial orchestrations for their clients. For
example, these could include linking an instrument to a network stub that terminates on a
local high speed network, or perhaps to a distant exchange point. Rather than offering
the only instrument as a web service, it may be more practical to have a partial
orchestration such as this, which can then linked to other network APNs or lightpaths to
enable an end-to-end orchestration.
It is unlikely that Neptune administrator would allow any external user direct control over
a given instrument. Similarly no network manager will give entire control of a switch or
network to an external user. Instead it is assumed that the administrative user will
orchestrate a number of internal services in to a single web service instance that would be
exposed to external users. For example the administrator may only want to expose only
specific control functions of the instrument or network device to end users and attach a
AAA service to allow only authorized users access to that specific instrument or network
device. In addition the administrative-user may want to incorporate local archiving and
calibration of the data from the instrument before the data is forwarded to the end user.
Each one of these functions can be exposed as web services, or in some cases they may
be recursive web service instances of more complex orchestrations of other services.
As noted previously many instruments will not have the intelligence to support web
services natively and a proxy or “shell” will have to be built around the instrument on
some shore based computer that would represent the instrument as a web service. This is
the same with optical networks where in the past it was assumed only one network
manager would have access to the device. The same principles of this scenario apply
regardless of the location and form of the web service.
The administrative user would have a host of icons on their orchestration graphics
desktop representing the various instruments and services that would be internal to the
operations of Neptune or the network. The administrative user job is to basically link
these internal web service together into an orchestration exposed as a single web service
to an external user.
5.2.4 Administrative orchestration for Neptune or ORION
In this scenario the Neptune administrative user has linked the data output of the
instrument web service (which was created for Neptune internal purposes) and linked it
to a local area network service and then to a archiving/forking web service.
A simple web service instrument proxy is shown in Figure 5.3. In this example the
instrument is connected locally, or more likely remotely, to a small computer running a
web services platform such as the popular open source tool set Tomcat/Apache
Axis/Linux. It is assumed in this example that the instrument may have an existing Java
control interface, but the control interface could be as simple as a serial port or
alternatively be complex facility with many control buttons and knobs. In either case, the
Neptune/ORION administrator may want to pass on some of the controls, (or a subset of
the functionality of a control) through to another user or application. In this example the
direct instrument controls are not made available but the simple “datapathConnectionPT”
port type which allows a user to select the path has been exposed.
Neptune Instrument WS
Instrument Path A LAN Archive
WS Proxy WS & Fork
WS 1 Data Flow Path
Figure 5.2 ORION/Neptune Instrument Orchestration
In the scenario described in this document the Neptune/ORION administrative operator
does not pass on the “dataPathConnection” port type to an end user, but instead
orchestrates a direct link to the next web services process which in this is a local area
network connection between the instrument web service and an archiving/fork web
service as illustrated in Figure 5.2.
Data Path A
Data Path B
Figure 5.3 Simple web service instrument proxy
The web service orchestration, in essence, is the control plane mechanism for the
connection of the data forwarding path between the instrument web service, LAN
network service and archiving fork service. The resulting orchestration in itself becomes
a new WSDL service that can be linked to other services as detailed previously in Figure
5.1. Note that instrument controls, that the administrative user does not want exposed to
an external user can be easily removed from the external WSDL description of the
Independently the administrative user may advertise additional services which an
external user can optionally invoke. One example may be a AAA or MyProxy service to
authenticate users and another is an OSPF daemon web service which an external user
can invoke to bind to the end user’s network to permit routing of packets across private
The administrative user, through the graphics interface, will also define the partner link
requirements, the flow process and the invocation procedure and other requirements to
build the internal web service orchestration.
18.104.22.168 Orchestration of Power Consumption and Instrument Usage
One of the big challenges for cable observatories like Neptune is the management of
power consumption for various instruments. Neptune has a complex high voltage
distribution architecture [NEPTUNE-POWER] for providing power to the various
instruments and devices connected to the network. However the Neptune administrators
still have to carefully monitor the power utilization by various instruments and in some
cases may only authorize certain instruments or devices to be turned on, if they are
satisfied there is sufficient power available at given node.
Data Path A
Data Path B
Power Enabler WSDL
New Instrument WSDL
To user’s WSDL
Figure 5.4 Orchestration of power and instrumentation control
The power distribution decision process would be an ideal environment in which to
implement a BPEL work flow process. As the power system is quite complex different
nodes can have different power resources based on their distance from the shore station
and local consumption in use by different instruments. The whole power system could be
represented as a single WSDL web service, which may or may not be a BPEL
orchestration of a more complex set of web services for the power distribution at each
node and other sites. Initially a single WSDL abstraction using a shore based proxy
system of the power system may be the most practical implementation giving the critical
timing and reliability issues that surround the architecture of the power system. In any
event, which ever approach is taken, the Neptune administrator can use BPEL to link
together the power web service with the instrument web service to enable the activation
of an instrument. A simple illustrative example of how this may be implemented is
shown in Figure 5.4.
5.2.6 Administrative orchestration for UCLP network
The UCLP administrator can offer two types of web services –a set of pre-configured
APNs that are made available to a specific community or custom made APNs manually
created for a specific end user upon demand. As well, some current UCLP
implementations will allow users to set up an end-to-end lightpath on demand.
Some typical APNs may include pre-configured end to end lighpath from Chicago to
Amsterdam for access by researchers in Chicago, a layer 1 transit lightpath from NYC to
Chicago for international users, an Ethernet interface stub to a specific institution or ISP
at an Internet exchange point, or a set of APNs along a given network path and so forth.
An external user would discover all these pre-configured APNs, for which they have been
authorized, when they wanted to orchestrate a new APN.
The administrative user of a given UCLP domain would create a WSDL web service for
all APN cross connects and lightpaths that they wish to expose to external users. In some
cases the cross connects and lightpaths can be orchestrations of much more complex
internal web service APNs For example some users may want to represent a complex
network cloud as a single APN service where an external user would not see the
intermediary switches and lightpaths.
It is important to note that the orchestration engine of these web services resulting
in the expression of a new web service can be completely independent of the
engine(s) that support the component web services.
For example an administrative user can create APNs by dragging and clicking links
across a special graphics map. Every “click” could represent the instantiation of a new
web service that might be a constituent component of the larger APN. Each web service,
in fact could be a simple interface, factory or proxy to a single underlying UCLP system
or to multiple UCLPs for several independent instantiations. The APN web services
would actually be related to real physical SONET channels and cross connects.
These web services would be only available to certain users and have a given lifetime in
the order of weeks, months or year. Each web service that is exposed to an external user
would have 2 primary port types:
(a) the ability to connect another lightpath web service limited, of course by what is
physically possible; and
(b) the ability to create a child lightpath.
It is important to note that an APN would be made up of a concatentation of web
services representing each link and cross connect in the path of the APN. This way
users can “articulate” the APN from time to time by reconfiguring the web services to
create new topologies, to create child APNs and/or to interconnect to other APNs that
might be available to the user.
An example of some possible articulations would be to re-configure an APN for traffic
engineering purposes at an Internet exchange point. Initially an APN may be connected to
one peer, but at later time the end user may want to change the peering relationship to
another peer, or have 2 simultaneous peers by creating a child APN.
Another possible articulation would be to change the topology of an APN in the event of
a long time service outage on a given link.
5.2.6 Administrative orchestration for GMPLS or O-UNI network
The administrative requirements to configure a WSDL for the orchestration of a GMPLS,
JIT or O-UNI as would be used in OMNInet are very similar to creating an on-deamnd
end to end lightpath using UCLP. However, in this case most of the “orchestration” is
done by proprietary or hidden processes. The external user has no choice or influence
over the internal orchestration on how an end to end lightpath is setup across the network
cloud. The process results in binary outcome – successful or unsuccessful.
All the administrative user needs to create a simple web service describing the endpoints
and bandwidth for the underlying network interface. The port types for this web service
would be similar to those of UCLP in that the user has to define the inbound and next hop
requirements whether they be AS, IP number or slot/port/channel as well as the requested
Given the architecture of GMPLS, O-UNI, etc a number of features would not be
available as with UCLP such as the ability to partition a lightpath, cross connect a
lightpath with other third parties etc
5.2.7 Administrative orchestration IP routed network
The administrative requirements to configure a WSDL for the orchestration of an IP flow
network such as that would be used in BCnet would be similar to that for GMPLS. All
the administrative user needs to create a simple web service describing the IP endpoints
and the web service for the virtual router. Given the architecture of IP routers, a number
of features would not be available as with switches such as the ability to partition a
lightpath, cross connect a lightpath with other third parties etc. However some routing
manufacturers do allow for nested VPNs, and so in that case the web services interface
would allow the end user to create daughter lightpaths or VPNs but no daughter virtual
5.3 APN for a high energy physics network
In this scenario we will go through the steps that would be undertaken to allow Canada’s
high energy physicists to have their own discipline specific IP network for the
transmission of data from the CERN Large Hadron Collider to the Canadian Tier 1 site at
TRIUMF British Columbia, and from there distribution of the data to the two Tier 2 sites
and finally a private IP distribution network for physicists at the various universities can
access the data.
In 2005 CANARIE has acquired, in partnership with SURFnet in The Netherlands, a 10G
lambda from New York to the NetherLight in Amsterdam and from there an additional
10G facility on the GEANT 2 network to CERN in Switzerland.
In this scenario it is assumed that the 10G lightpath from New York through Netherlight
to CERN is part of the SURFnet management domain. But the facility is offered directly
to CANARIE and/or Canada’s High Physics community network – HEPnet.
Figure 5.5 is a conceptual screen shot of how a CANARIE super-user would create and
assign APN resources and lightpath objects to the HEPnet administrator. This is complex
diagram that attempts to illustrate a number of key concepts with UCLPv2.
Lightpath Object Creation
CANARIEFigure 5.4 Orchestration of power 1and instrumentation control
ONS Network YUL
Winnipeg YOW Edmonton
TRIUMF Chicago is hidden
CANARIE OME YCG
Network Resources ONS
ONS SURFnet APN
resources advertised to
Edmonton Toronto Ottawa CANARIE
MAN LAN HDX
Pwave HDX STAR LIGHT HDX
3 4 MAN LAN HDX
Chicago New York New York Geneva
Toronto STAR LIGHT HDX
Vancouver Edmonton Montreal
Victoria To Fermi
New York Geneva
New APN Resource list composition To Brookhaven
Figure 5.5 Screen shot of APN resources for high energy physics network
The dotted lines represent APN resources i.e. individual Web services that have not yet
been compiled into a functioning APN using BPEL. The solid lines represent lightpath
objects or existing running APNs that are BPEL scripts running as web services on a
CA*net 4 or other server.
The creation of the HEPnet APN involves the following steps:
(a) Identification by the physical network administrative user of those APN resources
that are to be assigned to a third party such as HEPnet
(b) Optionally aggregating some of these resources into lightpath objects and thereby
hiding some of the complexity to the third party such as HEPnet
(c) HEPnet accepting these resources including lightpath objects and concatenating
them into a working APN
(d) HEPnet partitioning the APN and offer child APNs or lighpath objects to third
First of all it should be noted that CA*net 4 is in fact made up of essentially 2 separate
physical networks: one network that contains Cisco ONS15454 switches and the other
made up of Nortel OME 6500s. Given CANARIE’s unique mandate of NOT attempting
to build a homogenous single manage network like a a traditional telecom company this
separation of the network by equipment manufacturer makes a lot of sense. These 2
separate physical networks can be treated as facilities in two separate management
domains in addition to the SURFnet domain.
The CA*net 4 network administrative staff would decide which windows and resources
that they wanted to display in the various windows on their APN screen. In this example
the administrator has show the 2 CA*net 4 networks in 2 separate windows as if they
were networks in two separate physical management domains.
The administrator user in each network domain must first identify those network
resources that are to made available to third parties. These resources, which are each
represented as an individual web service can be lightpaths, interfaces, virtual routers,
cross connects, instruments and aggregation of other resources. This can be done a
number of ways – for example by highlighting each resource and assigning permissions,
time to lease, SONET channel, etc and dragging or dropping them into the APN resource
list composition window as shown in Figure 5.5 (3) and (4). These selected APN
resources can then be advertised directly to the intended recipient, or included in a list of
at well known URL, and be “discovered” by the intended recipient through a directory or
In some cases the physical domain administrator may wish not to advertise all these
resources separately, but instead aggregate a number of resources into a single advertised
resource. This is illustrated in Figure 5.5 where the lightpath network elements between
Edmonton and Toronto are integrated into a single lightpath object (1) . This aggregation
can be a BPEL script window running on a local server that concatenates these resources
together. The resulting aggregation is in itself a Web service which can then be assigned
the appropriate permissions and dragged into the APN resource composition window (2).
Finally as shown in this example the SURFnet resources that were originally assigned to
CANARIE can also be re-assigned to HEPnet and dragged into the APN resource
composition window (5). The CA*net 4 administrator has two ways of doing this – they
can pass on the APN resources intact; or they can convert the resources into a running
web service lightpath object running on a CANARIE server. In this example it is
assumed that the resources have been passed through to HEPnet directly.
Note, at this point in time, no functioning APN has been created – only a set of lightpath
resources has been identified with permission assigned to the designated recipient. The
next stage is for the recipient of the APN resource list to compose these resources into a
functioning APN as illustrated in Figure 5.6.
The intended recipient, in this case HEPnet, also has an APN management screen/GUI
similar to CANARIE’s where they can now have two choices:
(a) they compose the various WS resources into one or more functioning APNs and/or
lightpath object WS; or
(b) they can re-assign some or more the WS resources including newly created lightpath
object WS to their parties
UoVic Campus UBC Campus
802.11 Lightpath CWDM Lightpath
Vancouver Edmonton Montreal
New York Geneva
Vancouver To Brookhaven
Lightpath Object for 2 Gbp
Composition Window Tiier 2between TRIUMF and UoVic
Figure 5.6 Screen shot of APN composition high energy physics network
The dotted lines in the largest window represent APN resources i.e. individual Web
services that have not yet been compiled into a functioning APN using BPEL. Solid lines
represent lightpath object WS or existing running APNs
In this scenario the HEPnet administrator drags APN resources from the resources
provided by CA*net 4 as well as other resources from UoVic and UBC campus to create
a lightpath object WS or running APN. The APN or lightpath object WS is a 2 Gbps
circuit between the UoVic and UBC campuses. It is assumed that this circuit will be used
for the Tier 2 connection between TRIUMF and UoVic.
Note that UoVic and UBC resources are campus LAN or DWM resources assigned
to HEPnet. It is assumed the UoVIC and UBC network administrators have their own
UCLP systems with their own GUIs in which they can assign their network resources to
whatever third party they wish.
The HEPnet administrator can continue to create a number of additional APNs including
a 5 Gbps lightpath between TRIUMF and CERN, additional Tier 2 lightpaths and finally
a 1GBps routed network between the various Tier 3 high energy physics labs across the
country as illustrated in Figure 5.7
TRIUMF UoToronto Physics 1G HEPnet daisy chain
Tier 1 Tier 2 routed
Physics 5G Tier 1 data
UoVictoria Physics 2G Tier 2 data
Tier 2 Carleton
Victoria CA*net 4 Ottawa
To other physics users at
New York Geneav
smaller universities Optional
Note: Typical View on Chicago
TRIUMF UCLP GUI
Tier 1 Brookhaven Tier 0
Figure 5.7 Resultant APN compositions for high energy physics network
In this scenario there are 3 APN compositions that were created by the HEPnet
(a) one 5 Gbps lightpath from TRIUMF to CERN;
(b) two 2 GBps Tier 2 lightpaths from TRIUMF to UoVic and UoToronto
(c) one 1 Gbps routed lightpath connecting the various Tier 3 sites across the country
linking together virtual router web services on the CA*net 4 substrate routers.
HEPnet, if it so wishes can assign management and control of each one of these APNs to
different entities. For example it would make sense for TRIUMF to manage the 5 GBps
APN to CERN – that way it could control the re-routing of the APN to Brookhaven or
Fermi Lab if there was a disruption in the direct connection to CERN.
Similarly the two 2 Gbps lightpaths for the Tier 2 sites could be handed to the
administrator of those sites so that they could have more flexibility in the re-routing and
partitioning of those APNs.
It is important to note that the HEPnet administrator has two choices in how they
hand off APNs to third parties like TRIUMF or the Tier 2 site administrators:
(a) HEPnet could re-assign the permissions on those specific resources it
received from CA*net 4; or
(b) HEPnet could create the APNs or lightpath object WS themselves and hand
these over to the designated recipients
The first approach provides the greatest flexibility to the recipient as they get a collection
of web services that they can then integrate into their own workflow schema and/or
combines with other web services. The disadvantage is that they require their own BPEL
server in which they convert the BPEL scripts into running web services
The second approach is much simpler for the recipient as they don’t need their own
BPEL server, but they are then restricted in the flexibility they have in terms of
manipulating the APN and integrating it with other web services.
6.0 Conclusions and Next Steps
In the spring of 2005 CANARIE issued a call for proposals for the next generation of
UCLP based on the framework described in this document. Three teams have been
selected to do development of 3 different versions of UCLP. These projects are now in
their early prototype development stage with full functional products expected to be
available in the first quarter of 2006. All 3 versions of UCLPv2 will be available as open
Those who are interested in participating in the UCLPv2 program are encouraged to
contact the author.
The CANARIE CA*net 4 program is funded by a grant from the Government of Canada
through Industry Canada.
Thanks to the following individuals for their comments, suggestions and editorial
Dave Macneil, Rene Hatem, Darcy Quesnel of CANARIE
Eric Byres of BCIT
Loki Jorgenson of Apparent Networks
Bill Rutherford of RXX Network Inc
Michel Savoie, Scott Campbell, Mathieu Lemay, Jing Wu of CRC