1. Advance Network Reservation
and Provisioning for Science
Mehmet Balman
Scientific Data Management Research Group
(with Arie Shoshani and Alex Sim)
Lawrence Berkeley National Lab
Summer Presentations - Aug 13, 2009
3. Introduction
We are in a new era that offers new oppurtunities to
conduct scientific research with the help of
computation
Computational intensive science: particle physics,
climate modelling, bio-informatics simulations
Scientific simulations and experimental facilities
generate massive data sets
Climate modelling data
35 terabytes shared by more then 2500 users worldwide,
Next generation archive will be more than 650 terabytes
Large Hadron Collider
Expected to generate 100gigabits per second
4. Introduction
Large scale application necessitate collaborations
Data need to be tranferred to remote sites for
further analysis (validate with simulations)
Need on demand high speed data access between
collaborating parties
High performance visualization
Large volume data analysis
Require mass storage systems
Need coordination and management of resources
( BeStMan: Berkeley Storage Manager)
5. ESNet (Energy Sciences Network)
Provides high bandwidth network interconnect
between more than 40 sites
Connecting experimental facilities,
supercomputing centers and thousands DOE
scientists
Delivering network as a service
Predictable performance
Efficient resource utilization
6. OSCARS
The ESNet On-Demand Secure Circuits and Advance
Reservation System (OSCARS)
Conducts a QoS path for guaranteed bandwidth
End-to-end provisioning between multiple domains
Guaranteed bandwidth (at certain time, for a certain
bandwidth and length of time)
OSCARS components include reservation
manager, Bandwidth scheduler, and path setup
system
Needs to have information about current and future
states of the network
7. Data Movement
End-to-end High Performance Data Movement
Network reservation
Provisioning in client sites
Storage Allocation
Therefore, we need coordination between Storage
Resource Managers and Network Resource
Allocation
8. Network Reservation
Users make reservation over a web service
interface
Reservation request:
source/destination end-points
Requested bandwidth
start/end times
What is requested bandwidth can not be
guarantted?
Try-and-error until get an available reservation
9. Reservation request
Not aware of possible options
Cant make an optimal choice
Ineffective use of overall system
How can we enhance the OSCARS reservation
system?
Submit constraints and the system suggests
possible reservations satisfying requirements
10. A new service
Source / destination end-points
Maximum bandwidth that can be used
Amount of data requested to be transferred
(Volume)
Earliest start time
Latest completion time
Criteria (reserver a path for earliest completion,
reserve a path shortest transfer duration)
11. Challange
Dynamic max-bandwidth problem!
Time dependent dynamic network
The bandwidth value for every link is time
dependent
Discrete time algorithms: time is modeled as a
set of discrete values and a static graph is
constructed for every time interval.
12. Example
A vehicle travelling from city A to city B
There are multiple cities between A and B
connected with separate highways.
Each highway has a specific speed limit
(maximum bandwidth)
But we need to reduce our speed if there is
high traffic load on the road
We know the load on each highway for every
time period (reservations)
13. Example
The first question is which path the vehicle
should follow in order to reach city B from city A
as early as possible?
Or, we can delay our journey and start later if
the total travel time would be reduced. Thus,
the second question is to find the route along
with the starting time for shortest travel
duration.
14. Example
But, we are dealing with bandwidth reservation
where allocation should be set in advance
when a request is received.
We have to set the speed limit before starting
and cannot change that during the journey.
Advance Bandwitdth Reservation
15. A new algorithm
Search interval is divided into time windows
A time window represents a period of time
where we have a stable status of available
bandwidth of all related links
A snaphots of the network topology
The algorithm should be fast and scalable. Presenting
clients/users possible reservations requests and
alternate options
16. What I did?
Developed a new approach
Analyze the algorithm and tested with large
graphs
Implemented a library to be integrated into
OSCARS
If interested, a technical talk with a short demo
on Monday at 2pm in 50F-1647
17. Special Thanks to
Arie Shoshani
Alex Sim
Evangelos Chaniotakis
David Robertson
Mary Thompson
ESNet team
Scientific Data Management Research Group