Presentation summerstudent 2009-aug09-lbl-summer


Published on

Published in: Technology, Education
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Presentation summerstudent 2009-aug09-lbl-summer

  1. 1. Advance Network Reservation and Provisioning for Science  Mehmet Balman Scientific Data Management Research Group (with Arie Shoshani and Alex Sim) Lawrence Berkeley National Lab Summer Presentations - Aug 13, 2009
  2. 2. Outline  Introduction  Resource Management  ESNet and OSCARS  Network Reservation  Problem, Solution, Methodology
  3. 3. Introduction  We are in a new era that offers new oppurtunities to conduct scientific research with the help of computation  Computational intensive science: particle physics, climate modelling, bio-informatics simulations  Scientific simulations and experimental facilities generate massive data sets  Climate modelling data  35 terabytes shared by more then 2500 users worldwide,  Next generation archive will be more than 650 terabytes  Large Hadron Collider  Expected to generate 100gigabits per second
  4. 4. Introduction  Large scale application necessitate collaborations  Data need to be tranferred to remote sites for further analysis (validate with simulations)  Need on demand high speed data access between collaborating parties  High performance visualization  Large volume data analysis  Require mass storage systems  Need coordination and management of resources ( BeStMan: Berkeley Storage Manager)
  5. 5. ESNet (Energy Sciences Network)  Provides high bandwidth network interconnect between more than 40 sites  Connecting experimental facilities, supercomputing centers and thousands DOE scientists  Delivering network as a service  Predictable performance  Efficient resource utilization
  6. 6. OSCARS  The ESNet On-Demand Secure Circuits and Advance Reservation System (OSCARS)  Conducts a QoS path for guaranteed bandwidth  End-to-end provisioning between multiple domains  Guaranteed bandwidth (at certain time, for a certain bandwidth and length of time)  OSCARS components include reservation manager, Bandwidth scheduler, and path setup system  Needs to have information about current and future states of the network
  7. 7. Data Movement  End-to-end High Performance Data Movement  Network reservation  Provisioning in client sites  Storage Allocation  Therefore, we need coordination between Storage Resource Managers and Network Resource Allocation
  8. 8. Network Reservation  Users make reservation over a web service interface  Reservation request:  source/destination end-points  Requested bandwidth  start/end times   What is requested bandwidth can not be guarantted?  Try-and-error until get an available reservation
  9. 9. Reservation request  Not aware of possible options  Cant make an optimal choice  Ineffective use of overall system  How can we enhance the OSCARS reservation system?  Submit constraints and the system suggests possible reservations satisfying requirements
  10. 10. A new service  Source / destination end-points  Maximum bandwidth that can be used  Amount of data requested to be transferred (Volume)  Earliest start time  Latest completion time  Criteria (reserver a path for earliest completion, reserve a path shortest transfer duration)
  11. 11. Challange  Dynamic max-bandwidth problem!  Time dependent dynamic network  The bandwidth value for every link is time dependent  Discrete time algorithms: time is modeled as a set of discrete values and a static graph is constructed for every time interval.
  12. 12. Example  A vehicle travelling from city A to city B  There are multiple cities between A and B connected with separate highways.  Each highway has a specific speed limit (maximum bandwidth)  But we need to reduce our speed if there is high traffic load on the road  We know the load on each highway for every time period (reservations)
  13. 13. Example  The first question is which path the vehicle should follow in order to reach city B from city A as early as possible?  Or, we can delay our journey and start later if the total travel time would be reduced. Thus, the second question is to find the route along with the starting time for shortest travel duration.
  14. 14. Example  But, we are dealing with bandwidth reservation where allocation should be set in advance when a request is received.  We have to set the speed limit before starting and cannot change that during the journey.  Advance Bandwitdth Reservation
  15. 15. A new algorithm  Search interval is divided into time windows  A time window represents a period of time where we have a stable status of available bandwidth of all related links  A snaphots of the network topology  The algorithm should be fast and scalable. Presenting clients/users possible reservations requests and alternate options
  16. 16. What I did?  Developed a new approach  Analyze the algorithm and tested with large graphs  Implemented a library to be integrated into OSCARS  If interested, a technical talk with a short demo on Monday at 2pm in 50F-1647
  17. 17. Special Thanks to  Arie Shoshani  Alex Sim  Evangelos Chaniotakis  David Robertson  Mary Thompson  ESNet team  Scientific Data Management Research Group