Federating experimental platforms
            as a playground for the future
                       internet



                                    Serge Fdida
                                http://www-rp.lip6.fr/~sf/

                             UPMC – Sorbonne Universités
                               Laboratoire LIP6 – CNRS
                                       France


FIBRE Workshop- Salvador - Nov 27 2012                       1
Past and current testbeds success




FIBRE Workshop- Salvador - Nov 27 2012                 2
Why?

    ● Do you/we TRUST the results that you/we produce?
    ● Can you REPRODUCE the conditions for which you
        obtained these results, or reproduce the results? Or
        check the robustness of your assumptions?
    ● Which Solution(s)?
            Simulation, Emulation, Testbeds
    ● A testbed for what type of problem?
            One or multiple testbeds?
    ● What about the environment?
            Monitoring, Control and Experimental plane

FIBRE Workshop- Salvador - Nov 27 2012                         3
What is our target?
                 Can we achieve this?




FIBRE Workshop- Salvador - Nov 27 2012   4
Or this, eg FEDERATION


                                         Overlays

               Sensors



                                                       Multi-Hop
                                                       radio



              Optical


                                            Services


FIBRE Workshop- Salvador - Nov 27 2012                             5
Some possible scenario ….

FIBRE Workshop- Salvador - Nov 27 2012                               6
The Polymorphic Internet :
                  Some Internet Future(s)

     • The Network is a Database (“CCN”)
     • The (Access) Network is Wireless
     • The Network is the People
     • The Network is a global Virtualized resource
     • And many others … The Social Computer …


     • They’re all Federated (Inter-domain)


FIBRE Workshop- Salvador - Nov 27 2012                7
Enabling continuous deployment of New
                  architectures
                                         Policies

                                         Monitoring

                                         Security/Trust


                                   Federation
                                                           IP
                                                          world




                                 Virtualized substrate
FIBRE Workshop- Salvador - Nov 27 2012                            8
The Federation Concept!




FIBRE Workshop- Salvador - Nov 27 2012      9
Federation (dictionary definition!)

           A federation is a union comprising a number of partially
           self-governing regions united by a central ("federal")
           government under a common set of objectives.




FIBRE Workshop- Salvador - Nov 27 2012                                10
Why to federate?
    ● Many benefits …. For users, testbed providers, …
    ● But also challenges
            Complexity, Legal and trust issues
            Policies
    ● How?
            Trust testbed owners
            Describe, control and manage distributed testbeds
             resources
    ● Challenges
            Modelling and description of the resources
            Policies, Monitoring, Scheduling, “Scale”, Diversity,
             …
FIBRE Workshop- Salvador - Nov 27 2012                               11
Federation Architecture

    ● Who is concerned?
    ● Management Authority
            Own the resources
            Legal entity
    ● Operation Engineers
            Operate the testbeds and their associated resources
    ● Experimenters
            Motivated by the launch of an experiment across the
             resources


FIBRE Workshop- Salvador - Nov 27 2012                             12
Architecture: Definition (from GENI)

    ● An aggregate represents a resource or set of
        resources that can be offered for inclusion in some
        customer specified topology. These typically fall
        into the broad categories of Computation,
        Communication and Storage resources.
    ● A sliver is a (real or virtual) resource group provided
        by the aggregate via the “Aggregate Manager” (AM)
        API.
    ● A slice is a collection of slivers gathered for a
        common purpose that are configured into a
        topology on which to deploy experiments or
        applications in some degree of isolation from other
        slices.
FIBRE Workshop- Salvador - Nov 27 2012                          13
Federation: main entities


       Facility A                                                              Facility B



  Researchers


                      Slice Management                      Slice Management



  Resources




   Authority A                                                                        Authority B
                                    Administrative border



FIBRE Workshop- Salvador - Nov 27 2012                                                         14
Federation (1)

       Facility A                                                         Facility B
                                    1. Share user credentials




                      Slice Management                 Slice Management




FIBRE Workshop- Salvador - Nov 27 2012                                                 15
Federation (2,3)

       Facility A                                                          Facility B
                                    1. Share user credentials




                      Slice Management                  Slice Management




                                   2. Share resource descriptions

                                 3. Allow experiments to run across facilities

FIBRE Workshop- Salvador - Nov 27 2012                                                  16
Federation: more than resource
                                         integration


  Facility A                                                             Facility B




                                         Policies
                Slice Management                      Slice Management
                                         Monitoring

                                         Security




FIBRE Workshop- Salvador - Nov 27 2012                                                17
PlanetLab Europe
                 testbed




         PlanetLab nodes worldwide and (insert) PlanetLab nodes
                          located within Europe
FIBRE Workshop- Salvador - Nov 27 2012                            18
PlanetLab Terminology

    ● Site A site is a physical location where PlanetLab nodes
      are located (e.g. Fraunhofer Institute or UCL).
    ● Node A node is a dedicated server that runs components
      of PlanetLab services.
    ● Slice A slice is a set of allocated resources distributed
      across PlanetLab. Slices are assigned to some number of
      PlanetLab nodes. Slices have a finite lifetime and must be
      periodically renewed to remain valid.
    ● Sliver A sliver is a slice running on a specific node. You can
        use ssh to login to a sliver on a specific node.
    ● MyPLC: PL control framework


                                  19
FIBRE Workshop- Salvador - Nov 27 2012                                 19
Distributed Virtualization

    ● As a user you want to isolated from other activities
        on those nodes on which you run. The PL provides
        a level of isolation which gives you your own file
        system, and process control.
    ● You share CPU cycles and network bandwidth with
        other active slivers on each node.
    ● The concept of slice aggregates the presence of
        your slivers within the system




                                  20
FIBRE Workshop- Salvador - Nov 27 2012                       20
Nodes




                                  21
FIBRE Workshop- Salvador - Nov 27 2012   21
Slices – hujiple_isis




                                  22
FIBRE Workshop- Salvador - Nov 27 2012    22
Slices – upmcple_paristr




                                  23
FIBRE Workshop- Salvador - Nov 27 2012       23
Slices




                                  24
FIBRE Workshop- Salvador - Nov 27 2012   24
Federation (& Legal aspects)


    ● Local consortium agreement defines
        responsibilities and liabilities of each partner
    ● Federation integrates the consortiums into a
        seamless global authority
    ● Formal Trust Relationships are the basis for this
        integration




                                  25
FIBRE Workshop- Salvador - Nov 27 2012                     25
Global Federation


                                                                        Princeton

        UPMC
                                         Europe
                                                                  USA
                                          PLC
                                                                  PLC




                                                  Japan
                                                   PLC




                                                          Kyoto


                                  26
FIBRE Workshop- Salvador - Nov 27 2012                                              26
In Practice …

    What is SFA, OMF, MySlice,
                …



FIBRE Workshop- Salvador - Nov 27 2012     27
What researchers want
Researchers




Testbeds
What researchers want
Researchers




              To have access to a testbed




Testbeds
What researchers want
Researchers




              To access many instances of the
              same type of testbed / resource




Testbeds
What researchers want
Researchers




              To access many instances of the
              same type of testbed / resource
               • For geographic diversity
               • For reproducing experiments




Testbeds
What researchers want
Researchers




              To access different types of
              testbeds and resources




Testbeds
What researchers want
Researchers




              To access different types of
              testbeds and resources
               • For combined experiments
               • For moving between simulation,
                   emulation, and the real-world



Testbeds
What researchers want
Researchers




                         Authority




Testbeds
What researchers want
Researchers




                                         Authority

      Administrative simplicity:
      A single place to sign up
      in order to use all the testbeds




Testbeds
What researchers want
Researchers




 Experiment
 control tool




Testbeds
What researchers want
Researchers




 Experiment       Ease of access:
 control tool     The ability to use a small
                  number of familiar tools to
                  access and control the testbeds




Testbeds
Where we were recently
Researchers



Authorities




Tools




Testbeds
Where we were recently
Researchers



Authorities   ✗



Tools




Testbeds
Where we were recently
Researchers



Authorities




Tools

                ✗ ✗ ✗




Testbeds
Where we were recently
Researchers



Authorities




Tools




              ✗    ✗
                          ✗
Testbeds
Problems to be solved
Researchers




                         Impossible to agree on who
                         will be the single trusted
                         authority at a global scale




Testbeds
Problems to be solved
Researchers




                         Impractical to scale a
                         single database to global
                         scale




Testbeds
Researchers




                           A secure and
              Common API   distributed
                           thin waist




Testbeds
SFA: Slice Facility Architecture
 Researchers




 ims to provide a secure common API with the minimum
ble fonctionality to enable a global testbed federation




  Testbeds
Experiment lifecycle
Researchers



Authorities      Authenticate




Tools




Testbeds
Experiment lifecycle
Researchers



Authorities




Tools




                        Resource discovery


Testbeds
Experiment lifecycle
Researchers



Authorities




Tools




                        Resource reservation


Testbeds
Experiment lifecycle
Researchers



Authorities




Tools




                        Configure resources


Testbeds
Experiment lifecycle
Researchers



Authorities




Tools




                        Run experiment


Testbeds
Experiment lifecycle
Researchers



Authorities




Tools




                        Repatriate results


Testbeds
Experiment lifecycle
Researchers



Authorities




Tools




                        Release resources


Testbeds
SFA and the experiment lifecycle
Researchers



Authorities         Authenticate    Yes*




Tools




Testbeds
              *This is at the heart of SFA
SFA and the experiment lifecycle
Researchers



Authorities




Tools




                              Resource discovery
                                                   Yes*


Testbeds
              *Without going into all details
SFA and the experiment lifecycle
Researchers



Authorities




Tools




                        Resource reservation
                                               Yes


Testbeds
SFA and the experiment lifecycle
Researchers



Authorities




Tools




                              Configure resources
                                                    No*


Testbeds
              *Use the tool of your choice
SFA and the experiment lifecycle
Researchers



Authorities




Tools




                              Run experiment
                                               No*


Testbeds
              *Use the tool of your choice
SFA and the experiment lifecycle
Researchers



Authorities




Tools




                              Repatriate results
                                                   No*


Testbeds
              *Use the tool of your choice
SFA and the experiment lifecycle
Researchers



Authorities




Tools




                        Release resources
                                            Yes


Testbeds
60
       SFA

     
         Naming (slices, users, resources, authorities)
          
              Hierarchical naming space
     
         Authentication and authorization
          
              X.509 certificates and signed credentials
          
              Federation links through the exchange of certificates
     
         Resource descriptions (RSpecs)
          
              Only the language (XML) not the semantics
     
         Standard API (XMLRPC)
          
              To manage objects (users, resources, slices, authorities)
          
              To manage resources (browse, acquire, manifest)
          
              To manage slices (create, delete, start, stop)
 FIBRE Workshop- Salvador - Nov 27 2012                                   60
OMF/OML software




            OMF supports users over the full cycle of their
           experiments, from description to instrumentation, to
         execution and measurement collection and visualization.
FIBRE Workshop- Salvador - Nov 27 2012                         61
MySlice

    ● A common abstraction to help the user browse through
        and interact with a large amount of data and sources
    ● Grounded on related work on data integration and large
        dataset navigation.
    ● Formulate semantic queries, requesting ltering and
        annotations
    ● Propose enhanced visualizations on received results
    ● Allow to balance homogeneity and heterogeneity in the
        GUI




FIBRE Workshop- Salvador - Nov 27 2012                         62
MySlice




FIBRE Workshop- Salvador - Nov 27 2012   63
FIBRE Workshop- Salvador - Nov 27 2012   64
FIBRE Workshop- Salvador - Nov 27 2012   65

Federating experimental platforms as a playground for the future internet

  • 1.
    Federating experimental platforms as a playground for the future internet Serge Fdida http://www-rp.lip6.fr/~sf/ UPMC – Sorbonne Universités Laboratoire LIP6 – CNRS France FIBRE Workshop- Salvador - Nov 27 2012 1
  • 2.
    Past and currenttestbeds success FIBRE Workshop- Salvador - Nov 27 2012 2
  • 3.
    Why? ● Do you/we TRUST the results that you/we produce? ● Can you REPRODUCE the conditions for which you obtained these results, or reproduce the results? Or check the robustness of your assumptions? ● Which Solution(s)?  Simulation, Emulation, Testbeds ● A testbed for what type of problem?  One or multiple testbeds? ● What about the environment?  Monitoring, Control and Experimental plane FIBRE Workshop- Salvador - Nov 27 2012 3
  • 4.
    What is ourtarget? Can we achieve this? FIBRE Workshop- Salvador - Nov 27 2012 4
  • 5.
    Or this, egFEDERATION Overlays Sensors Multi-Hop radio Optical Services FIBRE Workshop- Salvador - Nov 27 2012 5
  • 6.
    Some possible scenario…. FIBRE Workshop- Salvador - Nov 27 2012 6
  • 7.
    The Polymorphic Internet: Some Internet Future(s) • The Network is a Database (“CCN”) • The (Access) Network is Wireless • The Network is the People • The Network is a global Virtualized resource • And many others … The Social Computer … • They’re all Federated (Inter-domain) FIBRE Workshop- Salvador - Nov 27 2012 7
  • 8.
    Enabling continuous deploymentof New architectures Policies Monitoring Security/Trust Federation IP world Virtualized substrate FIBRE Workshop- Salvador - Nov 27 2012 8
  • 9.
    The Federation Concept! FIBREWorkshop- Salvador - Nov 27 2012 9
  • 10.
    Federation (dictionary definition!) A federation is a union comprising a number of partially self-governing regions united by a central ("federal") government under a common set of objectives. FIBRE Workshop- Salvador - Nov 27 2012 10
  • 11.
    Why to federate? ● Many benefits …. For users, testbed providers, … ● But also challenges  Complexity, Legal and trust issues  Policies ● How?  Trust testbed owners  Describe, control and manage distributed testbeds resources ● Challenges  Modelling and description of the resources  Policies, Monitoring, Scheduling, “Scale”, Diversity, … FIBRE Workshop- Salvador - Nov 27 2012 11
  • 12.
    Federation Architecture ● Who is concerned? ● Management Authority  Own the resources  Legal entity ● Operation Engineers  Operate the testbeds and their associated resources ● Experimenters  Motivated by the launch of an experiment across the resources FIBRE Workshop- Salvador - Nov 27 2012 12
  • 13.
    Architecture: Definition (fromGENI) ● An aggregate represents a resource or set of resources that can be offered for inclusion in some customer specified topology. These typically fall into the broad categories of Computation, Communication and Storage resources. ● A sliver is a (real or virtual) resource group provided by the aggregate via the “Aggregate Manager” (AM) API. ● A slice is a collection of slivers gathered for a common purpose that are configured into a topology on which to deploy experiments or applications in some degree of isolation from other slices. FIBRE Workshop- Salvador - Nov 27 2012 13
  • 14.
    Federation: main entities Facility A Facility B Researchers Slice Management Slice Management Resources Authority A Authority B Administrative border FIBRE Workshop- Salvador - Nov 27 2012 14
  • 15.
    Federation (1) Facility A Facility B 1. Share user credentials Slice Management Slice Management FIBRE Workshop- Salvador - Nov 27 2012 15
  • 16.
    Federation (2,3) Facility A Facility B 1. Share user credentials Slice Management Slice Management 2. Share resource descriptions 3. Allow experiments to run across facilities FIBRE Workshop- Salvador - Nov 27 2012 16
  • 17.
    Federation: more thanresource integration Facility A Facility B Policies Slice Management Slice Management Monitoring Security FIBRE Workshop- Salvador - Nov 27 2012 17
  • 18.
    PlanetLab Europe testbed PlanetLab nodes worldwide and (insert) PlanetLab nodes located within Europe FIBRE Workshop- Salvador - Nov 27 2012 18
  • 19.
    PlanetLab Terminology ● Site A site is a physical location where PlanetLab nodes are located (e.g. Fraunhofer Institute or UCL). ● Node A node is a dedicated server that runs components of PlanetLab services. ● Slice A slice is a set of allocated resources distributed across PlanetLab. Slices are assigned to some number of PlanetLab nodes. Slices have a finite lifetime and must be periodically renewed to remain valid. ● Sliver A sliver is a slice running on a specific node. You can use ssh to login to a sliver on a specific node. ● MyPLC: PL control framework 19 FIBRE Workshop- Salvador - Nov 27 2012 19
  • 20.
    Distributed Virtualization ● As a user you want to isolated from other activities on those nodes on which you run. The PL provides a level of isolation which gives you your own file system, and process control. ● You share CPU cycles and network bandwidth with other active slivers on each node. ● The concept of slice aggregates the presence of your slivers within the system 20 FIBRE Workshop- Salvador - Nov 27 2012 20
  • 21.
    Nodes 21 FIBRE Workshop- Salvador - Nov 27 2012 21
  • 22.
    Slices – hujiple_isis 22 FIBRE Workshop- Salvador - Nov 27 2012 22
  • 23.
    Slices – upmcple_paristr 23 FIBRE Workshop- Salvador - Nov 27 2012 23
  • 24.
    Slices 24 FIBRE Workshop- Salvador - Nov 27 2012 24
  • 25.
    Federation (& Legalaspects) ● Local consortium agreement defines responsibilities and liabilities of each partner ● Federation integrates the consortiums into a seamless global authority ● Formal Trust Relationships are the basis for this integration 25 FIBRE Workshop- Salvador - Nov 27 2012 25
  • 26.
    Global Federation Princeton UPMC Europe USA PLC PLC Japan PLC Kyoto 26 FIBRE Workshop- Salvador - Nov 27 2012 26
  • 27.
    In Practice … What is SFA, OMF, MySlice, … FIBRE Workshop- Salvador - Nov 27 2012 27
  • 28.
  • 29.
    What researchers want Researchers To have access to a testbed Testbeds
  • 30.
    What researchers want Researchers To access many instances of the same type of testbed / resource Testbeds
  • 31.
    What researchers want Researchers To access many instances of the same type of testbed / resource • For geographic diversity • For reproducing experiments Testbeds
  • 32.
    What researchers want Researchers To access different types of testbeds and resources Testbeds
  • 33.
    What researchers want Researchers To access different types of testbeds and resources • For combined experiments • For moving between simulation, emulation, and the real-world Testbeds
  • 34.
  • 35.
    What researchers want Researchers Authority Administrative simplicity: A single place to sign up in order to use all the testbeds Testbeds
  • 36.
    What researchers want Researchers Experiment control tool Testbeds
  • 37.
    What researchers want Researchers Experiment Ease of access: control tool The ability to use a small number of familiar tools to access and control the testbeds Testbeds
  • 38.
    Where we wererecently Researchers Authorities Tools Testbeds
  • 39.
    Where we wererecently Researchers Authorities ✗ Tools Testbeds
  • 40.
    Where we wererecently Researchers Authorities Tools ✗ ✗ ✗ Testbeds
  • 41.
    Where we wererecently Researchers Authorities Tools ✗ ✗ ✗ Testbeds
  • 42.
    Problems to besolved Researchers Impossible to agree on who will be the single trusted authority at a global scale Testbeds
  • 43.
    Problems to besolved Researchers Impractical to scale a single database to global scale Testbeds
  • 44.
    Researchers A secure and Common API distributed thin waist Testbeds
  • 45.
    SFA: Slice FacilityArchitecture Researchers ims to provide a secure common API with the minimum ble fonctionality to enable a global testbed federation Testbeds
  • 46.
  • 47.
  • 48.
  • 49.
  • 50.
  • 51.
  • 52.
  • 53.
    SFA and theexperiment lifecycle Researchers Authorities Authenticate Yes* Tools Testbeds *This is at the heart of SFA
  • 54.
    SFA and theexperiment lifecycle Researchers Authorities Tools Resource discovery Yes* Testbeds *Without going into all details
  • 55.
    SFA and theexperiment lifecycle Researchers Authorities Tools Resource reservation Yes Testbeds
  • 56.
    SFA and theexperiment lifecycle Researchers Authorities Tools Configure resources No* Testbeds *Use the tool of your choice
  • 57.
    SFA and theexperiment lifecycle Researchers Authorities Tools Run experiment No* Testbeds *Use the tool of your choice
  • 58.
    SFA and theexperiment lifecycle Researchers Authorities Tools Repatriate results No* Testbeds *Use the tool of your choice
  • 59.
    SFA and theexperiment lifecycle Researchers Authorities Tools Release resources Yes Testbeds
  • 60.
    60 SFA  Naming (slices, users, resources, authorities)  Hierarchical naming space  Authentication and authorization  X.509 certificates and signed credentials  Federation links through the exchange of certificates  Resource descriptions (RSpecs)  Only the language (XML) not the semantics  Standard API (XMLRPC)  To manage objects (users, resources, slices, authorities)  To manage resources (browse, acquire, manifest)  To manage slices (create, delete, start, stop) FIBRE Workshop- Salvador - Nov 27 2012 60
  • 61.
    OMF/OML software OMF supports users over the full cycle of their experiments, from description to instrumentation, to execution and measurement collection and visualization. FIBRE Workshop- Salvador - Nov 27 2012 61
  • 62.
    MySlice ● A common abstraction to help the user browse through and interact with a large amount of data and sources ● Grounded on related work on data integration and large dataset navigation. ● Formulate semantic queries, requesting ltering and annotations ● Propose enhanced visualizations on received results ● Allow to balance homogeneity and heterogeneity in the GUI FIBRE Workshop- Salvador - Nov 27 2012 62
  • 63.
  • 64.
    FIBRE Workshop- Salvador- Nov 27 2012 64
  • 65.
    FIBRE Workshop- Salvador- Nov 27 2012 65

Editor's Notes

  • #11 Devolution
  • #12 Global research community: exchange of experiences between researchers, credibility and comparability of experiments. Networking experiments benefit always for more resources, without finishing earlier (as computing ones)
  • #15 Less optimistic « federation » scenario could be one that allows researchers to access resources belonging to different facilities (but not allowing them to directly communicate). Also sharing of experiment data (input, output).
  • #16 Less optimistic « federation » scenario could be one that allows researchers to access resources belonging to different facilities (but not allowing them to directly communicate). Also sharing of experiment data (input, output).
  • #17 Less optimistic « federation » scenario could be one that allows researchers to access resources belonging to different facilities (but not allowing them to directly communicate). Also sharing of experiment data (input, output).
  • #18 Less optimistic « federation » scenario could be one that allows researchers to access resources belonging to different facilities (but not allowing them to directly communicate). Also sharing of experiment data (input, output).