SlideShare a Scribd company logo

SPLAY: Distributed Systems Made Simple

SPLAY simplifies the prototyping and development of large-scale distributed applications and overlay networks. SPLAY covers the complete chain of distributed system design, development and testing: from coding and local runs to controlled deployment, experiment control and monitoring. Developers specify their applications using Lua, which is executed in a sandboxed environment. The talk will include a short live demo of a simple distributed protocol in Lua deployed on the PlanetLab. SPLAY is the outcome of research and development activities at the Computer Science Department of the University of Neuchatel, Switzerland.

1 of 60
Download to read offline
!"#$%
   Distributed Systems
      Made Simple
Pascal Felber, Lorenzo Leonini, Etienne Rivière,
         Valerio Schiavoni, José Valerio
      Lua Workshop, 8 September 2011
             Frick, Switzerland

                     !"#$%
About Me
•   2010-now : PhD student at the University of
    Neuchatel, Switzerland (and Lua user since then)

    •   Topic: large-scale distributed systems, cloud
        computing. Daily job:
        1. Invent new protocols for cloud applications
        2. Do experiments, write papers, go to 1

•   2007-2009: Research engineer at INRIA, France

•   BSc and MSc in Computer Engineering at the
    Università degli Studi Roma Tre, Italy
    !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
Motivations




!"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
Motivations

• Developing, testing and tuning distributed
  applications is hard
• In Computer Science research, fixing the
  gap of simplicity between pseudocode
  description and implementation is hard
• Using worldwide testbeds is hard
   !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
What is




  !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
What is




  !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland

Recommended

Open al bivio fra software e webware
Open al bivio fra software e webwareOpen al bivio fra software e webware
Open al bivio fra software e webwareDavide Carboni
 
Organizzazione e tipologie di progetto crossmediale
Organizzazione e tipologie di progetto crossmedialeOrganizzazione e tipologie di progetto crossmediale
Organizzazione e tipologie di progetto crossmedialeAdele Savarese
 
Inspec: Turn your compliance, security, and other policy requirements into au...
Inspec: Turn your compliance, security, and other policy requirements into au...Inspec: Turn your compliance, security, and other policy requirements into au...
Inspec: Turn your compliance, security, and other policy requirements into au...Kangaroot
 
InSpec - June 2018 at Open28.be
InSpec - June 2018 at Open28.beInSpec - June 2018 at Open28.be
InSpec - June 2018 at Open28.beMandi Walls
 
Kurt baumgartner lan_deskse2012
Kurt baumgartner lan_deskse2012Kurt baumgartner lan_deskse2012
Kurt baumgartner lan_deskse2012Kurt Baumgartner
 
Software Lifecycle Management in Asemantics
Software Lifecycle Management in AsemanticsSoftware Lifecycle Management in Asemantics
Software Lifecycle Management in Asemanticssimonetripodi
 

More Related Content

Similar to SPLAY: Distributed Systems Made Simple

Reverse engineering
Reverse engineeringReverse engineering
Reverse engineeringSaswat Padhi
 
Workflows in the Virtual Observatory
Workflows in the Virtual ObservatoryWorkflows in the Virtual Observatory
Workflows in the Virtual ObservatoryJose Enrique Ruiz
 
Shannon McFarland OpenStack/Cisco Intro
Shannon McFarland OpenStack/Cisco IntroShannon McFarland OpenStack/Cisco Intro
Shannon McFarland OpenStack/Cisco IntroShannon McFarland
 
OpenDaylight Developers Experience 1.5: Eclipse Setup, HOT reload, future plans
OpenDaylight Developers Experience 1.5: Eclipse Setup, HOT reload, future plansOpenDaylight Developers Experience 1.5: Eclipse Setup, HOT reload, future plans
OpenDaylight Developers Experience 1.5: Eclipse Setup, HOT reload, future plansMichael Vorburger
 
Blackhat USA 2011 - Cesar Cerrudo - Easy and quick vulnerability hunting in W...
Blackhat USA 2011 - Cesar Cerrudo - Easy and quick vulnerability hunting in W...Blackhat USA 2011 - Cesar Cerrudo - Easy and quick vulnerability hunting in W...
Blackhat USA 2011 - Cesar Cerrudo - Easy and quick vulnerability hunting in W...Michael Boman
 
Rugged DevOps Will help you build ur cloudz
Rugged DevOps Will help you build ur cloudzRugged DevOps Will help you build ur cloudz
Rugged DevOps Will help you build ur cloudzJames Wickett
 
Infinispan, a distributed in-memory key/value data grid and cache
 Infinispan, a distributed in-memory key/value data grid and cache Infinispan, a distributed in-memory key/value data grid and cache
Infinispan, a distributed in-memory key/value data grid and cacheSebastian Andrasoni
 
Implementing Continuous Deployment
Implementing Continuous DeploymentImplementing Continuous Deployment
Implementing Continuous DeploymentAndrew Phillips
 
Implementing continuous deployment JFall 2010
Implementing continuous deployment JFall 2010Implementing continuous deployment JFall 2010
Implementing continuous deployment JFall 2010XebiaLabs
 
Unified Deployment: Including the Mainframe in Enterprise DevOps
Unified Deployment: Including the Mainframe in Enterprise DevOpsUnified Deployment: Including the Mainframe in Enterprise DevOps
Unified Deployment: Including the Mainframe in Enterprise DevOpsXebiaLabs
 
2018 ABRF Tools for improving rigor and reproducibility in bioinformatics
2018 ABRF Tools for improving rigor and reproducibility in bioinformatics2018 ABRF Tools for improving rigor and reproducibility in bioinformatics
2018 ABRF Tools for improving rigor and reproducibility in bioinformaticsStephen Turner
 
EclipseCon 2016 - OCCIware : one Cloud API to rule them all
EclipseCon 2016 - OCCIware : one Cloud API to rule them allEclipseCon 2016 - OCCIware : one Cloud API to rule them all
EclipseCon 2016 - OCCIware : one Cloud API to rule them allMarc Dutoo
 
OCCIware Project at EclipseCon France 2016, by Marc Dutoo, Open Wide
OCCIware Project at EclipseCon France 2016, by Marc Dutoo, Open WideOCCIware Project at EclipseCon France 2016, by Marc Dutoo, Open Wide
OCCIware Project at EclipseCon France 2016, by Marc Dutoo, Open WideOCCIware
 
Lessons Learned: Novell Open Enterprise Server Upgrades Made Easy
Lessons Learned: Novell Open Enterprise Server Upgrades Made EasyLessons Learned: Novell Open Enterprise Server Upgrades Made Easy
Lessons Learned: Novell Open Enterprise Server Upgrades Made EasyNovell
 
Using Puppet To Manage Splunk
Using Puppet To Manage SplunkUsing Puppet To Manage Splunk
Using Puppet To Manage SplunkSplunk
 
Software Engineering - chp8- deployment
Software Engineering - chp8- deploymentSoftware Engineering - chp8- deployment
Software Engineering - chp8- deploymentLilia Sfaxi
 

Similar to SPLAY: Distributed Systems Made Simple (20)

Reverse engineering
Reverse engineeringReverse engineering
Reverse engineering
 
Workflows in the Virtual Observatory
Workflows in the Virtual ObservatoryWorkflows in the Virtual Observatory
Workflows in the Virtual Observatory
 
Shannon McFarland OpenStack/Cisco Intro
Shannon McFarland OpenStack/Cisco IntroShannon McFarland OpenStack/Cisco Intro
Shannon McFarland OpenStack/Cisco Intro
 
OpenDaylight Developers Experience 1.5: Eclipse Setup, HOT reload, future plans
OpenDaylight Developers Experience 1.5: Eclipse Setup, HOT reload, future plansOpenDaylight Developers Experience 1.5: Eclipse Setup, HOT reload, future plans
OpenDaylight Developers Experience 1.5: Eclipse Setup, HOT reload, future plans
 
Blackhat USA 2011 - Cesar Cerrudo - Easy and quick vulnerability hunting in W...
Blackhat USA 2011 - Cesar Cerrudo - Easy and quick vulnerability hunting in W...Blackhat USA 2011 - Cesar Cerrudo - Easy and quick vulnerability hunting in W...
Blackhat USA 2011 - Cesar Cerrudo - Easy and quick vulnerability hunting in W...
 
Rugged DevOps Will help you build ur cloudz
Rugged DevOps Will help you build ur cloudzRugged DevOps Will help you build ur cloudz
Rugged DevOps Will help you build ur cloudz
 
Infinispan, a distributed in-memory key/value data grid and cache
 Infinispan, a distributed in-memory key/value data grid and cache Infinispan, a distributed in-memory key/value data grid and cache
Infinispan, a distributed in-memory key/value data grid and cache
 
Implementing Continuous Deployment
Implementing Continuous DeploymentImplementing Continuous Deployment
Implementing Continuous Deployment
 
Implementing continuous deployment JFall 2010
Implementing continuous deployment JFall 2010Implementing continuous deployment JFall 2010
Implementing continuous deployment JFall 2010
 
Ionic in 30
Ionic in 30Ionic in 30
Ionic in 30
 
Unified Deployment: Including the Mainframe in Enterprise DevOps
Unified Deployment: Including the Mainframe in Enterprise DevOpsUnified Deployment: Including the Mainframe in Enterprise DevOps
Unified Deployment: Including the Mainframe in Enterprise DevOps
 
2018 ABRF Tools for improving rigor and reproducibility in bioinformatics
2018 ABRF Tools for improving rigor and reproducibility in bioinformatics2018 ABRF Tools for improving rigor and reproducibility in bioinformatics
2018 ABRF Tools for improving rigor and reproducibility in bioinformatics
 
Cloud iaa s-labs- ubuntu canonical- fossa2010
Cloud iaa s-labs- ubuntu canonical- fossa2010Cloud iaa s-labs- ubuntu canonical- fossa2010
Cloud iaa s-labs- ubuntu canonical- fossa2010
 
EclipseCon 2016 - OCCIware : one Cloud API to rule them all
EclipseCon 2016 - OCCIware : one Cloud API to rule them allEclipseCon 2016 - OCCIware : one Cloud API to rule them all
EclipseCon 2016 - OCCIware : one Cloud API to rule them all
 
OCCIware Project at EclipseCon France 2016, by Marc Dutoo, Open Wide
OCCIware Project at EclipseCon France 2016, by Marc Dutoo, Open WideOCCIware Project at EclipseCon France 2016, by Marc Dutoo, Open Wide
OCCIware Project at EclipseCon France 2016, by Marc Dutoo, Open Wide
 
DevOps tools for winning agility
DevOps tools for winning agilityDevOps tools for winning agility
DevOps tools for winning agility
 
XS Boston 2008 OVF
XS Boston 2008 OVFXS Boston 2008 OVF
XS Boston 2008 OVF
 
Lessons Learned: Novell Open Enterprise Server Upgrades Made Easy
Lessons Learned: Novell Open Enterprise Server Upgrades Made EasyLessons Learned: Novell Open Enterprise Server Upgrades Made Easy
Lessons Learned: Novell Open Enterprise Server Upgrades Made Easy
 
Using Puppet To Manage Splunk
Using Puppet To Manage SplunkUsing Puppet To Manage Splunk
Using Puppet To Manage Splunk
 
Software Engineering - chp8- deployment
Software Engineering - chp8- deploymentSoftware Engineering - chp8- deployment
Software Engineering - chp8- deployment
 

More from vschiavoni

Shielding Federated Learning Systems against Inference Attacks with ARM Trust...
Shielding Federated Learning Systems against Inference Attacks with ARM Trust...Shielding Federated Learning Systems against Inference Attacks with ARM Trust...
Shielding Federated Learning Systems against Inference Attacks with ARM Trust...vschiavoni
 
Labri 2021-invited-talk
Labri 2021-invited-talkLabri 2021-invited-talk
Labri 2021-invited-talkvschiavoni
 
SafeFS: A Modular Architecture for Secure User-Space File Systems (One FUSE t...
SafeFS: A Modular Architecture for Secure User-Space File Systems (One FUSE t...SafeFS: A Modular Architecture for Secure User-Space File Systems (One FUSE t...
SafeFS: A Modular Architecture for Secure User-Space File Systems (One FUSE t...vschiavoni
 
X-Search: Revisiting private web search using Intel SGX
X-Search: Revisiting private web search using Intel SGXX-Search: Revisiting private web search using Intel SGX
X-Search: Revisiting private web search using Intel SGXvschiavoni
 
Actor concurrency for the JVM: a case study
Actor concurrency for the JVM: a case studyActor concurrency for the JVM: a case study
Actor concurrency for the JVM: a case studyvschiavoni
 
FraSCAti: An Open SCA Platform
FraSCAti: An Open SCA PlatformFraSCAti: An Open SCA Platform
FraSCAti: An Open SCA Platformvschiavoni
 
Scorware - Spring Introduction
Scorware - Spring IntroductionScorware - Spring Introduction
Scorware - Spring Introductionvschiavoni
 
BindingFactory
BindingFactoryBindingFactory
BindingFactoryvschiavoni
 
Maven: Convention over Configuration
Maven: Convention over ConfigurationMaven: Convention over Configuration
Maven: Convention over Configurationvschiavoni
 

More from vschiavoni (12)

DEBS-2023.pdf
DEBS-2023.pdfDEBS-2023.pdf
DEBS-2023.pdf
 
Shielding Federated Learning Systems against Inference Attacks with ARM Trust...
Shielding Federated Learning Systems against Inference Attacks with ARM Trust...Shielding Federated Learning Systems against Inference Attacks with ARM Trust...
Shielding Federated Learning Systems against Inference Attacks with ARM Trust...
 
Labri 2021-invited-talk
Labri 2021-invited-talkLabri 2021-invited-talk
Labri 2021-invited-talk
 
SafeFS: A Modular Architecture for Secure User-Space File Systems (One FUSE t...
SafeFS: A Modular Architecture for Secure User-Space File Systems (One FUSE t...SafeFS: A Modular Architecture for Secure User-Space File Systems (One FUSE t...
SafeFS: A Modular Architecture for Secure User-Space File Systems (One FUSE t...
 
X-Search: Revisiting private web search using Intel SGX
X-Search: Revisiting private web search using Intel SGXX-Search: Revisiting private web search using Intel SGX
X-Search: Revisiting private web search using Intel SGX
 
Actor concurrency for the JVM: a case study
Actor concurrency for the JVM: a case studyActor concurrency for the JVM: a case study
Actor concurrency for the JVM: a case study
 
DHT and NAT
DHT and NATDHT and NAT
DHT and NAT
 
FraSCAti: An Open SCA Platform
FraSCAti: An Open SCA PlatformFraSCAti: An Open SCA Platform
FraSCAti: An Open SCA Platform
 
Spring Intro
Spring IntroSpring Intro
Spring Intro
 
Scorware - Spring Introduction
Scorware - Spring IntroductionScorware - Spring Introduction
Scorware - Spring Introduction
 
BindingFactory
BindingFactoryBindingFactory
BindingFactory
 
Maven: Convention over Configuration
Maven: Convention over ConfigurationMaven: Convention over Configuration
Maven: Convention over Configuration
 

Recently uploaded

iOncologi_Pitch Deck_2024 slide show for hostinger
iOncologi_Pitch Deck_2024 slide show for hostingeriOncologi_Pitch Deck_2024 slide show for hostinger
iOncologi_Pitch Deck_2024 slide show for hostingerssuser9354ce
 
TrustArc Webinar - TrustArc's Latest AI Innovations
TrustArc Webinar - TrustArc's Latest AI InnovationsTrustArc Webinar - TrustArc's Latest AI Innovations
TrustArc Webinar - TrustArc's Latest AI InnovationsTrustArc
 
KUBRICK Graphs: A journey from in vogue to success-ion
KUBRICK Graphs: A journey from in vogue to success-ionKUBRICK Graphs: A journey from in vogue to success-ion
KUBRICK Graphs: A journey from in vogue to success-ionNeo4j
 
How We Grew Up with CloudStack and its Journey – Dilip Singh, DataHub
How We Grew Up with CloudStack and its Journey – Dilip Singh, DataHubHow We Grew Up with CloudStack and its Journey – Dilip Singh, DataHub
How We Grew Up with CloudStack and its Journey – Dilip Singh, DataHubShapeBlue
 
Unleash the Solace Pub Sub connector | Banaglore MuleSoft Meetup #31
Unleash the Solace Pub Sub connector | Banaglore MuleSoft Meetup #31Unleash the Solace Pub Sub connector | Banaglore MuleSoft Meetup #31
Unleash the Solace Pub Sub connector | Banaglore MuleSoft Meetup #31shyamraj55
 
Establishing data sharing standards to promote global industry development
Establishing data sharing standards to promote global industry developmentEstablishing data sharing standards to promote global industry development
Establishing data sharing standards to promote global industry developmentThorsten Huelsmann
 
CloudStack Authentication Methods – Harikrishna Patnala, ShapeBlue
CloudStack Authentication Methods – Harikrishna Patnala, ShapeBlueCloudStack Authentication Methods – Harikrishna Patnala, ShapeBlue
CloudStack Authentication Methods – Harikrishna Patnala, ShapeBlueShapeBlue
 
Elevating Cloud Infrastructure with Object Storage, DRS, VM Scheduling, and D...
Elevating Cloud Infrastructure with Object Storage, DRS, VM Scheduling, and D...Elevating Cloud Infrastructure with Object Storage, DRS, VM Scheduling, and D...
Elevating Cloud Infrastructure with Object Storage, DRS, VM Scheduling, and D...ShapeBlue
 
Enterprise Architecture As Strategy - Book Review
Enterprise Architecture As Strategy - Book ReviewEnterprise Architecture As Strategy - Book Review
Enterprise Architecture As Strategy - Book ReviewAshraf Fouad
 
CloudStack Tooling Ecosystem – Kiran Chavala, ShapeBlue
CloudStack Tooling Ecosystem – Kiran Chavala, ShapeBlueCloudStack Tooling Ecosystem – Kiran Chavala, ShapeBlue
CloudStack Tooling Ecosystem – Kiran Chavala, ShapeBlueShapeBlue
 
Trending now: Book subjects on the move in the Canadian market - Tech Forum 2024
Trending now: Book subjects on the move in the Canadian market - Tech Forum 2024Trending now: Book subjects on the move in the Canadian market - Tech Forum 2024
Trending now: Book subjects on the move in the Canadian market - Tech Forum 2024BookNet Canada
 
Key projects in AI, ML and Generative AI
Key projects in AI, ML and Generative AIKey projects in AI, ML and Generative AI
Key projects in AI, ML and Generative AIVijayananda Mohire
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & EsriGeospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & EsriSafe Software
 
ChatGPT's Code Interpreter: Your secret weapon for SEO automation success - S...
ChatGPT's Code Interpreter: Your secret weapon for SEO automation success - S...ChatGPT's Code Interpreter: Your secret weapon for SEO automation success - S...
ChatGPT's Code Interpreter: Your secret weapon for SEO automation success - S...SearchNorwich
 
AI improves software testing to be more fault tolerant, focused and efficient
AI improves software testing to be more fault tolerant, focused and efficientAI improves software testing to be more fault tolerant, focused and efficient
AI improves software testing to be more fault tolerant, focused and efficientKari Kakkonen
 
AGFM - Toyota Coaster 1HZ Install Guide.pdf
AGFM - Toyota Coaster 1HZ Install Guide.pdfAGFM - Toyota Coaster 1HZ Install Guide.pdf
AGFM - Toyota Coaster 1HZ Install Guide.pdfRodneyThomas28
 
Microsoft x 2toLead Webinar Session 1 - How Employee Communication and Connec...
Microsoft x 2toLead Webinar Session 1 - How Employee Communication and Connec...Microsoft x 2toLead Webinar Session 1 - How Employee Communication and Connec...
Microsoft x 2toLead Webinar Session 1 - How Employee Communication and Connec...2toLead Limited
 
Artificial Intelligence, Design, and More-than-Human Justice
Artificial Intelligence, Design, and More-than-Human JusticeArtificial Intelligence, Design, and More-than-Human Justice
Artificial Intelligence, Design, and More-than-Human JusticeJosh Gellers
 
Leonis Insights: The State of AI (7 trends for 2023 and 7 predictions for 2024)
Leonis Insights: The State of AI (7 trends for 2023 and 7 predictions for 2024)Leonis Insights: The State of AI (7 trends for 2023 and 7 predictions for 2024)
Leonis Insights: The State of AI (7 trends for 2023 and 7 predictions for 2024)Jay Zhao
 

Recently uploaded (20)

iOncologi_Pitch Deck_2024 slide show for hostinger
iOncologi_Pitch Deck_2024 slide show for hostingeriOncologi_Pitch Deck_2024 slide show for hostinger
iOncologi_Pitch Deck_2024 slide show for hostinger
 
TrustArc Webinar - TrustArc's Latest AI Innovations
TrustArc Webinar - TrustArc's Latest AI InnovationsTrustArc Webinar - TrustArc's Latest AI Innovations
TrustArc Webinar - TrustArc's Latest AI Innovations
 
KUBRICK Graphs: A journey from in vogue to success-ion
KUBRICK Graphs: A journey from in vogue to success-ionKUBRICK Graphs: A journey from in vogue to success-ion
KUBRICK Graphs: A journey from in vogue to success-ion
 
How We Grew Up with CloudStack and its Journey – Dilip Singh, DataHub
How We Grew Up with CloudStack and its Journey – Dilip Singh, DataHubHow We Grew Up with CloudStack and its Journey – Dilip Singh, DataHub
How We Grew Up with CloudStack and its Journey – Dilip Singh, DataHub
 
Unleash the Solace Pub Sub connector | Banaglore MuleSoft Meetup #31
Unleash the Solace Pub Sub connector | Banaglore MuleSoft Meetup #31Unleash the Solace Pub Sub connector | Banaglore MuleSoft Meetup #31
Unleash the Solace Pub Sub connector | Banaglore MuleSoft Meetup #31
 
Establishing data sharing standards to promote global industry development
Establishing data sharing standards to promote global industry developmentEstablishing data sharing standards to promote global industry development
Establishing data sharing standards to promote global industry development
 
CloudStack Authentication Methods – Harikrishna Patnala, ShapeBlue
CloudStack Authentication Methods – Harikrishna Patnala, ShapeBlueCloudStack Authentication Methods – Harikrishna Patnala, ShapeBlue
CloudStack Authentication Methods – Harikrishna Patnala, ShapeBlue
 
Elevating Cloud Infrastructure with Object Storage, DRS, VM Scheduling, and D...
Elevating Cloud Infrastructure with Object Storage, DRS, VM Scheduling, and D...Elevating Cloud Infrastructure with Object Storage, DRS, VM Scheduling, and D...
Elevating Cloud Infrastructure with Object Storage, DRS, VM Scheduling, and D...
 
Enterprise Architecture As Strategy - Book Review
Enterprise Architecture As Strategy - Book ReviewEnterprise Architecture As Strategy - Book Review
Enterprise Architecture As Strategy - Book Review
 
CloudStack Tooling Ecosystem – Kiran Chavala, ShapeBlue
CloudStack Tooling Ecosystem – Kiran Chavala, ShapeBlueCloudStack Tooling Ecosystem – Kiran Chavala, ShapeBlue
CloudStack Tooling Ecosystem – Kiran Chavala, ShapeBlue
 
Trending now: Book subjects on the move in the Canadian market - Tech Forum 2024
Trending now: Book subjects on the move in the Canadian market - Tech Forum 2024Trending now: Book subjects on the move in the Canadian market - Tech Forum 2024
Trending now: Book subjects on the move in the Canadian market - Tech Forum 2024
 
Key projects in AI, ML and Generative AI
Key projects in AI, ML and Generative AIKey projects in AI, ML and Generative AI
Key projects in AI, ML and Generative AI
 
Geospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & EsriGeospatial Synergy: Amplifying Efficiency with FME & Esri
Geospatial Synergy: Amplifying Efficiency with FME & Esri
 
In sharing we trust. Taking advantage of a diverse consortium to build a tran...
In sharing we trust. Taking advantage of a diverse consortium to build a tran...In sharing we trust. Taking advantage of a diverse consortium to build a tran...
In sharing we trust. Taking advantage of a diverse consortium to build a tran...
 
ChatGPT's Code Interpreter: Your secret weapon for SEO automation success - S...
ChatGPT's Code Interpreter: Your secret weapon for SEO automation success - S...ChatGPT's Code Interpreter: Your secret weapon for SEO automation success - S...
ChatGPT's Code Interpreter: Your secret weapon for SEO automation success - S...
 
AI improves software testing to be more fault tolerant, focused and efficient
AI improves software testing to be more fault tolerant, focused and efficientAI improves software testing to be more fault tolerant, focused and efficient
AI improves software testing to be more fault tolerant, focused and efficient
 
AGFM - Toyota Coaster 1HZ Install Guide.pdf
AGFM - Toyota Coaster 1HZ Install Guide.pdfAGFM - Toyota Coaster 1HZ Install Guide.pdf
AGFM - Toyota Coaster 1HZ Install Guide.pdf
 
Microsoft x 2toLead Webinar Session 1 - How Employee Communication and Connec...
Microsoft x 2toLead Webinar Session 1 - How Employee Communication and Connec...Microsoft x 2toLead Webinar Session 1 - How Employee Communication and Connec...
Microsoft x 2toLead Webinar Session 1 - How Employee Communication and Connec...
 
Artificial Intelligence, Design, and More-than-Human Justice
Artificial Intelligence, Design, and More-than-Human JusticeArtificial Intelligence, Design, and More-than-Human Justice
Artificial Intelligence, Design, and More-than-Human Justice
 
Leonis Insights: The State of AI (7 trends for 2023 and 7 predictions for 2024)
Leonis Insights: The State of AI (7 trends for 2023 and 7 predictions for 2024)Leonis Insights: The State of AI (7 trends for 2023 and 7 predictions for 2024)
Leonis Insights: The State of AI (7 trends for 2023 and 7 predictions for 2024)
 

SPLAY: Distributed Systems Made Simple

  • 1. !"#$% Distributed Systems Made Simple Pascal Felber, Lorenzo Leonini, Etienne Rivière, Valerio Schiavoni, José Valerio Lua Workshop, 8 September 2011 Frick, Switzerland !"#$%
  • 2. About Me • 2010-now : PhD student at the University of Neuchatel, Switzerland (and Lua user since then) • Topic: large-scale distributed systems, cloud computing. Daily job: 1. Invent new protocols for cloud applications 2. Do experiments, write papers, go to 1 • 2007-2009: Research engineer at INRIA, France • BSc and MSc in Computer Engineering at the Università degli Studi Roma Tre, Italy !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 3. Motivations !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 4. Motivations • Developing, testing and tuning distributed applications is hard • In Computer Science research, fixing the gap of simplicity between pseudocode description and implementation is hard • Using worldwide testbeds is hard !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 5. What is !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 6. What is !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 7. What is • Machines contributed by universities, companies,etc. • 1098 nodes at 531 sites (02/09/2011) • Shared resources, no privileged access • University-quality Internet links • High resource contention • Faults, churn, packet-loss is the norm • Challenging conditions !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 8. Daily Job With Distributed Systems !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 9. Daily Job With Distributed Systems 1 code 1 • Write (testbed specific) code • Local tests, in-house cluster, PlanetLab... !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 10. Daily Job With Distributed Systems 2 1 code debug 2 • Debug (in this context, a nightmare) !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 11. Daily Job With Distributed Systems 2 1 code debug 2 • Debug (in this context, a nightmare) !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 12. Daily Job With Distributed Systems 3 1 2 code debug deploy 3 • Deploy, with testbed specific scripts !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 13. Daily Job With Distributed Systems 3 1 2 code debug deploy 3 • Deploy, with testbed specific scripts !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 14. Daily Job With Distributed Systems 4 1 2 3 code debug deploy get logs 4 • Get logs, with testbed specific scripts !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 15. Daily Job With Distributed Systems 4 1 2 3 code debug deploy get logs 4 • Get logs, with testbed specific scripts !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 16. Daily Job With Distributed Systems 5 1 2 3 4 code debug deploy get logs plots 5 • Produce plots, hopefully !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 17. Daily Job With Distributed Systems 1 2 3 4 5 code debug deploy get logs plots 5 • Produce plots, hopefully !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 18. !"#$% At Glance code debug deploy get logs plots • Supports the development, evaluation, testing, and tuning of distributed applications on any testbed: • In-house cluster, shared testbeds, emulated environments • Provides an easy-to-use pseudocode-like language implemented in Lua !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 19. !"#$% At Glance code debug deploy get logs plots • Supports the development, evaluation, testing, and tuning of distributed applications on any testbed: • In-house cluster, shared testbeds, emulated environments • Provides an easy-to-use pseudocode-like language implemented in Lua !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 20. !"#$% At Glance code debug deploy get logs plots l ot is gnup riend ur f • Supports the development, evaluation, yo testing, and tuning of distributed applications on any testbed: • In-house cluster, shared testbeds, emulated environments • Provides an easy-to-use pseudocode-like language implemented in Lua !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 21. The Big Picture require”splay.base” splayd splayd splayd splayd Splay Controller SQL DB !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 22. The Big Picture splayd splayd require”splay.base” splayd splayd Splay Controller SQL DB !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 23. The Big Picture splayd splayd require”splay.base” splayd splayd Splay Controller SQL DB !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 24. Why ? !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 25. Why ? • Light & Fast • (Very) Close to equivalent code in C !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 26. Why ? • Light & Fast • (Very) Close to equivalent code in C • Concise • Allowimplementation focus on ideas more than developers to details • Key for researchers !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 27. Why ? • Light & Fast • (Very) Close to equivalent code in C • Concise • Allowimplementation focus on ideas more than developers to details • Key for researchers • Sandbox thanks to the possibility of easily redefine (even built-in) functions !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 28. Concise !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 29. Concise Pseudo code as published on original paper Executable code using SPLAY libraries !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 30. Sandbox: Motivations • Experiments should access only their own resources • Required for non-dedicated testbeds • In universities, companies • Totally available at night/holiday time • Memory-allocation, filesystem, network resources are restricted !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 31. Sandboxing LuaSocket ! ! • Same API as plain LuaSocket if sock.send then ! ! new_sock.send = function(self, data) ! local len = #data • On-demand sandboxed sockets ! ! ! if total_sent + len > max_send then ! ! ! ! l_o:warn("Send restricted (total: "..total_sent..")") ! ! ! ! return nil, "restricted" • Very easy thanks to Lua’s metatable ! ! ! end ! ! ! local n, status ! ! ! if math.random(1000) > udp_drop_ratio then ! ! ! ! n, status = sock:send(data) ! ! ! • Limits chosen by the SPLAYd deployer ! ! ! ! ! ! else ! end n = len • Both TCP and UDP ! ! ! if n then ! ! ! ! total_sent = total_sent + len ! ! ! end • Example: UDP’s socket.send ! ! ! return n, status ! ! end ! end !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 32. Sandboxing LuaSocket ! ! • Same API as plain LuaSocket if sock.send then ! ! new_sock.send = function(self, data) ! local len = #data • On-demand sandboxed sockets ! ! ! if total_sent + len > max_send then ! ! ! ! l_o:warn("Send restricted (total: "..total_sent..")") ! ! ! ! return nil, "restricted" • Very easy thanks to Lua’s metatable ! ! ! end ! ! ! local n, status ! ! ! if math.random(1000) > udp_drop_ratio then ! ! ! ! n, status = sock:send(data) ! ! ! • Limits chosen by the SPLAYd deployer ! ! ! ! ! ! else ! end n = len • Both TCP and UDP ! ! ! if n then ! ! ! ! total_sent = total_sent + len ! ! ! end • Example: UDP’s socket.send ! ! ! return n, status ! ! end ! end !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 33. for _,module in pairs(splay) !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 34. for _,module in pairs(splay) luasocket events io crypto llenc/benc json misc log rpc !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 35. for _,module in pairs(splay) luasocket events io crypto llenc/benc json misc log rpc Modules sandboxed to prevent access to sensible resources !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 36. splay.events luasocket events io crypto llenc/benc json misc log rpc Modules sandboxed to prevent access to sensible resources !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 37. splay.events events !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 38. splay.events • Distributed protocols can use message- passing paradigm to communicate • Nodes react to events events • Local, incoming, outgoing messages • The core of the Splay runtime • splay.socket, splay.io designed to provide a non-blocking operational mode • Based on Lua’s coroutines !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 39. splay.rpc luasocket events io crypto llenc/benc json misc log rpc !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 40. splay.rpc rpc !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 41. splay.rpc • Default mechanism to communicate between nodes • Support for UDP/TCP • Efficient BitTorrent-like encoding rpc • Experimental binary encoding !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 42. Life Before !"#$% • Time spent on developing testbed specific protocols • Or fallback on simulations... • The focus should be on ideas • Researchers usually have no time to produce industrial-quality code !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 43. public void declareNeighbourInactive(final DHTNode node) { // DO NOTHING HERE: Life Before !"#$% } public void addToLeafSet(final DHTNode node) { // DO NOTHING } /** * Empty UDP message is 64 bytes, to which we must add 1 bytes for the header, • Time spent on developing testbed * 1 bytes for the messageId, 28 bytes (IP,port,Bamboo ID) for message src, 28 * bytes for message dest + bytes for the specific data carried in the specific protocols • Or fallback on simulations... * message. NOTE: the size of informations to ack a message is 10 bytes, */ @Override • The focus should be on ideas public long calculateMessageBytes(final Message msg) { long result = 122; // 64 + 1 + 1 + 28 + 28; • Researchers usually have no time to if (msg.header == MessageType.PING) { /* nothing to add */ return result; produce industrial-quality code } if (msg.header == MessageType.PONG) { /* some bytes for the informations about which message is being acked */ result += ACK_INFORMATIONS_SIZE; return result; } if (msg.header == MessageType.BAMBOO_JOIN_LOOKUP_REQUEST !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland || msg.header == MessageType.BAMBOO_LOOKUP_REQUEST
  • 44. responseCallback.onResponseReceived(msg); }/* * CAN BE COMMENTED AS SEND RETRIAL OF MESSAGES IS OFF else { it is a Life Before !"#$% * duplicate lookup response, send a PONG to (hopefully) stop source to * send us again that message final Message pong = new * Message(MessageType.PONG, msg.messageId, this, msg.src); pong.ackedMsg * = msg; simulator.send(pong); } */ • Time spent on developing testbed } /* ELSE IF IS SOMETHING NOT A LOOKUP RESPONSE */ else { final Message beingAcked = msg.ackedMsg; specific protocols • Or fallback on simulations... final ResponseArrivedCallback handler = this.sentMessagesCallbacks .remove(beingAcked); • The focus should be on ideas /* if there is a mapping */ if (handler != null) { handler.onResponseReceived(msg); • Researchers usually have no time to }/* * else { this may happen in case of a response arrive later than * expected... log .info("%%% " + this + " received the response => " + produce industrial-quality code * msg + * " but no handler was expecting it! So, simply send back a PONG to make it stop sending again..n" * ); send a PONG to (hopefully) stop source to send us again final * Message pong = new Message(MessageType.PONG, msg.messageId, this, there is more :-( * msg.src); pong.ackedMsg = msg; simulator.send(pong); } */ } // /* it replied at last, so "reset" the value */ !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland // this.nodeConsecutiveTimeouts.put((DHTNode) msg.src, 0);
  • 45. such as epidemic diffusion on Erd¨ s-Renyi random o graphs [16] and various types of distribution trees [10] Life With !"#$% (n-ary trees, parallel trees). As one can note from the fol- lowing figure, all implementations are extremely concise 7 in terms of lines of code (LOC): Chord (base) 59 (base) + 17 (FT) + 26 (leafset) = 101 Pastry 265 Scribe Pastry 79 SplitStream Pastry Scribe 58 BitTorrent 420 Cyclon 93 Epidemic 35 Trees 47 Although the number of lines is clearly just a rough • Lines thepseudocode ~== Lines of executableitcodestill a indicator of of expressiveness of a system, is 5 Splay daemons have been continuously running on these hosts for !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 46. Live Demo www.splay-project.org !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 47. ay aw e- e ak lid T S • Distributed systems raise a number of issues related to their evaluation • Their implementation, debug, deployment and tuning is hard • !"#$% leverages Lua to produce an easy to use yet powerful working environment to solve these issues !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 48. Backup Slides !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 49. The Big Picture !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 50. The Big Picture Splay Controller SQL DB !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 51. The Big Picture REST API Splay Controller SQL DB !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 52. The Big Picture Shell Web REST API Splay Controller SQL DB !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 53. The Big Picture Shell Web REST API Splay Controller SQL DB !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 54. The Big Picture Shell Web REST API Splay churn Controller jobs SQL DB blacklist !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 55. The Big Picture alpha require”splay.base” ... Shell Web REST API Splay churn Controller jobs SQL DB blacklist !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 56. splayd The Big Picture splayd alpha splayd require”splay.base” ... Shell Web REST API Splay churn Controller jobs SQL DB blacklist !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 57. splayd The Big Picture splayd alpha splayd require”splay.base” ... Shell Web REST API Splay churn Controller log jobs log SQL DB blacklist log !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 58. splayd The Big Picture splayd alpha splayd require”splay.base” ... Shell Web ctrl REST API ctrl ctrl Splay churn Controller log jobs log SQL DB blacklist log !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 59. splayd The Big Picture splayd alpha splayd require”splay.base” ... Shell Web ctrl REST API ctrl ctrl Splay churn Controller log jobs log SQL DB blacklist log !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland
  • 60. splayd The Big Picture splayd alpha splayd require”splay.base” ... Shell Web ctrl REST API ctrl ctrl Splay churn Controller log jobs log SQL DB blacklist log !"#$% Distributed Systems Made Simple - V. Schiavoni - Lua Workshop 2011, Frick, Switzerland