Disnix: A toolset for distributed deployment

Sander van der Burg
Sander van der BurgSoftware Engineer and Researcher at Mendix
Disnix: A toolset for distributed deployment
Sander van der Burg Eelco Dolstra
Delft University of Technology, EEMCS,
Department of Software Technology
September 20, 2010
Sander van der Burg, Eelco Dolstra Disnix: A toolset for distributed deployment
Disnix
Disnix is a tool to automatically deploy a distributed system in a
network of machines
Sander van der Burg, Eelco Dolstra Disnix: A toolset for distributed deployment
Research issues
General issues:
Automatic deployment
Reliable deployment
Efficient deployment
Atomic upgrades/rollbacks
Domain specific issues:
Security
Privacy
Performance
Sander van der Burg, Eelco Dolstra Disnix: A toolset for distributed deployment
Research issues
It is complex to implement and maintain a toolset dealing with
these issues!
Sander van der Burg, Eelco Dolstra Disnix: A toolset for distributed deployment
Challenges
How to develop this toolset in a convenient way?
Some non-functional requirements of the domain cannot be
solved generically
Maintaining everything ourselves is a lot of effort:
Programming language compilers/interpreters: C, Java,
Python, PHP, ...
Libraries: Apache Axis2, libxml2, ...
Infrastructure components: Apache Tomcat, MySQL, ...
The tool is used in a distributed setting. How to test?
Sander van der Burg, Eelco Dolstra Disnix: A toolset for distributed deployment
Architecture
Sander van der Burg, Eelco Dolstra Disnix: A toolset for distributed deployment
Architecture
Sander van der Burg, Eelco Dolstra Disnix: A toolset for distributed deployment
Architecture
Architecture is mainly composition of processes
(command-line tools)
Each tool performs a separate deployment task in the
deployment process:
Building
Transferring
Activating
Inspired by Art of UNIX programming book
Architecture
Sander van der Burg, Eelco Dolstra Disnix: A toolset for distributed deployment
Architecture
A user can easily perform steps seperately
Testing and debugging is relatively easy; things can be
easily scripted
Components can be implemented using various
programming languages: Nix, C, shell scripts
Prototyping is relatively easy. First use a high-level
language, implement later in a low-level language
Architecture
Extensions with custom modules are implemented by composing
processes:
Virtualization extension
Dynamic deployment
Service abstraction layer
Sander van der Burg, Eelco Dolstra Disnix: A toolset for distributed deployment
Testing
Generate cheap virtual networks from declarative specifications
Automatically performs testcases on them
Completely scripted
Disnix is continuously integrated and tested by Hydra
Sander van der Burg, Eelco Dolstra Disnix: A toolset for distributed deployment
Community
Sander van der Burg, Eelco Dolstra Disnix: A toolset for distributed deployment
Community
Disnix is part of the Nix project
Tools are released under free and open-source licenses
About 25 contributors
Cross compiling
Nixpkgs: contains compilers, libraries, tools which can
be used by Disnix
Lessons learned
Architecture works well for supporting development
Easy prototyping, debugging, testing, extendable
Our test approach works well; we can automatically perform
distributed testcases
Community is crucial; too much effort to maintain everything
ourselves
Sander van der Burg, Eelco Dolstra Disnix: A toolset for distributed deployment
Website
Disnix, and other related Nix tooling: Hydra, NixOS can be freely
downloaded from: http://nixos.org
Sander van der Burg, Eelco Dolstra Disnix: A toolset for distributed deployment
Questions
Sander van der Burg, Eelco Dolstra Disnix: A toolset for distributed deployment
1 of 14

Recommended

Truly Secure: The Steps a Security Practitioner Took to Build a Secure Public... by
Truly Secure: The Steps a Security Practitioner Took to Build a Secure Public...Truly Secure: The Steps a Security Practitioner Took to Build a Secure Public...
Truly Secure: The Steps a Security Practitioner Took to Build a Secure Public...John Kinsella
381 views44 slides
Applied Security for Containers, OW2con'18, June 7-8, 2018, Paris by
Applied Security for Containers, OW2con'18, June 7-8, 2018, ParisApplied Security for Containers, OW2con'18, June 7-8, 2018, Paris
Applied Security for Containers, OW2con'18, June 7-8, 2018, ParisOW2
131 views21 slides
SBA Security Meetup: Building a Secure Architecture – A Deep-Dive into Securi... by
SBA Security Meetup: Building a Secure Architecture – A Deep-Dive into Securi...SBA Security Meetup: Building a Secure Architecture – A Deep-Dive into Securi...
SBA Security Meetup: Building a Secure Architecture – A Deep-Dive into Securi...SBA Research
87 views19 slides
Pull Deployment of Services by
Pull Deployment of ServicesPull Deployment of Services
Pull Deployment of ServicesSander van der Burg
408 views16 slides
Using NixOS for declarative deployment and testing by
Using NixOS for declarative deployment and testingUsing NixOS for declarative deployment and testing
Using NixOS for declarative deployment and testingSander van der Burg
1.4K views31 slides
Hydra: Continuous Integration and Testing for Demanding People: The Details by
Hydra: Continuous Integration and Testing for Demanding People: The DetailsHydra: Continuous Integration and Testing for Demanding People: The Details
Hydra: Continuous Integration and Testing for Demanding People: The DetailsSander van der Burg
3.8K views43 slides

More Related Content

Similar to Disnix: A toolset for distributed deployment

A Reference Architecture for Distributed Software Deployment by
A Reference Architecture for Distributed Software DeploymentA Reference Architecture for Distributed Software Deployment
A Reference Architecture for Distributed Software DeploymentSander van der Burg
1.6K views56 slides
Deploying .NET applications with the Nix package manager by
Deploying .NET applications with the Nix package managerDeploying .NET applications with the Nix package manager
Deploying .NET applications with the Nix package managerSander van der Burg
1.8K views36 slides
Developing multi-platform microservices using .NET core by
 Developing multi-platform microservices using .NET core Developing multi-platform microservices using .NET core
Developing multi-platform microservices using .NET coreDror Helper
408 views18 slides
DockerDay2015: Keynote by
DockerDay2015: KeynoteDockerDay2015: Keynote
DockerDay2015: KeynoteDocker-Hanoi
441 views84 slides
Introduction to android by
Introduction to androidIntroduction to android
Introduction to androidAravindharamanan S
119 views25 slides
Session 2 beccse by
Session 2 beccseSession 2 beccse
Session 2 beccsevin123456gangal
390 views27 slides

Similar to Disnix: A toolset for distributed deployment(20)

A Reference Architecture for Distributed Software Deployment by Sander van der Burg
A Reference Architecture for Distributed Software DeploymentA Reference Architecture for Distributed Software Deployment
A Reference Architecture for Distributed Software Deployment
Sander van der Burg1.6K views
Deploying .NET applications with the Nix package manager by Sander van der Burg
Deploying .NET applications with the Nix package managerDeploying .NET applications with the Nix package manager
Deploying .NET applications with the Nix package manager
Sander van der Burg1.8K views
Developing multi-platform microservices using .NET core by Dror Helper
 Developing multi-platform microservices using .NET core Developing multi-platform microservices using .NET core
Developing multi-platform microservices using .NET core
Dror Helper408 views
DockerDay2015: Keynote by Docker-Hanoi
DockerDay2015: KeynoteDockerDay2015: Keynote
DockerDay2015: Keynote
Docker-Hanoi441 views
Automated Deployment of Hetergeneous Service-Oriented System by Sander van der Burg
Automated Deployment of Hetergeneous Service-Oriented SystemAutomated Deployment of Hetergeneous Service-Oriented System
Automated Deployment of Hetergeneous Service-Oriented System
DockerCon 2017 - General Session Day 1 - Solomon Hykes by Docker, Inc.
DockerCon 2017 - General Session Day 1 - Solomon HykesDockerCon 2017 - General Session Day 1 - Solomon Hykes
DockerCon 2017 - General Session Day 1 - Solomon Hykes
Docker, Inc.5.8K views
NUX Presentation from TechMixer Birmingham 2011 by Michael Heydt
NUX Presentation from TechMixer Birmingham 2011NUX Presentation from TechMixer Birmingham 2011
NUX Presentation from TechMixer Birmingham 2011
Michael Heydt156 views
Security: The Value of SBOMs by Weaveworks
Security: The Value of SBOMsSecurity: The Value of SBOMs
Security: The Value of SBOMs
Weaveworks339 views
Machine Learning , Analytics & Cyber Security the Next Level Threat Analytics... by PranavPatil822557
Machine Learning , Analytics & Cyber Security the Next Level Threat Analytics...Machine Learning , Analytics & Cyber Security the Next Level Threat Analytics...
Machine Learning , Analytics & Cyber Security the Next Level Threat Analytics...
DSD-INT 2015 - Delft3D 4 open source workshop - Adri Mourits by Deltares
DSD-INT 2015 - Delft3D 4 open source workshop - Adri MouritsDSD-INT 2015 - Delft3D 4 open source workshop - Adri Mourits
DSD-INT 2015 - Delft3D 4 open source workshop - Adri Mourits
Deltares1.6K views
DESIGNS & IMPLEMENTATIONS TO OVERCOME CHALLENGES IN THE UTILITY INDUSTRY by iQHub
DESIGNS & IMPLEMENTATIONS TO OVERCOME CHALLENGES IN THE UTILITY INDUSTRYDESIGNS & IMPLEMENTATIONS TO OVERCOME CHALLENGES IN THE UTILITY INDUSTRY
DESIGNS & IMPLEMENTATIONS TO OVERCOME CHALLENGES IN THE UTILITY INDUSTRY
iQHub13 views
A Summary about Hykes' Keynote on Dockercon 2015 by Henry Huang
A Summary about Hykes' Keynote on Dockercon 2015A Summary about Hykes' Keynote on Dockercon 2015
A Summary about Hykes' Keynote on Dockercon 2015
Henry Huang1.1K views
Linux Environment- Linux vs Unix by Trinity Dwarka
Linux Environment- Linux vs UnixLinux Environment- Linux vs Unix
Linux Environment- Linux vs Unix
Trinity Dwarka1.1K views

More from Sander van der Burg

The Monitoring Playground by
The Monitoring PlaygroundThe Monitoring Playground
The Monitoring PlaygroundSander van der Burg
1.2K views37 slides
nix-processmgmt: An experimental Nix-based process manager-agnostic framework by
nix-processmgmt: An experimental Nix-based process manager-agnostic frameworknix-processmgmt: An experimental Nix-based process manager-agnostic framework
nix-processmgmt: An experimental Nix-based process manager-agnostic frameworkSander van der Burg
1.8K views30 slides
Using Nix and Docker as automated deployment solutions by
Using Nix and Docker as automated deployment solutionsUsing Nix and Docker as automated deployment solutions
Using Nix and Docker as automated deployment solutionsSander van der Burg
6.4K views52 slides
Dysnomia: complementing Nix deployments with state deployment by
Dysnomia: complementing Nix deployments with state deploymentDysnomia: complementing Nix deployments with state deployment
Dysnomia: complementing Nix deployments with state deploymentSander van der Burg
743 views30 slides
Automating Mendix application deployments with Nix by
Automating Mendix application deployments with NixAutomating Mendix application deployments with Nix
Automating Mendix application deployments with NixSander van der Burg
2.4K views33 slides
Deploying NPM packages with the Nix package manager by
Deploying NPM packages with the Nix package managerDeploying NPM packages with the Nix package manager
Deploying NPM packages with the Nix package managerSander van der Burg
5.4K views51 slides

More from Sander van der Burg(20)

nix-processmgmt: An experimental Nix-based process manager-agnostic framework by Sander van der Burg
nix-processmgmt: An experimental Nix-based process manager-agnostic frameworknix-processmgmt: An experimental Nix-based process manager-agnostic framework
nix-processmgmt: An experimental Nix-based process manager-agnostic framework
Sander van der Burg1.8K views
Using Nix and Docker as automated deployment solutions by Sander van der Burg
Using Nix and Docker as automated deployment solutionsUsing Nix and Docker as automated deployment solutions
Using Nix and Docker as automated deployment solutions
Sander van der Burg6.4K views
Dysnomia: complementing Nix deployments with state deployment by Sander van der Burg
Dysnomia: complementing Nix deployments with state deploymentDysnomia: complementing Nix deployments with state deployment
Dysnomia: complementing Nix deployments with state deployment
Automating Mendix application deployments with Nix by Sander van der Burg
Automating Mendix application deployments with NixAutomating Mendix application deployments with Nix
Automating Mendix application deployments with Nix
Sander van der Burg2.4K views
Deploying NPM packages with the Nix package manager by Sander van der Burg
Deploying NPM packages with the Nix package managerDeploying NPM packages with the Nix package manager
Deploying NPM packages with the Nix package manager
Sander van der Burg5.4K views
The NixOS project and deploying systems declaratively by Sander van der Burg
The NixOS project and deploying systems declarativelyThe NixOS project and deploying systems declaratively
The NixOS project and deploying systems declaratively
Sander van der Burg35.8K views
Hydra: Continuous Integration and Testing for Demanding People: The Basics by Sander van der Burg
Hydra: Continuous Integration and Testing for Demanding People: The BasicsHydra: Continuous Integration and Testing for Demanding People: The Basics
Hydra: Continuous Integration and Testing for Demanding People: The Basics
A Reference Architecture for Distributed Software Deployment by Sander van der Burg
A Reference Architecture for Distributed Software DeploymentA Reference Architecture for Distributed Software Deployment
A Reference Architecture for Distributed Software Deployment
A Generic Approach for Deploying and Upgrading Mutable Software Components by Sander van der Burg
A Generic Approach for Deploying and Upgrading Mutable Software ComponentsA Generic Approach for Deploying and Upgrading Mutable Software Components
A Generic Approach for Deploying and Upgrading Mutable Software Components
A Self-Adaptive Deployment Framework for Service-Oriented Systems by Sander van der Burg
A Self-Adaptive Deployment Framework for Service-Oriented SystemsA Self-Adaptive Deployment Framework for Service-Oriented Systems
A Self-Adaptive Deployment Framework for Service-Oriented Systems
Pull Deployment of Services: Introduction, Progress and Challenges by Sander van der Burg
Pull Deployment of Services: Introduction, Progress and ChallengesPull Deployment of Services: Introduction, Progress and Challenges
Pull Deployment of Services: Introduction, Progress and Challenges
Model-driven Distributed Software Deployment laymen's talk by Sander van der Burg
Model-driven Distributed Software Deployment laymen's talkModel-driven Distributed Software Deployment laymen's talk
Model-driven Distributed Software Deployment laymen's talk

Recently uploaded

Discovery of therapeutic agents targeting PKLR for NAFLD using drug repositio... by
Discovery of therapeutic agents targeting PKLR for NAFLD using drug repositio...Discovery of therapeutic agents targeting PKLR for NAFLD using drug repositio...
Discovery of therapeutic agents targeting PKLR for NAFLD using drug repositio...Trustlife
207 views17 slides
Krishna VSC 692 Credit Seminar.pptx by
Krishna VSC 692 Credit Seminar.pptxKrishna VSC 692 Credit Seminar.pptx
Krishna VSC 692 Credit Seminar.pptxKrishnaSharma682993
11 views54 slides
ELECTRON TRANSPORT CHAIN by
ELECTRON TRANSPORT CHAINELECTRON TRANSPORT CHAIN
ELECTRON TRANSPORT CHAINDEEKSHA RANI
16 views16 slides
Presentation on experimental laboratory animal- Hamster by
Presentation on experimental laboratory animal- HamsterPresentation on experimental laboratory animal- Hamster
Presentation on experimental laboratory animal- HamsterKanika13641
6 views8 slides
별헤는 사람들 2023년 12월호 전명원 교수 자료 by
별헤는 사람들 2023년 12월호 전명원 교수 자료별헤는 사람들 2023년 12월호 전명원 교수 자료
별헤는 사람들 2023년 12월호 전명원 교수 자료sciencepeople
68 views30 slides
Indian council for child welfare by
Indian council for child welfareIndian council for child welfare
Indian council for child welfareRenuWaghmare2
7 views21 slides

Recently uploaded(20)

Discovery of therapeutic agents targeting PKLR for NAFLD using drug repositio... by Trustlife
Discovery of therapeutic agents targeting PKLR for NAFLD using drug repositio...Discovery of therapeutic agents targeting PKLR for NAFLD using drug repositio...
Discovery of therapeutic agents targeting PKLR for NAFLD using drug repositio...
Trustlife207 views
ELECTRON TRANSPORT CHAIN by DEEKSHA RANI
ELECTRON TRANSPORT CHAINELECTRON TRANSPORT CHAIN
ELECTRON TRANSPORT CHAIN
DEEKSHA RANI16 views
Presentation on experimental laboratory animal- Hamster by Kanika13641
Presentation on experimental laboratory animal- HamsterPresentation on experimental laboratory animal- Hamster
Presentation on experimental laboratory animal- Hamster
Kanika136416 views
별헤는 사람들 2023년 12월호 전명원 교수 자료 by sciencepeople
별헤는 사람들 2023년 12월호 전명원 교수 자료별헤는 사람들 2023년 12월호 전명원 교수 자료
별헤는 사람들 2023년 12월호 전명원 교수 자료
sciencepeople68 views
Indian council for child welfare by RenuWaghmare2
Indian council for child welfareIndian council for child welfare
Indian council for child welfare
RenuWaghmare27 views
Factors affecting fluorescence and phosphorescence.pptx by SamarthGiri1
Factors affecting fluorescence and phosphorescence.pptxFactors affecting fluorescence and phosphorescence.pptx
Factors affecting fluorescence and phosphorescence.pptx
SamarthGiri18 views
Determination of color fastness to rubbing(wet and dry condition) by crockmeter. by ShadmanSakib63
Determination of color fastness to rubbing(wet and dry condition) by crockmeter.Determination of color fastness to rubbing(wet and dry condition) by crockmeter.
Determination of color fastness to rubbing(wet and dry condition) by crockmeter.
ShadmanSakib636 views
Vegetable grafting: A new crop improvement approach.pptx by Himul Suthar
Vegetable grafting: A new crop improvement approach.pptxVegetable grafting: A new crop improvement approach.pptx
Vegetable grafting: A new crop improvement approach.pptx
Himul Suthar8 views
A Ready-to-Analyze High-Plex Spatial Signature Development Workflow for Cance... by InsideScientific
A Ready-to-Analyze High-Plex Spatial Signature Development Workflow for Cance...A Ready-to-Analyze High-Plex Spatial Signature Development Workflow for Cance...
A Ready-to-Analyze High-Plex Spatial Signature Development Workflow for Cance...
InsideScientific121 views
Ellagic Acid and Its Metabolites as Potent and Selective Allosteric Inhibitor... by Trustlife
Ellagic Acid and Its Metabolites as Potent and Selective Allosteric Inhibitor...Ellagic Acid and Its Metabolites as Potent and Selective Allosteric Inhibitor...
Ellagic Acid and Its Metabolites as Potent and Selective Allosteric Inhibitor...
Trustlife154 views
Small ruminant keepers’ knowledge, attitudes and practices towards peste des ... by ILRI
Small ruminant keepers’ knowledge, attitudes and practices towards peste des ...Small ruminant keepers’ knowledge, attitudes and practices towards peste des ...
Small ruminant keepers’ knowledge, attitudes and practices towards peste des ...
ILRI9 views
Note on the Riemann Hypothesis by vegafrank2
Note on the Riemann HypothesisNote on the Riemann Hypothesis
Note on the Riemann Hypothesis
vegafrank28 views
Experimental animal Guinea pigs.pptx by Mansee Arya
Experimental animal Guinea pigs.pptxExperimental animal Guinea pigs.pptx
Experimental animal Guinea pigs.pptx
Mansee Arya42 views

Disnix: A toolset for distributed deployment

  • 1. Disnix: A toolset for distributed deployment Sander van der Burg Eelco Dolstra Delft University of Technology, EEMCS, Department of Software Technology September 20, 2010 Sander van der Burg, Eelco Dolstra Disnix: A toolset for distributed deployment
  • 2. Disnix Disnix is a tool to automatically deploy a distributed system in a network of machines Sander van der Burg, Eelco Dolstra Disnix: A toolset for distributed deployment
  • 3. Research issues General issues: Automatic deployment Reliable deployment Efficient deployment Atomic upgrades/rollbacks Domain specific issues: Security Privacy Performance Sander van der Burg, Eelco Dolstra Disnix: A toolset for distributed deployment
  • 4. Research issues It is complex to implement and maintain a toolset dealing with these issues! Sander van der Burg, Eelco Dolstra Disnix: A toolset for distributed deployment
  • 5. Challenges How to develop this toolset in a convenient way? Some non-functional requirements of the domain cannot be solved generically Maintaining everything ourselves is a lot of effort: Programming language compilers/interpreters: C, Java, Python, PHP, ... Libraries: Apache Axis2, libxml2, ... Infrastructure components: Apache Tomcat, MySQL, ... The tool is used in a distributed setting. How to test? Sander van der Burg, Eelco Dolstra Disnix: A toolset for distributed deployment
  • 6. Architecture Sander van der Burg, Eelco Dolstra Disnix: A toolset for distributed deployment
  • 7. Architecture Sander van der Burg, Eelco Dolstra Disnix: A toolset for distributed deployment Architecture Architecture is mainly composition of processes (command-line tools) Each tool performs a separate deployment task in the deployment process: Building Transferring Activating Inspired by Art of UNIX programming book
  • 8. Architecture Sander van der Burg, Eelco Dolstra Disnix: A toolset for distributed deployment Architecture A user can easily perform steps seperately Testing and debugging is relatively easy; things can be easily scripted Components can be implemented using various programming languages: Nix, C, shell scripts Prototyping is relatively easy. First use a high-level language, implement later in a low-level language
  • 9. Architecture Extensions with custom modules are implemented by composing processes: Virtualization extension Dynamic deployment Service abstraction layer Sander van der Burg, Eelco Dolstra Disnix: A toolset for distributed deployment
  • 10. Testing Generate cheap virtual networks from declarative specifications Automatically performs testcases on them Completely scripted Disnix is continuously integrated and tested by Hydra Sander van der Burg, Eelco Dolstra Disnix: A toolset for distributed deployment
  • 11. Community Sander van der Burg, Eelco Dolstra Disnix: A toolset for distributed deployment Community Disnix is part of the Nix project Tools are released under free and open-source licenses About 25 contributors Cross compiling Nixpkgs: contains compilers, libraries, tools which can be used by Disnix
  • 12. Lessons learned Architecture works well for supporting development Easy prototyping, debugging, testing, extendable Our test approach works well; we can automatically perform distributed testcases Community is crucial; too much effort to maintain everything ourselves Sander van der Burg, Eelco Dolstra Disnix: A toolset for distributed deployment
  • 13. Website Disnix, and other related Nix tooling: Hydra, NixOS can be freely downloaded from: http://nixos.org Sander van der Burg, Eelco Dolstra Disnix: A toolset for distributed deployment
  • 14. Questions Sander van der Burg, Eelco Dolstra Disnix: A toolset for distributed deployment