SlideShare a Scribd company logo

Hydra: Continuous Integration and Testing for Demanding People: The Details

Presentation given at Conference Compass about the details on Hydra: the Nix-based continuous integration server

1 of 43
Download to read offline
Hydra: Continuous Integration and Testing for
Demanding People: The Details
Sander van der Burg
Conference Compass
July 15, 2014
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
Continuous integration
We want to deliver and test software rapidly
We quickly want to see the impact of changes to the source
code and its dependencies.
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
Hydra
Hydra: A Nix-based continuous integration server:
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
Hydra
Hydra: A Nix-based continuous integration server:
Generic. Supports multiple programming language
environments and component technologies.
Deployment. Build and test environments are deployed
automatically and all dependencies are ensured to be present
and correct.
Variability. Multiple versions/variants of dependencies can
safely coexist.
Multi platform support. Builds can be easily delegated to
machines with a different operating system.
Scalability. Builds are transparently delegated to any machine
in a cluster capable of building it.
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
Hydra
How to use Hydra to build or test stuff?
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
Hydra overview
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People

Recommended

Hydra: Continuous Integration and Testing for Demanding People: The Basics
Hydra: Continuous Integration and Testing for Demanding People: The BasicsHydra: Continuous Integration and Testing for Demanding People: The Basics
Hydra: Continuous Integration and Testing for Demanding People: The BasicsSander van der Burg
 
Deploying (micro)services with Disnix
Deploying (micro)services with DisnixDeploying (micro)services with Disnix
Deploying (micro)services with DisnixSander van der Burg
 
The NixOS project and deploying systems declaratively
The NixOS project and deploying systems declarativelyThe NixOS project and deploying systems declaratively
The NixOS project and deploying systems declarativelySander van der Burg
 
Deploying NPM packages with the Nix package manager
Deploying NPM packages with the Nix package managerDeploying NPM packages with the Nix package manager
Deploying NPM packages with the Nix package managerSander van der Burg
 
Using NixOS for declarative deployment and testing
Using NixOS for declarative deployment and testingUsing NixOS for declarative deployment and testing
Using NixOS for declarative deployment and testingSander van der Burg
 
Nix for Python developers
Nix for Python developersNix for Python developers
Nix for Python developersAsko Soukka
 

More Related Content

What's hot

IPexpo - What is DevOps, and why should infrastructure operations care?
IPexpo - What is DevOps, and why should infrastructure operations care?IPexpo - What is DevOps, and why should infrastructure operations care?
IPexpo - What is DevOps, and why should infrastructure operations care?Chris Swan
 
Security in a containerized world - Jessie Frazelle
Security in a containerized world - Jessie FrazelleSecurity in a containerized world - Jessie Frazelle
Security in a containerized world - Jessie FrazelleParis Container Day
 
LlinuxKit security, Security Scanning and Notary
LlinuxKit security, Security Scanning and NotaryLlinuxKit security, Security Scanning and Notary
LlinuxKit security, Security Scanning and NotaryDocker, Inc.
 
There is no container - Ori Pekelman
There is no container - Ori PekelmanThere is no container - Ori Pekelman
There is no container - Ori PekelmanParis Container Day
 
Using Docker Containers to Improve Reproducibility in Software and Web Engine...
Using Docker Containers to Improve Reproducibility in Software and Web Engine...Using Docker Containers to Improve Reproducibility in Software and Web Engine...
Using Docker Containers to Improve Reproducibility in Software and Web Engine...Vincenzo Ferme
 
O'Reilly Software Architecture Conference London 2017: Building Resilient Mic...
O'Reilly Software Architecture Conference London 2017: Building Resilient Mic...O'Reilly Software Architecture Conference London 2017: Building Resilient Mic...
O'Reilly Software Architecture Conference London 2017: Building Resilient Mic...Ambassador Labs
 
Docker and kernel security
Docker and kernel securityDocker and kernel security
Docker and kernel securitysmart_bit
 
Securing Applications and Pipelines on a Container Platform
Securing Applications and Pipelines on a Container PlatformSecuring Applications and Pipelines on a Container Platform
Securing Applications and Pipelines on a Container PlatformAll Things Open
 
Swimming upstream
Swimming upstreamSwimming upstream
Swimming upstreamDave Neary
 
Ryan Koop's Docker Chicago Meetup Demo March 12 2014
Ryan Koop's Docker Chicago Meetup Demo March 12 2014Ryan Koop's Docker Chicago Meetup Demo March 12 2014
Ryan Koop's Docker Chicago Meetup Demo March 12 2014Cohesive Networks
 
CI/CD with Kubernetes
CI/CD with KubernetesCI/CD with Kubernetes
CI/CD with KubernetesHart Hoover
 
Continuous Security in DevOps
Continuous Security in DevOpsContinuous Security in DevOps
Continuous Security in DevOpsMaciej Lasyk
 
Introduction to Docker and all things containers, Docker Meetup at RelateIQ
Introduction to Docker and all things containers, Docker Meetup at RelateIQIntroduction to Docker and all things containers, Docker Meetup at RelateIQ
Introduction to Docker and all things containers, Docker Meetup at RelateIQdotCloud
 
Mirantis Contributions to Kubernetes Ecosystem
Mirantis Contributions to Kubernetes EcosystemMirantis Contributions to Kubernetes Ecosystem
Mirantis Contributions to Kubernetes EcosystemMoscowKubernetes
 
Deployment with Ruby on Rails
Deployment with Ruby on RailsDeployment with Ruby on Rails
Deployment with Ruby on RailsJonathan Weiss
 
Getting started with docker
Getting started with dockerGetting started with docker
Getting started with dockerJEMLI Fathi
 
Kubernetes and container security
Kubernetes and container securityKubernetes and container security
Kubernetes and container securityVolodymyr Shynkar
 

What's hot (20)

IPexpo - What is DevOps, and why should infrastructure operations care?
IPexpo - What is DevOps, and why should infrastructure operations care?IPexpo - What is DevOps, and why should infrastructure operations care?
IPexpo - What is DevOps, and why should infrastructure operations care?
 
Security in a containerized world - Jessie Frazelle
Security in a containerized world - Jessie FrazelleSecurity in a containerized world - Jessie Frazelle
Security in a containerized world - Jessie Frazelle
 
LlinuxKit security, Security Scanning and Notary
LlinuxKit security, Security Scanning and NotaryLlinuxKit security, Security Scanning and Notary
LlinuxKit security, Security Scanning and Notary
 
There is no container - Ori Pekelman
There is no container - Ori PekelmanThere is no container - Ori Pekelman
There is no container - Ori Pekelman
 
Using Docker Containers to Improve Reproducibility in Software and Web Engine...
Using Docker Containers to Improve Reproducibility in Software and Web Engine...Using Docker Containers to Improve Reproducibility in Software and Web Engine...
Using Docker Containers to Improve Reproducibility in Software and Web Engine...
 
O'Reilly Software Architecture Conference London 2017: Building Resilient Mic...
O'Reilly Software Architecture Conference London 2017: Building Resilient Mic...O'Reilly Software Architecture Conference London 2017: Building Resilient Mic...
O'Reilly Software Architecture Conference London 2017: Building Resilient Mic...
 
Docker and kernel security
Docker and kernel securityDocker and kernel security
Docker and kernel security
 
Securing Applications and Pipelines on a Container Platform
Securing Applications and Pipelines on a Container PlatformSecuring Applications and Pipelines on a Container Platform
Securing Applications and Pipelines on a Container Platform
 
Containers 101
Containers 101Containers 101
Containers 101
 
Swimming upstream
Swimming upstreamSwimming upstream
Swimming upstream
 
Ryan Koop's Docker Chicago Meetup Demo March 12 2014
Ryan Koop's Docker Chicago Meetup Demo March 12 2014Ryan Koop's Docker Chicago Meetup Demo March 12 2014
Ryan Koop's Docker Chicago Meetup Demo March 12 2014
 
CI/CD with Kubernetes
CI/CD with KubernetesCI/CD with Kubernetes
CI/CD with Kubernetes
 
Continuous Security in DevOps
Continuous Security in DevOpsContinuous Security in DevOps
Continuous Security in DevOps
 
Introduction to Docker and all things containers, Docker Meetup at RelateIQ
Introduction to Docker and all things containers, Docker Meetup at RelateIQIntroduction to Docker and all things containers, Docker Meetup at RelateIQ
Introduction to Docker and all things containers, Docker Meetup at RelateIQ
 
OpenStack Days Krakow
OpenStack Days KrakowOpenStack Days Krakow
OpenStack Days Krakow
 
Mirantis Contributions to Kubernetes Ecosystem
Mirantis Contributions to Kubernetes EcosystemMirantis Contributions to Kubernetes Ecosystem
Mirantis Contributions to Kubernetes Ecosystem
 
Deployment with Ruby on Rails
Deployment with Ruby on RailsDeployment with Ruby on Rails
Deployment with Ruby on Rails
 
Docker & Daily DevOps
Docker & Daily DevOpsDocker & Daily DevOps
Docker & Daily DevOps
 
Getting started with docker
Getting started with dockerGetting started with docker
Getting started with docker
 
Kubernetes and container security
Kubernetes and container securityKubernetes and container security
Kubernetes and container security
 

Similar to Hydra: Continuous Integration and Testing for Demanding People: The Details

Deploying .NET applications with the Nix package manager
Deploying .NET applications with the Nix package managerDeploying .NET applications with the Nix package manager
Deploying .NET applications with the Nix package managerSander van der Burg
 
nix-processmgmt: An experimental Nix-based process manager-agnostic framework
nix-processmgmt: An experimental Nix-based process manager-agnostic frameworknix-processmgmt: An experimental Nix-based process manager-agnostic framework
nix-processmgmt: An experimental Nix-based process manager-agnostic frameworkSander van der Burg
 
A Reference Architecture for Distributed Software Deployment
A Reference Architecture for Distributed Software DeploymentA Reference Architecture for Distributed Software Deployment
A Reference Architecture for Distributed Software DeploymentSander van der Burg
 
Techniques and lessons for improvement of deployment processes
Techniques and lessons for improvement of deployment processesTechniques and lessons for improvement of deployment processes
Techniques and lessons for improvement of deployment processesSander van der Burg
 
Automating Mendix application deployments with Nix
Automating Mendix application deployments with NixAutomating Mendix application deployments with Nix
Automating Mendix application deployments with NixSander van der Burg
 
Automate drupal deployments with linux containers, docker and vagrant
Automate drupal deployments with linux containers, docker and vagrant Automate drupal deployments with linux containers, docker and vagrant
Automate drupal deployments with linux containers, docker and vagrant Ricardo Amaro
 
Dysnomia: complementing Nix deployments with state deployment
Dysnomia: complementing Nix deployments with state deploymentDysnomia: complementing Nix deployments with state deployment
Dysnomia: complementing Nix deployments with state deploymentSander van der Burg
 
Fosdem_Using_SELinux_with_container_runtimes.pdf
Fosdem_Using_SELinux_with_container_runtimes.pdfFosdem_Using_SELinux_with_container_runtimes.pdf
Fosdem_Using_SELinux_with_container_runtimes.pdfnicerussianpainter
 
Présentation de Docker
Présentation de DockerPrésentation de Docker
Présentation de DockerProto204
 
drbd9_and_drbdmanage_may_2015
drbd9_and_drbdmanage_may_2015drbd9_and_drbdmanage_may_2015
drbd9_and_drbdmanage_may_2015Alexandre Huynh
 
Андрей Володин — Как подружиться с роботом
Андрей Володин — Как подружиться с роботомАндрей Володин — Как подружиться с роботом
Андрей Володин — Как подружиться с роботомCocoaHeads
 
Enhancing Network and Runtime Security with Cilium and Tetragon by Raymond De...
Enhancing Network and Runtime Security with Cilium and Tetragon by Raymond De...Enhancing Network and Runtime Security with Cilium and Tetragon by Raymond De...
Enhancing Network and Runtime Security with Cilium and Tetragon by Raymond De...ContainerDay Security 2023
 
Container Monitoring with Sysdig
Container Monitoring with SysdigContainer Monitoring with Sysdig
Container Monitoring with SysdigSreenivas Makam
 
Functional Operations (Functional Programming at Comcast Labs Connect)
Functional Operations (Functional Programming at Comcast Labs Connect)Functional Operations (Functional Programming at Comcast Labs Connect)
Functional Operations (Functional Programming at Comcast Labs Connect)Susan Potter
 
Making Security Invisible
Making Security InvisibleMaking Security Invisible
Making Security InvisibleJ On The Beach
 
Cross-compilation native sous android
Cross-compilation native sous androidCross-compilation native sous android
Cross-compilation native sous androidThierry Gayet
 
Our way of systems monitoring in application development
Our way of systems monitoring in application developmentOur way of systems monitoring in application development
Our way of systems monitoring in application developmentOCoderFest
 
Demystifying docker networking black magic - Lorenzo Fontana, Kiratech
Demystifying docker networking black magic - Lorenzo Fontana, KiratechDemystifying docker networking black magic - Lorenzo Fontana, Kiratech
Demystifying docker networking black magic - Lorenzo Fontana, KiratechCodemotion Tel Aviv
 
Developing MIPS Exploits to Hack Routers
Developing MIPS Exploits to Hack RoutersDeveloping MIPS Exploits to Hack Routers
Developing MIPS Exploits to Hack RoutersBGA Cyber Security
 

Similar to Hydra: Continuous Integration and Testing for Demanding People: The Details (20)

Deploying .NET applications with the Nix package manager
Deploying .NET applications with the Nix package managerDeploying .NET applications with the Nix package manager
Deploying .NET applications with the Nix package manager
 
nix-processmgmt: An experimental Nix-based process manager-agnostic framework
nix-processmgmt: An experimental Nix-based process manager-agnostic frameworknix-processmgmt: An experimental Nix-based process manager-agnostic framework
nix-processmgmt: An experimental Nix-based process manager-agnostic framework
 
A Reference Architecture for Distributed Software Deployment
A Reference Architecture for Distributed Software DeploymentA Reference Architecture for Distributed Software Deployment
A Reference Architecture for Distributed Software Deployment
 
Techniques and lessons for improvement of deployment processes
Techniques and lessons for improvement of deployment processesTechniques and lessons for improvement of deployment processes
Techniques and lessons for improvement of deployment processes
 
Automating Mendix application deployments with Nix
Automating Mendix application deployments with NixAutomating Mendix application deployments with Nix
Automating Mendix application deployments with Nix
 
Drbd
DrbdDrbd
Drbd
 
Automate drupal deployments with linux containers, docker and vagrant
Automate drupal deployments with linux containers, docker and vagrant Automate drupal deployments with linux containers, docker and vagrant
Automate drupal deployments with linux containers, docker and vagrant
 
Dysnomia: complementing Nix deployments with state deployment
Dysnomia: complementing Nix deployments with state deploymentDysnomia: complementing Nix deployments with state deployment
Dysnomia: complementing Nix deployments with state deployment
 
Fosdem_Using_SELinux_with_container_runtimes.pdf
Fosdem_Using_SELinux_with_container_runtimes.pdfFosdem_Using_SELinux_with_container_runtimes.pdf
Fosdem_Using_SELinux_with_container_runtimes.pdf
 
Présentation de Docker
Présentation de DockerPrésentation de Docker
Présentation de Docker
 
drbd9_and_drbdmanage_may_2015
drbd9_and_drbdmanage_may_2015drbd9_and_drbdmanage_may_2015
drbd9_and_drbdmanage_may_2015
 
Андрей Володин — Как подружиться с роботом
Андрей Володин — Как подружиться с роботомАндрей Володин — Как подружиться с роботом
Андрей Володин — Как подружиться с роботом
 
Enhancing Network and Runtime Security with Cilium and Tetragon by Raymond De...
Enhancing Network and Runtime Security with Cilium and Tetragon by Raymond De...Enhancing Network and Runtime Security with Cilium and Tetragon by Raymond De...
Enhancing Network and Runtime Security with Cilium and Tetragon by Raymond De...
 
Container Monitoring with Sysdig
Container Monitoring with SysdigContainer Monitoring with Sysdig
Container Monitoring with Sysdig
 
Functional Operations (Functional Programming at Comcast Labs Connect)
Functional Operations (Functional Programming at Comcast Labs Connect)Functional Operations (Functional Programming at Comcast Labs Connect)
Functional Operations (Functional Programming at Comcast Labs Connect)
 
Making Security Invisible
Making Security InvisibleMaking Security Invisible
Making Security Invisible
 
Cross-compilation native sous android
Cross-compilation native sous androidCross-compilation native sous android
Cross-compilation native sous android
 
Our way of systems monitoring in application development
Our way of systems monitoring in application developmentOur way of systems monitoring in application development
Our way of systems monitoring in application development
 
Demystifying docker networking black magic - Lorenzo Fontana, Kiratech
Demystifying docker networking black magic - Lorenzo Fontana, KiratechDemystifying docker networking black magic - Lorenzo Fontana, Kiratech
Demystifying docker networking black magic - Lorenzo Fontana, Kiratech
 
Developing MIPS Exploits to Hack Routers
Developing MIPS Exploits to Hack RoutersDeveloping MIPS Exploits to Hack Routers
Developing MIPS Exploits to Hack Routers
 

More from Sander van der Burg

Using Nix and Docker as automated deployment solutions
Using Nix and Docker as automated deployment solutionsUsing Nix and Docker as automated deployment solutions
Using Nix and Docker as automated deployment solutionsSander van der Burg
 
A Reference Architecture for Distributed Software Deployment
A Reference Architecture for Distributed Software DeploymentA Reference Architecture for Distributed Software Deployment
A Reference Architecture for Distributed Software DeploymentSander van der Burg
 
A Generic Approach for Deploying and Upgrading Mutable Software Components
A Generic Approach for Deploying and Upgrading Mutable Software ComponentsA Generic Approach for Deploying and Upgrading Mutable Software Components
A Generic Approach for Deploying and Upgrading Mutable Software ComponentsSander van der Burg
 
Deploying .NET services with Disnix
Deploying .NET services with DisnixDeploying .NET services with Disnix
Deploying .NET services with DisnixSander van der Burg
 
A Self-Adaptive Deployment Framework for Service-Oriented Systems
A Self-Adaptive Deployment Framework for Service-Oriented SystemsA Self-Adaptive Deployment Framework for Service-Oriented Systems
A Self-Adaptive Deployment Framework for Service-Oriented SystemsSander van der Burg
 
Disnix: A toolset for distributed deployment
Disnix: A toolset for distributed deploymentDisnix: A toolset for distributed deployment
Disnix: A toolset for distributed deploymentSander van der Burg
 
Automated Deployment of Hetergeneous Service-Oriented System
Automated Deployment of Hetergeneous Service-Oriented SystemAutomated Deployment of Hetergeneous Service-Oriented System
Automated Deployment of Hetergeneous Service-Oriented SystemSander van der Burg
 
Pull Deployment of Services: Introduction, Progress and Challenges
Pull Deployment of Services: Introduction, Progress and ChallengesPull Deployment of Services: Introduction, Progress and Challenges
Pull Deployment of Services: Introduction, Progress and ChallengesSander van der Burg
 
Software Deployment in a Dynamic Cloud
Software Deployment in a Dynamic CloudSoftware Deployment in a Dynamic Cloud
Software Deployment in a Dynamic CloudSander van der Burg
 
Atomic Upgrading of Distributed Systems
Atomic Upgrading of Distributed SystemsAtomic Upgrading of Distributed Systems
Atomic Upgrading of Distributed SystemsSander van der Burg
 
Model-driven Distributed Software Deployment
Model-driven Distributed Software DeploymentModel-driven Distributed Software Deployment
Model-driven Distributed Software DeploymentSander van der Burg
 
Model-driven Distributed Software Deployment
Model-driven Distributed Software DeploymentModel-driven Distributed Software Deployment
Model-driven Distributed Software DeploymentSander van der Burg
 
Model-driven Distributed Software Deployment laymen's talk
Model-driven Distributed Software Deployment laymen's talkModel-driven Distributed Software Deployment laymen's talk
Model-driven Distributed Software Deployment laymen's talkSander van der Burg
 

More from Sander van der Burg (15)

The Monitoring Playground
The Monitoring PlaygroundThe Monitoring Playground
The Monitoring Playground
 
Using Nix and Docker as automated deployment solutions
Using Nix and Docker as automated deployment solutionsUsing Nix and Docker as automated deployment solutions
Using Nix and Docker as automated deployment solutions
 
A Reference Architecture for Distributed Software Deployment
A Reference Architecture for Distributed Software DeploymentA Reference Architecture for Distributed Software Deployment
A Reference Architecture for Distributed Software Deployment
 
A Generic Approach for Deploying and Upgrading Mutable Software Components
A Generic Approach for Deploying and Upgrading Mutable Software ComponentsA Generic Approach for Deploying and Upgrading Mutable Software Components
A Generic Approach for Deploying and Upgrading Mutable Software Components
 
Deploying .NET services with Disnix
Deploying .NET services with DisnixDeploying .NET services with Disnix
Deploying .NET services with Disnix
 
A Self-Adaptive Deployment Framework for Service-Oriented Systems
A Self-Adaptive Deployment Framework for Service-Oriented SystemsA Self-Adaptive Deployment Framework for Service-Oriented Systems
A Self-Adaptive Deployment Framework for Service-Oriented Systems
 
Pull Deployment of Services
Pull Deployment of ServicesPull Deployment of Services
Pull Deployment of Services
 
Disnix: A toolset for distributed deployment
Disnix: A toolset for distributed deploymentDisnix: A toolset for distributed deployment
Disnix: A toolset for distributed deployment
 
Automated Deployment of Hetergeneous Service-Oriented System
Automated Deployment of Hetergeneous Service-Oriented SystemAutomated Deployment of Hetergeneous Service-Oriented System
Automated Deployment of Hetergeneous Service-Oriented System
 
Pull Deployment of Services: Introduction, Progress and Challenges
Pull Deployment of Services: Introduction, Progress and ChallengesPull Deployment of Services: Introduction, Progress and Challenges
Pull Deployment of Services: Introduction, Progress and Challenges
 
Software Deployment in a Dynamic Cloud
Software Deployment in a Dynamic CloudSoftware Deployment in a Dynamic Cloud
Software Deployment in a Dynamic Cloud
 
Atomic Upgrading of Distributed Systems
Atomic Upgrading of Distributed SystemsAtomic Upgrading of Distributed Systems
Atomic Upgrading of Distributed Systems
 
Model-driven Distributed Software Deployment
Model-driven Distributed Software DeploymentModel-driven Distributed Software Deployment
Model-driven Distributed Software Deployment
 
Model-driven Distributed Software Deployment
Model-driven Distributed Software DeploymentModel-driven Distributed Software Deployment
Model-driven Distributed Software Deployment
 
Model-driven Distributed Software Deployment laymen's talk
Model-driven Distributed Software Deployment laymen's talkModel-driven Distributed Software Deployment laymen's talk
Model-driven Distributed Software Deployment laymen's talk
 

Recently uploaded

Product Manager vs Product Owner – Why Do Companies Still Struggle 23 Years A...
Product Manager vs Product Owner – Why Do Companies Still Struggle 23 Years A...Product Manager vs Product Owner – Why Do Companies Still Struggle 23 Years A...
Product Manager vs Product Owner – Why Do Companies Still Struggle 23 Years A...ISPMAIndia
 
DBA Fundamentals Group: Continuous SQL with Kafka and Flink
DBA Fundamentals Group: Continuous SQL with Kafka and FlinkDBA Fundamentals Group: Continuous SQL with Kafka and Flink
DBA Fundamentals Group: Continuous SQL with Kafka and FlinkTimothy Spann
 
100 TOOLS TO MEASURE AND ANALYSE YOUR DIGITAL MARKETING EFFORTS
100 TOOLS TO MEASURE AND ANALYSE YOUR DIGITAL MARKETING EFFORTS100 TOOLS TO MEASURE AND ANALYSE YOUR DIGITAL MARKETING EFFORTS
100 TOOLS TO MEASURE AND ANALYSE YOUR DIGITAL MARKETING EFFORTSi-engage
 
Software Testing life cycle (STLC) Importance, Phases, Benefits...
Software Testing life cycle (STLC) Importance, Phases, Benefits...Software Testing life cycle (STLC) Importance, Phases, Benefits...
Software Testing life cycle (STLC) Importance, Phases, Benefits...Flexsin
 
LLMOps with Azure Machine Learning prompt flow
LLMOps with Azure Machine Learning prompt flowLLMOps with Azure Machine Learning prompt flow
LLMOps with Azure Machine Learning prompt flowNaoki (Neo) SATO
 
sql ppt for students who preparing for sql
sql ppt for students who preparing for sqlsql ppt for students who preparing for sql
sql ppt for students who preparing for sqlbharatjanadharwarud
 
Open Sprintera (Where Open Source Sparks a Sprint of Possibilities)
Open Sprintera (Where Open Source Sparks a Sprint of Possibilities)Open Sprintera (Where Open Source Sparks a Sprint of Possibilities)
Open Sprintera (Where Open Source Sparks a Sprint of Possibilities)GDSCNiT
 
Embracing Change - The Impact of Generative AI on Strategic Portfolio Management
Embracing Change - The Impact of Generative AI on Strategic Portfolio ManagementEmbracing Change - The Impact of Generative AI on Strategic Portfolio Management
Embracing Change - The Impact of Generative AI on Strategic Portfolio ManagementOnePlan Solutions
 
killingcamp longest common subsequence.pdf
killingcamp longest common subsequence.pdfkillingcamp longest common subsequence.pdf
killingcamp longest common subsequence.pdfssuser82c38d
 
"Discovery and Delivery through Product IntelliGenAI framework" by Ramkumar A...
"Discovery and Delivery through Product IntelliGenAI framework" by Ramkumar A..."Discovery and Delivery through Product IntelliGenAI framework" by Ramkumar A...
"Discovery and Delivery through Product IntelliGenAI framework" by Ramkumar A...ISPMAIndia
 
AUTOKEYUNLOCKER-BRANDS-SUPPORT-STANDARD-VERSION.pdf
AUTOKEYUNLOCKER-BRANDS-SUPPORT-STANDARD-VERSION.pdfAUTOKEYUNLOCKER-BRANDS-SUPPORT-STANDARD-VERSION.pdf
AUTOKEYUNLOCKER-BRANDS-SUPPORT-STANDARD-VERSION.pdfAutokey
 
Essence of Requirements Engineering: Pragmatic Insights for 2024
Essence of Requirements Engineering: Pragmatic Insights for 2024Essence of Requirements Engineering: Pragmatic Insights for 2024
Essence of Requirements Engineering: Pragmatic Insights for 2024Asher Sterkin
 
The Game-Changer_ How Software Development Outsource Can Catapult Your Growth...
The Game-Changer_ How Software Development Outsource Can Catapult Your Growth...The Game-Changer_ How Software Development Outsource Can Catapult Your Growth...
The Game-Changer_ How Software Development Outsource Can Catapult Your Growth...emili denli
 
Sql server types of joins with example.pptx
Sql server types of joins with example.pptxSql server types of joins with example.pptx
Sql server types of joins with example.pptxsameer gaikwad
 
SPM 2024 – Overview of and benefits of AI in Product Management
SPM 2024 – Overview of and benefits of AI in Product ManagementSPM 2024 – Overview of and benefits of AI in Product Management
SPM 2024 – Overview of and benefits of AI in Product ManagementISPMAIndia
 
maximum subarray ppt for killing camp students
maximum subarray ppt for killing camp studentsmaximum subarray ppt for killing camp students
maximum subarray ppt for killing camp studentsssuser82c38d
 
Getting Started with Trello for Beginners.pptx
Getting Started with Trello for Beginners.pptxGetting Started with Trello for Beginners.pptx
Getting Started with Trello for Beginners.pptxmavinoikein
 
The Age of AI: Elevating Experiences & Delivering Customer Value!
The Age of AI: Elevating Experiences & Delivering Customer Value!The Age of AI: Elevating Experiences & Delivering Customer Value!
The Age of AI: Elevating Experiences & Delivering Customer Value!ISPMAIndia
 
"Taking an idea to a Product in Health diagnostics" by Dr. Geetha Manjunath, ...
"Taking an idea to a Product in Health diagnostics" by Dr. Geetha Manjunath, ..."Taking an idea to a Product in Health diagnostics" by Dr. Geetha Manjunath, ...
"Taking an idea to a Product in Health diagnostics" by Dr. Geetha Manjunath, ...ISPMAIndia
 

Recently uploaded (20)

eLearning Content Development Company Code and Pixels.pdf
eLearning Content Development Company Code and Pixels.pdfeLearning Content Development Company Code and Pixels.pdf
eLearning Content Development Company Code and Pixels.pdf
 
Product Manager vs Product Owner – Why Do Companies Still Struggle 23 Years A...
Product Manager vs Product Owner – Why Do Companies Still Struggle 23 Years A...Product Manager vs Product Owner – Why Do Companies Still Struggle 23 Years A...
Product Manager vs Product Owner – Why Do Companies Still Struggle 23 Years A...
 
DBA Fundamentals Group: Continuous SQL with Kafka and Flink
DBA Fundamentals Group: Continuous SQL with Kafka and FlinkDBA Fundamentals Group: Continuous SQL with Kafka and Flink
DBA Fundamentals Group: Continuous SQL with Kafka and Flink
 
100 TOOLS TO MEASURE AND ANALYSE YOUR DIGITAL MARKETING EFFORTS
100 TOOLS TO MEASURE AND ANALYSE YOUR DIGITAL MARKETING EFFORTS100 TOOLS TO MEASURE AND ANALYSE YOUR DIGITAL MARKETING EFFORTS
100 TOOLS TO MEASURE AND ANALYSE YOUR DIGITAL MARKETING EFFORTS
 
Software Testing life cycle (STLC) Importance, Phases, Benefits...
Software Testing life cycle (STLC) Importance, Phases, Benefits...Software Testing life cycle (STLC) Importance, Phases, Benefits...
Software Testing life cycle (STLC) Importance, Phases, Benefits...
 
LLMOps with Azure Machine Learning prompt flow
LLMOps with Azure Machine Learning prompt flowLLMOps with Azure Machine Learning prompt flow
LLMOps with Azure Machine Learning prompt flow
 
sql ppt for students who preparing for sql
sql ppt for students who preparing for sqlsql ppt for students who preparing for sql
sql ppt for students who preparing for sql
 
Open Sprintera (Where Open Source Sparks a Sprint of Possibilities)
Open Sprintera (Where Open Source Sparks a Sprint of Possibilities)Open Sprintera (Where Open Source Sparks a Sprint of Possibilities)
Open Sprintera (Where Open Source Sparks a Sprint of Possibilities)
 
Embracing Change - The Impact of Generative AI on Strategic Portfolio Management
Embracing Change - The Impact of Generative AI on Strategic Portfolio ManagementEmbracing Change - The Impact of Generative AI on Strategic Portfolio Management
Embracing Change - The Impact of Generative AI on Strategic Portfolio Management
 
killingcamp longest common subsequence.pdf
killingcamp longest common subsequence.pdfkillingcamp longest common subsequence.pdf
killingcamp longest common subsequence.pdf
 
"Discovery and Delivery through Product IntelliGenAI framework" by Ramkumar A...
"Discovery and Delivery through Product IntelliGenAI framework" by Ramkumar A..."Discovery and Delivery through Product IntelliGenAI framework" by Ramkumar A...
"Discovery and Delivery through Product IntelliGenAI framework" by Ramkumar A...
 
AUTOKEYUNLOCKER-BRANDS-SUPPORT-STANDARD-VERSION.pdf
AUTOKEYUNLOCKER-BRANDS-SUPPORT-STANDARD-VERSION.pdfAUTOKEYUNLOCKER-BRANDS-SUPPORT-STANDARD-VERSION.pdf
AUTOKEYUNLOCKER-BRANDS-SUPPORT-STANDARD-VERSION.pdf
 
Essence of Requirements Engineering: Pragmatic Insights for 2024
Essence of Requirements Engineering: Pragmatic Insights for 2024Essence of Requirements Engineering: Pragmatic Insights for 2024
Essence of Requirements Engineering: Pragmatic Insights for 2024
 
The Game-Changer_ How Software Development Outsource Can Catapult Your Growth...
The Game-Changer_ How Software Development Outsource Can Catapult Your Growth...The Game-Changer_ How Software Development Outsource Can Catapult Your Growth...
The Game-Changer_ How Software Development Outsource Can Catapult Your Growth...
 
Sql server types of joins with example.pptx
Sql server types of joins with example.pptxSql server types of joins with example.pptx
Sql server types of joins with example.pptx
 
SPM 2024 – Overview of and benefits of AI in Product Management
SPM 2024 – Overview of and benefits of AI in Product ManagementSPM 2024 – Overview of and benefits of AI in Product Management
SPM 2024 – Overview of and benefits of AI in Product Management
 
maximum subarray ppt for killing camp students
maximum subarray ppt for killing camp studentsmaximum subarray ppt for killing camp students
maximum subarray ppt for killing camp students
 
Getting Started with Trello for Beginners.pptx
Getting Started with Trello for Beginners.pptxGetting Started with Trello for Beginners.pptx
Getting Started with Trello for Beginners.pptx
 
The Age of AI: Elevating Experiences & Delivering Customer Value!
The Age of AI: Elevating Experiences & Delivering Customer Value!The Age of AI: Elevating Experiences & Delivering Customer Value!
The Age of AI: Elevating Experiences & Delivering Customer Value!
 
"Taking an idea to a Product in Health diagnostics" by Dr. Geetha Manjunath, ...
"Taking an idea to a Product in Health diagnostics" by Dr. Geetha Manjunath, ..."Taking an idea to a Product in Health diagnostics" by Dr. Geetha Manjunath, ...
"Taking an idea to a Product in Health diagnostics" by Dr. Geetha Manjunath, ...
 

Hydra: Continuous Integration and Testing for Demanding People: The Details

  • 1. Hydra: Continuous Integration and Testing for Demanding People: The Details Sander van der Burg Conference Compass July 15, 2014 Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
  • 2. Continuous integration We want to deliver and test software rapidly We quickly want to see the impact of changes to the source code and its dependencies. Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
  • 3. Hydra Hydra: A Nix-based continuous integration server: Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
  • 4. Hydra Hydra: A Nix-based continuous integration server: Generic. Supports multiple programming language environments and component technologies. Deployment. Build and test environments are deployed automatically and all dependencies are ensured to be present and correct. Variability. Multiple versions/variants of dependencies can safely coexist. Multi platform support. Builds can be easily delegated to machines with a different operating system. Scalability. Builds are transparently delegated to any machine in a cluster capable of building it. Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
  • 5. Hydra How to use Hydra to build or test stuff? Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
  • 6. Hydra overview Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
  • 7. Hydra overview Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People Components Queue runner: Regularly checks what has changed and what to build Evaluator: Builds the jobs Server: Web application making builds and test results available Nix: Package mananger responsible for the actual builds and depedency management
  • 8. Hydra overview Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
  • 9. The Nix package manager A package manager borrowing concepts from purely functional programming languages. x = y ⇒ f (x) = f (y) Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
  • 10. Nix store Main idea: store all packages in isolation from each other: /nix/store/40awryfqzp46m... -disnix-0.3 Paths contain a 160-bit cryptographic hash of all inputs used to build the package: Sources Libraries Compilers Build scripts . . . /nix/store 40awryfqzp...-disnix-0.3 bin disnix-env disnix-manifest disnix-service kjlv4klmra...-getopt-1.1.4 bin getopt am13rq9ka...-dbus-glib-0.102 lib libdbus-glib-1.so.2 94n64qy99...-glibc-2.19 lib libc.so.6 Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
  • 11. Nix expressions disnix.nix { stdenv, fetchurl, pkgconfig, dbus_glib , libxml2, libxslt, getopt, nix, dysnomia }: stdenv.mkDerivation { name = "disnix-0.3"; src = fetchurl { url = http://.../disnix-0.3.tar.bz2; sha256 = "1jjmzdd7fac6isq5wdaqjbwwnsnzjag5s4..."; }; buildInputs = [ pkgconfig dbus_glib libxml2 libxslt getopt nix dysnomia ]; buildCommand = ’’ tar xjf $src ./configure --prefix=$out make; make install ’’; } Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
  • 12. Nix expressions disnix.nix { stdenv, fetchurl, pkgconfig, dbus_glib , libxml2, libxslt, getopt, nix, dysnomia }: stdenv.mkDerivation { name = "disnix-0.3"; src = fetchurl { url = http://.../disnix-0.3.tar.bz2; sha256 = "1jjmzdd7fac6isq5wdaqjbwwnsnzjag5s4..."; }; buildInputs = [ pkgconfig dbus_glib libxml2 libxslt getopt nix dysnomia ]; buildCommand = ’’ tar xjf $src ./configure --prefix=$out make; make install ’’; } Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People Environments Expression defines a function that composes an environment in which a build is executed Nearly any type of build can be performed inside it, e.g. C/C++, Java, Perl, Python... We can also run tests inside these environments The buildInputs parameters are used to configure all settings to make a build find its dependencies, e.g. setting PATH, PYTHONPATH, CLASSPATH ... There are also function abstractions for different kinds of packages If no buildCommand is given, it executes the default GNU Autotools build procedure: ./configure; make; make install.
  • 13. Nix expressions all-packages.nix {system ? builtins.currentSystem}: rec { stdenv = ... { inherit system; }; fetchurl = ...; pkgconfig = ...; dbus_glib = ...; libxml2 = ...; libxslt = ...; getopt = ...; nix = callPackage ../pkgs/tools/package-management/nix { }; dysnomia = import ../pkgs/tools/package-management/dysnomia { inherit stdenv fetchurl getopt; }; disnix = import ../pkgs/tools/package-management/disnix { inherit stdenv fetchurl pkgconfig dbus_glib; inherit libxml2 libxslt getopt nix dysnomia; }; ... } Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
  • 14. Nix expressions all-packages.nix {system ? builtins.currentSystem}: rec { stdenv = ... { inherit system; }; fetchurl = ...; pkgconfig = ...; dbus_glib = ...; libxml2 = ...; libxslt = ...; getopt = ...; nix = callPackage ../pkgs/tools/package-management/nix { }; dysnomia = import ../pkgs/tools/package-management/dysnomia { inherit stdenv fetchurl getopt; }; disnix = import ../pkgs/tools/package-management/disnix { inherit stdenv fetchurl pkgconfig dbus_glib; inherit libxml2 libxslt getopt nix dysnomia; }; ... } Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People Environments Composes packages by calling them with the required function arguments. Function invocations are lazy – they are only evaluated if needed. Previous expression for Disnix that defines a function is imported here. All dependencies are composed in the same expression as well.
  • 15. Building Nix expressions Building a Nix package: $ nix-build all-packages.nix -A disnix /nix/store/40awryfqzp46mjzm1rwy9qa8vxscjhgx-disnix-0.3 Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
  • 16. Building Nix expressions Building a Nix package: $ nix-build all-packages.nix -A disnix /nix/store/40awryfqzp46mjzm1rwy9qa8vxscjhgx-disnix-0.3 The Nix package manager builds disnix and all its dependencies that have not been built yet. Hash component is derived from all build inputs used to build the package. Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
  • 18. Finding runtime dependencies /nix/store 40awryfqzp...-disnix-0.3 bin disnix-env disnix-manifest disnix-service kjlv4klmra...-getopt-1.1.4 bin getopt am13rq9ka...-dbus-glib-0.102 lib libdbus-glib-1.so.2 94n64qy99...-glibc-2.19 lib libc.so.6 Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People Contents of 40aw...-disnix-0.3/bin/disnix-service ... 72 74 00 5f 65 6e 64 00 2f 6e 69 78 2f 73 74 6f |rt._end./nix/sto| 72 65 2f 61 6d 31 33 72 71 39 6b 61 7a 6d 31 78 |re/am13rq9kazm1x| 34 30 71 37 67 6b 70 71 77 31 35 62 37 33 32 6d |40q7gkpqw15b732m| 63 62 69 2d 64 62 75 73 2d 67 6c 69 62 2d 30 2e |cbi-dbus-glib-0.| 31 30 32 2f 6c 69 62 2f 6c 69 62 64 62 75 73 2d |102/lib/libdbus-| 67 6c 69 62 2d 31 2e 73 6f 3a 2f 6e 69 78 2f 73 |glib-1.so:/nix/s| 74 6f 72 65 2f 30 39 33 66 61 69 64 32 6d 78 69 |tore/093faid2mxi| 63 72 33 36 38 79 39 36 63 35 61 6a 6b 6c 39 6b |cr368y96c5ajkl9k| ...
  • 19. Finding runtime dependencies /nix/store 40awryfqzp...-disnix-0.3 bin disnix-env disnix-manifest disnix-service kjlv4klmra...-getopt-1.1.4 bin getopt am13rq9ka...-dbus-glib-0.102 lib libdbus-glib-1.so.2 94n64qy99...-glibc-2.19 lib libc.so.6 Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People Contents of 40aw...-disnix-0.3/bin/disnix-service ... 72 74 00 5f 65 6e 64 00 2f 6e 69 78 2f 73 74 6f |rt._end./nix/sto| 72 65 2f 61 6d 31 33 72 71 39 6b 61 7a 6d 31 78 |re/am13rq9kazm1x| 34 30 71 37 67 6b 70 71 77 31 35 62 37 33 32 6d |40q7gkpqw15b732m| 63 62 69 2d 64 62 75 73 2d 67 6c 69 62 2d 30 2e |cbi-dbus-glib-0.| 31 30 32 2f 6c 69 62 2f 6c 69 62 64 62 75 73 2d |102/lib/libdbus-| 67 6c 69 62 2d 31 2e 73 6f 3a 2f 6e 69 78 2f 73 |glib-1.so:/nix/s| 74 6f 72 65 2f 30 39 33 66 61 69 64 32 6d 78 69 |tore/093faid2mxi| 63 72 33 36 38 79 39 36 63 35 61 6a 6b 6c 39 6b |cr368y96c5ajkl9k| ...
  • 20. Building Nix expressions During a build of package many side effects are removed: Most environment variables are initially cleared or set to dummy values, such as PATH. Environment variables, such as PATH, are configured to only contain the specified dependencies. Nix store paths prevent packages to be implicitly found in many cases (unlike “traditional” systems using /usr/lib, /usr/bin or C:WINDOWSSystem32). Timestamps are set to 1 second after the epoch Files in the Nix store are made read-only. Optionally, builds can be performed in a chroot() environment, improving purity Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
  • 21. User environments Users can have different sets of installed applications. PATH /nix/.../profiles current 42 /nix/store pp56i0a01si5...-user-env bin firefox disnix-env b9w6q73mqm...-disnix-0.2 bin disnix-env mr8f62946...-firefox-30.0 bin firefox Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
  • 22. User environments Users can have different sets of installed applications. nix-env operations create new user environments in the store. PATH /nix/.../profiles current 42 /nix/store pp56i0a01si5...-user-env bin firefox disnix-env b9w6q73mqm...-disnix-0.2 bin disnix-env mr8f62946...-firefox-30.0 bin firefox 40awryfq...-disnix-0.3 bin disnix-env (nix-env -u disnix) Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
  • 23. User environments Users can have different sets of installed applications. nix-env operations create new user environments in the store. PATH /nix/.../profiles current 42 /nix/store pp56i0a01si5...-user-env bin firefox disnix-env b9w6q73mqm...-disnix-0.2 bin disnix-env mr8f62946...-firefox-30.0 bin firefox 40awryfq...-disnix-0.3 bin disnix-env i3d9vh6d8ip1...-user-env bin disnix-env firefox (nix-env -u disnix) Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
  • 24. User environments Users can have different sets of installed applications. nix-env operations create new user environments in the store. PATH /nix/.../profiles current 42 43 /nix/store pp56i0a01si5...-user-env bin firefox disnix-env b9w6q73mqm...-disnix-0.2 bin disnix-env mr8f62946...-firefox-30.0 bin firefox 40awryfq...-disnix-0.3 bin disnix-env i3d9vh6d8ip1...-user-env bin disnix-env firefox (nix-env -u disnix) Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
  • 25. User environments Users can have different sets of installed applications. nix-env operations create new user environments in the store. We can atomically switch between them. PATH /nix/.../profiles current 42 43 /nix/store pp56i0a01si5...-user-env bin firefox disnix-env b9w6q73mqm...-disnix-0.2 bin disnix-env mr8f62946...-firefox-30.0 bin firefox 40awryfq...-disnix-0.3 bin disnix-env i3d9vh6d8ip1...-user-env bin disnix-env firefox (nix-env -u disnix) Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
  • 26. User environments Users can have different sets of installed applications. nix-env operations create new user environments in the store. We can atomically switch between them. These are roots of the garbage collector. PATH /nix/.../profiles current 43 /nix/store pp56i0a01si5...-user-env bin firefox disnix-env b9w6q73mqm...-disnix-0.2 bin disnix-env mr8f62946...-firefox-30.0 bin firefox 40awryfq...-disnix-0.3 bin disnix-env i3d9vh6d8ip1...-user-env bin disnix-env firefox (nix-env --remove-generations old) Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
  • 27. User environments Users can have different sets of installed applications. nix-env operations create new user environments in the store. We can atomically switch between them. These are roots of the garbage collector. PATH /nix/.../profiles current 43 /nix/store mr8f62946...-firefox-30.0 bin firefox 40awryfq...-disnix-0.3 bin disnix-env i3d9vh6d8ip1...-user-env bin disnix-env firefox (nix-collect-garbage) Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
  • 28. Hydra expression release.nix { nixpkgs ? <nixpkgs>, systems ? [ "x86_64-linux" "x86_64-darwin" ] , dysnomia ? { outPath = ./.; rev = 1234; } }: let pkgs = import nixpkgs {}; in rec { tarball = pkgs.releaseTools.sourceTarball { name = "dysnomia-tarball"; version = builtins.readFile ./version; src = dysnomia; buildInputs = [ pkgs.getopt ]; }; build = pkgs.lib.genAttrs systems (system: let pkgs = import nixpkgs { inherit system; }; in pkgs.releaseTools.nixBuild { name = "dysnomia"; version = builtins.readFile ./version; src = tarball; buildInputs = [ pkgs.getopt ]; }); ... } Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
  • 29. Hydra expression release.nix { nixpkgs ? <nixpkgs>, systems ? [ "x86_64-linux" "x86_64-darwin" ] , dysnomia ? { outPath = ./.; rev = 1234; } }: let pkgs = import nixpkgs {}; in rec { tarball = pkgs.releaseTools.sourceTarball { name = "dysnomia-tarball"; version = builtins.readFile ./version; src = dysnomia; buildInputs = [ pkgs.getopt ]; }; build = pkgs.lib.genAttrs systems (system: let pkgs = import nixpkgs { inherit system; }; in pkgs.releaseTools.nixBuild { name = "dysnomia"; version = builtins.readFile ./version; src = tarball; buildInputs = [ pkgs.getopt ]; }); ... } Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People Jobs An Hydra expression is a function returing an attribute set: rec{attr1 = value1; ...; attrn = valuen; } Function parameters define variability points: Locations of the Dysnomia, Nixpkgs collection Git repositories Target system architectures Each attribute corresponds to a job. Each value refers to a function performing a build or test. File is typically placed in the root folder of a source package.
  • 30. Building jobs from the command-line Building a source tarball: $ nix-build release.nix -A tarball Building Dysnomia for 64-bit AMD Linux: $ nix-build release.nix -A build.x86 64-linux Building Dysnomia for Mac OS X (Nix delegates the build to a Mac machine if the build is run on Linux and an external machine is configured): $ nix-build release.nix -A build.x86 64-darwin Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
  • 31. Building jobs from Hydra Create a project: Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
  • 32. Building jobs from Hydra Create a jobset: Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
  • 33. Building jobs from Hydra Create a jobset: Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People Jobs All (but one) inputs are provided as function arguments to the release expression. One input is the package itself (dysnomia) that contains the release.nix expression
  • 34. Building jobs from Hydra Evaluation results (job names correspond to those defined in release.nix): Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
  • 35. Hydra expression referring to other jobsets release.nix { nixpkgs ? <nixpkgs>, systems ? [ "x86_64-linux" "x86_64-darwin" ] , dysnomiaJobset ? import ../dysnomia/release.nix { inherit nixpkgs systems; } , disnix ? { outPath = ./.; rev = 1234; } }: let pkgs = import nixpkgs {}; in rec { tarball = ... build = pkgs.lib.genAttrs systems (system: let dysnomia = builtins.getAttr system (dysnomiaJobset.build); in with import nixpkgs { inherit system; }; releaseTools.nixBuild { name = "disnix"; src = tarball; buildInputs = [ pkgconfig dbus_glib libxml2 libxslt getopt nix dysnomia ]; }; ... } Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
  • 36. Building jobs from Hydra Use an input of type: ’Previous Hydra evaluation’: Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
  • 37. Hydra jobs I typically write Source packages. Jobs that assemble tarballs, Zip files or other archives containing the source code. Binary packages. The actual builds for a variety of architectures, such as i686-linux, x86 64-linux, x86 64-darwin. Program manuals. For example, building the manual from Docbook. Program documentation catalogs. Generating a documentation catalog from the source code, e.g. using javadoc, doxygen or JSDuck. Unit tests. Running Unit tests, for example with JUnit or mocha and producing a coverage report. System integration tests. Composing NixOS Linux VMs with all environmental dependencies, e.g. DBMS, web server etc, and run tests inside them. Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
  • 38. Nix channel Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
  • 39. Nix channel Adding a channel: $ nix-channel --add http://localhost/jobset/disnix/disnix-master/channel/latest $ nix-channel --update When running: $ nix-env -i disnix installing ‘disnix-0.3pre174e883b7b09da822494876d2f297736f33707a7’ these paths will be fetched (0.31 MiB download, 0.91 MiB unpacked): /nix/store/70rkq38r69fwrz90ayc4fyg823z92nmf-disnix-0.3 fetching path ‘/nix/store/70rkq38r69fwrz90ayc4fyg823z92nmf-disnix-0.3’... The build gets downloaded from the Hydra server, instead of being built from source code. Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
  • 40. Nix package manager: Exercises Check it out yourself!!! Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
  • 41. Availability Nix and Hydra are available as free and open source software under the LGPLv2 and the GPLv3 licenses: Nix: http://nixos.org/nix Hydra: http://nixos.org/hydra NixOS’ Hydra server: http://hydra.nixos.org Nix can be used on any Linux distribution, NixOS, Mac OS X, FreeBSD, and Windows (through Cygwin) Hydra can be used on any Linux distribution Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
  • 42. Related work Using Nix while doing development: Deploy development packages and composing an environment in which they can be found NixOS: http://nixos.org/nixos Deploy an entire system configuration (Linux distribution) with Nix. System integration testing with NixOS Efficiently compose networks of NixOS machines within a build in which system integration tests can be performed Disnix: http://nixos/disnix (Re)deploy service-oriented systems into networks of machines NixOps: http://nixos/nixops Deploy networks of NixOS configurations to physical machines or into the cloud Automatically creates VM instances if needed NiJS: https://www.npmjs.org/package/nijs Compose Nix packages in JavaScript Invoke JavaScript functions from Nix expressions Very primitive stand-alone package manager Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
  • 43. Questions Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People