What do you do when your python project has dependencies beyond Python? Nix is a "purely functional" package manager, which can also be used like "virtualenv", but for everything. I would cover single user installation of Nix package manager on Linux or Mac, defining Nix based development environments, generating Nix expressions for large Python projects using buildout, and finally, building minimal Docker images from scratch by using those generated expressions.
What do you do when your python project has dependencies beyond Python? Nix is a "purely functional" package manager, which can also be used like "virtualenv", but for everything. I would cover single user installation of Nix package manager on Linux or Mac, defining Nix based development environments, generating Nix expressions for large Python projects using buildout, and finally, building minimal Docker images from scratch by using those generated expressions.
Using Docker Containers to Improve Reproducibility in Software and Web Engine...Vincenzo Ferme
The ability to replicate and reproduce scientific results has become an increasingly important topic for many academic disciplines. In computer science and, more specifically, software and Web engineering, contributions of scientific work rely on developed algorithms, tools and prototypes, quantitative evaluations, and other computational analyses. Published code and data come with many undocumented assumptions, dependencies, and configurations that are internal knowledge and make reproducibility hard to achieve. This tutorial presents how Docker containers can overcome these issues and aid the reproducibility of research artefacts in software engineering and discusses their applications in the field.
Cite us: http://link.springer.com/chapter/10.1007/978-3-319-38791-8_58
Securing Applications and Pipelines on a Container PlatformAll Things Open
Presented at: Open Source 101 at Home
Presented by: Veer Muchandi, Red Hat Inc
Abstract: While everyone wants to do Containers and Kubernetes, they don’t know what they are getting into from Security perspective. This session intends to take you from “I don’t know what I don’t know” to “I know what I don’t know”. This helps you to make informed choices on Application Security.
Kubernetes as a Container Platform is becoming a de facto for every enterprise. In my interactions with enterprises adopting container platform, I come across common questions:
- How does application security work on this platform? What all do I need to secure?
- How do I implement security in pipelines?
- What about vulnerabilities discovered at a later point in time?
- What are newer technologies like Istio Service Mesh bring to table?
In this session, I will be addressing these commonly asked questions that every enterprise trying to adopt an Enterprise Kubernetes Platform needs to know so that they can make informed decisions.
Dockerized containers are the current wave that promising to revolutionize IT. Everybody is talking about containers, but a lot of people remain confused on how they work and why they are different or better than virtual machines. In this session, Black Duck container and virtualization expert Tim Mackey will demystify containers, explain their core concepts, and compare and contrast them with the virtual machine architectures that have been the staple of IT for the last decade.
A presentation on how and why to engage upstream projects productively, and ensure that work is accepted upstream first.
Originally delivered at Linux Foundation Collaboration Summit 2015 in Santa Rosa.
Ryan Koop's Docker Chicago Meetup Demo March 12 2014Cohesive Networks
CohesiveFT's Director of Products & Marketing, Ryan Koop, presented on how CohesiveFT is incorporating Docker containers in our latest version of the virtual networking appliance, VNS3.
Docker Meetup #2 was held on March 12, 2014 at Mediafly
Introduction to Docker and all things containers, Docker Meetup at RelateIQdotCloud
Docker is an open-source project to easily create lightweight, portable, self-sufficient containers from any application. The same container that a developer builds and tests on a laptop can run at scale, in production, on VMs, bare metal, OpenStack clusters, public clouds and more.
We're really happy to say that today we made the first meetup about Kubernetes in Russia! Thanks to all speakers and guests! Join us: https://twitter.com/kubernetesMSK
Introduction to deployment with Ruby on Rails presented at JAX09 in Mainz by Jonathan Weiss.
Learn about the deployment architectures and setups (web and app tier) and using Capistrano/Webistrano.
Presentation on Pesantren Kilat Code Security
Tangerang, 2016-06-06
We talk about docker. What it is? Why it matters? and how it can benefit us?
This presentation is an introduction and delivered to local meetup in Indonesia.
Presentation about Docker:
2016 Trends:
* Microservices: load balancing and orchestration
* Cloud
* Continuos integration
* Environment-less deployment
What are containers?
Why Docker?
Docker project
Docker. Inc
Docker VS VM
Docker basics
Some statistics about Docker and some Docker use case insights
Docker compose configuration file:
http://www.mediafire.com/download/lfmfzrkgn9wzegm/docker-compose.yml
Présentation link:
https://docs.google.com/presentation/d/1x11EgUqBVLAl70p53rZ-nJoLlL6FoZd2KbvTRxyVp1g/pub?start=false&loop=false&delayms=3000
Everyone heard about Kubernetes. Everyone wants to use this tool. However, sometimes we forget about security, which is essential throughout the container lifecycle.
Therefore, our journey with Kubernetes security should begin in the build stage when writing the code becomes the container image.
Kubernetes provides innate security advantages, and together with solid container protection, it will be invincible.
During the sessions, we will review all those features and highlight which are mandatory to use. We will discuss the main vulnerabilities which may cause compromising your system.
Contacts:
LinkedIn - https://www.linkedin.com/in/vshynkar/
GitHub - https://github.com/sqerison
-------------------------------------------------------------------------------------
Materials from the video:
The policies and docker files examples:
https://gist.github.com/sqerison/43365e30ee62298d9757deeab7643a90
The repo with the helm chart used in a demo:
https://github.com/sqerison/argo-rollouts-demo
Tools that showed in the last section:
https://github.com/armosec/kubescape
https://github.com/aquasecurity/kube-bench
https://github.com/controlplaneio/kubectl-kubesec
https://github.com/Shopify/kubeaudit#installation
https://github.com/eldadru/ksniff
Further learning.
A book released by CISA (Cybersecurity and Infrastructure Security Agency):
https://media.defense.gov/2021/Aug/03/2002820425/-1/-1/1/CTR_KUBERNETES%20HARDENING%20GUIDANCE.PDF
O`REILLY Kubernetes Security:
https://kubernetes-security.info/
O`REILLY Container Security:
https://info.aquasec.com/container-security-book
Thanks for watching!
nix-processmgmt: An experimental Nix-based process manager-agnostic frameworkSander van der Burg
NixCon 2020 talk about an experimental framework that integrates the Nix package manager with all kinds of process managers, such as : sysvinit, systemd, launchd, and even Docker
Using Docker Containers to Improve Reproducibility in Software and Web Engine...Vincenzo Ferme
The ability to replicate and reproduce scientific results has become an increasingly important topic for many academic disciplines. In computer science and, more specifically, software and Web engineering, contributions of scientific work rely on developed algorithms, tools and prototypes, quantitative evaluations, and other computational analyses. Published code and data come with many undocumented assumptions, dependencies, and configurations that are internal knowledge and make reproducibility hard to achieve. This tutorial presents how Docker containers can overcome these issues and aid the reproducibility of research artefacts in software engineering and discusses their applications in the field.
Cite us: http://link.springer.com/chapter/10.1007/978-3-319-38791-8_58
Securing Applications and Pipelines on a Container PlatformAll Things Open
Presented at: Open Source 101 at Home
Presented by: Veer Muchandi, Red Hat Inc
Abstract: While everyone wants to do Containers and Kubernetes, they don’t know what they are getting into from Security perspective. This session intends to take you from “I don’t know what I don’t know” to “I know what I don’t know”. This helps you to make informed choices on Application Security.
Kubernetes as a Container Platform is becoming a de facto for every enterprise. In my interactions with enterprises adopting container platform, I come across common questions:
- How does application security work on this platform? What all do I need to secure?
- How do I implement security in pipelines?
- What about vulnerabilities discovered at a later point in time?
- What are newer technologies like Istio Service Mesh bring to table?
In this session, I will be addressing these commonly asked questions that every enterprise trying to adopt an Enterprise Kubernetes Platform needs to know so that they can make informed decisions.
Dockerized containers are the current wave that promising to revolutionize IT. Everybody is talking about containers, but a lot of people remain confused on how they work and why they are different or better than virtual machines. In this session, Black Duck container and virtualization expert Tim Mackey will demystify containers, explain their core concepts, and compare and contrast them with the virtual machine architectures that have been the staple of IT for the last decade.
A presentation on how and why to engage upstream projects productively, and ensure that work is accepted upstream first.
Originally delivered at Linux Foundation Collaboration Summit 2015 in Santa Rosa.
Ryan Koop's Docker Chicago Meetup Demo March 12 2014Cohesive Networks
CohesiveFT's Director of Products & Marketing, Ryan Koop, presented on how CohesiveFT is incorporating Docker containers in our latest version of the virtual networking appliance, VNS3.
Docker Meetup #2 was held on March 12, 2014 at Mediafly
Introduction to Docker and all things containers, Docker Meetup at RelateIQdotCloud
Docker is an open-source project to easily create lightweight, portable, self-sufficient containers from any application. The same container that a developer builds and tests on a laptop can run at scale, in production, on VMs, bare metal, OpenStack clusters, public clouds and more.
We're really happy to say that today we made the first meetup about Kubernetes in Russia! Thanks to all speakers and guests! Join us: https://twitter.com/kubernetesMSK
Introduction to deployment with Ruby on Rails presented at JAX09 in Mainz by Jonathan Weiss.
Learn about the deployment architectures and setups (web and app tier) and using Capistrano/Webistrano.
Presentation on Pesantren Kilat Code Security
Tangerang, 2016-06-06
We talk about docker. What it is? Why it matters? and how it can benefit us?
This presentation is an introduction and delivered to local meetup in Indonesia.
Presentation about Docker:
2016 Trends:
* Microservices: load balancing and orchestration
* Cloud
* Continuos integration
* Environment-less deployment
What are containers?
Why Docker?
Docker project
Docker. Inc
Docker VS VM
Docker basics
Some statistics about Docker and some Docker use case insights
Docker compose configuration file:
http://www.mediafire.com/download/lfmfzrkgn9wzegm/docker-compose.yml
Présentation link:
https://docs.google.com/presentation/d/1x11EgUqBVLAl70p53rZ-nJoLlL6FoZd2KbvTRxyVp1g/pub?start=false&loop=false&delayms=3000
Everyone heard about Kubernetes. Everyone wants to use this tool. However, sometimes we forget about security, which is essential throughout the container lifecycle.
Therefore, our journey with Kubernetes security should begin in the build stage when writing the code becomes the container image.
Kubernetes provides innate security advantages, and together with solid container protection, it will be invincible.
During the sessions, we will review all those features and highlight which are mandatory to use. We will discuss the main vulnerabilities which may cause compromising your system.
Contacts:
LinkedIn - https://www.linkedin.com/in/vshynkar/
GitHub - https://github.com/sqerison
-------------------------------------------------------------------------------------
Materials from the video:
The policies and docker files examples:
https://gist.github.com/sqerison/43365e30ee62298d9757deeab7643a90
The repo with the helm chart used in a demo:
https://github.com/sqerison/argo-rollouts-demo
Tools that showed in the last section:
https://github.com/armosec/kubescape
https://github.com/aquasecurity/kube-bench
https://github.com/controlplaneio/kubectl-kubesec
https://github.com/Shopify/kubeaudit#installation
https://github.com/eldadru/ksniff
Further learning.
A book released by CISA (Cybersecurity and Infrastructure Security Agency):
https://media.defense.gov/2021/Aug/03/2002820425/-1/-1/1/CTR_KUBERNETES%20HARDENING%20GUIDANCE.PDF
O`REILLY Kubernetes Security:
https://kubernetes-security.info/
O`REILLY Container Security:
https://info.aquasec.com/container-security-book
Thanks for watching!
nix-processmgmt: An experimental Nix-based process manager-agnostic frameworkSander van der Burg
NixCon 2020 talk about an experimental framework that integrates the Nix package manager with all kinds of process managers, such as : sysvinit, systemd, launchd, and even Docker
This presentation is on the DRBD product. At eNovance, we're using it for several years. In those slides, you will find informations on how we use it, use cases and Ninja tricks.
This document has been realized with a lot of feedbacks and thanks to strong knowledges on that technology that eNovance is able to provide.
Dysnomia: complementing Nix deployments with state deploymentSander van der Burg
This talk covers Dysnomia, a state deployment tool that complements various tools in the Nix project, such as NixOS and Disnix, with state management facilities.
Présentation du système Docker animée par Sebastien Binet (CNRS/IN2P3/LPC) en Décembre 2015 au Proto204
http://reseau-loops.github.io/journee_2015_12.html
Session at ContainerDay Security 2023 on the 8th of March in Hamburg.
Cilium is the next generation, eBPF powered open-source Cloud Native Networking solution, providing security, observability, scalability, and superior performance. Cilium is an incubating project under CNCF and the leading CNI for Kubernetes. In this session we will introduce the fundamentals of Cilium Network Policies and the basics of application-aware and Identity-based Security. We will discuss the default-allow and default-deny approaches and visualize the corresponding ingress and egress connections. Using the Network Policy Editor we will be able to demonstrate how a Cilium Network Policy looks like and what they mean on a given Kubernetes cluster. Additionally, we will walk through different examples and demonstrate how application traffic can be observed with Hubble and show how you can use the Network Policy Editor to apply new Cilium Network Policies for your workloads. Finally, we’ll demonstrate how Tetragon provides eBPF-based transparent security observability combined with real-time runtime enforcement.
Functional Operations (Functional Programming at Comcast Labs Connect)Susan Potter
Functional Operations: Packaging, system/configuration building, and testing infrastructure with [Nix] lambda
Maintaining configurations for different kinds of nodes and cloud resources in a [micro]service architecture can be an operational nightmare, especially if not managed with the application codebase. CI and CD job environments diverge from production configuration yielding their results unpredictable at best or produce false positives in the worst case. Code pushes to staging and production can have unintended consequences that can't be reasoned about before deploy and often can’t be inspected thoroughly on a dry run. Leading to unhappy users when problems do arise.
This session will demonstrate the use of the Nix and NixOS ecosystem to define and build packages in a referentially transparent way which can be leveraged as a solid foundation to configure systems and test multiple [virtual] machines with coordinated scenarios. We also look at how reliable packaging allows us to build a consistent CI/CD pipeline where upgrading your version of the JVM doesn't break your CI build servers for days.
This talk will cover the problems currently with why applications are not being sandboxed to lessen the attack surface. Mostly this is based upon the existing tools being not user friendly and requiring a low level knowledge of syscalls that is hard to find in application developers.
Seccomp is one of these tools. It defines syscall filters that allow an application to define what syscalls it allows or denies. It is commonly used in the highly-regarded Chrome sandbox.
Integrating things like seccomp filters into programming languages at build time could allow for creating a perfect set of filters based off the application code. In practice, some try to mock this behavior at runtime but it often fails due to certain functions not being called during testing and missing specific syscalls. Therefore causing the user to turn it off completely. By integrating it into the code at build time we can ensure that all the syscalls are accounted for.
This talk will also show a proof of concept with this in Golang.
Explains how Docker and Nix work as deployment solutions, in what ways they are similar and different, and how they can be combined to achieve interesting results.
Need for Speed: Removing speed bumps from your Symfony projects ⚡️Łukasz Chruściel
No one wants their application to drag like a car stuck in the slow lane! Yet it’s all too common to encounter bumpy, pothole-filled solutions that slow the speed of any application. Symfony apps are not an exception.
In this talk, I will take you for a spin around the performance racetrack. We’ll explore common pitfalls - those hidden potholes on your application that can cause unexpected slowdowns. Learn how to spot these performance bumps early, and more importantly, how to navigate around them to keep your application running at top speed.
We will focus in particular on tuning your engine at the application level, making the right adjustments to ensure that your system responds like a well-oiled, high-performance race car.
Essentials of Automations: The Art of Triggers and Actions in FMESafe Software
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
What is Augmented Reality Image Trackingpavan998932
Augmented Reality (AR) Image Tracking is a technology that enables AR applications to recognize and track images in the real world, overlaying digital content onto them. This enhances the user's interaction with their environment by providing additional information and interactive elements directly tied to physical images.
Software Engineering, Software Consulting, Tech Lead, Spring Boot, Spring Cloud, Spring Core, Spring JDBC, Spring Transaction, Spring MVC, OpenShift Cloud Platform, Kafka, REST, SOAP, LLD & HLD.
Quarkus Hidden and Forbidden ExtensionsMax Andersen
Quarkus has a vast extension ecosystem and is known for its subsonic and subatomic feature set. Some of these features are not as well known, and some extensions are less talked about, but that does not make them less interesting - quite the opposite.
Come join this talk to see some tips and tricks for using Quarkus and some of the lesser known features, extensions and development techniques.
Do you want Software for your Business? Visit Deuglo
Deuglo has top Software Developers in India. They are experts in software development and help design and create custom Software solutions.
Deuglo follows seven steps methods for delivering their services to their customers. They called it the Software development life cycle process (SDLC).
Requirement — Collecting the Requirements is the first Phase in the SSLC process.
Feasibility Study — after completing the requirement process they move to the design phase.
Design — in this phase, they start designing the software.
Coding — when designing is completed, the developers start coding for the software.
Testing — in this phase when the coding of the software is done the testing team will start testing.
Installation — after completion of testing, the application opens to the live server and launches!
Maintenance — after completing the software development, customers start using the software.
Code reviews are vital for ensuring good code quality. They serve as one of our last lines of defense against bugs and subpar code reaching production.
Yet, they often turn into annoying tasks riddled with frustration, hostility, unclear feedback and lack of standards. How can we improve this crucial process?
In this session we will cover:
- The Art of Effective Code Reviews
- Streamlining the Review Process
- Elevating Reviews with Automated Tools
By the end of this presentation, you'll have the knowledge on how to organize and improve your code review proces
AI Pilot Review: The World’s First Virtual Assistant Marketing SuiteGoogle
AI Pilot Review: The World’s First Virtual Assistant Marketing Suite
👉👉 Click Here To Get More Info 👇👇
https://sumonreview.com/ai-pilot-review/
AI Pilot Review: Key Features
✅Deploy AI expert bots in Any Niche With Just A Click
✅With one keyword, generate complete funnels, websites, landing pages, and more.
✅More than 85 AI features are included in the AI pilot.
✅No setup or configuration; use your voice (like Siri) to do whatever you want.
✅You Can Use AI Pilot To Create your version of AI Pilot And Charge People For It…
✅ZERO Manual Work With AI Pilot. Never write, Design, Or Code Again.
✅ZERO Limits On Features Or Usages
✅Use Our AI-powered Traffic To Get Hundreds Of Customers
✅No Complicated Setup: Get Up And Running In 2 Minutes
✅99.99% Up-Time Guaranteed
✅30 Days Money-Back Guarantee
✅ZERO Upfront Cost
See My Other Reviews Article:
(1) TubeTrivia AI Review: https://sumonreview.com/tubetrivia-ai-review
(2) SocioWave Review: https://sumonreview.com/sociowave-review
(3) AI Partner & Profit Review: https://sumonreview.com/ai-partner-profit-review
(4) AI Ebook Suite Review: https://sumonreview.com/ai-ebook-suite-review
Hand Rolled Applicative User ValidationCode KataPhilip Schwarz
Could you use a simple piece of Scala validation code (granted, a very simplistic one too!) that you can rewrite, now and again, to refresh your basic understanding of Applicative operators <*>, <*, *>?
The goal is not to write perfect code showcasing validation, but rather, to provide a small, rough-and ready exercise to reinforce your muscle-memory.
Despite its grandiose-sounding title, this deck consists of just three slides showing the Scala 3 code to be rewritten whenever the details of the operators begin to fade away.
The code is my rough and ready translation of a Haskell user-validation program found in a book called Finding Success (and Failure) in Haskell - Fall in love with applicative functors.
Utilocate offers a comprehensive solution for locate ticket management by automating and streamlining the entire process. By integrating with Geospatial Information Systems (GIS), it provides accurate mapping and visualization of utility locations, enhancing decision-making and reducing the risk of errors. The system's advanced data analytics tools help identify trends, predict potential issues, and optimize resource allocation, making the locate ticket management process smarter and more efficient. Additionally, automated ticket management ensures consistency and reduces human error, while real-time notifications keep all relevant personnel informed and ready to respond promptly.
The system's ability to streamline workflows and automate ticket routing significantly reduces the time taken to process each ticket, making the process faster and more efficient. Mobile access allows field technicians to update ticket information on the go, ensuring that the latest information is always available and accelerating the locate process. Overall, Utilocate not only enhances the efficiency and accuracy of locate ticket management but also improves safety by minimizing the risk of utility damage through precise and timely locates.
Takashi Kobayashi and Hironori Washizaki, "SWEBOK Guide and Future of SE Education," First International Symposium on the Future of Software Engineering (FUSE), June 3-6, 2024, Okinawa, Japan
Artificia Intellicence and XPath Extension FunctionsOctavian Nadolu
The purpose of this presentation is to provide an overview of how you can use AI from XSLT, XQuery, Schematron, or XML Refactoring operations, the potential benefits of using AI, and some of the challenges we face.
May Marketo Masterclass, London MUG May 22 2024.pdfAdele Miller
Can't make Adobe Summit in Vegas? No sweat because the EMEA Marketo Engage Champions are coming to London to share their Summit sessions, insights and more!
This is a MUG with a twist you don't want to miss.
OpenMetadata Community Meeting - 5th June 2024OpenMetadata
The OpenMetadata Community Meeting was held on June 5th, 2024. In this meeting, we discussed about the data quality capabilities that are integrated with the Incident Manager, providing a complete solution to handle your data observability needs. Watch the end-to-end demo of the data quality features.
* How to run your own data quality framework
* What is the performance impact of running data quality frameworks
* How to run the test cases in your own ETL pipelines
* How the Incident Manager is integrated
* Get notified with alerts when test cases fail
Watch the meeting recording here - https://www.youtube.com/watch?v=UbNOje0kf6E
Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...Mind IT Systems
Healthcare providers often struggle with the complexities of chronic conditions and remote patient monitoring, as each patient requires personalized care and ongoing monitoring. Off-the-shelf solutions may not meet these diverse needs, leading to inefficiencies and gaps in care. It’s here, custom healthcare software offers a tailored solution, ensuring improved care and effectiveness.
openEuler Case Study - The Journey to Supply Chain Security
Hydra: Continuous Integration and Testing for Demanding People: The Details
1. Hydra: Continuous Integration and Testing for
Demanding People: The Details
Sander van der Burg
Conference Compass
July 15, 2014
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
2. Continuous integration
We want to deliver and test software rapidly
We quickly want to see the impact of changes to the source
code and its dependencies.
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
3. Hydra
Hydra: A Nix-based continuous integration server:
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
4. Hydra
Hydra: A Nix-based continuous integration server:
Generic. Supports multiple programming language
environments and component technologies.
Deployment. Build and test environments are deployed
automatically and all dependencies are ensured to be present
and correct.
Variability. Multiple versions/variants of dependencies can
safely coexist.
Multi platform support. Builds can be easily delegated to
machines with a different operating system.
Scalability. Builds are transparently delegated to any machine
in a cluster capable of building it.
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
5. Hydra
How to use Hydra to build or test stuff?
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
7. Hydra overview
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
Components
Queue runner: Regularly checks what has changed and
what to build
Evaluator: Builds the jobs
Server: Web application making builds and test results
available
Nix: Package mananger responsible for the actual
builds and depedency management
9. The Nix package manager
A package manager borrowing concepts from purely functional
programming languages.
x = y ⇒ f (x) = f (y)
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
10. Nix store
Main idea: store all packages
in isolation from each other:
/nix/store/40awryfqzp46m...
-disnix-0.3
Paths contain a 160-bit
cryptographic hash of all
inputs used to build the
package:
Sources
Libraries
Compilers
Build scripts
. . .
/nix/store
40awryfqzp...-disnix-0.3
bin
disnix-env
disnix-manifest
disnix-service
kjlv4klmra...-getopt-1.1.4
bin
getopt
am13rq9ka...-dbus-glib-0.102
lib
libdbus-glib-1.so.2
94n64qy99...-glibc-2.19
lib
libc.so.6
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
11. Nix expressions
disnix.nix
{ stdenv, fetchurl, pkgconfig, dbus_glib
, libxml2, libxslt, getopt, nix, dysnomia }:
stdenv.mkDerivation {
name = "disnix-0.3";
src = fetchurl {
url = http://.../disnix-0.3.tar.bz2;
sha256 = "1jjmzdd7fac6isq5wdaqjbwwnsnzjag5s4...";
};
buildInputs = [ pkgconfig dbus_glib libxml2 libxslt
getopt nix dysnomia ];
buildCommand = ’’
tar xjf $src
./configure --prefix=$out
make; make install
’’;
}
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
12. Nix expressions
disnix.nix
{ stdenv, fetchurl, pkgconfig, dbus_glib
, libxml2, libxslt, getopt, nix, dysnomia }:
stdenv.mkDerivation {
name = "disnix-0.3";
src = fetchurl {
url = http://.../disnix-0.3.tar.bz2;
sha256 = "1jjmzdd7fac6isq5wdaqjbwwnsnzjag5s4...";
};
buildInputs = [ pkgconfig dbus_glib libxml2 libxslt
getopt nix dysnomia ];
buildCommand = ’’
tar xjf $src
./configure --prefix=$out
make; make install
’’;
}
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
Environments
Expression defines a function that composes an
environment in which a build is executed
Nearly any type of build can be performed inside it,
e.g. C/C++, Java, Perl, Python...
We can also run tests inside these environments
The buildInputs parameters are used to configure all
settings to make a build find its dependencies, e.g.
setting PATH, PYTHONPATH, CLASSPATH ...
There are also function abstractions for different kinds
of packages
If no buildCommand is given, it executes the default
GNU Autotools build procedure: ./configure;
make; make install.
14. Nix expressions
all-packages.nix
{system ? builtins.currentSystem}:
rec {
stdenv = ... { inherit system; };
fetchurl = ...;
pkgconfig = ...;
dbus_glib = ...;
libxml2 = ...;
libxslt = ...;
getopt = ...;
nix = callPackage ../pkgs/tools/package-management/nix { };
dysnomia = import ../pkgs/tools/package-management/dysnomia {
inherit stdenv fetchurl getopt;
};
disnix = import ../pkgs/tools/package-management/disnix {
inherit stdenv fetchurl pkgconfig dbus_glib;
inherit libxml2 libxslt getopt nix dysnomia;
};
...
}
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
Environments
Composes packages by calling them with the required
function arguments.
Function invocations are lazy – they are only evaluated
if needed.
Previous expression for Disnix that defines a function
is imported here.
All dependencies are composed in the same expression
as well.
15. Building Nix expressions
Building a Nix package:
$ nix-build all-packages.nix -A disnix
/nix/store/40awryfqzp46mjzm1rwy9qa8vxscjhgx-disnix-0.3
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
16. Building Nix expressions
Building a Nix package:
$ nix-build all-packages.nix -A disnix
/nix/store/40awryfqzp46mjzm1rwy9qa8vxscjhgx-disnix-0.3
The Nix package manager builds disnix and all its
dependencies that have not been built yet.
Hash component is derived from all build inputs used to build
the package.
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
20. Building Nix expressions
During a build of package many side effects are removed:
Most environment variables are initially cleared or set to
dummy values, such as PATH.
Environment variables, such as PATH, are configured to only
contain the specified dependencies.
Nix store paths prevent packages to be implicitly found in
many cases (unlike “traditional” systems using /usr/lib,
/usr/bin or C:WINDOWSSystem32).
Timestamps are set to 1 second after the epoch
Files in the Nix store are made read-only.
Optionally, builds can be performed in a chroot()
environment, improving purity
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
21. User environments
Users can have
different sets of
installed applications.
PATH
/nix/.../profiles
current
42
/nix/store
pp56i0a01si5...-user-env
bin
firefox
disnix-env
b9w6q73mqm...-disnix-0.2
bin
disnix-env
mr8f62946...-firefox-30.0
bin
firefox
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
22. User environments
Users can have
different sets of
installed applications.
nix-env operations
create new user
environments in the
store.
PATH
/nix/.../profiles
current
42
/nix/store
pp56i0a01si5...-user-env
bin
firefox
disnix-env
b9w6q73mqm...-disnix-0.2
bin
disnix-env
mr8f62946...-firefox-30.0
bin
firefox
40awryfq...-disnix-0.3
bin
disnix-env
(nix-env -u disnix)
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
23. User environments
Users can have
different sets of
installed applications.
nix-env operations
create new user
environments in the
store.
PATH
/nix/.../profiles
current
42
/nix/store
pp56i0a01si5...-user-env
bin
firefox
disnix-env
b9w6q73mqm...-disnix-0.2
bin
disnix-env
mr8f62946...-firefox-30.0
bin
firefox
40awryfq...-disnix-0.3
bin
disnix-env
i3d9vh6d8ip1...-user-env
bin
disnix-env
firefox
(nix-env -u disnix)
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
24. User environments
Users can have
different sets of
installed applications.
nix-env operations
create new user
environments in the
store.
PATH
/nix/.../profiles
current
42
43
/nix/store
pp56i0a01si5...-user-env
bin
firefox
disnix-env
b9w6q73mqm...-disnix-0.2
bin
disnix-env
mr8f62946...-firefox-30.0
bin
firefox
40awryfq...-disnix-0.3
bin
disnix-env
i3d9vh6d8ip1...-user-env
bin
disnix-env
firefox
(nix-env -u disnix)
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
25. User environments
Users can have
different sets of
installed applications.
nix-env operations
create new user
environments in the
store.
We can atomically
switch between them.
PATH
/nix/.../profiles
current
42
43
/nix/store
pp56i0a01si5...-user-env
bin
firefox
disnix-env
b9w6q73mqm...-disnix-0.2
bin
disnix-env
mr8f62946...-firefox-30.0
bin
firefox
40awryfq...-disnix-0.3
bin
disnix-env
i3d9vh6d8ip1...-user-env
bin
disnix-env
firefox
(nix-env -u disnix)
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
26. User environments
Users can have
different sets of
installed applications.
nix-env operations
create new user
environments in the
store.
We can atomically
switch between them.
These are roots of the
garbage collector.
PATH
/nix/.../profiles
current
43
/nix/store
pp56i0a01si5...-user-env
bin
firefox
disnix-env
b9w6q73mqm...-disnix-0.2
bin
disnix-env
mr8f62946...-firefox-30.0
bin
firefox
40awryfq...-disnix-0.3
bin
disnix-env
i3d9vh6d8ip1...-user-env
bin
disnix-env
firefox
(nix-env --remove-generations old)
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
27. User environments
Users can have
different sets of
installed applications.
nix-env operations
create new user
environments in the
store.
We can atomically
switch between them.
These are roots of the
garbage collector.
PATH
/nix/.../profiles
current
43
/nix/store
mr8f62946...-firefox-30.0
bin
firefox
40awryfq...-disnix-0.3
bin
disnix-env
i3d9vh6d8ip1...-user-env
bin
disnix-env
firefox
(nix-collect-garbage)
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
28. Hydra expression
release.nix
{ nixpkgs ? <nixpkgs>, systems ? [ "x86_64-linux" "x86_64-darwin" ]
, dysnomia ? { outPath = ./.; rev = 1234; } }:
let pkgs = import nixpkgs {}; in
rec {
tarball = pkgs.releaseTools.sourceTarball {
name = "dysnomia-tarball";
version = builtins.readFile ./version;
src = dysnomia;
buildInputs = [ pkgs.getopt ];
};
build = pkgs.lib.genAttrs systems (system:
let pkgs = import nixpkgs { inherit system; }; in
pkgs.releaseTools.nixBuild {
name = "dysnomia";
version = builtins.readFile ./version;
src = tarball;
buildInputs = [ pkgs.getopt ];
});
...
}
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
29. Hydra expression
release.nix
{ nixpkgs ? <nixpkgs>, systems ? [ "x86_64-linux" "x86_64-darwin" ]
, dysnomia ? { outPath = ./.; rev = 1234; } }:
let pkgs = import nixpkgs {}; in
rec {
tarball = pkgs.releaseTools.sourceTarball {
name = "dysnomia-tarball";
version = builtins.readFile ./version;
src = dysnomia;
buildInputs = [ pkgs.getopt ];
};
build = pkgs.lib.genAttrs systems (system:
let pkgs = import nixpkgs { inherit system; }; in
pkgs.releaseTools.nixBuild {
name = "dysnomia";
version = builtins.readFile ./version;
src = tarball;
buildInputs = [ pkgs.getopt ];
});
...
}
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
Jobs
An Hydra expression is a function returing an attribute
set: rec{attr1 = value1; ...; attrn = valuen; }
Function parameters define variability points:
Locations of the Dysnomia, Nixpkgs collection Git
repositories
Target system architectures
Each attribute corresponds to a job.
Each value refers to a function performing a build or
test.
File is typically placed in the root folder of a source
package.
30. Building jobs from the command-line
Building a source tarball:
$ nix-build release.nix -A tarball
Building Dysnomia for 64-bit AMD Linux:
$ nix-build release.nix -A build.x86 64-linux
Building Dysnomia for Mac OS X (Nix delegates the build to
a Mac machine if the build is run on Linux and an external
machine is configured):
$ nix-build release.nix -A build.x86 64-darwin
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
31. Building jobs from Hydra
Create a project:
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
32. Building jobs from Hydra
Create a jobset:
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
33. Building jobs from Hydra
Create a jobset:
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
Jobs
All (but one) inputs are provided as function
arguments to the release expression.
One input is the package itself (dysnomia) that
contains the release.nix expression
34. Building jobs from Hydra
Evaluation results (job names correspond to those defined in
release.nix):
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
35. Hydra expression referring to other jobsets
release.nix
{ nixpkgs ? <nixpkgs>, systems ? [ "x86_64-linux" "x86_64-darwin" ]
, dysnomiaJobset ?
import ../dysnomia/release.nix { inherit nixpkgs systems; }
, disnix ? { outPath = ./.; rev = 1234; } }:
let pkgs = import nixpkgs {}; in
rec {
tarball = ...
build = pkgs.lib.genAttrs systems (system:
let dysnomia = builtins.getAttr system (dysnomiaJobset.build); in
with import nixpkgs { inherit system; };
releaseTools.nixBuild {
name = "disnix";
src = tarball;
buildInputs = [ pkgconfig dbus_glib libxml2 libxslt
getopt nix dysnomia ];
};
...
}
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
36. Building jobs from Hydra
Use an input of type: ’Previous Hydra evaluation’:
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
37. Hydra jobs I typically write
Source packages. Jobs that assemble tarballs, Zip files or
other archives containing the source code.
Binary packages. The actual builds for a variety of
architectures, such as i686-linux, x86 64-linux,
x86 64-darwin.
Program manuals. For example, building the manual from
Docbook.
Program documentation catalogs. Generating a
documentation catalog from the source code, e.g. using
javadoc, doxygen or JSDuck.
Unit tests. Running Unit tests, for example with JUnit or
mocha and producing a coverage report.
System integration tests. Composing NixOS Linux VMs with
all environmental dependencies, e.g. DBMS, web server etc,
and run tests inside them.
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
38. Nix channel
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
39. Nix channel
Adding a channel:
$ nix-channel --add http://localhost/jobset/disnix/disnix-master/channel/latest
$ nix-channel --update
When running:
$ nix-env -i disnix
installing ‘disnix-0.3pre174e883b7b09da822494876d2f297736f33707a7’
these paths will be fetched (0.31 MiB download, 0.91 MiB unpacked):
/nix/store/70rkq38r69fwrz90ayc4fyg823z92nmf-disnix-0.3
fetching path ‘/nix/store/70rkq38r69fwrz90ayc4fyg823z92nmf-disnix-0.3’...
The build gets downloaded from the Hydra server, instead of being
built from source code.
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
40. Nix package manager: Exercises
Check it out yourself!!!
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
41. Availability
Nix and Hydra are available as free and open source software under
the LGPLv2 and the GPLv3 licenses:
Nix: http://nixos.org/nix
Hydra: http://nixos.org/hydra
NixOS’ Hydra server: http://hydra.nixos.org
Nix can be used on any Linux distribution, NixOS, Mac OS X,
FreeBSD, and Windows (through Cygwin)
Hydra can be used on any Linux distribution
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People
42. Related work
Using Nix while doing development:
Deploy development packages and composing an environment
in which they can be found
NixOS: http://nixos.org/nixos
Deploy an entire system configuration (Linux distribution) with
Nix.
System integration testing with NixOS
Efficiently compose networks of NixOS machines within a build
in which system integration tests can be performed
Disnix: http://nixos/disnix
(Re)deploy service-oriented systems into networks of machines
NixOps: http://nixos/nixops
Deploy networks of NixOS configurations to physical machines
or into the cloud
Automatically creates VM instances if needed
NiJS: https://www.npmjs.org/package/nijs
Compose Nix packages in JavaScript
Invoke JavaScript functions from Nix expressions
Very primitive stand-alone package manager
Sander van der Burg Hydra: Continuous Integration and Testing for Demanding People