Explains how Docker and Nix work as deployment solutions, in what ways they are similar and different, and how they can be combined to achieve interesting results.
nix-processmgmt: An experimental Nix-based process manager-agnostic frameworkSander van der Burg
NixCon 2020 talk about an experimental framework that integrates the Nix package manager with all kinds of process managers, such as : sysvinit, systemd, launchd, and even Docker
Small, Simple, and Secure: Alpine Linux under the MicroscopeDocker, Inc.
Alpine Linux is a distro that has become popular for Docker images. Why do we need another distro? Why does Alpine matter? How does it differ from other distros?
In this talk, we'll answer all these questions – and a few more.
nix-processmgmt: An experimental Nix-based process manager-agnostic frameworkSander van der Burg
NixCon 2020 talk about an experimental framework that integrates the Nix package manager with all kinds of process managers, such as : sysvinit, systemd, launchd, and even Docker
Small, Simple, and Secure: Alpine Linux under the MicroscopeDocker, Inc.
Alpine Linux is a distro that has become popular for Docker images. Why do we need another distro? Why does Alpine matter? How does it differ from other distros?
In this talk, we'll answer all these questions – and a few more.
Containerization helps us bundle dependencies with applications instead of having to use configuration management to prepare machines for running them, hence making build once run anywhere easy. For legacy applications this can be quite hard though when they spread persistent data across the file system.
In this talk I'll show how we can quickly set up a Go.CD server and agents for our Continuous Delivery pipelines on Google Cloud. The infrastructure creation is handled by Terraform, the server and agents are custom built Docker containers.
Build Your Own CaaS (Container as a Service)HungWei Chiu
In this slide, I introduce the kubernetes and show an example what is CaaS and what it can provides.
Besides, I also introduce how to setup a continuous integration and continuous deployment for the CaaS platform.
The last decade belonged to virtual machines and the next one belongs to containers. CoreOS is a new Linux distribution designed specifically for application containers and running them at scale. This talk will examine all the major components of CoreOS (etcd, fleet, docker, systemd) and how these components work together.
15 kubernetes failure points you should watchSysdig
When operating a production platform we should prepare for failure and in addition to monitoring working metrics, we cannot forget about the most common failure points. From monitoring solution agnostic perspective, and following a use-case driven approach, we will learn the most common failure points in a Kubernetes infrastructure and how to detect them (metrics, events, checks, etc).
Networking and Go: An Engineer's Journey (Strangeloop 2019)Sneha Inguva
My foray into Go began a few years ago when I started working at DigitalOcean. While building an abstraction layer on top of Kubernetes and familiarizing myself with the language, I began to love it. Syntactically simple, with amazing concurrency primitives and a wonderful community, Go was an excellent choice for a cloud-hosting company with a variety of low-level, server-side microservices.
In the last year, however, I've joined the software-defined networking team and learned of another application of Go; networking services. The networking team at DigitalOcean uses Go for a variety of purposes - from DHCP servers to IP address management services..to even wrappers around virtual switch tooling. Intrigued, I decided to also investigate how Go could be used to build other services such as port scanners and load-balancers.
This session will highlight my networking journey via Go. I will discuss useful packages, key learnings, and even struggles faced while building a variety networking services within and outside of DigitalOcean. I will discuss both relevant packages within the standard library and open source packages used to implement key network protocols. As a result, listeners will gain an understanding of how to specifically leverage Go for their own networking needs.
AtlasCamp 2015: The age of orchestration: From Docker basics to cluster manag...Atlassian
Nicola Paolucci, Atlassian
Containers hit the collective developer mind with great force the past two years and created a space of fervent innovation. Now work is moving towards orchestration. In this session we'll cover an overview of the container orchestration landscape, give an introduction to Docker's own tools - machine, swarm and compose - and show a (semi)live demo of how they work in practice.
What Have Syscalls Done for you Lately?Docker, Inc.
If you've ever written any code - even just Hello World - you've used some syscalls. In this talk we'll explore what syscalls are, how they are used to set up containers, and how to make your deployment more secure at runtime by limiting the syscalls your containers can make thanks to seccomp and Linux security modules like AppArmor.
We'll also discuss how, if your architecture is broken into containerized microservices, this gives you a great opportunity to improve security by limiting what each container can do. This is where containerized microservices really shine over traditional monoliths from a security perspective - so it's helpful to know about if you're trying to convince your security team that containers are a good idea.
There will be lots of live demos!
Scaling Development Environments with DockerDocker, Inc.
We set out to solve the problems of quickly building high quality games for a fragmented mobile market. Taking advantage of HTML5 allowed a fast, familiar and highly iterative local development process, and a hybrid build process for native apps meant high performance games on mobile. Our product is designed to comprehensively handle complex UI flows, related server tasks as well as deep integrations with any social platform. This is necessarily complex piece of engineering, with dozens of large dependencies, and 5 local web servers powering a single user’s experience. When we set out to make this easily available to 3rd parties, we used Docker to solve to major challenges: 1) Fitting many users, each with a unique development environment, on to one machine; 2) Managing all of these development environments in a scalable way.
Invited to introduce Docker to the Dept. for Information and Communication Services (Informations- und Kommunikationsdienste - IuK) at the University of Rostock.
Containerization helps us bundle dependencies with applications instead of having to use configuration management to prepare machines for running them, hence making build once run anywhere easy. For legacy applications this can be quite hard though when they spread persistent data across the file system.
In this talk I'll show how we can quickly set up a Go.CD server and agents for our Continuous Delivery pipelines on Google Cloud. The infrastructure creation is handled by Terraform, the server and agents are custom built Docker containers.
Build Your Own CaaS (Container as a Service)HungWei Chiu
In this slide, I introduce the kubernetes and show an example what is CaaS and what it can provides.
Besides, I also introduce how to setup a continuous integration and continuous deployment for the CaaS platform.
The last decade belonged to virtual machines and the next one belongs to containers. CoreOS is a new Linux distribution designed specifically for application containers and running them at scale. This talk will examine all the major components of CoreOS (etcd, fleet, docker, systemd) and how these components work together.
15 kubernetes failure points you should watchSysdig
When operating a production platform we should prepare for failure and in addition to monitoring working metrics, we cannot forget about the most common failure points. From monitoring solution agnostic perspective, and following a use-case driven approach, we will learn the most common failure points in a Kubernetes infrastructure and how to detect them (metrics, events, checks, etc).
Networking and Go: An Engineer's Journey (Strangeloop 2019)Sneha Inguva
My foray into Go began a few years ago when I started working at DigitalOcean. While building an abstraction layer on top of Kubernetes and familiarizing myself with the language, I began to love it. Syntactically simple, with amazing concurrency primitives and a wonderful community, Go was an excellent choice for a cloud-hosting company with a variety of low-level, server-side microservices.
In the last year, however, I've joined the software-defined networking team and learned of another application of Go; networking services. The networking team at DigitalOcean uses Go for a variety of purposes - from DHCP servers to IP address management services..to even wrappers around virtual switch tooling. Intrigued, I decided to also investigate how Go could be used to build other services such as port scanners and load-balancers.
This session will highlight my networking journey via Go. I will discuss useful packages, key learnings, and even struggles faced while building a variety networking services within and outside of DigitalOcean. I will discuss both relevant packages within the standard library and open source packages used to implement key network protocols. As a result, listeners will gain an understanding of how to specifically leverage Go for their own networking needs.
AtlasCamp 2015: The age of orchestration: From Docker basics to cluster manag...Atlassian
Nicola Paolucci, Atlassian
Containers hit the collective developer mind with great force the past two years and created a space of fervent innovation. Now work is moving towards orchestration. In this session we'll cover an overview of the container orchestration landscape, give an introduction to Docker's own tools - machine, swarm and compose - and show a (semi)live demo of how they work in practice.
What Have Syscalls Done for you Lately?Docker, Inc.
If you've ever written any code - even just Hello World - you've used some syscalls. In this talk we'll explore what syscalls are, how they are used to set up containers, and how to make your deployment more secure at runtime by limiting the syscalls your containers can make thanks to seccomp and Linux security modules like AppArmor.
We'll also discuss how, if your architecture is broken into containerized microservices, this gives you a great opportunity to improve security by limiting what each container can do. This is where containerized microservices really shine over traditional monoliths from a security perspective - so it's helpful to know about if you're trying to convince your security team that containers are a good idea.
There will be lots of live demos!
Scaling Development Environments with DockerDocker, Inc.
We set out to solve the problems of quickly building high quality games for a fragmented mobile market. Taking advantage of HTML5 allowed a fast, familiar and highly iterative local development process, and a hybrid build process for native apps meant high performance games on mobile. Our product is designed to comprehensively handle complex UI flows, related server tasks as well as deep integrations with any social platform. This is necessarily complex piece of engineering, with dozens of large dependencies, and 5 local web servers powering a single user’s experience. When we set out to make this easily available to 3rd parties, we used Docker to solve to major challenges: 1) Fitting many users, each with a unique development environment, on to one machine; 2) Managing all of these development environments in a scalable way.
Invited to introduce Docker to the Dept. for Information and Communication Services (Informations- und Kommunikationsdienste - IuK) at the University of Rostock.
AtlasCamp 2015 Docker continuous integration trainingSteve Smith
A 2-hour training session delivered at AtlasCamp in Prague, June 9th 2015.
* Docker vs virtual machines
* Docker concepts
* Docker for testing
* Automation with Docker Compose
* Continuous integration with Bamboo Docker support
* Extracting test results from Docker containers
* Continuous deployment with deployment environments
Использование Docker в CI / Александр Акбашев (HERE Technologies)Ontico
РИТ++ 2017, Root Conf
Зал Пекин + Шанхай, 6 июня, 17:00
Тезисы:
http://rootconf.ru/2017/abstracts/2504.html
В своём докладе я расскажу о том, почему мы решили использовать Docker в рамках Continuous Integration: ускорить тесты, повысить стабильность, улучшить контроль над окружением и используемыми библиотеками.
Доклад так же содержит подробности о многих сложностях, с которыми пришлось столкнуться в ходе миграции на Docker: борьба с растущим числом и размером образов, бесконтрольные обновления образов, нестабильное поведение, и другие.
В конце доклада я покажу, как именно мы следим за стабильностью Docker в нашей инфраструктуре. И насколько Docker стабилен на больших объемах (больше 100k билдов в сутки).
Instrumentación de entrega continua con GitlabSoftware Guru
Mostraremos el caso real de cómo tenemos implementado en nuestra empresa el flujo de desarrollo para integración y entrega continua, instrumentado con GitLab.
Sesión presentada por David Padilla en SG Next 2017
Docker Practice for beginner.
- docker install on ubuntu 18.04 LTS
- docker pull/push
- making docker-compose file which serving spring-boot+ mySql application
DCEU 18: Tips and Tricks of the Docker CaptainsDocker, Inc.
Brandon Mitchell - Solutions Architect, BoxBoat
Docker Captain Brandon Mitchell will help you accelerate your adoption of Docker containers by delivering tips and tricks on getting the most out of Docker. Topics include managing disk usage, preventing subnet collisions, debugging container networking, understanding image layers, getting more value out of the default volume driver, and solving the UID/GID permission issues with volumes in a way that allows images to be portable from any developer laptop and to production.
Streamline your development environment with dockerGiacomo Bagnoli
These days applications are getting more and more complex. It's becoming quite
difficult to keep track of all the different components an application needs in order to
function (a database, a message queueing system, a web server, a document
store, a search engine, you name it.). How many times we heard 'it worked on my
machine'?. In this talk we are going to explore Docker, what it is, how it works
and how much it can benefit in keeping the development environment consistent.
We are going to talk about Dockerfiles, best practices, tools like fig and vagrant,
and finally show an example of how it applies to a ruby on rails
application.
Be a better developer with Docker (revision 3)Nicola Paolucci
Be a better developer with Docker: tricks of the trade (revision 3)
The talk will teach developers how to approach their development environment setups using Docker, covering awesome tricks to make the experience smooth, fast, powerful and repeatable. The talk is logically divided in five parts:
- What is Docker
- Why Docker makes developers happier
- Workflows and techniques
- Tips and tricks
- Future developments
AWS re:Invent 2016: Amazon ECR Deep Dive on Image Optimization (CON401)Amazon Web Services
“Are you struggling with bulky images or slow push and pull times? In this session we will walk through the anatomy of a Docker image and provide techniques you can use to optimize images for faster pushes and pulls and reduce your overall storage footprint. We will discuss Docker image building (build containers versus runtime containers to remove unnecessary software), Docker image composition (minimizing the number of layers), the Docker Remote API (optimizing how images are pushed and pulled), and CI/CD Integration (automate building, versioning, and deploying images to production). We’ll also examine the tools that ECR provides to make Docker image management easier so that you can focus on building your application. Finally, we'll hear from Pinterest about how they use ECR and Docker, some valuable experiences gained along the way, and best practices for using ECR with Apache Mesos.”
Cette présentation vous montrera comment utiliser et profiter rapidement de Docker, quelles commandes utiliser et quelles fonctionnalités sont disponibles.
sfPot de Lille - Le 15 janvier 2015
Similar to Using Nix and Docker as automated deployment solutions (20)
Dysnomia: complementing Nix deployments with state deploymentSander van der Burg
This talk covers Dysnomia, a state deployment tool that complements various tools in the Nix project, such as NixOS and Disnix, with state management facilities.
Check out the webinar slides to learn more about how XfilesPro transforms Salesforce document management by leveraging its world-class applications. For more details, please connect with sales@xfilespro.com
If you want to watch the on-demand webinar, please click here: https://www.xfilespro.com/webinars/salesforce-document-management-2-0-smarter-faster-better/
Providing Globus Services to Users of JASMIN for Environmental Data AnalysisGlobus
JASMIN is the UK’s high-performance data analysis platform for environmental science, operated by STFC on behalf of the UK Natural Environment Research Council (NERC). In addition to its role in hosting the CEDA Archive (NERC’s long-term repository for climate, atmospheric science & Earth observation data in the UK), JASMIN provides a collaborative platform to a community of around 2,000 scientists in the UK and beyond, providing nearly 400 environmental science projects with working space, compute resources and tools to facilitate their work. High-performance data transfer into and out of JASMIN has always been a key feature, with many scientists bringing model outputs from supercomputers elsewhere in the UK, to analyse against observational or other model data in the CEDA Archive. A growing number of JASMIN users are now realising the benefits of using the Globus service to provide reliable and efficient data movement and other tasks in this and other contexts. Further use cases involve long-distance (intercontinental) transfers to and from JASMIN, and collecting results from a mobile atmospheric radar system, pushing data to JASMIN via a lightweight Globus deployment. We provide details of how Globus fits into our current infrastructure, our experience of the recent migration to GCSv5.4, and of our interest in developing use of the wider ecosystem of Globus services for the benefit of our user community.
Enhancing Project Management Efficiency_ Leveraging AI Tools like ChatGPT.pdfJay Das
With the advent of artificial intelligence or AI tools, project management processes are undergoing a transformative shift. By using tools like ChatGPT, and Bard organizations can empower their leaders and managers to plan, execute, and monitor projects more effectively.
Understanding Globus Data Transfers with NetSageGlobus
NetSage is an open privacy-aware network measurement, analysis, and visualization service designed to help end-users visualize and reason about large data transfers. NetSage traditionally has used a combination of passive measurements, including SNMP and flow data, as well as active measurements, mainly perfSONAR, to provide longitudinal network performance data visualization. It has been deployed by dozens of networks world wide, and is supported domestically by the Engagement and Performance Operations Center (EPOC), NSF #2328479. We have recently expanded the NetSage data sources to include logs for Globus data transfers, following the same privacy-preserving approach as for Flow data. Using the logs for the Texas Advanced Computing Center (TACC) as an example, this talk will walk through several different example use cases that NetSage can answer, including: Who is using Globus to share data with my institution, and what kind of performance are they able to achieve? How many transfers has Globus supported for us? Which sites are we sharing the most data with, and how is that changing over time? How is my site using Globus to move data internally, and what kind of performance do we see for those transfers? What percentage of data transfers at my institution used Globus, and how did the overall data transfer performance compare to the Globus users?
In software engineering, the right architecture is essential for robust, scalable platforms. Wix has undergone a pivotal shift from event sourcing to a CRUD-based model for its microservices. This talk will chart the course of this pivotal journey.
Event sourcing, which records state changes as immutable events, provided robust auditing and "time travel" debugging for Wix Stores' microservices. Despite its benefits, the complexity it introduced in state management slowed development. Wix responded by adopting a simpler, unified CRUD model. This talk will explore the challenges of event sourcing and the advantages of Wix's new "CRUD on steroids" approach, which streamlines API integration and domain event management while preserving data integrity and system resilience.
Participants will gain valuable insights into Wix's strategies for ensuring atomicity in database updates and event production, as well as caching, materialization, and performance optimization techniques within a distributed system.
Join us to discover how Wix has mastered the art of balancing simplicity and extensibility, and learn how the re-adoption of the modest CRUD has turbocharged their development velocity, resilience, and scalability in a high-growth environment.
Globus Compute wth IRI Workflows - GlobusWorld 2024Globus
As part of the DOE Integrated Research Infrastructure (IRI) program, NERSC at Lawrence Berkeley National Lab and ALCF at Argonne National Lab are working closely with General Atomics on accelerating the computing requirements of the DIII-D experiment. As part of the work the team is investigating ways to speedup the time to solution for many different parts of the DIII-D workflow including how they run jobs on HPC systems. One of these routes is looking at Globus Compute as a way to replace the current method for managing tasks and we describe a brief proof of concept showing how Globus Compute could help to schedule jobs and be a tool to connect compute at different facilities.
TROUBLESHOOTING 9 TYPES OF OUTOFMEMORYERRORTier1 app
Even though at surface level ‘java.lang.OutOfMemoryError’ appears as one single error; underlyingly there are 9 types of OutOfMemoryError. Each type of OutOfMemoryError has different causes, diagnosis approaches and solutions. This session equips you with the knowledge, tools, and techniques needed to troubleshoot and conquer OutOfMemoryError in all its forms, ensuring smoother, more efficient Java applications.
Listen to the keynote address and hear about the latest developments from Rachana Ananthakrishnan and Ian Foster who review the updates to the Globus Platform and Service, and the relevance of Globus to the scientific community as an automation platform to accelerate scientific discovery.
Into the Box Keynote Day 2: Unveiling amazing updates and announcements for modern CFML developers! Get ready for exciting releases and updates on Ortus tools and products. Stay tuned for cutting-edge innovations designed to boost your productivity.
Large Language Models and the End of ProgrammingMatt Welsh
Talk by Matt Welsh at Craft Conference 2024 on the impact that Large Language Models will have on the future of software development. In this talk, I discuss the ways in which LLMs will impact the software industry, from replacing human software developers with AI, to replacing conventional software with models that perform reasoning, computation, and problem-solving.
SOCRadar Research Team: Latest Activities of IntelBrokerSOCRadar
The European Union Agency for Law Enforcement Cooperation (Europol) has suffered an alleged data breach after a notorious threat actor claimed to have exfiltrated data from its systems. Infamous data leaker IntelBroker posted on the even more infamous BreachForums hacking forum, saying that Europol suffered a data breach this month.
The alleged breach affected Europol agencies CCSE, EC3, Europol Platform for Experts, Law Enforcement Forum, and SIRIUS. Infiltration of these entities can disrupt ongoing investigations and compromise sensitive intelligence shared among international law enforcement agencies.
However, this is neither the first nor the last activity of IntekBroker. We have compiled for you what happened in the last few days. To track such hacker activities on dark web sources like hacker forums, private Telegram channels, and other hidden platforms where cyber threats often originate, you can check SOCRadar’s Dark Web News.
Stay Informed on Threat Actors’ Activity on the Dark Web with SOCRadar!
First Steps with Globus Compute Multi-User EndpointsGlobus
In this presentation we will share our experiences around getting started with the Globus Compute multi-user endpoint. Working with the Pharmacology group at the University of Auckland, we have previously written an application using Globus Compute that can offload computationally expensive steps in the researcher's workflows, which they wish to manage from their familiar Windows environments, onto the NeSI (New Zealand eScience Infrastructure) cluster. Some of the challenges we have encountered were that each researcher had to set up and manage their own single-user globus compute endpoint and that the workloads had varying resource requirements (CPUs, memory and wall time) between different runs. We hope that the multi-user endpoint will help to address these challenges and share an update on our progress here.
Top Features to Include in Your Winzo Clone App for Business Growth (4).pptxrickgrimesss22
Discover the essential features to incorporate in your Winzo clone app to boost business growth, enhance user engagement, and drive revenue. Learn how to create a compelling gaming experience that stands out in the competitive market.
Climate Science Flows: Enabling Petabyte-Scale Climate Analysis with the Eart...Globus
The Earth System Grid Federation (ESGF) is a global network of data servers that archives and distributes the planet’s largest collection of Earth system model output for thousands of climate and environmental scientists worldwide. Many of these petabyte-scale data archives are located in proximity to large high-performance computing (HPC) or cloud computing resources, but the primary workflow for data users consists of transferring data, and applying computations on a different system. As a part of the ESGF 2.0 US project (funded by the United States Department of Energy Office of Science), we developed pre-defined data workflows, which can be run on-demand, capable of applying many data reduction and data analysis to the large ESGF data archives, transferring only the resultant analysis (ex. visualizations, smaller data files). In this talk, we will showcase a few of these workflows, highlighting how Globus Flows can be used for petabyte-scale climate analysis.
Quarkus Hidden and Forbidden ExtensionsMax Andersen
Quarkus has a vast extension ecosystem and is known for its subsonic and subatomic feature set. Some of these features are not as well known, and some extensions are less talked about, but that does not make them less interesting - quite the opposite.
Come join this talk to see some tips and tricks for using Quarkus and some of the lesser known features, extensions and development techniques.
We describe the deployment and use of Globus Compute for remote computation. This content is aimed at researchers who wish to compute on remote resources using a unified programming interface, as well as system administrators who will deploy and operate Globus Compute services on their research computing infrastructure.
Paketo Buildpacks : la meilleure façon de construire des images OCI? DevopsDa...Anthony Dahanne
Les Buildpacks existent depuis plus de 10 ans ! D’abord, ils étaient utilisés pour détecter et construire une application avant de la déployer sur certains PaaS. Ensuite, nous avons pu créer des images Docker (OCI) avec leur dernière génération, les Cloud Native Buildpacks (CNCF en incubation). Sont-ils une bonne alternative au Dockerfile ? Que sont les buildpacks Paketo ? Quelles communautés les soutiennent et comment ?
Venez le découvrir lors de cette session ignite
48. $ docker history nginxexp:test
IMAGE CREATED CREATED BY SIZE COMMENT
b91799a04b99 50 years ago 1.47kB store paths: ['/nix/store/snxpdsksd4wxcn3niiyck0fry3wzri96-nginxexp-customisation-layer']
<missing> 50 years ago 200B store paths: ['/nix/store/6npz42nl2hhsrs98bq45aqkqsndpwvp1-nginx-root.conf']
<missing> 50 years ago 1.79MB store paths: ['/nix/store/qsq6ni4lxd8i4g9g4dvh3y7v1f43fqsp-nginx-1.18.0']
<missing> 50 years ago 492kB store paths: ['/nix/store/kdrdxhswaqm4dgdqs1vs2l4b4md7djma-pcre-8.44']
<missing> 50 years ago 4.17MB store paths: ['/nix/store/6glpgx3pypxzb09wxdqyagv33rrj03qp-openssl-1.1.1g']
...
<missing> 50 years ago 123kB store paths: ['/nix/store/5x6l9xm5dp6v113dpfv673qvhwjyb7p5-zlib-1.2.11']
<missing> 50 years ago 30.9MB store paths: ['/nix/store/bqbg6hb2jsl3kvf6jgmgfdqy06fpjrrn-glibc-2.30']
<missing> 50 years ago 209kB store paths: ['/nix/store/fhg84pzckx2igmcsvg92x1wpvl1dmybf-libidn2-2.3.0']
<missing> 50 years ago 1.63MB store paths: ['/nix/store/y8n2b9nwjrgfx3kvi3vywvfib2cw5xa6-libunistring-0.9.10']