Docker and Radio Astronomy - Containing Fragile Scientific Software by Gijs Molenaar (University of Amsterdam)
•
2 likes•8,384 views
As a scientific software engineer in the field of radio astronomy, I'm involved in creating, improving and maintaining a broad range of tools used by scientists. These tools are used to process and analyse data coming from various radio telescopes, particularly LOFAR (LOw Frequency ARray) and SKA (Square Kilometre Array). Radio astronomy has a long and rich computing related history, and with that comes a big pile of poorly written, fragile, and badly maintained legacy code. This problem is not unique to radio astronomy but is a common in various scientific fields. Although no quick solution exists when it comes to re-writing lots of legacy code, Docker helps to contain this fragile software, simplifies the installation and ensures that the software works. In this talk I will discuss how Papino, our umbrella project for containing various radio astronomy libraries and RODRIGUES, an online radio telescope simulator leverage Docker.
Report
Share
Report
Share
Download to read offline
Recommended
Opinionated containers and the future of game servers by Brendan Fosberry
The document discusses opinionated containers and containerizing game server applications. Containerizing simple applications like TeamSpeak is straightforward, but complex games like Team Fortress 2 pose unique challenges due to requirements for specific configuration files, port mappings, and handling of persisted state. While containerizing opinionated applications is becoming more common, the game server ecosystem presents difficulties around proprietary assets, obscured documentation, and hacker behavior that promote inflexible environments. Standardizing configuration methods and allowing external persisted state could help containerized game servers become more compatible and deployable at scale.
Migrating a large code-base to containers by Doug Johnson and Jonathan Lozins...
This document discusses strategies for migrating a large Ruby on Rails codebase to Docker containers. It provides examples of Dockerfile configurations for building container images with the Rails application and its dependencies. It also demonstrates commands for building, running, and committing container images to prepare the application and restore the original startup command.
Docker at Shopify: From This-Looks-Fun to Production by Simon Eskildsen (Shop...
Since July 2014 Shopify's been serving thousands of requests per second of production web traffic from Docker containers. This was an 8 month effort, with multiple pivots of direction from the team—and we're only getting started. This talk covers the lessons learned through the trial and error of an in-flight architecture redesign, spanning hundreds of hosts, as well as the technical vision of the future of our platform.
Enable Fig to deploy to multiple Docker servers by Willy Kuo
Fig (http://www.fig.sh/) is an Docker-based development environment tool which is owned by Docker. Originally, we can only deploy to one host at one time. My hack in Docker Global Hack Day #2 is to enable Fig to deploy multiple hosts at one time. In this talk, I'll give a brief introduction to Fig first. Then describe my hack in the hack day. Finally I'll give a short demo about deploying apps to multi hosts at one time.
DockerCon SF 2015: Ben Golub's Keynote Day 1
The summary provides an overview of the key points from DockerCon Day 1:
- Ben Golub, Docker CEO, welcomed attendees and encouraged them to use hashtags and share photos from the conference to win prizes.
- The keynote discussed how Docker has grown significantly over the past year in terms of contributors, projects, job openings, dockerized applications, and downloads.
- It was noted that applications have fundamentally changed to be more distributed and composed of loosely coupled microservices, and Docker is helping enable this new model of application development and deployment.
- The keynote discussed Docker's vision for the future of distributed applications and highlighted what Docker in production means in terms of community, ecosystem, and solutions for security
Open Design at large scale by Solomon Hykes
This document discusses Docker's open design and governance at large scales of contributors. It notes that Docker had over 4,800 pull requests in October 2014, most of which were merged within a day. It proposes a new governance structure with differentiated roles like chief architect, operators, and maintainers to scale Docker's open source development to tens of thousands of contributors and multiple sub-projects. This model aims to programmatically address issues like bottlenecks and inefficiencies that can arise at large scales.
Evaluating and ranking genome assemblers by Michael Barton (Joint Genome Inst...
The document discusses the benefits of exercise for both physical and mental health. Regular exercise can help reduce the risk of diseases like heart disease and diabetes, and improve mood and cognitive function. It recommends that adults get at least 150 minutes of moderate exercise or 75 minutes of vigorous exercise per week to gain these health benefits.
DockerCon 15 Keynote - Day 2
The document summarizes Day 2 of DockerCon. It discusses Docker being ready for production use with solutions for building, shipping, and running containers. It highlights Docker Hub growth and improvements to quality. Business Insider's journey with Docker is presented, covering lessons learned around local development and using Puppet and Docker Hub. Future directions discussed include orchestration tools and image security.
Recommended
Opinionated containers and the future of game servers by Brendan Fosberry
The document discusses opinionated containers and containerizing game server applications. Containerizing simple applications like TeamSpeak is straightforward, but complex games like Team Fortress 2 pose unique challenges due to requirements for specific configuration files, port mappings, and handling of persisted state. While containerizing opinionated applications is becoming more common, the game server ecosystem presents difficulties around proprietary assets, obscured documentation, and hacker behavior that promote inflexible environments. Standardizing configuration methods and allowing external persisted state could help containerized game servers become more compatible and deployable at scale.
Migrating a large code-base to containers by Doug Johnson and Jonathan Lozins...
This document discusses strategies for migrating a large Ruby on Rails codebase to Docker containers. It provides examples of Dockerfile configurations for building container images with the Rails application and its dependencies. It also demonstrates commands for building, running, and committing container images to prepare the application and restore the original startup command.
Docker at Shopify: From This-Looks-Fun to Production by Simon Eskildsen (Shop...
Since July 2014 Shopify's been serving thousands of requests per second of production web traffic from Docker containers. This was an 8 month effort, with multiple pivots of direction from the team—and we're only getting started. This talk covers the lessons learned through the trial and error of an in-flight architecture redesign, spanning hundreds of hosts, as well as the technical vision of the future of our platform.
Enable Fig to deploy to multiple Docker servers by Willy Kuo
Fig (http://www.fig.sh/) is an Docker-based development environment tool which is owned by Docker. Originally, we can only deploy to one host at one time. My hack in Docker Global Hack Day #2 is to enable Fig to deploy multiple hosts at one time. In this talk, I'll give a brief introduction to Fig first. Then describe my hack in the hack day. Finally I'll give a short demo about deploying apps to multi hosts at one time.
DockerCon SF 2015: Ben Golub's Keynote Day 1
The summary provides an overview of the key points from DockerCon Day 1:
- Ben Golub, Docker CEO, welcomed attendees and encouraged them to use hashtags and share photos from the conference to win prizes.
- The keynote discussed how Docker has grown significantly over the past year in terms of contributors, projects, job openings, dockerized applications, and downloads.
- It was noted that applications have fundamentally changed to be more distributed and composed of loosely coupled microservices, and Docker is helping enable this new model of application development and deployment.
- The keynote discussed Docker's vision for the future of distributed applications and highlighted what Docker in production means in terms of community, ecosystem, and solutions for security
Open Design at large scale by Solomon Hykes
This document discusses Docker's open design and governance at large scales of contributors. It notes that Docker had over 4,800 pull requests in October 2014, most of which were merged within a day. It proposes a new governance structure with differentiated roles like chief architect, operators, and maintainers to scale Docker's open source development to tens of thousands of contributors and multiple sub-projects. This model aims to programmatically address issues like bottlenecks and inefficiencies that can arise at large scales.
Evaluating and ranking genome assemblers by Michael Barton (Joint Genome Inst...
The document discusses the benefits of exercise for both physical and mental health. Regular exercise can help reduce the risk of diseases like heart disease and diabetes, and improve mood and cognitive function. It recommends that adults get at least 150 minutes of moderate exercise or 75 minutes of vigorous exercise per week to gain these health benefits.
DockerCon 15 Keynote - Day 2
The document summarizes Day 2 of DockerCon. It discusses Docker being ready for production use with solutions for building, shipping, and running containers. It highlights Docker Hub growth and improvements to quality. Business Insider's journey with Docker is presented, covering lessons learned around local development and using Puppet and Docker Hub. Future directions discussed include orchestration tools and image security.
DockerCon SF 2015: Keynote Day 1
The document summarizes key points from Day 1 of DockerCon. It discusses Docker's mission to build tools for mass innovation and how they are taking an incremental approach to reinventing the programmer's toolbox. New tools like Docker, Docker Compose, Docker Machine, Docker Swarm, Docker Networking plugins and Notary were introduced to help solve problems around runtimes, packaging, service composition, machine management, clustering, networking and security. Docker is also donating runC to the Open Container Project and forming the Open Container Project to establish open standards around container formats and governance.
The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCl...
The ModCloth Platform team has been building a Docker-based continuous delivery pipeline. This presentation discusses that project and how we build containers at ModCloth. The topics include what goes into our containers; how to optimize builds to use the Docker build cache effectively; useful development workflows (including using fig); and the key decision to treat containers as processes instead of mini-vms. This presentation will also discuss (and demo!) the workflow we’ve adopted for building containers and how we’ve integrated container builds with our CI.
DockerCon SF 2015: AWS w/ Deepak Singh
This 3 sentence summary provides the high-level information from the document:
Running the command "docker-compose up" will start and run all the services defined in a docker-compose.yml file; docker-compose up will create and start containers for the services defined; once the containers are running, docker-compose up will continue to run in the foreground and stream the log output of each container.
DockerCon SF 2015: Resilient Routing and Discovery
This document discusses resilient routing and discovery in large distributed systems. It describes Shopify's experience building reliable systems from unreliable components by focusing on resiliency, service discovery, routing and orchestration. It provides examples of techniques used like circuit breakers and failure injection tools. It also discusses the tradeoffs of different discovery solutions and Shopify's current approach of using DNS with Zookeeper for orchestration.
BBC: CI Problems and our Solutions by Simon Thulbourn
At BBC News, we've had a lot off issues with using CI environments for building and testing. Recently, we've taken a lot of time in order to solve these issues. We've created a new Jenkins setup to simplify our environment and in build & test our software inside on containers, even if ultimately we're not deploying in a container. This workflow has saved us potentially days of lost developer time.
A new model for Docker image distribution
This document provides an overview of Docker Registry 2.0, which implements a new API (V2) for distributing Docker images. The key points are:
1) Registry API V1 had problems with performance, security, and implementation; API V2 addresses these with a content-addressable, cryptographically verifiable model.
2) API V2 treats image layers as content-addressable blobs identified by cryptographic digests, allowing for independent verification.
3) Manifests describe the components of an image in a single object, allowing layers to be fetched in parallel for better performance.
4) The implementation of API V2 in Docker Registry 2.0 has improved pull performance and security while
Revamping Development and Testing Using Docker – Transforming Enterprise IT b...
Docker doesn't only revolutionize your application hosting, it also revolutionizes your development pipeline. ING has over 250DevOps teams, thousands of applications and a complex application landscape. ING is simplifying its application landscape in record time while introducing a webscale architecture based on anti-fragility patterns. Speed is vital in this transformation and one of ING’s key assets is it Continuous Delivery Pipeline. In this talk, we will show how ING uses test containers for confidence checks, integration testing on up and downstream services, creating dev/test environments for every feature branch, reverse proxy and CI servers. As a result, we are able to automate test processes and reduce our integration testing costs.
Docker Compose by Aanand Prasad
Docker Compose allows users to define and run multi-container Docker applications with a single command (docker up). It uses a YAML file to configure the application's services and Docker to automatically build images and link containers. With Compose, complex applications can be started and stopped with a single command, rather than multiple docker run commands. It also integrates with the Docker API, allowing it to work with tools like Docker Swarm for multi-host clusters.
Orchestrating Docker with Terraform and Consul by Mitchell Hashimoto
Terraform is a tool for building and safely iterating on infrastructure, while Consul provides service discovery, monitoring and orchestration. In this talk we discuss using Terraform and Consul together to build a Docker-based Service Oriented Architecture at scale. We use Consul to provide the runtime control plane for the datacenter, and Terraform is used to modify the underlying infrastructure to allow for elastic scalability.
Building Web Scale Apps with Docker and Mesos by Alex Rukletsov (Mesosphere)
Operating apps at web scale has become the new normal, but has been out of reach for most companies. Join us as we show you how to deploy and manage your Docker containers at scale. See how easy it is to build highly-available, fault-tolerant web scale apps using Docker with the Mesos cluster scheduler. Docker plus Mesos is a new way to scale applications. Together they give you capabilities similar to Google’s Borg, the Googleplex’s secret weapon of scalability and fault tolerance.
Docker Registry V2
This document discusses Docker Registry API V2, a new model for image distribution that addresses limitations in the previous V1 API. Key changes include making layers content-addressable using cryptographic digests for identification and verification. Images are now described by manifests containing layer digests. The registry stores content in repositories and no longer exposes internal image details. Early adoption shows V2 providing significantly better performance than V1 with 80% fewer requests and 60% less bandwidth used. Future goals include improving documentation, adding features like pull-through caching, and developing the Docker distribution components to provide a foundation for more advanced distribution models.
Docker 101: Introduction to Docker
This document provides an introduction to Docker. It discusses why Docker is useful for isolation, being lightweight, simplicity, workflow, and community. It describes the Docker engine, daemon, and CLI. It explains how Docker Hub provides image storage and automated builds. It outlines the Docker installation process and common workflows like finding images, pulling, running, stopping, and removing containers and images. It promotes Docker for building local images and using host volumes.
Authentication and Policy Enforcement for Docker
Docker has become an important tool for both startups and the enterprise; providing an easy way to package up and deploy applications and services. As organizations prepare to move to production, they are looking for ways to extend Docker and employ additional security and policy controls for their deployments. This talk will cover:
* Adding first class authentication using OAuth2, LDAP, Crowd and Kerberos
* Extending Docker deployments with policy and governance
* Securing network perimeter of Docker workloads while providing multiple multi-host networking options.
The Apcera Platform has always strived to be the best environment for deploying Docker workloads in production. This talk will cover our thinking around secure deployments and how we have extended Docker for our customers.
Dockerizing Stashboard
The document discusses dockerizing Stashboard, an open-source status dashboard for APIs and software services. It provides instructions for downloading Docker and the necessary Stashboard Dockerfiles. The Data Dockerfile creates a volume container to store status data. The Stashboard Dockerfile installs Stashboard and its dependencies in an Ubuntu container. Finally, it explains how to build and run the dockerized Stashboard application using the data volume container to persist status updates.
A Gentle Introduction to Docker and Containers
This document provides an introduction to Docker and containers. It outlines that Docker is an open source tool that makes it easy to deploy applications by using containers. Containers allow applications to be isolated for easier management and deployment. The document discusses how Docker builds on existing container technologies and provides a standardized way to build, share, and run application containers.
DockerCon 14 Keynote Day 2
The document provides an agenda and highlights from Day 2 of a DockerCon conference. It includes details on upcoming keynote speakers from IBM and Google, as well as highlights from the previous day including top speakers, quotes, and photos from the event. Winners of the hackathon are also announced, which include projects that focus on instrumentation and logging of Docker hosts as well as facilitating open source slideshow authoring.
The Future of Hardware-Backed Keys
The document discusses hardware-backed authentication keys called YubiKeys. It describes the YubiKey's durable design with no batteries or moving parts that allows it to communicate over USB and support multiple authentication protocols like one-time passwords, smart cards, and FIDO U2F. The YubiKey 4 can support multiple configurations and protocols simultaneously. YubiKeys can be used across many applications and services for two-factor authentication, code signing, and encryption. Yubico, the company that created YubiKeys, has millions of users in 150 countries and provides resources and documentation to support integration and use of YubiKeys.
Building Images from dockerfiles
The document describes how to build Docker images from Dockerfiles. It explains the main Dockerfile instructions like FROM, MAINTAINER, ADD, ENV, RUN, CMD, ENTRYPOINT, EXPOSE and VOLUME and provides an example Dockerfile to build a MongoDB image. The Dockerfile specifies the base image, copies configuration files, installs MongoDB, sets the entrypoint and exposes ports. Images built from this Dockerfile can then be run as Docker containers with data volumes mounted.
DockerCon EU 2015: It's in the game: the path to micro-services at Electronic...
This document discusses Electronic Arts' use of Docker containers to improve the scalability and deployment of their mobile game server infrastructure. It describes how EA was facing challenges with manual and complex deployment processes. With Docker, EA was able to break their application into microservices, package them as containers, and use tools like Jenkins and Fleet for automated builds, deployment, and orchestration. This resulted in faster and more reliable deployments while improving scalability to meet the demands of live service games.
cv-guegant-jean
Jean Guegant is a C++ backend developer for Qliro in Stockholm, Sweden. He has worked on fraud detection engines and high performance, scalable architectures using C++14 and Python. Guegant has also designed monitoring GUIs, worked on low latency trading systems, and researched SDN and home networks. He has a master's degree in computer science and interests include game development, computer security, and outdoor activities.
Scaling Up Machine Learning: How to Benchmark GraphLab Create on Huge Datasets
This document discusses benchmarks for GraphLab Create, a machine learning library. It summarizes benchmarking GraphLab Create on large datasets by running PageRank on a graph with 3.5 billion nodes and 128 billion links, and gradient boosted trees on a dataset with 4.3 billion rows and 39 features. The document also provides instructions for instantiating an Amazon EC2 instance with 32 cores and 244GB RAM to run the benchmarks, and includes a link to download GraphLab Create and access the benchmark notebooks on GitHub.
Docker for the Internet of Things (IoT): An Introduction
This document introduces Docker for Internet of Things (IoT) applications using a Raspberry Pi. It discusses:
1. What Docker is and why it is useful for IoT by providing consistent environments for applications from development to production.
2. How to set up Docker on a Raspberry Pi by installing Raspbian, Docker CE, and verifying the installation.
3. Examples of running Docker containers for MQTT, Python, and controlling GPIO pins with Python GPIO Zero to blink an LED.
More Related Content
Viewers also liked
DockerCon SF 2015: Keynote Day 1
The document summarizes key points from Day 1 of DockerCon. It discusses Docker's mission to build tools for mass innovation and how they are taking an incremental approach to reinventing the programmer's toolbox. New tools like Docker, Docker Compose, Docker Machine, Docker Swarm, Docker Networking plugins and Notary were introduced to help solve problems around runtimes, packaging, service composition, machine management, clustering, networking and security. Docker is also donating runC to the Open Container Project and forming the Open Container Project to establish open standards around container formats and governance.
The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCl...
The ModCloth Platform team has been building a Docker-based continuous delivery pipeline. This presentation discusses that project and how we build containers at ModCloth. The topics include what goes into our containers; how to optimize builds to use the Docker build cache effectively; useful development workflows (including using fig); and the key decision to treat containers as processes instead of mini-vms. This presentation will also discuss (and demo!) the workflow we’ve adopted for building containers and how we’ve integrated container builds with our CI.
DockerCon SF 2015: AWS w/ Deepak Singh
This 3 sentence summary provides the high-level information from the document:
Running the command "docker-compose up" will start and run all the services defined in a docker-compose.yml file; docker-compose up will create and start containers for the services defined; once the containers are running, docker-compose up will continue to run in the foreground and stream the log output of each container.
DockerCon SF 2015: Resilient Routing and Discovery
This document discusses resilient routing and discovery in large distributed systems. It describes Shopify's experience building reliable systems from unreliable components by focusing on resiliency, service discovery, routing and orchestration. It provides examples of techniques used like circuit breakers and failure injection tools. It also discusses the tradeoffs of different discovery solutions and Shopify's current approach of using DNS with Zookeeper for orchestration.
BBC: CI Problems and our Solutions by Simon Thulbourn
At BBC News, we've had a lot off issues with using CI environments for building and testing. Recently, we've taken a lot of time in order to solve these issues. We've created a new Jenkins setup to simplify our environment and in build & test our software inside on containers, even if ultimately we're not deploying in a container. This workflow has saved us potentially days of lost developer time.
A new model for Docker image distribution
This document provides an overview of Docker Registry 2.0, which implements a new API (V2) for distributing Docker images. The key points are:
1) Registry API V1 had problems with performance, security, and implementation; API V2 addresses these with a content-addressable, cryptographically verifiable model.
2) API V2 treats image layers as content-addressable blobs identified by cryptographic digests, allowing for independent verification.
3) Manifests describe the components of an image in a single object, allowing layers to be fetched in parallel for better performance.
4) The implementation of API V2 in Docker Registry 2.0 has improved pull performance and security while
Revamping Development and Testing Using Docker – Transforming Enterprise IT b...
Docker doesn't only revolutionize your application hosting, it also revolutionizes your development pipeline. ING has over 250DevOps teams, thousands of applications and a complex application landscape. ING is simplifying its application landscape in record time while introducing a webscale architecture based on anti-fragility patterns. Speed is vital in this transformation and one of ING’s key assets is it Continuous Delivery Pipeline. In this talk, we will show how ING uses test containers for confidence checks, integration testing on up and downstream services, creating dev/test environments for every feature branch, reverse proxy and CI servers. As a result, we are able to automate test processes and reduce our integration testing costs.
Docker Compose by Aanand Prasad
Docker Compose allows users to define and run multi-container Docker applications with a single command (docker up). It uses a YAML file to configure the application's services and Docker to automatically build images and link containers. With Compose, complex applications can be started and stopped with a single command, rather than multiple docker run commands. It also integrates with the Docker API, allowing it to work with tools like Docker Swarm for multi-host clusters.
Orchestrating Docker with Terraform and Consul by Mitchell Hashimoto
Terraform is a tool for building and safely iterating on infrastructure, while Consul provides service discovery, monitoring and orchestration. In this talk we discuss using Terraform and Consul together to build a Docker-based Service Oriented Architecture at scale. We use Consul to provide the runtime control plane for the datacenter, and Terraform is used to modify the underlying infrastructure to allow for elastic scalability.
Building Web Scale Apps with Docker and Mesos by Alex Rukletsov (Mesosphere)
Operating apps at web scale has become the new normal, but has been out of reach for most companies. Join us as we show you how to deploy and manage your Docker containers at scale. See how easy it is to build highly-available, fault-tolerant web scale apps using Docker with the Mesos cluster scheduler. Docker plus Mesos is a new way to scale applications. Together they give you capabilities similar to Google’s Borg, the Googleplex’s secret weapon of scalability and fault tolerance.
Docker Registry V2
This document discusses Docker Registry API V2, a new model for image distribution that addresses limitations in the previous V1 API. Key changes include making layers content-addressable using cryptographic digests for identification and verification. Images are now described by manifests containing layer digests. The registry stores content in repositories and no longer exposes internal image details. Early adoption shows V2 providing significantly better performance than V1 with 80% fewer requests and 60% less bandwidth used. Future goals include improving documentation, adding features like pull-through caching, and developing the Docker distribution components to provide a foundation for more advanced distribution models.
Docker 101: Introduction to Docker
This document provides an introduction to Docker. It discusses why Docker is useful for isolation, being lightweight, simplicity, workflow, and community. It describes the Docker engine, daemon, and CLI. It explains how Docker Hub provides image storage and automated builds. It outlines the Docker installation process and common workflows like finding images, pulling, running, stopping, and removing containers and images. It promotes Docker for building local images and using host volumes.
Authentication and Policy Enforcement for Docker
Docker has become an important tool for both startups and the enterprise; providing an easy way to package up and deploy applications and services. As organizations prepare to move to production, they are looking for ways to extend Docker and employ additional security and policy controls for their deployments. This talk will cover:
* Adding first class authentication using OAuth2, LDAP, Crowd and Kerberos
* Extending Docker deployments with policy and governance
* Securing network perimeter of Docker workloads while providing multiple multi-host networking options.
The Apcera Platform has always strived to be the best environment for deploying Docker workloads in production. This talk will cover our thinking around secure deployments and how we have extended Docker for our customers.
Dockerizing Stashboard
The document discusses dockerizing Stashboard, an open-source status dashboard for APIs and software services. It provides instructions for downloading Docker and the necessary Stashboard Dockerfiles. The Data Dockerfile creates a volume container to store status data. The Stashboard Dockerfile installs Stashboard and its dependencies in an Ubuntu container. Finally, it explains how to build and run the dockerized Stashboard application using the data volume container to persist status updates.
A Gentle Introduction to Docker and Containers
This document provides an introduction to Docker and containers. It outlines that Docker is an open source tool that makes it easy to deploy applications by using containers. Containers allow applications to be isolated for easier management and deployment. The document discusses how Docker builds on existing container technologies and provides a standardized way to build, share, and run application containers.
DockerCon 14 Keynote Day 2
The document provides an agenda and highlights from Day 2 of a DockerCon conference. It includes details on upcoming keynote speakers from IBM and Google, as well as highlights from the previous day including top speakers, quotes, and photos from the event. Winners of the hackathon are also announced, which include projects that focus on instrumentation and logging of Docker hosts as well as facilitating open source slideshow authoring.
The Future of Hardware-Backed Keys
The document discusses hardware-backed authentication keys called YubiKeys. It describes the YubiKey's durable design with no batteries or moving parts that allows it to communicate over USB and support multiple authentication protocols like one-time passwords, smart cards, and FIDO U2F. The YubiKey 4 can support multiple configurations and protocols simultaneously. YubiKeys can be used across many applications and services for two-factor authentication, code signing, and encryption. Yubico, the company that created YubiKeys, has millions of users in 150 countries and provides resources and documentation to support integration and use of YubiKeys.
Building Images from dockerfiles
The document describes how to build Docker images from Dockerfiles. It explains the main Dockerfile instructions like FROM, MAINTAINER, ADD, ENV, RUN, CMD, ENTRYPOINT, EXPOSE and VOLUME and provides an example Dockerfile to build a MongoDB image. The Dockerfile specifies the base image, copies configuration files, installs MongoDB, sets the entrypoint and exposes ports. Images built from this Dockerfile can then be run as Docker containers with data volumes mounted.
DockerCon EU 2015: It's in the game: the path to micro-services at Electronic...
This document discusses Electronic Arts' use of Docker containers to improve the scalability and deployment of their mobile game server infrastructure. It describes how EA was facing challenges with manual and complex deployment processes. With Docker, EA was able to break their application into microservices, package them as containers, and use tools like Jenkins and Fleet for automated builds, deployment, and orchestration. This resulted in faster and more reliable deployments while improving scalability to meet the demands of live service games.
Viewers also liked (19)
The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCl...
The Tale of a Docker-based Continuous Delivery Pipeline by Rafe Colton (ModCl...
DockerCon SF 2015: Resilient Routing and Discovery
DockerCon SF 2015: Resilient Routing and Discovery
BBC: CI Problems and our Solutions by Simon Thulbourn
BBC: CI Problems and our Solutions by Simon Thulbourn
Revamping Development and Testing Using Docker – Transforming Enterprise IT b...
Revamping Development and Testing Using Docker – Transforming Enterprise IT b...
Orchestrating Docker with Terraform and Consul by Mitchell Hashimoto
Orchestrating Docker with Terraform and Consul by Mitchell Hashimoto
Building Web Scale Apps with Docker and Mesos by Alex Rukletsov (Mesosphere)
Building Web Scale Apps with Docker and Mesos by Alex Rukletsov (Mesosphere)
DockerCon EU 2015: It's in the game: the path to micro-services at Electronic...
DockerCon EU 2015: It's in the game: the path to micro-services at Electronic...
Similar to Docker and Radio Astronomy - Containing Fragile Scientific Software by Gijs Molenaar (University of Amsterdam)
cv-guegant-jean
Jean Guegant is a C++ backend developer for Qliro in Stockholm, Sweden. He has worked on fraud detection engines and high performance, scalable architectures using C++14 and Python. Guegant has also designed monitoring GUIs, worked on low latency trading systems, and researched SDN and home networks. He has a master's degree in computer science and interests include game development, computer security, and outdoor activities.
Scaling Up Machine Learning: How to Benchmark GraphLab Create on Huge Datasets
This document discusses benchmarks for GraphLab Create, a machine learning library. It summarizes benchmarking GraphLab Create on large datasets by running PageRank on a graph with 3.5 billion nodes and 128 billion links, and gradient boosted trees on a dataset with 4.3 billion rows and 39 features. The document also provides instructions for instantiating an Amazon EC2 instance with 32 cores and 244GB RAM to run the benchmarks, and includes a link to download GraphLab Create and access the benchmark notebooks on GitHub.
Docker for the Internet of Things (IoT): An Introduction
This document introduces Docker for Internet of Things (IoT) applications using a Raspberry Pi. It discusses:
1. What Docker is and why it is useful for IoT by providing consistent environments for applications from development to production.
2. How to set up Docker on a Raspberry Pi by installing Raspbian, Docker CE, and verifying the installation.
3. Examples of running Docker containers for MQTT, Python, and controlling GPIO pins with Python GPIO Zero to blink an LED.
IoT Session Thomas More
The document provides an overview of Internet of Things (IoT) technologies including hardware platforms, communication protocols, and programming languages that can be used to develop IoT solutions. It discusses common hardware devices like Arduino, Raspberry Pi, and Intel Edison. It also covers the LoRa wireless communication protocol and how it can be used with sensors. Finally, it gives examples of IoT applications and envisions future advancements in areas like battery life, networking, and data analytics.
Cc internet of things @ Thomas More
The document provides an overview of Internet of Things (IoT) technologies including hardware platforms, communication protocols, and programming languages that can be used to develop IoT solutions. It discusses common hardware devices like Arduino, Raspberry Pi, and Intel Edison. It also covers the LoRa wireless protocol for long range communication and provides examples of using Java and Node.js on Raspberry Pi for IoT. Finally, it discusses potential business applications of IoT and envisions continued improvements in areas like batteries, networks, sensors, and data analytics.
Nanaji_Sahukara
Nanaji Sahukara has over 5 years of experience in software development using C++, OpenGL, and Unity3D. He has worked on projects involving visualization, virtual reality, and avionics systems. His experience includes developing real-time simulation and visualization software, virtual reality applications, and handling all stages of the software development life cycle.
NVIDIA Deep Learning Institute 2017 基調講演
このスライドは 2017 年 1 月 17 日 (火)、ベルサール高田馬場で開催された「NVIDIA Deep Learning Institute 2017」の基調講演にて、NVIDIA Chief Scientist and SVP of Research の Bill Dally が講演したものです。
_Embedded-Eng.Heba-Abdelhady_
Heba Abdelhady Ahmed is seeking opportunities in embedded systems. She has a B.Eng. in computer science and engineering from Menufia University, with experience including scholarships in embedded C programming, microcontrollers, RTOS, software engineering and more. Her projects include Mos3ef and an incubator project using PIC microcontrollers. She has certifications in PIC, temperature sensors, PLC and A+.
Open Racing
The document discusses the potential for an open source driving simulator and AI research platform called OpenRacing. It notes that proprietary code from past competitions like the DARPA Grand Challenge was abandoned and not built upon. The author argues that an open source simulator could help build a community to collaboratively develop self-driving car software. Key points include the importance of simulators for testing vision and driving logic code, leveraging the Rigs of Rods driving simulator, and the potential to run a "Virtual DARPA Grand Challenge" competition to help generate interest and contributions to shared open source code.
Mikhail_Tchernychev_Resume_t_2
Mikhail Tchernychev is a software engineer with over 30 years of experience developing complex multi-threaded applications in C++, C, Python, and Fortran. He has worked on projects involving data acquisition, processing, and mapping for geophysics, including magnetometer surveys and GIS applications. Currently he is a senior software engineer at Geometrics Inc where he develops acquisition and processing software for magnetometer data and has created new techniques that outperform competitors.
Echelon Indonesia 2016 - Innovation Through Opportunities in IoT & Arduino
In Echelon Indonesia 2016, April 5, I talked about what IoT and Arduino are all about, technological perspective in IoT, and some opportunities around IoT where we can innovate. I also gave my view on IoT market and players in Indonesia.
It's exciting time!
NOVA IMS Microsoft on Innovation
This document discusses several innovations from Microsoft including:
1. Project Natick, which developed prepackaged underwater datacenters that can be rapidly deployed and operate autonomously on the seafloor for years.
2. The Xbox Adaptive Controller, which evolved to allow gamers with mobility needs to customize their controller through additional buttons and switches.
3. InnerEye, an AI system developed by Microsoft to efficiently segment medical images and identify anatomy and pathology, which received FDA clearance.
4. Microsoft's approach to quantum computing, which includes developing a scalable quantum computer, taking a full stack approach, and leveraging quantum technologies today through inspiration, simulation, and developer tools.
Sanjay kumar joshi
Sanjay Kumar Joshi is seeking a position in research and development utilizing his knowledge and skills in programming. He has experience developing applications in C/C++, Java, Qt, and Android. Some of his projects include developing GUI applications, an RFID application, and software for emulating military equipment and distributing data between systems for the Ministry of Defence. He has expertise in data structures, multithreading, networking, and operating systems like Linux and Windows.
cv-2016-23
This document is a CV that outlines Sergei Vorobyov's skills and experience. It details his PhD in Computer Science, 25+ years of experience in software development, and expertise in areas such as C, C++, Java, databases, cloud computing, and functional programming. Recent projects described include developing AWS cloud infrastructure, a large MySQL database project, and a digital rights management subsystem. Contact information and links to additional online profiles are also provided.
End-to-End Big Data AI with Analytics Zoo
The document discusses Analytics Zoo, an open-source software platform for building end-to-end big data AI applications. It provides distributed deep learning frameworks like TensorFlow and PyTorch on Apache Spark. Analytics Zoo allows seamless scaling of AI models from laptop to distributed big data and includes features like automated machine learning, time series forecasting, and serving models in production. It aims to simplify development of end-to-end big data AI solutions.
ownR platform technical introduction
The document describes ownR, an enterprise analytics platform for R and Python. It consists of three main components: laiR, an application repository; roveR, for containerized R environments; and exposeR, a REST API for R/Python functions. Together they aim to address common challenges in data science workflows by providing version control, dependency management, reusable packages/code, scalable computing, and easy deployment of models to production. The platform facilitates collaboration, reproducibility of analyses, and integration of analytics code with other applications.
Роман Родоманський, «Досвід побудови Contact Tracing рішення»
The document discusses requirements for a COVID-19 tracking mobile app, including that it be reliable, cost-effective, privacy-friendly, user-friendly, and have good performance worldwide. It asks for estimates of development time, monthly infrastructure costs, and what tools and languages would be used.
Use r 2013 tutorial - r and cloud computing for higher education and research
This document discusses using R and cloud computing for higher education and research. It introduces Elastic-R, a platform that aims to provide a universal platform for data science in the cloud. Elastic-R allows users to access R and other data science tools on-demand in the cloud. It provides capabilities for collaboration, sharing resources, and publishing web services and applications without needing administrators. The document outlines Elastic-R's design, technologies, and scripting framework to make data science resources and tools available on-demand in the cloud.
PDE2011 pythonOCC project status and plans
Sldeshow presented at the latest NASA/ESA Product Data Exchange conference. Deals with pythonocc project status and midterm plans: WebGl renderer, high level API over the low level builtin data model.
IOT with Drupal 8 - Webinar Hyderabad Drupal Community
This document discusses the Internet of Things (IoT) and provides examples of IoT applications and platforms. It defines IoT as integrated systems with monitoring, sensing and computing capabilities that allow both analog and digital devices to transmit data over a network. The document notes that IoT projects are often more difficult than expected due to the complexity of working with different organizations and technologies. It then provides examples of using Raspberry Pi, sensors, and cloud platforms like ThingSpeak to build basic IoT applications. Finally, it discusses how Drupal can be used as a platform for IoT applications and content services.
Similar to Docker and Radio Astronomy - Containing Fragile Scientific Software by Gijs Molenaar (University of Amsterdam) (20)
Scaling Up Machine Learning: How to Benchmark GraphLab Create on Huge Datasets
Scaling Up Machine Learning: How to Benchmark GraphLab Create on Huge Datasets
Docker for the Internet of Things (IoT): An Introduction
Docker for the Internet of Things (IoT): An Introduction
Echelon Indonesia 2016 - Innovation Through Opportunities in IoT & Arduino
Echelon Indonesia 2016 - Innovation Through Opportunities in IoT & Arduino
Роман Родоманський, «Досвід побудови Contact Tracing рішення»
Роман Родоманський, «Досвід побудови Contact Tracing рішення»
Use r 2013 tutorial - r and cloud computing for higher education and research
Use r 2013 tutorial - r and cloud computing for higher education and research
IOT with Drupal 8 - Webinar Hyderabad Drupal Community
IOT with Drupal 8 - Webinar Hyderabad Drupal Community
More from Docker, Inc.
Containerize Your Game Server for the Best Multiplayer Experience
Raymond Arifianto, AccelByte and
Mark Mandel, Google -
We have been deploying containerized micro-services for our Game Backend Services for a while. Now we are tackling the challenge to scale up fleets of game dedicated servers in multiple regions, multiple data centers and multiple providers - some in bare metal, some in Cloud. So we leverage docker containerization to deploy Game Servers to achieve Portability, Fast Deployment and Predictability, enabling us to scale up to thousands of servers, on demand, without a sweat.
How to Improve Your Image Builds Using Advance Docker Build
Nicholas Dille, Haufe-Lexware + Docker Captain -
Docker continues to be the standard tool for building container images. For more than a year Docker ships with BuildKit as an alternative image builder, providing advanced features for secret and cache management. These features help to make image builds faster and more secure. In this session, Docker Captain Nicholas Dille will teach you how to use Buildkit features to your advantage.
Build & Deploy Multi-Container Applications to AWS
Lukonde Mwila, Entelect -
As the cloud-native approach to development and deployment becomes more prevalent, it's an exciting time for software engineers to be equipped on how to dockerize multi-container applications and deploy them to the cloud.
In this talk, Lukonde Mwila, Software Engineer at Entelect, will cover the following topics:
- Docker Compose
- Containerizing an Nginx Server
- Containerizing an React App
- Containerizing an Node.JS App
- Containerizing anMongoDB App
- Runing Multi-Container App Locally
- Creating a CI/CD Pipeline
- Adding a build stage to test containers and push images to Docker Hub
- Deploying Multi-Container App to AWS Elastic Beanstalk
Lukonde will start by giving an overview of how Docker Compose works and how it makes it very easy and straightforward to startup multiple Docker containers at the same time and automatically connect them together with some form of networking.
After that, Lukonde will take a hands on approach to containerize an Nginx server, a React app, a NodeJS app and a MongoDB instance to demonstrate the power of Docker Compose. He'll demonstrate usage of two Docker files for an application, one production grade and the other for local development and running of tests. Lastly, he'll demonstrate creating a CI/CD pipeline in AWS to build and test our Docker images before pushing them to Docker Hub or AWS ECR, and finally deploying our multi-container application AWS Elastic Beanstalk.
Securing Your Containerized Applications with NGINX
The document summarizes Kevin Jones' presentation on securing containerized applications with NGINX. It discusses the benefits of using a reverse proxy for security, NGINX best practices for TLS configuration, and deploying NGINX in Docker containers. It also provides code examples and configurations for setting up NGINX as a reverse proxy, optimizing TLS, and using NGINX as a sidecar proxy.
How To Build and Run Node Apps with Docker and Compose
Kathleen Juell, Digital Ocean -
Containers are an essential part of today's microservice ecosystem, as they allow developers and operators to maintain standards of reliability and reproducibility in fast-paced deployment scenarios. And while there are best practices that extend across stacks in containerized environments, there are also things that make each stack distinct, starting with the application image itself.
This talk will dive into some of these particularities, both at the image and service level, while also covering general best practices for building and running Node applications with database backends using Docker and Compose.
Hands-on Helm
Jessica Deen, Microsoft -
Helm 3 is here; let's go hands-on! In this demo-fueled session, I'll walk you through the differences between Helm 2 and Helm 3. I'll offer tips for a successful rollout or upgrade, go over how to easily use charts created for Helm 2 with Helm 3 (without changing your syntax), and review opportunities where you can participate in the project's future.
Distributed Deep Learning with Docker at Salesforce
Jeff Hajewski, Salesforce -
There is a wealth of information on building deep learning models with PyTorch or TensorFlow. Anyone interested in building a deep learning model is only a quick search away from a number of clear and well written tutorials that will take them from zero knowledge to having a working image classifier. But what happens when you need to deploy these models in a production setting? At Salesforce, we use TensorFlow models to help us provide customers with insights into their data, and we do this as close to real-time as possible. Designing these systems in a scalable manner requires overcoming a number of design challenges, but the core component is Docker. Docker enables us to design highly scalable systems by allowing us to focus on service interactions, rather than how our services will interact with the hardware. Docker is also at the core of our test infrastructure, allowing developers and data scientists to build and test the system in an end to end manner on their local machines. While some of this may sound complex, the core message is simplicity - Docker allows us to focus on the aspects of the system that matter, greatly simplifying our lives.
The First 10M Pulls: Building The Official Curl Image for Docker Hub
James Fuller, webcomposite s.r.o. -
Curl is the venerable (yet very modern) 'swiss army knife' command line tool and library for transferring data with URLs. Recently we (the Curl team) decided to build a release for Docker Hub. This talk will outline our current development workflow with respect to the docker image and provide insights on what it takes to build a docker image for mass public consumption. We are also keen to learn from users and other developers how we might improve and enhance the official curl docker image.
Monitoring in a Microservices World
Fabian Stäber, Instana -
In recent years, we saw a great paradigm shift in software engineering away from static monolithic applications towards dynamic distributed horizontally scalable architectures. Docker is one of the key technologies enabling this development. This shift poses a lot of new challenges for application monitoring, ranging from practical issues (need for automation) to technical challenges (Docker networking) to organizational topics (blurring line between software engineers and operations) to fundamental questions (define what is an application). In this talk we show how Docker changed the way we do monitoring, how modern application monitoring systems work, and what future developments we expect.
COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...
Clemente Biondo, Engineering Ingegneria Informatica -
When the COVID 19 pandemic started, Engineering Ingegneria Informatica Group (1.25 billion euros of revenues, 65 offices around the world, 12.000 employees) was forced to put their digital transformation to the test in order to maintain operational continuity. In this session, Clemente Biondo, the Tech Lead of the Information Systems Department, will share how his company is reacting to this unforeseeable scenario and how Docker-driven digital transformation had paved the path for work to continue remotely. Clemente will discuss learnings moving from colocated teams, manual approaches, email based-business processes, and a monolithic application to a mature DevOps culture characterized by a distributed autonomous workforce and a continuous deployment process that deploys backward-compatible Docker containerized microservices into hybrid multi cloud datacenters an average of twice a day with zero-downtime. He will detail how they use Docker to unify dev, test and production environments, and as an efficient and automated mechanism for deploying applications. Lastly, Clemente shares how, in our darkest hour, he and others are working to shine their brightest light.
Predicting Space Weather with Docker
The document discusses how NOAA's Space Weather Prediction Center transitioned from a monolithic architecture to microservices using Docker. It describes how they started with a small verification project, then replaced their critical GOES satellite data source. This improved developers' morale and delivery speed. They encountered some security issues initially but learned from them. The transition was very successful and allowed them to quickly expand their mission to forecast aviation impacts using scientists' models packaged as Docker services.
Become a Docker Power User With Microsoft Visual Studio Code
Brian Christner, 56k + Docker Captain -
In this session, we will unlock the full potential of using Microsoft Visual Studio Code (VS Code) and Docker Desktop to turn you into a Docker Power User. When we expand and utilize the VS Code Docker plugin, we can take our projects and Docker skills to the next level. In addition to using VS Code, we streamline our Docker Desktop development workflow with less context switching and built-in shortcuts. You will learn how to bootstrap new projects, quickly write Dockerfiles utilizing templates, build, run, and interact with containers all from VS Code.
How to Use Mirroring and Caching to Optimize your Container Registry
Brandon Mitchell, Boxboat + Docker Captain -
How do you make your builds more performant? This talk looks at options to configure caching and mirroring of images that you need to save on bandwidth costs and to keep running even if something goes down upstream.
Monolithic to Microservices + Docker = SDLC on Steroids!
Ashish Sharma, SS&C Eze -
SS&C Eze provides various products in the stock market domain. We spent the last couple of years building Eclipse which is an investment suite born in cloud. The journey so far has been very interesting. The very first version of the product were a bunch of monolithic windows services and deployed using Octopus tool. We successfully managed to bring all the monolithic problem to the cloud and created a nightmare for ourselves. We then started applying microservices architecture principles and started breaking the monolithic into small services. Very soon we realized that we need a better packaging/deployment tool. Docker looked like a magical solution to our problem. Since its adoption, It has not only solved the deployment problem for us but has made a deep impact on different aspects of SDLC. It allowed us to use heterogeneous technology stacks, simplified development environment setup, simplified our testing strategy, improved our speed of delivery, and made our developers more productive. In this talk I would like to share our experience of using Docker and its positive impact on our SDLC.
Kubernetes at Datadog Scale
Kubernetes networking can be complex to scale due to issues like growing iptables rules, but newer solutions are helping. Pod networking uses CNI plugins like flannel or Calico to assign each pod an IP and allow communication. Service networking uses kube-proxy and iptables or IPVS for load balancing to pods. DNS is used to resolve service names to IPs. While Kubernetes networking brings flexibility, operators must learn the nuances of their specific CNI plugin and issues can arise, but the ecosystem adapts quickly to new needs and changes don't impact all workloads.
Labels, Labels, Labels
Andy Clemenko, StackRox -
One underutilized, and amazing, thing about the docker image scheme is labels. Labels are a built in way to document all aspects about the image itself. Think about all the information that the tags inside your clothing carry. If you care to look you can find out everything about the garment. All that information can be very valuable. Now think about how we can leverage labels to carry similar information. We can even use the labels to contain Docker Compose or even Kubernetes Yaml. We can even include labels into the CI/CD process making things more secure and smoother. Come find out some fun techniques on how to leverage labels to do some fun and amazing things.
Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment Model
Micro Focus uses Docker Hub at scale to support its software delivery and deployment model. Some key points:
- Docker Hub is used as the registry service for Micro Focus container images
- It allows for optimized, secure, reliable and cost-effective software delivery through deployments and updates of container images to customers and partners
- Micro Focus leverages features like private repositories, offline/online access, signing and scanning of images, and integration with CI/CD pipelines
- Over 1,650 organizations, 450 repositories, and 18 teams are used on Docker Hub to manage access and deliver software from Micro Focus
Build & Deploy Multi-Container Applications to AWS
Lukonde Mwila, Entelect
As the cloud-native approach to development and deployment becomes more prevalent, it's an exciting time for software engineers to be equipped on how to dockerize multi-container applications and deploy them to the cloud.
In this talk, Lukonde Mwila, Software Engineer at Entelect, will cover the following topics:
- Docker Compose
- Containerizing an Nginx Server
- Containerizing an React App
- Containerizing an Node.JS App
- Containerizing anMongoDB App
- Runing Multi-Container App Locally
- Creating a CI/CD Pipeline
- Adding a build stage to test containers and push images to Docker Hub
- Deploying Multi-Container App to AWS Elastic Beanstalk
Lukonde will start by giving an overview of how Docker Compose works and how it makes it very easy and straightforward to startup multiple Docker containers at the same time and automatically connect them together with some form of networking.
After that, Lukonde will take a hands on approach to containerize an Nginx server, a React app, a NodeJS app and a MongoDB instance to demonstrate the power of Docker Compose. He'll demonstrate usage of two Docker files for an application, one production grade and the other for local development and running of tests. Lastly, he'll demonstrate creating a CI/CD pipeline in AWS to build and test our Docker images before pushing them to Docker Hub or AWS ECR, and finally deploying our multi-container application AWS Elastic Beanstalk.
From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...
Elton Stoneman, Docker Captain + Container Consultant and Trainer
How do you provide a SaaS offering when your product is a 10-year old Fortran app, currently built to run on Windows 10? With Docker and Kubernetes of course - and you can do it in a week (... to prototype level at least).
In this session I'll walk through the processes and practicalities of taking an older Windows app, making it run in containers with Kubernetes, and then building a simple API wrapper to host the whole stack as a cloud-based SaaS product.
There's a lot of technology here from a real world case study, and I'll focus on:
- running Windows apps in Docker containers
- building a .NET Core API which can run in Linux or Windows containers
- running the stack in Kubernetes with Docker Desktop locally and AKS in the cloud
- configuring AKS workloads in Azure to burst out to Azure Container Instances
And there's a core theme to this session: Docker and Kubernetes are complex technologies, but they're the key to modern development. If you invest time learning them, they make projects like this simple, portable, fast and fun.
Developing with Docker for the Arm Architecture
This virtual meetup introduces the concepts and best practices of using Docker containers for software development for the Arm architecture across a variety of hardware systems. Using Docker Desktop on Windows or Mac, Amazon Web Services (AWS) A1 instances, and embedded Linux, we will demonstrate the latest Docker features to build, share, and run multi-architecture images with transparent support for Arm.
More from Docker, Inc. (20)
Containerize Your Game Server for the Best Multiplayer Experience
Containerize Your Game Server for the Best Multiplayer Experience
How to Improve Your Image Builds Using Advance Docker Build
How to Improve Your Image Builds Using Advance Docker Build
Build & Deploy Multi-Container Applications to AWS
Build & Deploy Multi-Container Applications to AWS
Securing Your Containerized Applications with NGINX
Securing Your Containerized Applications with NGINX
How To Build and Run Node Apps with Docker and Compose
How To Build and Run Node Apps with Docker and Compose
Distributed Deep Learning with Docker at Salesforce
Distributed Deep Learning with Docker at Salesforce
The First 10M Pulls: Building The Official Curl Image for Docker Hub
The First 10M Pulls: Building The Official Curl Image for Docker Hub
COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...
COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...
Become a Docker Power User With Microsoft Visual Studio Code
Become a Docker Power User With Microsoft Visual Studio Code
How to Use Mirroring and Caching to Optimize your Container Registry
How to Use Mirroring and Caching to Optimize your Container Registry
Monolithic to Microservices + Docker = SDLC on Steroids!
Monolithic to Microservices + Docker = SDLC on Steroids!
Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment Model
Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment Model
Build & Deploy Multi-Container Applications to AWS
Build & Deploy Multi-Container Applications to AWS
From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...
From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...
Recently uploaded
20240609 QFM020 Irresponsible AI Reading List May 2024
Everything I found interesting about the irresponsible use of machine intelligence in May 2024
20240607 QFM018 Elixir Reading List May 2024
Everything I found interesting about the Elixir programming ecosystem in May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
Everything I found interesting about machines behaving intelligently during May 2024
Microsoft - Power Platform_G.Aspiotis.pdf
Revolutionizing Application Development
with AI-powered low-code, presentation by George Aspiotis, Sr. Partner Development Manager, Microsoft
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar
Video Streaming: Then, Now, and in the Future
In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.
Full-RAG: A modern architecture for hyper-personalization
Mike Del Balso, CEO & Co-Founder at Tecton, presents "Full RAG," a novel approach to AI recommendation systems, aiming to push beyond the limitations of traditional models through a deep integration of contextual insights and real-time data, leveraging the Retrieval-Augmented Generation architecture. This talk will outline Full RAG's potential to significantly enhance personalization, address engineering challenges such as data management and model training, and introduce data enrichment with reranking as a key solution. Attendees will gain crucial insights into the importance of hyperpersonalization in AI, the capabilities of Full RAG for advanced personalization, and strategies for managing complex data integrations for deploying cutting-edge AI solutions.
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
Leonard Jayamohan, Partner & Generative AI Lead, Deloitte
This keynote will reveal how Deloitte leverages Neo4j’s graph power for groundbreaking digital twin solutions, achieving a staggering 100x performance boost. Discover the essential role knowledge graphs play in successful generative AI implementations. Plus, get an exclusive look at an innovative Neo4j + Generative AI solution Deloitte is developing in-house.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Communications Mining Series - Zero to Hero - Session 1
This session provides introduction to UiPath Communication Mining, importance and platform overview. You will acquire a good understand of the phases in Communication Mining as we go over the platform with you. Topics covered:
• Communication Mining Overview
• Why is it important?
• How can it help today’s business and the benefits
• Phases in Communication Mining
• Demo on Platform overview
• Q/A
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
Sudheer Mechineni, Head of Application Frameworks, Standard Chartered Bank
Discover how Standard Chartered Bank harnessed the power of Neo4j to transform complex data access challenges into a dynamic, scalable graph database solution. This keynote will cover their journey from initial adoption to deploying a fully automated, enterprise-grade causal cluster, highlighting key strategies for modelling organisational changes and ensuring robust disaster recovery. Learn how these innovations have not only enhanced Standard Chartered Bank’s data infrastructure but also positioned them as pioneers in the banking sector’s adoption of graph technology.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
“I’m still / I’m still / Chaining from the Block”
“An Outlook of the Ongoing and Future Relationship between Blockchain Technologies and Process-aware Information Systems.” Invited talk at the joint workshop on Blockchain for Information Systems (BC4IS) and Blockchain for Trusted Data Sharing (B4TDS), co-located with with the 36th International Conference on Advanced Information Systems Engineering (CAiSE), 3 June 2024, Limassol, Cyprus.
Elizabeth Buie - Older adults: Are we really designing for our future selves?
Elizabeth Buie - Older adults: Are we really designing for our future selves?
Pushing the limits of ePRTC: 100ns holdover for 100 days
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
Mind map of terminologies used in context of Generative AI
Mind map of common terms used in context of Generative AI.
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
Maruthi Prithivirajan, Head of ASEAN & IN Solution Architecture, Neo4j
Get an inside look at the latest Neo4j innovations that enable relationship-driven intelligence at scale. Learn more about the newest cloud integrations and product enhancements that make Neo4j an essential choice for developers building apps with interconnected data and generative AI.
Removing Uninteresting Bytes in Software Fuzzing
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
Recently uploaded (20)
20240609 QFM020 Irresponsible AI Reading List May 2024
20240609 QFM020 Irresponsible AI Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
20240605 QFM017 Machine Intelligence Reading List May 2024
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf
Full-RAG: A modern architecture for hyper-personalization
Full-RAG: A modern architecture for hyper-personalization
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024
Communications Mining Series - Zero to Hero - Session 1
Communications Mining Series - Zero to Hero - Session 1
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
GraphSummit Singapore | Graphing Success: Revolutionising Organisational Stru...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...
Elizabeth Buie - Older adults: Are we really designing for our future selves?
Elizabeth Buie - Older adults: Are we really designing for our future selves?
Pushing the limits of ePRTC: 100ns holdover for 100 days
Pushing the limits of ePRTC: 100ns holdover for 100 days
Mind map of terminologies used in context of Generative AI
Mind map of terminologies used in context of Generative AI
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
GraphSummit Singapore | Neo4j Product Vision & Roadmap - Q2 2024
Docker and Radio Astronomy - Containing Fragile Scientific Software by Gijs Molenaar (University of Amsterdam)
- 1. Docker & radio astronomy 1
- 2. Who Am I Gijs Molenaar Scientific Software Engineer in NL PhD student in SA MSc Artificial Intelligence @gijzelaerr http://pythonic.nl 2
- 3. 3
- 4. 4
- 5. 5
- 6. 6
- 7. 7
- 8. 8
- 10. Scientist Q wants to crunch Data X with software Y on Platform Z 10
- 11. Scientist Q Smart person Self educated on the field of software engineer Gets the job done Good at duct tape programming 11
- 12. Data X Really big data Datasets of peta, exa bytes LOFAR - 50 GB/s SKA - 10 times global Internet traffic 12
- 13. Software Y 13 Complex math heavy software Written during PhD of scientist Q Often bad coding practices (Unit) tests? Version control? ‘Runs on my computer’
- 14. Platform Z 14
- 15. Solution until now 15
- 16. Solution until now 16 Compile everything by hand Still common practice (!)
- 17. 17
- 18. 18
- 21. $ docker run skasa/papino ipython notebook 21
- 22. Rodrigues - telescope simulator 22
- 23. 23
- 24. 24
- 25. 25
- 26. 26
- 27. 27
- 28. science vs scalable reliable web apps Much less users Much more data High availability less important Security less important Performance very important Durability very important 28
- 29. Lessons learned Don’t build virtual machine images use small minimal containers use exec to examine problems inside container 29
- 30. Problems to solve / future work GPU acceleration Adaptation ‘Mixing’ containers (nixos?) Container size (South Africa) Container aware programming language / library? 30
- 31. Thank You. 31 @gijzelaerr - http://pythonic.nl