DCSF 19 Online Feature Extraction and Event Generation for Computer-Animal Interaction

•

1 like•228 views

This talk will present an architecture developed to investigate the interaction with and between animals. The architecture allows online processing of multimedia streams and the generation, storing and visualizing of events using feature extraction. It allows biologists to analyze the events by monitoring live or by replaying streams through a web interface Docker swarm is the central component of the architecture and serves as infrastructure for stream processing, event generation, event processing and visualization. The main entry point for users is a web interface that spins up one container per user and allows independent replay of streams. This talk will focus on the architecture and on technical details concerning its implementation as well as how docker is utilized to process, store and visualize events. Some time will be spent explaining details about custom made docker solutions.

Technology

Online feature extraction
and event generation for
computer-animal
interaction
Leon Bonde Larsen & Mathias Neerup
Advanced Computer Systems Group

Server side playing > Multiple users > Containers

ATS v1.0> Limitations
▪ Janus WebRTC
▪ One user at a time

ATS v2.0
▪ Docker swarm
▪ React
▪ Redux
▪ SocketIO
▪ Semantic UI

ATS v2.0 > Janus
Janus
- Janus
- Input: Stream

ATS v2.0 > Backend
Backend
- Player
- Avahi Info.

ATS v2.0 > Plugin
Plugin
- A react
component
- Inputs
- Outputs
- Image

Container (service) pooling
▪ Pooler
▪ Service
▪ Acquired
▪ Released
▪ Policies
▪ Memory

Container (service) pooling
▪ Label on service
▪ SessionID
▪ Release policy
▪ 24 hours
▪ Closing session

Multicast traﬃc
▪ Multicast streams
▪ Enable IPv6
▪ Connect to bridge
https://github.com/mcproxy/mcproxy

Multicast traﬃc
▪ Multicast streams
▪ Avahi
▪ Avahi-daemon
▪ reflector
https://github.com/mcproxy/mcproxy

CD/CI - imageUpdater
▪ Gitlab docker registry

CD/CI - deploy strategy
▪ Gitlab docker registry

Docker swarm is suitable in biological research
Summary

Questions ?
Leon Bonde Larsen Mads Tilgaard Jensen Mathias Neerup
Ph.d Student programmer Research Assistant
https://acsg.sdu.dk

One of the common challenges for organizations that are moving more workloads to containers is running them inside a data center where you will still need to access both containerized and non-containerized workloads. In many organizations, that means you need to provide external access to and from a container and a traditional VLAN service or network via Layer 2. In this session, we’ll explain how you can interact with traditional network assets without routing through Layer 7 by using MACVLAN and Transparent Networks that allow you to use multiple MAC addresses on the same interface. We will outline how the concepts are similar but the implementations are different by demonstrating the same setup across both Linux and Windows using MACVLAN and Transparent Networks.

KubeCon EU 2016: Multi-Tenant Kubernetes

KubeAcademy

Docker Orchestration at Production Scale

Docker, Inc.

DCSF19 How Docker Simplifies Kubernetes for the Masses

Docker, Inc.

Jean Rouge & David Yu, Docker Kubernetes has taken the technology industry by storm these last few years. It delivers powerful orchestration and container management capabilities that have been leveraged by cloud-scale companies and small startups alike. But for many organizations, the learning curve for Kubernetes can be steep and organizations can't build up their skills fast enough. Luckily Docker has always had a history of making the complex easy - first with Linux containers and now with Kubernetes - both in our Desktop and Enterprise platform. In this session, we'll highlight some of the innovation Docker has added to Kubernetes to simplify configuration and ongoing operations while still providing a fully conformant Kubernetes environment. We'll cover areas like deploying applications on Kubernetes, managing access controls and multi-tenancy, end-to-end security and improved troubleshooting. Demos will highlight key comparisons to show you that you don't have to build it yourself.

DockerCon EU 2015: What's New with Docker Trusted Registry

Docker, Inc.

Production Ready Containers from IBM and Docker

Docker, Inc.

Containers are quickly becoming the default foundation for modern applications. As a public cloud provider, IBM has been an early champion of containers in the cloud and has built an enterprise ready container service as part of IBM Bluemix. IBM has a long heritage of supporting, contributing to, and building offerings on top of open technologies and IBM carries this commitment to the open development of container solutions by being an active/founding member of the Open Containers Initiative and Cloud Native Computing Foundation. In this session, we will explore the enduring commitment to open technology as well as the advantages of using a pure containers service where the user has access to total solution life cycle management through integration of lessons learned, cutting edge enhancements/development and end-to-end support on the user's underlying infrastructure. We will explore topics such as exploiting bare metal servers, applying overlay networking to containers, ensuring isolation and security in a truly multi-tenant container environment and managing a global service deployment.

Scale Kubernetes to support 50000 services

LinuxCon ContainerCon CloudOpen China

Kubernetes currently has two load balancing mode: userspace and IPTables. They both have limitation on scalability and performance. We introduced IPVS as third kube-proxy mode which scales kubernetes load balancer to support 50,000 services. Beyond that, control plane needs to be optimized in order to deploy 50,000 services. We will introduce alternative solutions and our prototypes with detailed performance data.

DockerCon EU 2015: The Latest in Docker Engine

Docker, Inc.

While working on Docker for AWS and Azure, we realized the need for a standard way to create and manage infrastructure state that was portable across any type of infrastructure, from different cloud providers to on-prem. One challenge is that each vendor has differentiated IP invested in how they handle certain aspects of their cloud infrastructure. It is not enough to just provision five servers; what IT ops teams need is a simple and consistent way to declare the number of servers, what size they should be, and what sort of base software configuration is required. And in the case of server failures (especially unplanned), that sudden change needs to be reconciled against the desired state to ensure that any required servers are re-provisioned with the necessary configuration. We started InfraKit to solves these problems and to provide the ability to create a self healing infrastructure for distributed systems.

Kubernetes 架構與虛擬化之差異

inwin stack

What's New in Docker 1.12 (June 20, 2016) by Mike Goelzer & Andrea Luzzardi

Mike Goelzer

runC: The little engine that could (run Docker containers) by Docker Captain ...

Docker, Inc.

With the announcement of the OCI by Solomon Hykes at last summer's DockerCon, a Docker-contributed reference implementation of the OCI spec, called runC, was born. While some of you may have tried runC or have a history of poking at the OS layer integration library to Linux namespaces, cgroups and the like (known as libcontainer), many of you may not know what runC offers. In this talk Phil Estes, Docker engine maintainer who has also contributed to libcontainer and runC, will show what's possible using runC as a lightweight and fast runtime environment to experiment with lower-level features of the container runtime. Phil will introduce a conversion tool called "riddler", which can inspect and convert container configurations from Docker into the proper OCI configuration bundle for easy conversion between the two environments. He'll also demonstrate how to make custom configurations for trying out security features like user namespaces and seccomp profiles.

Thinking Inside the Container: A Continuous Delivery Story by Maxfield Stewart

Docker, Inc.

Riot builds a lot of software. At the start of 2015 we were looking at 3000 build jobs over a hundred different applications and dozens of teams. We were handling nearly 750 jobs per hour and our build infrastructure needed to grow rapidly to meet demand. We needed to give teams total control of the “stack” used to build their applications and we needed a solution that enabled agile delivery to our players. On top of that, we needed a scalable system that would allow a team of four engineers to support over 250. After as few explorations, we built an integrated Docker solution using Jenkins that accepts docker images submitted as build environments by engineers around the company . Our “containerized” farm now creates over 10,000 containers a week and handles nearly 1000 jobs at a rate of about 100 jobs an hour. In this occasionally technical talk, we’ll explore the decisions that led Riot to consider Docker, the evolutionary stages of our build infrastructure, and how the open source and in-house software we combined to achieve our goals at scale. You’ll come away with some best practices, plenty of lessons learned, and insight into some of the more unique aspects of our system (like automated testing of submitted build environments, or testing node.js apps in containers with Chromium and xvfb).

Introduction to OCI Image Technologies Serving Container

LinuxCon ContainerCon CloudOpen China

Securing your Containers

Riyaz Faizullabhoy

The state of containerd

Docker, Inc.

Docker for Ops: Docker Networking Deep Dive, Considerations and Troubleshooti...

Docker, Inc.

AtlasCamp 2015: The age of orchestration: From Docker basics to cluster manag...

Atlassian

Nicola Paolucci, Atlassian Containers hit the collective developer mind with great force the past two years and created a space of fervent innovation. Now work is moving towards orchestration. In this session we'll cover an overview of the container orchestration landscape, give an introduction to Docker's own tools - machine, swarm and compose - and show a (semi)live demo of how they work in practice.

Sharding Containers: Make Go Apps Computer-Friendly Again by Andrey Sibiryov

Docker, Inc.

Go is, without doubt, a great language for writing massively concurrent programs. Nevertheless, our experience running Go under extreme load shows that there comes a point where assumptions and decisions made in Go runtime bite back on its users and lead to inferior performance, especially in high-throughput & high-load applications. This talk covers main reasons for this to happen and explores an interesting way to work around this issue: automatic local sharding with Docker. Using Docker, local load-balancer and creativity, we can automatically shard & pin our apps in such a way so that the external observer (client, another microservice) would never see any difference. The result is that apps run faster, resource utilization is better, engineers are not frustrated when their Go suddenly breaks down and runs slow because they have a solution!

KubeCon EU 2016: Heroku to Kubernetes

KubeAcademy

Arkena's video-on-demand platform is used as backend by major european channels (TF1 / beIN SPORTS / Elisa) to propose a non-linear experience to their customers. Previously hosted on Heroku, the number of our users is increasing constantly. In order to optimize resources we decided to move on a bare metal infrastructure powered by Kubernetes. We'll share thoughts, feedbacks and technical details about this successful transition. Sched Link:

Outside The Wire

Salo Shp

Persistent Data Storage for Docker Containers by Andre Moruga

Docker, Inc.

Bitbucket Pipelines - Powered by Kubernetes

Nathan Burrell

Testing kubernetes and_open_shift_at_scale_20170209

mffiedler

containerd the universal container runtime

Docker, Inc.

containerd is an industry-standard core container runtime with an emphasis on simplicity, robustness and portability. It is available as a daemon for Linux and Windows, which can manage the complete container lifecycle of its host system: image transfer and storage, container execution and supervision, low-level storage and network attachments, etc.. containerd is designed to be embedded into a larger system, rather than being used directly by developers or end-users. containerd includes a daemon exposing gRPC API over a local UNIX socket. The API is a low-level one designed for higher layers to wrap and extend. It also includes a barebone CLI (ctr) designed specifically for development and debugging purpose. It uses runC to run containers according to the OCI specification. The code can be found on GitHub, and here are the contribution guidelines. containerd is based on the Docker Engine’s core container runtime to benefit from its maturity and existing contributors.

DockerCon EU 2015: Shipping Manifests, Bill of Lading and Docker Metadata and...

Docker, Inc.

Presented by Gareth Rushgrove, Sr. Software Engineer, Puppet Labs The shipping container metaphor for Docker points to many of the advantages of building and running software using containers. But what about other essential parts of the shipping container ecosystem like the shipping manifest and bill of lading? Many of the most powerful features of traditional package management tools like apt or yum are based on metadata associated with the packages. You can find out who created a package and when, check where a particular file came from, whether the package has a known vulnerability and more. What would this capability look like for Docker containers? This talk will look at the power of metadata for containers, in particular: * Docker provides labels for associating metadata with images and containers but how best to use them?* What problems can be solved by agreeing on standards for container metadata?* Exposing standard commands and endpoints to expose metadata about what is inside a container* Demo some open source toolings and also look at the sort of tools we might build atop those standards and low-level tools.

Private cloud networking_cloudstack_days_austin

Chiradeep Vittal

Eduardo Silva - monkey http-server everywhereStarTech Conference

What's hot

Remote secured storage

Salo Shp

Virtualization inside kubernetes

inwin stack

Docker Online Meetup: Infrakit update and Q&A

Docker, Inc.

Kubernetes 架構與虛擬化之差異

inwin stack

What's New in Docker 1.12 (June 20, 2016) by Mike Goelzer & Andrea Luzzardi

Mike Goelzer

runC: The little engine that could (run Docker containers) by Docker Captain ...

Docker, Inc.

Thinking Inside the Container: A Continuous Delivery Story by Maxfield Stewart

Docker, Inc.

Introduction to OCI Image Technologies Serving Container

LinuxCon ContainerCon CloudOpen China

Securing your Containers

Riyaz Faizullabhoy

The state of containerd

Docker, Inc.

Docker for Ops: Docker Networking Deep Dive, Considerations and Troubleshooti...

Docker, Inc.

AtlasCamp 2015: The age of orchestration: From Docker basics to cluster manag...

Atlassian

Sharding Containers: Make Go Apps Computer-Friendly Again by Andrey Sibiryov

Docker, Inc.

KubeCon EU 2016: Heroku to Kubernetes

KubeAcademy

Outside The Wire

Salo Shp

Persistent Data Storage for Docker Containers by Andre Moruga

Docker, Inc.

Bitbucket Pipelines - Powered by Kubernetes

Nathan Burrell

Testing kubernetes and_open_shift_at_scale_20170209

mffiedler

containerd the universal container runtime

Docker, Inc.

DockerCon EU 2015: Shipping Manifests, Bill of Lading and Docker Metadata and...

Docker, Inc.

What's hot (20)

Remote secured storage

Virtualization inside kubernetes

Docker Online Meetup: Infrakit update and Q&A

Kubernetes 架構與虛擬化之差異

What's New in Docker 1.12 (June 20, 2016) by Mike Goelzer & Andrea Luzzardi

runC: The little engine that could (run Docker containers) by Docker Captain ...

Thinking Inside the Container: A Continuous Delivery Story by Maxfield Stewart

Introduction to OCI Image Technologies Serving Container

Securing your Containers

The state of containerd

Docker for Ops: Docker Networking Deep Dive, Considerations and Troubleshooti...

AtlasCamp 2015: The age of orchestration: From Docker basics to cluster manag...

Sharding Containers: Make Go Apps Computer-Friendly Again by Andrey Sibiryov

KubeCon EU 2016: Heroku to Kubernetes

Outside The Wire

Persistent Data Storage for Docker Containers by Andre Moruga

Bitbucket Pipelines - Powered by Kubernetes

Testing kubernetes and_open_shift_at_scale_20170209

containerd the universal container runtime

DockerCon EU 2015: Shipping Manifests, Bill of Lading and Docker Metadata and...

Similar to DCSF 19 Online Feature Extraction and Event Generation for Computer-Animal Interaction

Private cloud networking_cloudstack_days_austin

Chiradeep Vittal

Eduardo Silva - monkey http-server everywhereStarTech Conference

Realtime Messaging und verteilte Systeme mit SharePoint und Windows Azure Ser...

Damir Dobric

Data power v7 update - Ravi Katikalafloridawusergroup

Ajax Push For Revolutionary Enterprise Applicationselliando dias

Developing Revolutionary Web Applications using Comet and Ajax Push

Doris Chen

Join the asynchronous web revolution! Because Ajax-based applications are almost becoming the de facto technology for designing web-based applications, it is more and more important that such applications react on the fly, or in real time, to both client and server events. AJAX can be used to allow the browser to request information from the web server, but does not allow a server to push updates to a browser. Comet solves this problem. Comet is a technology that enables web clients and web servers to communicate asynchronously, allowing real-time operations and functions previously unheard of with traditional web applications to approach the capabilities of desktop applications. This session will start to provide an brief introduction to the asynchronous web, AJAX polling, long polling, and Streaming, explaining the Bayeux protocol, Cometd, Grizzly Comet implementation on GlassFish. Different approaches and best practices to develop comet application will also be discussed. You will learn how to develop the chat application, how to implement distance learning slideshow application, how to manage a chat application from the server and how to develop a two-player distributed game application. Attendees will take away the tactics they need in order to add multiuser collaboration, notification and other Comet features to their application, whether they develop with Dojo, jQuery, jMaki, or Prototype and whether they deploy on Jetty, Tomcat, or the GlassFish Application Server.

WebRTC Webinar & Q&A - Sumilcast Standards & Implementation

Amir Zmora

Best practices of building data streaming API

Constantine Slisenka

Interested in building real-time apps like gmail or facebook? In many cases we need to notify clients immediately when something happens: stock prices, online games, chats, betting apps, etc. In this talk we will discuss how to design and build data-streaming API using WebSockets and other related technologies. We plan to cover problems and challenges and overview several libraries and products in this area.

Introduction to WebSockets

Gunnar Hillert

WebSockets: The Current State of the Most Valuable HTML5 API for Java DevelopersViktor Gamov

LISA18: Hidden Linux Metrics with Prometheus eBPF Exporter

Ivan Babrou

Presented at LISA18: https://www.usenix.org/conference/lisa18/presentation/huynh While there are plenty of readily available metrics for monitoring Linux kernel, many gems remain hidden. With the help of recent developments in eBPF, it is now possible to run safe programs in the kernel to collect arbitrary information with little to no overhead. A few examples include: * Disk latency and io size histograms * Run queue (scheduler) latency * Page cache efficiency * Directory cache efficiency * LLC (aka L3 cache) efficiency * Kernel timer counters * System-wide TCP retransmits Practically any event from "perf list" output and any kernel function can be traced, analyzed and turned into a Prometheus metric with almost arbitrary labels attached to it. If you are already familiar with BCC tools, you may think if ebpf_exporter as bcc tools turned into prometheus metrics. In this tutorial we’ll go over eBPF basics, how to write programs and get insights into a running system.

Flink Forward Berlin 2018: Nico Kruber - "Improving throughput and latency wi...

Flink Forward

Flink's network stack is designed with two goals in mind: (a) having low latency for data passing through, and (b) achieving an optimal throughput. It already achieves a good trade-off between these two but we will continue to tune it further for an optimal one. Flink 1.5 did not only further reduce overhead inside the network stack, but also introduced some groundbreaking changes in the stack itself, introducing an own (credit-based) flow control mechanism and a general change towards a more network-event-driven pull approach. With our own flow control, we steer what data is sent on the wire and make sure that the receiver has capacity to handle it. If there is no capacity, we will backpressure earlier and allow checkpoint barriers to pass through more quickly which improves checkpoint alignment and overall checkpoint speed. In this talk, we will describe in detail how the network stack works with respect to (de)serialization, memory management and buffer pools, the different actor threads involved, configuration, as well as the design to improve throughput and latency trade-offs in the network and processing pipelines. We will go into detail on what is happening during checkpoint alignments and how credit-based flow control improves them. Additionally, we will present common pitfalls as well as debugging techniques to find network-related issues in your Flink jobs.

Harmonia open iris_basic_v0.1

Yongyoon Shin

OpenStack cloud for ConoHa, Z.com and GMO AppsCloud in okinawa opendays 2015 ...

Naoto Gohko

WebRTC Seminar Report

srinivasa teja

Rudder 3.0 and beyond

Jonathan Clarke

Rudder 3.0 was released in January 2015. This talk will bring attendees up to date on recent evolutions in Rudder and show off some of the latest and greatest features like the new compliance dashboard and graphs, redesigned web interface, built-in Technique editor (that automatically builds CFEngine code), basic command line interface, ... We will then discuss ideas for future features. Last but not least, we should have some time to dig deeper into any parts of Rudder attendees want to know more about - examples could include reporting, ncf, OS support, CFEngine integration, ...

OAuth-as-a-service using ASP.NET Web API and Windows Azure Access Control - W...

Maarten Balliauw

API’s are the new apps. They can be consumed by everyone using a web browser or a mobile application on their smartphone or tablet. How would you build your API if you want these apps to be a full-fledged front-end to your service without compromising security? In this session, Maarten will explain how to build an API using the ASP.NET Web API framework and how the Windows Azure Access Control service can be used to almost completely outsource all security and OAuth-related tasks.

AWS re:Invent 2016: Amazon CloudFront Flash Talks: Best Practices on Configur...

Amazon Web Services

StrongLoop Overview

Shubhra Kar

GMOインターネット様発表「OpenStackのモデルの最適化とConoHa, Z.comとGMOアプリクラウドへの適用」 - OpenStack最新情...

VirtualTech Japan Inc.

OpenStackのモデルの最適化とConoHa, Z.comとGMOアプリクラウドへの適用郷古直仁 (GMOインターネット株式会社) アジェンダ： - Using OpenStack at GMO Internet --- OpenStack Diablo cluster: Onamae.com VPS --- OpenStack Grizzly cluster: ConoHa --- OpenStack Havana cluster: GMO AppsCloud --- OpenStack Juno cluster: ConoHa (2), GMO AppsCloud (2) ------ OpenStack Authentication in Juno (V2 keystone domains) - How to using OpenStack at GMO Internet --- OpenStack Designate DNSaaS: ConoHa, Z.com(OEM) --- OpenStack Cinder Block storage: ConoHa: NexentaStor(SDS), AppsCloud: NetApp --- OpenStack Ironic: Only AppsCloud: Undercloud Ironic deploy, Multi-tenant Ironic deploy --- OpenStack Swift: shared cluster

Similar to DCSF 19 Online Feature Extraction and Event Generation for Computer-Animal Interaction (20)

Private cloud networking_cloudstack_days_austin

Eduardo Silva - monkey http-server everywhere

Realtime Messaging und verteilte Systeme mit SharePoint und Windows Azure Ser...

Data power v7 update - Ravi Katikala

Ajax Push For Revolutionary Enterprise Applications

Developing Revolutionary Web Applications using Comet and Ajax Push

WebRTC Webinar & Q&A - Sumilcast Standards & Implementation

Best practices of building data streaming API

Introduction to WebSockets

WebSockets: The Current State of the Most Valuable HTML5 API for Java Developers

LISA18: Hidden Linux Metrics with Prometheus eBPF Exporter

Flink Forward Berlin 2018: Nico Kruber - "Improving throughput and latency wi...

Harmonia open iris_basic_v0.1

OpenStack cloud for ConoHa, Z.com and GMO AppsCloud in okinawa opendays 2015 ...

WebRTC Seminar Report

Rudder 3.0 and beyond

OAuth-as-a-service using ASP.NET Web API and Windows Azure Access Control - W...

AWS re:Invent 2016: Amazon CloudFront Flash Talks: Best Practices on Configur...

StrongLoop Overview

GMOインターネット様発表「OpenStackのモデルの最適化とConoHa, Z.comとGMOアプリクラウドへの適用」 - OpenStack最新情...

More from Docker, Inc.

Containerize Your Game Server for the Best Multiplayer Experience

Docker, Inc.

Raymond Arifianto, AccelByte and Mark Mandel, Google - We have been deploying containerized micro-services for our Game Backend Services for a while. Now we are tackling the challenge to scale up fleets of game dedicated servers in multiple regions, multiple data centers and multiple providers - some in bare metal, some in Cloud. So we leverage docker containerization to deploy Game Servers to achieve Portability, Fast Deployment and Predictability, enabling us to scale up to thousands of servers, on demand, without a sweat.

How to Improve Your Image Builds Using Advance Docker Build

Docker, Inc.

Nicholas Dille, Haufe-Lexware + Docker Captain - Docker continues to be the standard tool for building container images. For more than a year Docker ships with BuildKit as an alternative image builder, providing advanced features for secret and cache management. These features help to make image builds faster and more secure. In this session, Docker Captain Nicholas Dille will teach you how to use Buildkit features to your advantage.

Build & Deploy Multi-Container Applications to AWS

Docker, Inc.

Lukonde Mwila, Entelect - As the cloud-native approach to development and deployment becomes more prevalent, it's an exciting time for software engineers to be equipped on how to dockerize multi-container applications and deploy them to the cloud. In this talk, Lukonde Mwila, Software Engineer at Entelect, will cover the following topics: - Docker Compose - Containerizing an Nginx Server - Containerizing an React App - Containerizing an Node.JS App - Containerizing anMongoDB App - Runing Multi-Container App Locally - Creating a CI/CD Pipeline - Adding a build stage to test containers and push images to Docker Hub - Deploying Multi-Container App to AWS Elastic Beanstalk Lukonde will start by giving an overview of how Docker Compose works and how it makes it very easy and straightforward to startup multiple Docker containers at the same time and automatically connect them together with some form of networking. After that, Lukonde will take a hands on approach to containerize an Nginx server, a React app, a NodeJS app and a MongoDB instance to demonstrate the power of Docker Compose. He'll demonstrate usage of two Docker files for an application, one production grade and the other for local development and running of tests. Lastly, he'll demonstrate creating a CI/CD pipeline in AWS to build and test our Docker images before pushing them to Docker Hub or AWS ECR, and finally deploying our multi-container application AWS Elastic Beanstalk.

Securing Your Containerized Applications with NGINX

Docker, Inc.

Kevin Jones, NGNIX - NGINX is one of the most popular images on Docker Hub and has been at the forefront of the web since the early 2000's. In this talk we will discuss how and why NGINX's lightweight and powerful architecture makes it a very popular choice for securing containerized applications as a sidecar reverse proxy within containers. We will highlight important aspects of application security that NGINX can help with, such as TLS, HTTP, AuthN, AuthZ and traffic control.

How To Build and Run Node Apps with Docker and Compose

Docker, Inc.

Kathleen Juell, Digital Ocean - Containers are an essential part of today's microservice ecosystem, as they allow developers and operators to maintain standards of reliability and reproducibility in fast-paced deployment scenarios. And while there are best practices that extend across stacks in containerized environments, there are also things that make each stack distinct, starting with the application image itself. This talk will dive into some of these particularities, both at the image and service level, while also covering general best practices for building and running Node applications with database backends using Docker and Compose.

Hands-on Helm

Docker, Inc.

Distributed Deep Learning with Docker at Salesforce

Docker, Inc.

Jeff Hajewski, Salesforce - There is a wealth of information on building deep learning models with PyTorch or TensorFlow. Anyone interested in building a deep learning model is only a quick search away from a number of clear and well written tutorials that will take them from zero knowledge to having a working image classifier. But what happens when you need to deploy these models in a production setting? At Salesforce, we use TensorFlow models to help us provide customers with insights into their data, and we do this as close to real-time as possible. Designing these systems in a scalable manner requires overcoming a number of design challenges, but the core component is Docker. Docker enables us to design highly scalable systems by allowing us to focus on service interactions, rather than how our services will interact with the hardware. Docker is also at the core of our test infrastructure, allowing developers and data scientists to build and test the system in an end to end manner on their local machines. While some of this may sound complex, the core message is simplicity - Docker allows us to focus on the aspects of the system that matter, greatly simplifying our lives.

The First 10M Pulls: Building The Official Curl Image for Docker Hub

Docker, Inc.

James Fuller, webcomposite s.r.o. - Curl is the venerable (yet very modern) 'swiss army knife' command line tool and library for transferring data with URLs. Recently we (the Curl team) decided to build a release for Docker Hub. This talk will outline our current development workflow with respect to the docker image and provide insights on what it takes to build a docker image for mass public consumption. We are also keen to learn from users and other developers how we might improve and enhance the official curl docker image.

Monitoring in a Microservices World

Docker, Inc.

Fabian Stäber, Instana - In recent years, we saw a great paradigm shift in software engineering away from static monolithic applications towards dynamic distributed horizontally scalable architectures. Docker is one of the key technologies enabling this development. This shift poses a lot of new challenges for application monitoring, ranging from practical issues (need for automation) to technical challenges (Docker networking) to organizational topics (blurring line between software engineers and operations) to fundamental questions (define what is an application). In this talk we show how Docker changed the way we do monitoring, how modern application monitoring systems work, and what future developments we expect.

COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...

Docker, Inc.

Clemente Biondo, Engineering Ingegneria Informatica - When the COVID 19 pandemic started, Engineering Ingegneria Informatica Group (1.25 billion euros of revenues, 65 offices around the world, 12.000 employees) was forced to put their digital transformation to the test in order to maintain operational continuity. In this session, Clemente Biondo, the Tech Lead of the Information Systems Department, will share how his company is reacting to this unforeseeable scenario and how Docker-driven digital transformation had paved the path for work to continue remotely. Clemente will discuss learnings moving from colocated teams, manual approaches, email based-business processes, and a monolithic application to a mature DevOps culture characterized by a distributed autonomous workforce and a continuous deployment process that deploys backward-compatible Docker containerized microservices into hybrid multi cloud datacenters an average of twice a day with zero-downtime. He will detail how they use Docker to unify dev, test and production environments, and as an efficient and automated mechanism for deploying applications. Lastly, Clemente shares how, in our darkest hour, he and others are working to shine their brightest light.

Predicting Space Weather with Docker

Docker, Inc.

Chris Lauer, NOAA Space Weather Prediction Center - This is the story of how adopting a containerized workflow changed the way our small software team works at NOAA’s Space Weather Prediction Center. Our old architecture, a big ball of mud shared-database integration, just wasn’t cutting it - it was killing our agility. Over the past two years, our small team has adopted a microservice style architecture, using Docker with docker-compose and environment files as our deployment strategy for all new development. We’ve discovered the joys of using containers for identical dev, staging, and production environments. We work closely with scientists: much of the code we’re running has complicated and conflicting library dependencies. Docker captures these beautifully - we’ve even had some success teaching our scientists to use it! I’ll share what we’ve learned, some of the persistent challenges we face, and one place we really got it wrong. This talk builds off of a popular hallway track from DockerCon 2019.

Become a Docker Power User With Microsoft Visual Studio Code

Docker, Inc.

Brian Christner, 56k + Docker Captain - In this session, we will unlock the full potential of using Microsoft Visual Studio Code (VS Code) and Docker Desktop to turn you into a Docker Power User. When we expand and utilize the VS Code Docker plugin, we can take our projects and Docker skills to the next level. In addition to using VS Code, we streamline our Docker Desktop development workflow with less context switching and built-in shortcuts. You will learn how to bootstrap new projects, quickly write Dockerfiles utilizing templates, build, run, and interact with containers all from VS Code.

How to Use Mirroring and Caching to Optimize your Container Registry

Docker, Inc.

Monolithic to Microservices + Docker = SDLC on Steroids!

Docker, Inc.

Ashish Sharma, SS&C Eze - SS&C Eze provides various products in the stock market domain. We spent the last couple of years building Eclipse which is an investment suite born in cloud. The journey so far has been very interesting. The very first version of the product were a bunch of monolithic windows services and deployed using Octopus tool. We successfully managed to bring all the monolithic problem to the cloud and created a nightmare for ourselves. We then started applying microservices architecture principles and started breaking the monolithic into small services. Very soon we realized that we need a better packaging/deployment tool. Docker looked like a magical solution to our problem. Since its adoption, It has not only solved the deployment problem for us but has made a deep impact on different aspects of SDLC. It allowed us to use heterogeneous technology stacks, simplified development environment setup, simplified our testing strategy, improved our speed of delivery, and made our developers more productive. In this talk I would like to share our experience of using Docker and its positive impact on our SDLC.

Kubernetes at Datadog Scale

Docker, Inc.

Ara Pulido, Datadog - Container technologies, although not new, have increased their popularity in the past few years, with container orchestrators allowing companies around the world to adopt these technologies to help them ship and scale microservices with precision and velocity. Kubernetes is currently the most popular container orchestration platform, and while many organizations are migrating their workloads to it, Kubernetes is still relatively immature. New corner cases, errors, and quirks are regularly discovered as users push the boundaries of size and scale. When Datadog adopted Kubernetes we discovered some of these boundaries the hard way, and we continuously challenge and modify our infrastructure decisions in order to fit our use case. Join me in this talk for our story on what we learned while we scaled our Kubernetes clusters, the contributions to Kubernetes we made along the way, and how you can apply those learnings when growing your Kubernetes clusters from a handful to hundreds or thousands of nodes.

Labels, Labels, Labels

Docker, Inc.

Andy Clemenko, StackRox - One underutilized, and amazing, thing about the docker image scheme is labels. Labels are a built in way to document all aspects about the image itself. Think about all the information that the tags inside your clothing carry. If you care to look you can find out everything about the garment. All that information can be very valuable. Now think about how we can leverage labels to carry similar information. We can even use the labels to contain Docker Compose or even Kubernetes Yaml. We can even include labels into the CI/CD process making things more secure and smoother. Come find out some fun techniques on how to leverage labels to do some fun and amazing things.

Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment Model

Docker, Inc.

Patrick Deloulay, Micro Focus - Micro Focus started their digital transformation 3 years ago, moving the entire portfolio into hundreds of container images. Leveraging Docker Hub as our primary registry service, we will cover how we ended up building a simple but secure push/pull model to publish and deliver our premium assets to our customers and partners to both meet the high agility of our DevOps teams while greatly simplifying the deployment of our applications.

Build & Deploy Multi-Container Applications to AWS

Docker, Inc.

Lukonde Mwila, Entelect As the cloud-native approach to development and deployment becomes more prevalent, it's an exciting time for software engineers to be equipped on how to dockerize multi-container applications and deploy them to the cloud. In this talk, Lukonde Mwila, Software Engineer at Entelect, will cover the following topics: - Docker Compose - Containerizing an Nginx Server - Containerizing an React App - Containerizing an Node.JS App - Containerizing anMongoDB App - Runing Multi-Container App Locally - Creating a CI/CD Pipeline - Adding a build stage to test containers and push images to Docker Hub - Deploying Multi-Container App to AWS Elastic Beanstalk Lukonde will start by giving an overview of how Docker Compose works and how it makes it very easy and straightforward to startup multiple Docker containers at the same time and automatically connect them together with some form of networking. After that, Lukonde will take a hands on approach to containerize an Nginx server, a React app, a NodeJS app and a MongoDB instance to demonstrate the power of Docker Compose. He'll demonstrate usage of two Docker files for an application, one production grade and the other for local development and running of tests. Lastly, he'll demonstrate creating a CI/CD pipeline in AWS to build and test our Docker images before pushing them to Docker Hub or AWS ECR, and finally deploying our multi-container application AWS Elastic Beanstalk.

From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...

Docker, Inc.

Elton Stoneman, Docker Captain + Container Consultant and Trainer How do you provide a SaaS offering when your product is a 10-year old Fortran app, currently built to run on Windows 10? With Docker and Kubernetes of course - and you can do it in a week (... to prototype level at least). In this session I'll walk through the processes and practicalities of taking an older Windows app, making it run in containers with Kubernetes, and then building a simple API wrapper to host the whole stack as a cloud-based SaaS product. There's a lot of technology here from a real world case study, and I'll focus on: - running Windows apps in Docker containers - building a .NET Core API which can run in Linux or Windows containers - running the stack in Kubernetes with Docker Desktop locally and AKS in the cloud - configuring AKS workloads in Azure to burst out to Azure Container Instances And there's a core theme to this session: Docker and Kubernetes are complex technologies, but they're the key to modern development. If you invest time learning them, they make projects like this simple, portable, fast and fun.

Developing with Docker for the Arm Architecture

Docker, Inc.

This virtual meetup introduces the concepts and best practices of using Docker containers for software development for the Arm architecture across a variety of hardware systems. Using Docker Desktop on Windows or Mac, Amazon Web Services (AWS) A1 instances, and embedded Linux, we will demonstrate the latest Docker features to build, share, and run multi-architecture images with transparent support for Arm.

More from Docker, Inc. (20)

Containerize Your Game Server for the Best Multiplayer Experience

How to Improve Your Image Builds Using Advance Docker Build

Build & Deploy Multi-Container Applications to AWS

Securing Your Containerized Applications with NGINX

How To Build and Run Node Apps with Docker and Compose

Hands-on Helm

Distributed Deep Learning with Docker at Salesforce

The First 10M Pulls: Building The Official Curl Image for Docker Hub

Monitoring in a Microservices World

COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...

Predicting Space Weather with Docker

Become a Docker Power User With Microsoft Visual Studio Code

How to Use Mirroring and Caching to Optimize your Container Registry

Monolithic to Microservices + Docker = SDLC on Steroids!

Kubernetes at Datadog Scale

Labels, Labels, Labels

Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment Model

Build & Deploy Multi-Container Applications to AWS

From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...

Developing with Docker for the Arm Architecture

Recently uploaded

Accelerate your Kubernetes clusters with Varnish Caching

Thijs Feryn

Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf

91mobiles

Free Complete Python - A step towards Data Science

RinaMondal9

SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdf

Peter Spielvogel

Building better applications for business users with SAP Fiori. • What is SAP Fiori and why it matters to you • How a better user experience drives measurable business benefits • How to get started with SAP Fiori today • How SAP Fiori elements accelerates application development • How SAP Build Code includes SAP Fiori tools and other generative artificial intelligence capabilities • How SAP Fiori paves the way for using AI in SAP apps

zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex Proofs

Alex Pruden

This paper presents Reef, a system for generating publicly verifiable succinct non-interactive zero-knowledge proofs that a committed document matches or does not match a regular expression. We describe applications such as proving the strength of passwords, the provenance of email despite redactions, the validity of oblivious DNS queries, and the existence of mutations in DNA. Reef supports the Perl Compatible Regular Expression syntax, including wildcards, alternation, ranges, capture groups, Kleene star, negations, and lookarounds. Reef introduces a new type of automata, Skipping Alternating Finite Automata (SAFA), that skips irrelevant parts of a document when producing proofs without undermining soundness, and instantiates SAFA with a lookup argument. Our experimental evaluation confirms that Reef can generate proofs for documents with 32M characters; the proofs are small and cheap to verify (under a second). Paper: https://eprint.iacr.org/2023/1886

Removing Uninteresting Bytes in Software Fuzzing

Aftab Hussain

Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process. In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds. - These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.

Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...

UiPathCommunity

💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™: See how to accelerate model training and optimize model performance with active learning Learn about the latest enhancements to out-of-the-box document processing – with little to no training required Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath. Speakers: 👨‍🏫 Andras Palfi, Senior Product Manager, UiPath 👩‍🏫 Lenka Dulovicova, Product Program Manager, UiPath

FIDO Alliance Osaka Seminar: Passkeys at Amazon.pdf

FIDO Alliance

UiPath Community Day Dubai: AI at Work..

UiPathCommunity

Welcome to the first live UiPath Community Day Dubai! Join us for this unique occasion to meet our local and global UiPath Community and leaders. You will get a full view of the MEA region's automation landscape and the AI Powered automation technology capabilities of UiPath. Also, hosted by our local partners Marc Ellis, you will enjoy a half-day packed with industry insights and automation peers networking. 📕 Curious on our agenda? Wait no more! 10:00 Welcome note - UiPath Community in Dubai Lovely Sinha, UiPath Community Chapter Leader, UiPath MVPx3, Hyper-automation Consultant, First Abu Dhabi Bank 10:20 A UiPath cross-region MEA overview Ashraf El Zarka, VP and Managing Director MEA, UiPath 10:35: Customer Success Journey Deepthi Deepak, Head of Intelligent Automation CoE, First Abu Dhabi Bank 11:15 The UiPath approach to GenAI with our three principles: improve accuracy, supercharge productivity, and automate more Boris Krumrey, Global VP, Automation Innovation, UiPath 12:15 To discover how Marc Ellis leverages tech-driven solutions in recruitment and managed services. Brendan Lingam, Director of Sales and Business Development, Marc Ellis

Generative AI Deep Dive: Advancing from Proof of Concept to Production

Aggregage

FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf

FIDO Alliance

Epistemic Interaction - tuning interfaces to provide information for AI support

Alan Dix

Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024 https://alandix.com/academic/papers/synergy2024-epistemic/ As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.

FIDO Alliance Osaka Seminar: Overview.pdf

FIDO Alliance

FIDO Alliance Osaka Seminar: Passkeys and the Road Ahead.pdf

FIDO Alliance

Elizabeth Buie - Older adults: Are we really designing for our future selves?

Nexer Digital

The Art of the Pitch: WordPress Relationships and Sales

Laura Byrne

Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes? All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.

Quantum Computing: Current Landscape and the Future Role of APIs

Vlad Stirbu

Monitoring Java Application Security with JDK Tools and JFR Events

Ana-Maria Mihalceanu

Assure Contact Center Experiences for Your Customers With ThousandEyes

ThousandEyes

FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf

FIDO Alliance