Repainting the Past with Distributed Machine Learning and Docker
•
2 likes•715 views
"A picture is worth a thousand words” – Frederick R Bernard. Video is worth thousands more. With millions of hours of black and white video footage circulating around the internet or locked indefinitely in storage archives, their true stories and colours have been lost forever. With the cost of breathing new colour into these fragments of history being $3000 per minute, the vast majority of this footage will never be truly appreciated. Wouldn’t it be revolutionary if we could combine the latest cutting edge research in machine learning with the power unleashed by distributed computing with Docker to solve this problem? Using recurrent convolutional neural networks combined with advanced scheduling, distribution, and orchestration made possible only by Docker, we are able to realistically hallucinate colour back into any black and white video. And with machine learning, we can cut the costs dramatically by automating the process of painting each frame, resulting in a realistic high-quality video in record time. We will demo this and much more in our talk “Repainting the past with distributed machine learning and Docker
Report
Share
Report
Share
Download to read offline
Recommended
My Journey To Go
Before becoming a Gopher, Ashley was a professional photographer, which explains her talent and dedication to creating unique Gopher artwork. However, she found that photography wasn’t paying the bills and with a family to feed, she turned to programming. Prior to enrolling at Hackbright Academy (a software engineering bootcamp for women), Ashley had done some front-end work building websites for photographers and working on SEO. Bootcamp had promised to teach her all of the things you need to know to be an engineer, but despite having the “official” business card 12 weeks later, she didn’t feel like a Software Engineer. Ashley found herself writing Python and working as a Community Manager at Rackspace, which is when she met a Gopher who would completely change her trajectory.
In 2014, Ashley met and became quick friends with Steve Francia @spf13, who made her abandon Python and never look back. Steve had been teaching an introductory course to Go, and urged Ashley to help teach the course, where her fresh perspective on Go was an asset. Ever since then, Ashley has been a valued member and a key contributor in building the Go community.
Join this session to learn how teaching others (even though Ashley was new herself) got her involved in an awesome open source community and ultimately changed her career path. Ashley will share how she contributed without committing code, and her lessons and tips on how anyone can get involved in OSS communities and make an impact.
Looking Under The Hood: containerD
As we move to our application units to containers most people are asking themselves the question about orchestrator choice. That is not the only choice that’s important, what about the underlying container runtime? In this talk, we will look at why you would use containerD with runC with both Swarm and Kubernetes, but other uses for ContainerD like container OS’s to ship immutable infrastructure.
Docker?!?! But I'm a SysAdmin
Your developers just walked into your cube and said: "Here's the new app, I built it with Docker, and it's ready to go live." What do you do next? In this session, we'll talk about what containers are and what they are not. And we'll step through a series of considerations that need to be examined when deploying containerized workloads - VMs or Container? Bare Metal or Cloud? What about capacity planning? Security? Disaster Recovery? How do I even get started?
Tales of Training: Scaling CodeLabs with Swarm Mode and Docker-Compose
Why does any "code lab workshop" or live demo are always such a challenge?
A wise sysadmin once told me: “Get your hand dirty with the production to learn”.
So I want to tell you a story of getting hand dirties, by creating a code lab environment considered as production.
This story will show that we can build a reproducible environment for code-labs workshops, by using the Docker “tools”: the Engine, Swarm Mode, Docker-Compose, Moby, LinuxKit.
Following the spirit of “Play With Docker”, but generalized at any service collection, this Codelab toolkit has been used on code-labs workshops of 120+ people.
That path was not a free lunch, but the lessons learned will give you an idea on how a training environment can be efficiently done with Compose and Swarm Mode, by treating it as a “production” platform, tackling the plumbing “youth” limitations for the better of your use case.
As a trainer, I never learned so much than building something to teach people someone else: this the story I want to tell you, the tale of using Docker as a tool of MASSIVE KNOWLEDGE SHARING, which is the root of growing our industry together.
The Fairy Tale of the One Command Build Script
Do you have this build script that with a single command builds your software? Does this still apply on a brand new PC?
This presentation takes you on the journey to construct complex build environments using Docker. The journey follows our lessons learned and experiences going from hand crafted to Dockerized build environments. We will look at different patterns to build modular containers, ways to chain containers and the specialties of Windows containers.
Cilium: Kernel Native Security & DDOS Mitigation for Microservices with BPF
We have introduced Cilium at DockerCon US 2017 this year. Cilium provides application-aware network connectivity, security, and load-balancing for containers. This talk will follow up on the introduction and deep dive into recent kernel developments that address two fundamental questions: How can I provide application-aware security and routing efficiently without overhead embedded into every service? How can container hosts protect themselves from internal and external DDoS attacks? The solutions include:
kproxy: a kernel-based socket proxy which allows for application-aware routing and security enforcement with minimal overhead.
XDP: A lightning-fast packet processing datapath using BPF. The technology is intended for DDoS mitigation, load-balancing, and forwarding.
This talk will deep dive into these exciting technologies and show how Cilium makes BPF and these kernel features available on Linux for your Docker containers.
The Value Of Diverse Experiences
Some things in life are within your control, while others are not. And if you belong to an underrepresented demographic, this translates into additional challenges when it comes to having the career you want. Finding a mentor, participating in resource groups, and building your network are all great ways to help you climb the corporate ladder (or more accurately, the jungle gym). For me, refusing to put invisible guardrails around myself led me down an interesting path and allowed me to take charge of my career.
In this session, I’ll share personal anecdotes about my journey from a hardware engineer working on 64 nm chips, to software engineer working on large scale distributed systems, to a manager, where I have the opportunity to make a greater impact by building a strong and diverse team. I’ll cover my learnings, and give attendees tips on how to define what a successful career is for you, and keys to building a diverse team.
Continuous Packaging is also Mandatory for DevOps
While DevOps are comfortable with continuous integration and automatic tests, the area of continuous packaging has not been given the attention it deserves.
Even with containers, delivering an application using software packages provides multiple advantages with regards to file-based installation: it allows to manage dependencies more easily, to provide metadata, checksum, and signature mechanisms, to deal with packages repositories.
But doing that in a continuous packaging approach means that the generation of these packages is fully automated and part of the build process of the software. As a consequence, it eases the various steps of a solution lifecycle (controlled impact of installation/uninstallation,
identical deliveries up to the customer, avoidance of code or metadata duplication)
This presentation will detail the methodological approach around continuous packaging and demonstrate how this can be put in place using an Open Source tool such as project-builder.org and how this allows the MondoRescue project to deliver packages at will for lots of distribution tuples through the same number of Docker containers.
Recommended
My Journey To Go
Before becoming a Gopher, Ashley was a professional photographer, which explains her talent and dedication to creating unique Gopher artwork. However, she found that photography wasn’t paying the bills and with a family to feed, she turned to programming. Prior to enrolling at Hackbright Academy (a software engineering bootcamp for women), Ashley had done some front-end work building websites for photographers and working on SEO. Bootcamp had promised to teach her all of the things you need to know to be an engineer, but despite having the “official” business card 12 weeks later, she didn’t feel like a Software Engineer. Ashley found herself writing Python and working as a Community Manager at Rackspace, which is when she met a Gopher who would completely change her trajectory.
In 2014, Ashley met and became quick friends with Steve Francia @spf13, who made her abandon Python and never look back. Steve had been teaching an introductory course to Go, and urged Ashley to help teach the course, where her fresh perspective on Go was an asset. Ever since then, Ashley has been a valued member and a key contributor in building the Go community.
Join this session to learn how teaching others (even though Ashley was new herself) got her involved in an awesome open source community and ultimately changed her career path. Ashley will share how she contributed without committing code, and her lessons and tips on how anyone can get involved in OSS communities and make an impact.
Looking Under The Hood: containerD
As we move to our application units to containers most people are asking themselves the question about orchestrator choice. That is not the only choice that’s important, what about the underlying container runtime? In this talk, we will look at why you would use containerD with runC with both Swarm and Kubernetes, but other uses for ContainerD like container OS’s to ship immutable infrastructure.
Docker?!?! But I'm a SysAdmin
Your developers just walked into your cube and said: "Here's the new app, I built it with Docker, and it's ready to go live." What do you do next? In this session, we'll talk about what containers are and what they are not. And we'll step through a series of considerations that need to be examined when deploying containerized workloads - VMs or Container? Bare Metal or Cloud? What about capacity planning? Security? Disaster Recovery? How do I even get started?
Tales of Training: Scaling CodeLabs with Swarm Mode and Docker-Compose
Why does any "code lab workshop" or live demo are always such a challenge?
A wise sysadmin once told me: “Get your hand dirty with the production to learn”.
So I want to tell you a story of getting hand dirties, by creating a code lab environment considered as production.
This story will show that we can build a reproducible environment for code-labs workshops, by using the Docker “tools”: the Engine, Swarm Mode, Docker-Compose, Moby, LinuxKit.
Following the spirit of “Play With Docker”, but generalized at any service collection, this Codelab toolkit has been used on code-labs workshops of 120+ people.
That path was not a free lunch, but the lessons learned will give you an idea on how a training environment can be efficiently done with Compose and Swarm Mode, by treating it as a “production” platform, tackling the plumbing “youth” limitations for the better of your use case.
As a trainer, I never learned so much than building something to teach people someone else: this the story I want to tell you, the tale of using Docker as a tool of MASSIVE KNOWLEDGE SHARING, which is the root of growing our industry together.
The Fairy Tale of the One Command Build Script
Do you have this build script that with a single command builds your software? Does this still apply on a brand new PC?
This presentation takes you on the journey to construct complex build environments using Docker. The journey follows our lessons learned and experiences going from hand crafted to Dockerized build environments. We will look at different patterns to build modular containers, ways to chain containers and the specialties of Windows containers.
Cilium: Kernel Native Security & DDOS Mitigation for Microservices with BPF
We have introduced Cilium at DockerCon US 2017 this year. Cilium provides application-aware network connectivity, security, and load-balancing for containers. This talk will follow up on the introduction and deep dive into recent kernel developments that address two fundamental questions: How can I provide application-aware security and routing efficiently without overhead embedded into every service? How can container hosts protect themselves from internal and external DDoS attacks? The solutions include:
kproxy: a kernel-based socket proxy which allows for application-aware routing and security enforcement with minimal overhead.
XDP: A lightning-fast packet processing datapath using BPF. The technology is intended for DDoS mitigation, load-balancing, and forwarding.
This talk will deep dive into these exciting technologies and show how Cilium makes BPF and these kernel features available on Linux for your Docker containers.
The Value Of Diverse Experiences
Some things in life are within your control, while others are not. And if you belong to an underrepresented demographic, this translates into additional challenges when it comes to having the career you want. Finding a mentor, participating in resource groups, and building your network are all great ways to help you climb the corporate ladder (or more accurately, the jungle gym). For me, refusing to put invisible guardrails around myself led me down an interesting path and allowed me to take charge of my career.
In this session, I’ll share personal anecdotes about my journey from a hardware engineer working on 64 nm chips, to software engineer working on large scale distributed systems, to a manager, where I have the opportunity to make a greater impact by building a strong and diverse team. I’ll cover my learnings, and give attendees tips on how to define what a successful career is for you, and keys to building a diverse team.
Continuous Packaging is also Mandatory for DevOps
While DevOps are comfortable with continuous integration and automatic tests, the area of continuous packaging has not been given the attention it deserves.
Even with containers, delivering an application using software packages provides multiple advantages with regards to file-based installation: it allows to manage dependencies more easily, to provide metadata, checksum, and signature mechanisms, to deal with packages repositories.
But doing that in a continuous packaging approach means that the generation of these packages is fully automated and part of the build process of the software. As a consequence, it eases the various steps of a solution lifecycle (controlled impact of installation/uninstallation,
identical deliveries up to the customer, avoidance of code or metadata duplication)
This presentation will detail the methodological approach around continuous packaging and demonstrate how this can be put in place using an Open Source tool such as project-builder.org and how this allows the MondoRescue project to deliver packages at will for lots of distribution tuples through the same number of Docker containers.
Android Meets Docker
How does awesome Android benefit from dazzling Docker? With an introduction to the Docker way of continuous integration for Android, it would help you get a consistent Android development and build environment with minimal effort.
Syllabus:
* Background story - where the problem happened
* Why Android dev should care?
* How to provision an Android environment to build project usually? How Docker is gonna solve it?
* Typical machine dependant failing unit tests examples, which could be revealed by using Docker
* Philosophy, features, improvements, comparison against traditional approach
* Limitations
* Quick setup of Jenkins using AndroidSDK docker slaves (live demo)
Docker to the Rescue of an Ops Team
In this talk, we'll discover how Docker comes to the rescue of the Ops Team, while rebuilding from scratch our monitoring infrastructure. We'll start by quickly describing the challenges, to focus on why and how using docker saved the project. From fixing dependencies and isolation issues, implementing rolling upgrades and new features hot addition, to building a completely modular, scalable and resilient infrastructure, we'll talk about why CI/CD workflows, docker tooling and Docker Swarm were the key to success.
Experience the Swarm API in Virtual Reality
Docker has a CLI that is a great starting point for programming and automation, but did you know that it's really just a wrapper for the Docker Engine REST API?
The Engine has an accessible, well-documented API that's super easy to use.
In this session, you'll learn how to consume the Docker API in different ways and be encouraged to build your own tools.
We built an app using VueJS and A-Frame that uses the API to visualize Docker Swarm in Web-VR.
The combination of these functional, declarative web technologies is a good way to build immersive, interactive experiences around infrastructure that are easily accessible to others.
More people should be building these kinds of tools.
We'll show how we started building the app in Swarm and how we're able to develop against the API with hot-module-reloading. (This totally works in VR!)
You'll also get a run down how we leveraged existing components from the community in order to quickly prototype our application using Docker Stacks.
Ultimately, everyone will have a chance to experience interacting with a Swarm through the magic window of their phones.
VR is a personal, educational medium. We should be building these virtual worlds around all the components of the Moby Project.
Our hope is that we can share ideas and motivate others to create more compelling Docker and infrastructure tooling with Virtual Reality.
Expect to leave the session wowed by the power of VR and informed about how to leverage Docker's REST API for your own endeavors.
The Truth Behind Serverless
We'll look at how to architect and build a serverless platform and what makes something "serverless". We will dive into the design patterns for serverless applications and how container management solutions must be architected around user requirements.
We will dive deep into how existing cloud-based serverless platforms leverage containers, how they're scheduled, managed, and sandboxed. We'll also look at what improvements we might expect or desire of new and existing serverless platforms.
Containerizing Hardware Accelerated Applications
Many applications allow you to use hardware such as GPUs and FPGAs for acceleration. Common examples include media processing and offloading highly parallel work to a GPU. Applications that use accelerators are resource heavy and have stacks spanning kernel and user space; accelerators often have their own requirements for operating system support and kernel versions. While it may not seem intuitive to containerize this type of application, the use of containers provides benefits such as reduced setup time from container reuse, reduction in dependency conflicts and dependency on a specific operating system, and easier updates.
In this session I show a media processing stack, making use of containers alongside a GPU. Specifically, I explain the kernel and user space divide of a hardware-accelerated transcode application using a device exposed to the container. This specific stack is an interesting case because of its dependency on hardware, use of a custom kernel and libraries, and operating system requirements. Our investigations have shown the use of containers has minimal performance overhead compared to running natively. Furthermore, we can quickly deploy on other machines with reduced configuration effort. There are some aspects of the application not suited to containerization, however. Since the application relies on a custom kernel, the use of containers does not necessarily increase portability. Improvement in this area would require rethinking how the applications are developed and distributed. Other areas of innovation include things such as Docker plugins to check for compatibility between the container software and host kernel.
Skynet vs. Planet of The Apes: Duel!
Singularity happened. Machines have risen. Skynet, the massively distributed AI will soon expand its grip and its army of robots will spread across the whole planet. Fortunately, a great ape army is standing across their way to put an end to this supremacy of steel.
The battle will be epic!
On one side we have Skynet, an automated, self-healed, hybrid Docker platform and skynet application that has become indestructible. Playing the army of Apes, is a dedicated platform hosting Netflix's Simian Army and other flavors.
Will Skynet resists the relentless assaults of the great ape army?
Through this fantasy, we'll first cover all the technologies concretely used to set up the platforms and run the battle (linuxkit, infrakit, & swarm mode, and even raspberry devices among others), while we'll step back in the second part to address the subsequent architecture stakes involved: reliability, scalability, edge computing, immutability, microservices, hybridization, distributed storage. Most of all, you'll understand the importance of the synergies implied between the platforms and the app's design to achieve such a result.
LinuxKit Deep Dive
We open-sourced LinuxKit in April 2017 at DockerCon in Austin. In this session, we'll take a detailed look at some advanced topics of LinuxKit ranging from the general read-only filesystem setup, multi-arch image support for x86_64 and arm64, custom network configuration, and kernel debugging and testing.
Taking Docker to Production: What You Need to Know and Decide
DevOps in the Real World is far from perfect, yet we all dream of that amazing auto-healing fully-automated CI/CD micro-service infrastructure that we'll have "someday." But until then, how can you really start using containers today, and what decisions do you need to make to get there? This session is designed for practitioners who are looking for ways to get started now with Docker and Swarm in production. This is not a Docker 101, but rather it's to help you be successful on your way to Dockerizing your production systems. Attendees will get tactics, example configs, real working infrastructure designs, and see the (sometimes messy) internals of Docker in production today.
How and Why Prometheus' New Storage Engine Pushes the Limits of Time Series D...
The Prometheus monitoring system collects and stores time series data to give valuable insights over hosts, containers, and applications. Its storage engine was designed to be multiple orders of magnitude faster and more space efficient than, say, RRD or SQL storage. However, with the rise of orchestration systems such as Docker Swarm and Kubernetes, and their extensive use of techniques like rolling updates and auto-scaling, environments are becoming increasingly dynamic. This increases the strain on metrics collection systems. To deal with the challenges, a new storage engine has been developed from scratch, bringing a sharp increase in performance and enabling new features.
This talk will describe this new storage engine, its architecture, its data structures, and explain why and how it is well suited to gracefully handle high turnover rates of monitoring targets and provide consistent query performance.
What's New in Docker
It’s the first breakout after the keynote and you need to know more about all the latest and greatest Docker announcements. We've got you covered! In this session, the Docker team will go deeper, looking into what's new with Docker, demoing the latest features and answering your questions.
Learning Docker from Square One
Being a newer technology, Docker has yet to make its way into some computer science training programs. College programs, bootcamps, and online resources have yet to jump onto the container train; so, what's the best way for newer engineers to learn Docker from square one? Chloe (former actress turned developer) tells her story about how she went from wondering "What's a Docker?" to helping teach others about Docker and instead asking "What?? You haven't heard of Docker?". This talk is perfect for anyone new to Docker looking for how to get started, or for those interested in learning how to teach Docker to new users.
Introduction to Docker - 2017
Docker is the world’s leading software container platform. Developers use Docker to eliminate “works on my machine” problems when collaborating on code with co-workers. Operators use Docker to run and manage apps side-by-side in isolated containers to get better compute density. Enterprises use Docker to build agile software delivery pipelines to ship new features faster, more securely and with confidence for both Linux and Windows Server apps.
Integrating Docker EE into Société Générale's Existing Enterprise IT Systems
Société Générale knows that containers and the cloud are the future of the IT industry and have been using Docker EE for over a year and a half. In this talk, we will share how Docker EE fits into our global strategy and our architecture for integrating the platform to our existing IT systems. We will go over tradeoffs of how we operationalized the platform to provide a highly available CAAS to our global enterprise. Finally, we will share how we are onboarding development teams and deploying their applications to production.
Introduction to computer vision with Convoluted Neural Networks
Introduction to computer vision with Convoluted Neural Networks - going over history of CNNs, describing basic concepts such as convolution and discussing applications of computer vision and image recognition technologies
MDEC Data Matters Series: machine learning and Deep Learning, A Primer
MDEC Data Matters Series Talk at Impiana KLCC Hotel, 21st September 2016
More Related Content
Viewers also liked
Android Meets Docker
How does awesome Android benefit from dazzling Docker? With an introduction to the Docker way of continuous integration for Android, it would help you get a consistent Android development and build environment with minimal effort.
Syllabus:
* Background story - where the problem happened
* Why Android dev should care?
* How to provision an Android environment to build project usually? How Docker is gonna solve it?
* Typical machine dependant failing unit tests examples, which could be revealed by using Docker
* Philosophy, features, improvements, comparison against traditional approach
* Limitations
* Quick setup of Jenkins using AndroidSDK docker slaves (live demo)
Docker to the Rescue of an Ops Team
In this talk, we'll discover how Docker comes to the rescue of the Ops Team, while rebuilding from scratch our monitoring infrastructure. We'll start by quickly describing the challenges, to focus on why and how using docker saved the project. From fixing dependencies and isolation issues, implementing rolling upgrades and new features hot addition, to building a completely modular, scalable and resilient infrastructure, we'll talk about why CI/CD workflows, docker tooling and Docker Swarm were the key to success.
Experience the Swarm API in Virtual Reality
Docker has a CLI that is a great starting point for programming and automation, but did you know that it's really just a wrapper for the Docker Engine REST API?
The Engine has an accessible, well-documented API that's super easy to use.
In this session, you'll learn how to consume the Docker API in different ways and be encouraged to build your own tools.
We built an app using VueJS and A-Frame that uses the API to visualize Docker Swarm in Web-VR.
The combination of these functional, declarative web technologies is a good way to build immersive, interactive experiences around infrastructure that are easily accessible to others.
More people should be building these kinds of tools.
We'll show how we started building the app in Swarm and how we're able to develop against the API with hot-module-reloading. (This totally works in VR!)
You'll also get a run down how we leveraged existing components from the community in order to quickly prototype our application using Docker Stacks.
Ultimately, everyone will have a chance to experience interacting with a Swarm through the magic window of their phones.
VR is a personal, educational medium. We should be building these virtual worlds around all the components of the Moby Project.
Our hope is that we can share ideas and motivate others to create more compelling Docker and infrastructure tooling with Virtual Reality.
Expect to leave the session wowed by the power of VR and informed about how to leverage Docker's REST API for your own endeavors.
The Truth Behind Serverless
We'll look at how to architect and build a serverless platform and what makes something "serverless". We will dive into the design patterns for serverless applications and how container management solutions must be architected around user requirements.
We will dive deep into how existing cloud-based serverless platforms leverage containers, how they're scheduled, managed, and sandboxed. We'll also look at what improvements we might expect or desire of new and existing serverless platforms.
Containerizing Hardware Accelerated Applications
Many applications allow you to use hardware such as GPUs and FPGAs for acceleration. Common examples include media processing and offloading highly parallel work to a GPU. Applications that use accelerators are resource heavy and have stacks spanning kernel and user space; accelerators often have their own requirements for operating system support and kernel versions. While it may not seem intuitive to containerize this type of application, the use of containers provides benefits such as reduced setup time from container reuse, reduction in dependency conflicts and dependency on a specific operating system, and easier updates.
In this session I show a media processing stack, making use of containers alongside a GPU. Specifically, I explain the kernel and user space divide of a hardware-accelerated transcode application using a device exposed to the container. This specific stack is an interesting case because of its dependency on hardware, use of a custom kernel and libraries, and operating system requirements. Our investigations have shown the use of containers has minimal performance overhead compared to running natively. Furthermore, we can quickly deploy on other machines with reduced configuration effort. There are some aspects of the application not suited to containerization, however. Since the application relies on a custom kernel, the use of containers does not necessarily increase portability. Improvement in this area would require rethinking how the applications are developed and distributed. Other areas of innovation include things such as Docker plugins to check for compatibility between the container software and host kernel.
Skynet vs. Planet of The Apes: Duel!
Singularity happened. Machines have risen. Skynet, the massively distributed AI will soon expand its grip and its army of robots will spread across the whole planet. Fortunately, a great ape army is standing across their way to put an end to this supremacy of steel.
The battle will be epic!
On one side we have Skynet, an automated, self-healed, hybrid Docker platform and skynet application that has become indestructible. Playing the army of Apes, is a dedicated platform hosting Netflix's Simian Army and other flavors.
Will Skynet resists the relentless assaults of the great ape army?
Through this fantasy, we'll first cover all the technologies concretely used to set up the platforms and run the battle (linuxkit, infrakit, & swarm mode, and even raspberry devices among others), while we'll step back in the second part to address the subsequent architecture stakes involved: reliability, scalability, edge computing, immutability, microservices, hybridization, distributed storage. Most of all, you'll understand the importance of the synergies implied between the platforms and the app's design to achieve such a result.
LinuxKit Deep Dive
We open-sourced LinuxKit in April 2017 at DockerCon in Austin. In this session, we'll take a detailed look at some advanced topics of LinuxKit ranging from the general read-only filesystem setup, multi-arch image support for x86_64 and arm64, custom network configuration, and kernel debugging and testing.
Taking Docker to Production: What You Need to Know and Decide
DevOps in the Real World is far from perfect, yet we all dream of that amazing auto-healing fully-automated CI/CD micro-service infrastructure that we'll have "someday." But until then, how can you really start using containers today, and what decisions do you need to make to get there? This session is designed for practitioners who are looking for ways to get started now with Docker and Swarm in production. This is not a Docker 101, but rather it's to help you be successful on your way to Dockerizing your production systems. Attendees will get tactics, example configs, real working infrastructure designs, and see the (sometimes messy) internals of Docker in production today.
How and Why Prometheus' New Storage Engine Pushes the Limits of Time Series D...
The Prometheus monitoring system collects and stores time series data to give valuable insights over hosts, containers, and applications. Its storage engine was designed to be multiple orders of magnitude faster and more space efficient than, say, RRD or SQL storage. However, with the rise of orchestration systems such as Docker Swarm and Kubernetes, and their extensive use of techniques like rolling updates and auto-scaling, environments are becoming increasingly dynamic. This increases the strain on metrics collection systems. To deal with the challenges, a new storage engine has been developed from scratch, bringing a sharp increase in performance and enabling new features.
This talk will describe this new storage engine, its architecture, its data structures, and explain why and how it is well suited to gracefully handle high turnover rates of monitoring targets and provide consistent query performance.
What's New in Docker
It’s the first breakout after the keynote and you need to know more about all the latest and greatest Docker announcements. We've got you covered! In this session, the Docker team will go deeper, looking into what's new with Docker, demoing the latest features and answering your questions.
Learning Docker from Square One
Being a newer technology, Docker has yet to make its way into some computer science training programs. College programs, bootcamps, and online resources have yet to jump onto the container train; so, what's the best way for newer engineers to learn Docker from square one? Chloe (former actress turned developer) tells her story about how she went from wondering "What's a Docker?" to helping teach others about Docker and instead asking "What?? You haven't heard of Docker?". This talk is perfect for anyone new to Docker looking for how to get started, or for those interested in learning how to teach Docker to new users.
Introduction to Docker - 2017
Docker is the world’s leading software container platform. Developers use Docker to eliminate “works on my machine” problems when collaborating on code with co-workers. Operators use Docker to run and manage apps side-by-side in isolated containers to get better compute density. Enterprises use Docker to build agile software delivery pipelines to ship new features faster, more securely and with confidence for both Linux and Windows Server apps.
Integrating Docker EE into Société Générale's Existing Enterprise IT Systems
Société Générale knows that containers and the cloud are the future of the IT industry and have been using Docker EE for over a year and a half. In this talk, we will share how Docker EE fits into our global strategy and our architecture for integrating the platform to our existing IT systems. We will go over tradeoffs of how we operationalized the platform to provide a highly available CAAS to our global enterprise. Finally, we will share how we are onboarding development teams and deploying their applications to production.
Viewers also liked (14)
Taking Docker to Production: What You Need to Know and Decide
Taking Docker to Production: What You Need to Know and Decide
How and Why Prometheus' New Storage Engine Pushes the Limits of Time Series D...
How and Why Prometheus' New Storage Engine Pushes the Limits of Time Series D...
Integrating Docker EE into Société Générale's Existing Enterprise IT Systems
Integrating Docker EE into Société Générale's Existing Enterprise IT Systems
Similar to Repainting the Past with Distributed Machine Learning and Docker
Introduction to computer vision with Convoluted Neural Networks
Introduction to computer vision with Convoluted Neural Networks - going over history of CNNs, describing basic concepts such as convolution and discussing applications of computer vision and image recognition technologies
MDEC Data Matters Series: machine learning and Deep Learning, A Primer
MDEC Data Matters Series Talk at Impiana KLCC Hotel, 21st September 2016
(Msc Thesis) Sparse Coral Classification Using Deep Convolutional Neural Netw...
Msc Thesis Presentation - June 2014
VIBOT Promotion 7 (2012-2014)
build a Convolutional Neural Network (CNN) using TensorFlow in Python
build Your CNN Using Tensorflow
Stateful Applications On the Cloud: A PayPal Journey
Anant Kumar and Jasdeep Singh's presentation from OpenStack Days East on August 23, 2016.
Cvpr 2018 papers review (efficient computing)
Overview of CVPR 2018 papers
For mobile DL
NetAdapt, ADC, Quantization and Training of ... (Tensorflow Lite quantization)
OpenCAPI-based Image Analysis Pipeline for 18 GB/s kilohertz-framerate X-ray ...
OpenCAPI-based Image Analysis Pipeline for 18 GB/s kilohertz-framerate X-ray ...Ganesan Narayanasamy
Macromolecular crystallography is an experimental technique allowing to explore 3D atomic structure of proteins, used by academics for research in biology and by pharmaceutical companies in rational drug design. While up to now development of the technique was limited by scientific instruments performance, recently computing performance becomes a key limitation. In my presentation I will present a computing challenge to handle 18 GB/s data stream coming from the new X-ray detector. I will show PSI experiences in applying conventional hardware for the task and why this attempt failed. I will then present how IC 922 server with OpenCAPI enabled FPGA boards allowed to build a sustainable and scalable solution for high speed data acquisition. Finally, I will give a perspective, how the advancement in hardware development will enable better science by users of the Swiss Light Source.Digitization Basics for Archives and Special Collections – Part 1: Select and...
Josh Hickman, Digital Resources Librarian, Beloit College
This is the first part of a two-part, full-day workshop introducing the core elements of creating digital collections of historic photographs, documents and other archival materials. Part 1 focuses on selecting materials to digitize and the basics of reformatting. We’ll start with some recommendations for planning a successful project and consider how your digital collections can fit into the statewide and national landscape of digital content. We’ll discuss copyright concerns in order to help you answer the question “CAN I put this online?” And we’ll explore the vocabulary of digital images, including pixels, resolution and bit depth as well as tools and best practices for scanning photographs and documents.
"Fast Deployment of Low-power Deep Learning on CEVA Vision Processors," a Pre...
"Fast Deployment of Low-power Deep Learning on CEVA Vision Processors," a Pre...Edge AI and Vision Alliance
For the full video of this presentation, please visit:
http://www.embedded-vision.com/platinum-members/ceva/embedded-vision-training/videos/pages/may-2016-embedded-vision-summit-siegel
For more information about embedded vision, please visit:
http://www.embedded-vision.com
Yair Siegel, Director of Segment Marketing at CEVA, presents the "Fast Deployment of Low-power Deep Learning on CEVA Vision Processors" tutorial at the May 2016 Embedded Vision Summit.
Image recognition capabilities enabled by deep learning are benefitting more and more applications, including automotive safety, surveillance and drones. This is driving a shift towards running neural networks inside embedded devices. But, there are numerous challenges in squeezing deep learning into resource-limited devices. This presentation details a fast path for taking a neural network from research into an embedded implementation on a CEVA vision processor core, making use of CEVA’s neural network software framework. Siegel explains how the CEVA framework integrates with existing deep learning development environments like Caffe, and how it can be used to create low-power embedded systems with neural network capabilities.Similar to Repainting the Past with Distributed Machine Learning and Docker (20)
Introduction to computer vision with Convoluted Neural Networks
Introduction to computer vision with Convoluted Neural Networks
MDEC Data Matters Series: machine learning and Deep Learning, A Primer
MDEC Data Matters Series: machine learning and Deep Learning, A Primer
High performace network of Cloud Native Taiwan User Group
High performace network of Cloud Native Taiwan User Group
(Msc Thesis) Sparse Coral Classification Using Deep Convolutional Neural Netw...
(Msc Thesis) Sparse Coral Classification Using Deep Convolutional Neural Netw...
build a Convolutional Neural Network (CNN) using TensorFlow in Python
build a Convolutional Neural Network (CNN) using TensorFlow in Python
Stateful Applications On the Cloud: A PayPal Journey
Stateful Applications On the Cloud: A PayPal Journey
OpenCAPI-based Image Analysis Pipeline for 18 GB/s kilohertz-framerate X-ray ...
OpenCAPI-based Image Analysis Pipeline for 18 GB/s kilohertz-framerate X-ray ...
Digitization Basics for Archives and Special Collections – Part 1: Select and...
Digitization Basics for Archives and Special Collections – Part 1: Select and...
"Fast Deployment of Low-power Deep Learning on CEVA Vision Processors," a Pre...
"Fast Deployment of Low-power Deep Learning on CEVA Vision Processors," a Pre...
More from Docker, Inc.
Containerize Your Game Server for the Best Multiplayer Experience
Raymond Arifianto, AccelByte and
Mark Mandel, Google -
We have been deploying containerized micro-services for our Game Backend Services for a while. Now we are tackling the challenge to scale up fleets of game dedicated servers in multiple regions, multiple data centers and multiple providers - some in bare metal, some in Cloud. So we leverage docker containerization to deploy Game Servers to achieve Portability, Fast Deployment and Predictability, enabling us to scale up to thousands of servers, on demand, without a sweat.
How to Improve Your Image Builds Using Advance Docker Build
Nicholas Dille, Haufe-Lexware + Docker Captain -
Docker continues to be the standard tool for building container images. For more than a year Docker ships with BuildKit as an alternative image builder, providing advanced features for secret and cache management. These features help to make image builds faster and more secure. In this session, Docker Captain Nicholas Dille will teach you how to use Buildkit features to your advantage.
Build & Deploy Multi-Container Applications to AWS
Lukonde Mwila, Entelect -
As the cloud-native approach to development and deployment becomes more prevalent, it's an exciting time for software engineers to be equipped on how to dockerize multi-container applications and deploy them to the cloud.
In this talk, Lukonde Mwila, Software Engineer at Entelect, will cover the following topics:
- Docker Compose
- Containerizing an Nginx Server
- Containerizing an React App
- Containerizing an Node.JS App
- Containerizing anMongoDB App
- Runing Multi-Container App Locally
- Creating a CI/CD Pipeline
- Adding a build stage to test containers and push images to Docker Hub
- Deploying Multi-Container App to AWS Elastic Beanstalk
Lukonde will start by giving an overview of how Docker Compose works and how it makes it very easy and straightforward to startup multiple Docker containers at the same time and automatically connect them together with some form of networking.
After that, Lukonde will take a hands on approach to containerize an Nginx server, a React app, a NodeJS app and a MongoDB instance to demonstrate the power of Docker Compose. He'll demonstrate usage of two Docker files for an application, one production grade and the other for local development and running of tests. Lastly, he'll demonstrate creating a CI/CD pipeline in AWS to build and test our Docker images before pushing them to Docker Hub or AWS ECR, and finally deploying our multi-container application AWS Elastic Beanstalk.
Securing Your Containerized Applications with NGINX
Kevin Jones, NGNIX -
NGINX is one of the most popular images on Docker Hub and has been at the forefront of the web since the early 2000's. In this talk we will discuss how and why NGINX's lightweight and powerful architecture makes it a very popular choice for securing containerized applications as a sidecar reverse proxy within containers. We will highlight important aspects of application security that NGINX can help with, such as TLS, HTTP, AuthN, AuthZ and traffic control.
How To Build and Run Node Apps with Docker and Compose
Kathleen Juell, Digital Ocean -
Containers are an essential part of today's microservice ecosystem, as they allow developers and operators to maintain standards of reliability and reproducibility in fast-paced deployment scenarios. And while there are best practices that extend across stacks in containerized environments, there are also things that make each stack distinct, starting with the application image itself.
This talk will dive into some of these particularities, both at the image and service level, while also covering general best practices for building and running Node applications with database backends using Docker and Compose.
Hands-on Helm
Jessica Deen, Microsoft -
Helm 3 is here; let's go hands-on! In this demo-fueled session, I'll walk you through the differences between Helm 2 and Helm 3. I'll offer tips for a successful rollout or upgrade, go over how to easily use charts created for Helm 2 with Helm 3 (without changing your syntax), and review opportunities where you can participate in the project's future.
Distributed Deep Learning with Docker at Salesforce
Jeff Hajewski, Salesforce -
There is a wealth of information on building deep learning models with PyTorch or TensorFlow. Anyone interested in building a deep learning model is only a quick search away from a number of clear and well written tutorials that will take them from zero knowledge to having a working image classifier. But what happens when you need to deploy these models in a production setting? At Salesforce, we use TensorFlow models to help us provide customers with insights into their data, and we do this as close to real-time as possible. Designing these systems in a scalable manner requires overcoming a number of design challenges, but the core component is Docker. Docker enables us to design highly scalable systems by allowing us to focus on service interactions, rather than how our services will interact with the hardware. Docker is also at the core of our test infrastructure, allowing developers and data scientists to build and test the system in an end to end manner on their local machines. While some of this may sound complex, the core message is simplicity - Docker allows us to focus on the aspects of the system that matter, greatly simplifying our lives.
The First 10M Pulls: Building The Official Curl Image for Docker Hub
James Fuller, webcomposite s.r.o. -
Curl is the venerable (yet very modern) 'swiss army knife' command line tool and library for transferring data with URLs. Recently we (the Curl team) decided to build a release for Docker Hub. This talk will outline our current development workflow with respect to the docker image and provide insights on what it takes to build a docker image for mass public consumption. We are also keen to learn from users and other developers how we might improve and enhance the official curl docker image.
Monitoring in a Microservices World
Fabian Stäber, Instana -
In recent years, we saw a great paradigm shift in software engineering away from static monolithic applications towards dynamic distributed horizontally scalable architectures. Docker is one of the key technologies enabling this development. This shift poses a lot of new challenges for application monitoring, ranging from practical issues (need for automation) to technical challenges (Docker networking) to organizational topics (blurring line between software engineers and operations) to fundamental questions (define what is an application). In this talk we show how Docker changed the way we do monitoring, how modern application monitoring systems work, and what future developments we expect.
COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...
Clemente Biondo, Engineering Ingegneria Informatica -
When the COVID 19 pandemic started, Engineering Ingegneria Informatica Group (1.25 billion euros of revenues, 65 offices around the world, 12.000 employees) was forced to put their digital transformation to the test in order to maintain operational continuity. In this session, Clemente Biondo, the Tech Lead of the Information Systems Department, will share how his company is reacting to this unforeseeable scenario and how Docker-driven digital transformation had paved the path for work to continue remotely. Clemente will discuss learnings moving from colocated teams, manual approaches, email based-business processes, and a monolithic application to a mature DevOps culture characterized by a distributed autonomous workforce and a continuous deployment process that deploys backward-compatible Docker containerized microservices into hybrid multi cloud datacenters an average of twice a day with zero-downtime. He will detail how they use Docker to unify dev, test and production environments, and as an efficient and automated mechanism for deploying applications. Lastly, Clemente shares how, in our darkest hour, he and others are working to shine their brightest light.
Predicting Space Weather with Docker
Chris Lauer, NOAA Space Weather Prediction Center -
This is the story of how adopting a containerized workflow changed the way our small software team works at NOAA’s Space Weather Prediction Center. Our old architecture, a big ball of mud shared-database integration, just wasn’t cutting it - it was killing our agility. Over the past two years, our small team has adopted a microservice style architecture, using Docker with docker-compose and environment files as our deployment strategy for all new development. We’ve discovered the joys of using containers for identical dev, staging, and production environments. We work closely with scientists: much of the code we’re running has complicated and conflicting library dependencies. Docker captures these beautifully - we’ve even had some success teaching our scientists to use it! I’ll share what we’ve learned, some of the persistent challenges we face, and one place we really got it wrong. This talk builds off of a popular hallway track from DockerCon 2019.
Become a Docker Power User With Microsoft Visual Studio Code
Brian Christner, 56k + Docker Captain -
In this session, we will unlock the full potential of using Microsoft Visual Studio Code (VS Code) and Docker Desktop to turn you into a Docker Power User. When we expand and utilize the VS Code Docker plugin, we can take our projects and Docker skills to the next level. In addition to using VS Code, we streamline our Docker Desktop development workflow with less context switching and built-in shortcuts. You will learn how to bootstrap new projects, quickly write Dockerfiles utilizing templates, build, run, and interact with containers all from VS Code.
How to Use Mirroring and Caching to Optimize your Container Registry
Brandon Mitchell, Boxboat + Docker Captain -
How do you make your builds more performant? This talk looks at options to configure caching and mirroring of images that you need to save on bandwidth costs and to keep running even if something goes down upstream.
Monolithic to Microservices + Docker = SDLC on Steroids!
Ashish Sharma, SS&C Eze -
SS&C Eze provides various products in the stock market domain. We spent the last couple of years building Eclipse which is an investment suite born in cloud. The journey so far has been very interesting. The very first version of the product were a bunch of monolithic windows services and deployed using Octopus tool. We successfully managed to bring all the monolithic problem to the cloud and created a nightmare for ourselves. We then started applying microservices architecture principles and started breaking the monolithic into small services. Very soon we realized that we need a better packaging/deployment tool. Docker looked like a magical solution to our problem. Since its adoption, It has not only solved the deployment problem for us but has made a deep impact on different aspects of SDLC. It allowed us to use heterogeneous technology stacks, simplified development environment setup, simplified our testing strategy, improved our speed of delivery, and made our developers more productive. In this talk I would like to share our experience of using Docker and its positive impact on our SDLC.
Kubernetes at Datadog Scale
Ara Pulido, Datadog -
Container technologies, although not new, have increased their popularity in the past few years, with container orchestrators allowing companies around the world to adopt these technologies to help them ship and scale microservices with precision and velocity. Kubernetes is currently the most popular container orchestration platform, and while many organizations are migrating their workloads to it, Kubernetes is still relatively immature. New corner cases, errors, and quirks are regularly discovered as users push the boundaries of size and scale. When Datadog adopted Kubernetes we discovered some of these boundaries the hard way, and we continuously challenge and modify our infrastructure decisions in order to fit our use case. Join me in this talk for our story on what we learned while we scaled our Kubernetes clusters, the contributions to Kubernetes we made along the way, and how you can apply those learnings when growing your Kubernetes clusters from a handful to hundreds or thousands of nodes.
Labels, Labels, Labels
Andy Clemenko, StackRox -
One underutilized, and amazing, thing about the docker image scheme is labels. Labels are a built in way to document all aspects about the image itself. Think about all the information that the tags inside your clothing carry. If you care to look you can find out everything about the garment. All that information can be very valuable. Now think about how we can leverage labels to carry similar information. We can even use the labels to contain Docker Compose or even Kubernetes Yaml. We can even include labels into the CI/CD process making things more secure and smoother. Come find out some fun techniques on how to leverage labels to do some fun and amazing things.
Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment Model
Patrick Deloulay, Micro Focus -
Micro Focus started their digital transformation 3 years ago, moving the entire portfolio into hundreds of container images. Leveraging Docker Hub as our primary registry service, we will cover how we ended up building a simple but secure push/pull model to publish and deliver our premium assets to our customers and partners to both meet the high agility of our DevOps teams while greatly simplifying the deployment of our applications.
Build & Deploy Multi-Container Applications to AWS
Lukonde Mwila, Entelect
As the cloud-native approach to development and deployment becomes more prevalent, it's an exciting time for software engineers to be equipped on how to dockerize multi-container applications and deploy them to the cloud.
In this talk, Lukonde Mwila, Software Engineer at Entelect, will cover the following topics:
- Docker Compose
- Containerizing an Nginx Server
- Containerizing an React App
- Containerizing an Node.JS App
- Containerizing anMongoDB App
- Runing Multi-Container App Locally
- Creating a CI/CD Pipeline
- Adding a build stage to test containers and push images to Docker Hub
- Deploying Multi-Container App to AWS Elastic Beanstalk
Lukonde will start by giving an overview of how Docker Compose works and how it makes it very easy and straightforward to startup multiple Docker containers at the same time and automatically connect them together with some form of networking.
After that, Lukonde will take a hands on approach to containerize an Nginx server, a React app, a NodeJS app and a MongoDB instance to demonstrate the power of Docker Compose. He'll demonstrate usage of two Docker files for an application, one production grade and the other for local development and running of tests. Lastly, he'll demonstrate creating a CI/CD pipeline in AWS to build and test our Docker images before pushing them to Docker Hub or AWS ECR, and finally deploying our multi-container application AWS Elastic Beanstalk.
From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...
Elton Stoneman, Docker Captain + Container Consultant and Trainer
How do you provide a SaaS offering when your product is a 10-year old Fortran app, currently built to run on Windows 10? With Docker and Kubernetes of course - and you can do it in a week (... to prototype level at least).
In this session I'll walk through the processes and practicalities of taking an older Windows app, making it run in containers with Kubernetes, and then building a simple API wrapper to host the whole stack as a cloud-based SaaS product.
There's a lot of technology here from a real world case study, and I'll focus on:
- running Windows apps in Docker containers
- building a .NET Core API which can run in Linux or Windows containers
- running the stack in Kubernetes with Docker Desktop locally and AKS in the cloud
- configuring AKS workloads in Azure to burst out to Azure Container Instances
And there's a core theme to this session: Docker and Kubernetes are complex technologies, but they're the key to modern development. If you invest time learning them, they make projects like this simple, portable, fast and fun.
Developing with Docker for the Arm Architecture
This virtual meetup introduces the concepts and best practices of using Docker containers for software development for the Arm architecture across a variety of hardware systems. Using Docker Desktop on Windows or Mac, Amazon Web Services (AWS) A1 instances, and embedded Linux, we will demonstrate the latest Docker features to build, share, and run multi-architecture images with transparent support for Arm.
More from Docker, Inc. (20)
Containerize Your Game Server for the Best Multiplayer Experience
Containerize Your Game Server for the Best Multiplayer Experience
How to Improve Your Image Builds Using Advance Docker Build
How to Improve Your Image Builds Using Advance Docker Build
Build & Deploy Multi-Container Applications to AWS
Build & Deploy Multi-Container Applications to AWS
Securing Your Containerized Applications with NGINX
Securing Your Containerized Applications with NGINX
How To Build and Run Node Apps with Docker and Compose
How To Build and Run Node Apps with Docker and Compose
Distributed Deep Learning with Docker at Salesforce
Distributed Deep Learning with Docker at Salesforce
The First 10M Pulls: Building The Official Curl Image for Docker Hub
The First 10M Pulls: Building The Official Curl Image for Docker Hub
COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...
COVID-19 in Italy: How Docker is Helping the Biggest Italian IT Company Conti...
Become a Docker Power User With Microsoft Visual Studio Code
Become a Docker Power User With Microsoft Visual Studio Code
How to Use Mirroring and Caching to Optimize your Container Registry
How to Use Mirroring and Caching to Optimize your Container Registry
Monolithic to Microservices + Docker = SDLC on Steroids!
Monolithic to Microservices + Docker = SDLC on Steroids!
Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment Model
Using Docker Hub at Scale to Support Micro Focus' Delivery and Deployment Model
Build & Deploy Multi-Container Applications to AWS
Build & Deploy Multi-Container Applications to AWS
From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...
From Fortran on the Desktop to Kubernetes in the Cloud: A Windows Migration S...
Recently uploaded
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Epistemic Interaction - tuning interfaces to provide information for AI support
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
Welocme to ViralQR, your best QR code generator.
Welcome to ViralQR, your best QR code generator available on the market!
At ViralQR, we design static and dynamic QR codes. Our mission is to make business operations easier and customer engagement more powerful through the use of QR technology. Be it a small-scale business or a huge enterprise, our easy-to-use platform provides multiple choices that can be tailored according to your company's branding and marketing strategies.
Our Vision
We are here to make the process of creating QR codes easy and smooth, thus enhancing customer interaction and making business more fluid. We very strongly believe in the ability of QR codes to change the world for businesses in their interaction with customers and are set on making that technology accessible and usable far and wide.
Our Achievements
Ever since its inception, we have successfully served many clients by offering QR codes in their marketing, service delivery, and collection of feedback across various industries. Our platform has been recognized for its ease of use and amazing features, which helped a business to make QR codes.
Our Services
At ViralQR, here is a comprehensive suite of services that caters to your very needs:
Static QR Codes: Create free static QR codes. These QR codes are able to store significant information such as URLs, vCards, plain text, emails and SMS, Wi-Fi credentials, and Bitcoin addresses.
Dynamic QR codes: These also have all the advanced features but are subscription-based. They can directly link to PDF files, images, micro-landing pages, social accounts, review forms, business pages, and applications. In addition, they can be branded with CTAs, frames, patterns, colors, and logos to enhance your branding.
Pricing and Packages
Additionally, there is a 14-day free offer to ViralQR, which is an exceptional opportunity for new users to take a feel of this platform. One can easily subscribe from there and experience the full dynamic of using QR codes. The subscription plans are not only meant for business; they are priced very flexibly so that literally every business could afford to benefit from our service.
Why choose us?
ViralQR will provide services for marketing, advertising, catering, retail, and the like. The QR codes can be posted on fliers, packaging, merchandise, and banners, as well as to substitute for cash and cards in a restaurant or coffee shop. With QR codes integrated into your business, improve customer engagement and streamline operations.
Comprehensive Analytics
Subscribers of ViralQR receive detailed analytics and tracking tools in light of having a view of the core values of QR code performance. Our analytics dashboard shows aggregate views and unique views, as well as detailed information about each impression, including time, device, browser, and estimated location by city and country.
So, thank you for choosing ViralQR; we have an offer of nothing but the best in terms of QR code services to meet business diversity!
DevOps and Testing slides at DASA Connect
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Leading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdf
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
The publishing industry has been selling digital audiobooks and ebooks for over a decade and has found its groove. What’s changed? What has stayed the same? Where do we go from here? Join a group of leading sales peers from across the industry for a conversation about the lessons learned since the popularization of digital books, best practices, digital book supply chain management, and more.
Link to video recording: https://bnctechforum.ca/sessions/selling-digital-books-in-2024-insights-from-industry-leaders/
Presented by BookNet Canada on May 28, 2024, with support from the Department of Canadian Heritage.
Securing your Kubernetes cluster_ a step-by-step guide to success !
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Le nuove frontiere dell'AI nell'RPA con UiPath Autopilot™
In questo evento online gratuito, organizzato dalla Community Italiana di UiPath, potrai esplorare le nuove funzionalità di Autopilot, il tool che integra l'Intelligenza Artificiale nei processi di sviluppo e utilizzo delle Automazioni.
📕 Vedremo insieme alcuni esempi dell'utilizzo di Autopilot in diversi tool della Suite UiPath:
Autopilot per Studio Web
Autopilot per Studio
Autopilot per Apps
Clipboard AI
GenAI applicata alla Document Understanding
👨🏫👨💻 Speakers:
Stefano Negro, UiPath MVPx3, RPA Tech Lead @ BSP Consultant
Flavio Martinelli, UiPath MVP 2023, Technical Account Manager @UiPath
Andrei Tasca, RPA Solutions Team Lead @NTT Data
When stars align: studies in data quality, knowledge graphs, and machine lear...
Keynote at DQMLKG workshop at the 21st European Semantic Web Conference 2024
UiPath Test Automation using UiPath Test Suite series, part 3
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
A tale of scale & speed: How the US Navy is enabling software delivery from l...
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
Elizabeth Buie - Older adults: Are we really designing for our future selves?
Elizabeth Buie - Older adults: Are we really designing for our future selves?
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
Elevating Tactical DDD Patterns Through Object Calisthenics
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
GraphRAG is All You need? LLM & Knowledge Graph
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
Recently uploaded (20)
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
Epistemic Interaction - tuning interfaces to provide information for AI support
Epistemic Interaction - tuning interfaces to provide information for AI support
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
State of ICS and IoT Cyber Threat Landscape Report 2024 preview
Leading Change strategies and insights for effective change management pdf 1.pdf
Leading Change strategies and insights for effective change management pdf 1.pdf
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Transcript: Selling digital books in 2024: Insights from industry leaders - T...
Securing your Kubernetes cluster_ a step-by-step guide to success !
Securing your Kubernetes cluster_ a step-by-step guide to success !
Le nuove frontiere dell'AI nell'RPA con UiPath Autopilot™
Le nuove frontiere dell'AI nell'RPA con UiPath Autopilot™
When stars align: studies in data quality, knowledge graphs, and machine lear...
When stars align: studies in data quality, knowledge graphs, and machine lear...
UiPath Test Automation using UiPath Test Suite series, part 3
UiPath Test Automation using UiPath Test Suite series, part 3
A tale of scale & speed: How the US Navy is enabling software delivery from l...
A tale of scale & speed: How the US Navy is enabling software delivery from l...
Assure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyes
Elizabeth Buie - Older adults: Are we really designing for our future selves?
Elizabeth Buie - Older adults: Are we really designing for our future selves?
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf
Elevating Tactical DDD Patterns Through Object Calisthenics
Elevating Tactical DDD Patterns Through Object Calisthenics
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf
Repainting the Past with Distributed Machine Learning and Docker
- 1. Student Oli Callaghan Student Repainting the Past with Distributed Machine Learning and Docker Finnian Anderson
- 2. • Quick introduction to machine learning • What we wanted to achieve • How we accomplished this • Problems we faced On the menu • Future developments
- 5. Problems with fully-connected NNs • Memory and performance issues • Impractical for image analysis • 10 x 13 pixels • 260 inputs • 260 weights per node • 30 nodes per layer • 6 layers • ~6M weights total • 600 x 800 pixels • ~1M inputs • ~1M weights per node • 240k nodes per layer • 6 layers • ~1.4 trillion weights total
- 6. Convolutional Neural Networks Filters of weights are slid across the input to produce output feature-maps.
- 7. Convolutional Neural Networks • 10 x 13 pixels • 260 inputs • 8 layers • 64, 3 x 3 filters / layer • 4608 weights total • 600 x 800 pixels • ~1M inputs • 8 layers • 64, 3 x 3 filters / layer • 4608 weights total
- 8. Convolutional Neural Networks • Trained like every other NN • Perform much better in image analysis
- 9. Baby steps
- 11. Current approaches to video colourisation • €400k / blockbuster • Automate process with neural networks
- 12. How did we do it Click to add text
- 14. Custom Solution • Took a long time to implement • Was slower than other frameworks • Gained an understanding of underlying process
- 15. Tensorflow • Easy to use • Optimised for GPU • Distributed learning $ docker run tensorflow/tensorflow
- 16. Caffe • Developed at Berkeley • Only CNNs • Superfast Richard Zhang et al. https://arxiv.org/pdf/ 1603.08511.pdf $ docker run bvlc/caffe
- 17. Problems we encountered ML takes a long time!
- 18. Hardware • Limited resources • PC useless whilst training • GPU? Solution?
- 19. Nvidia Docker CPU is slow Allows you to use GPU $ nvidia-docker run nvidia/caffe $ docker run bvlc/caffe
- 20. An Open Source Serverless Framework openfaas.com
- 22. Demo So how did it go?
- 23. Get tweeting!
- 24. @colorisebot
- 26. What next? • Run conversions faster • Use recurrent networks • Run on GPUs
- 27. Thank you! Questions? Oli @olicallaghan_ olicallaghan.com Finnian @developius finnian.io