A guest lecture at National University of Defense Technology (NUDT) in 2016 to postgraduate students in China about emerging technologies in the Linux operating system.
Describes what is lightweight virtualization and containers, and the low-level mechanisms in the Linux kernel that it relies on: namespaces, cgroups. It also gives details on AUFS. Those component together are the key to understanding how modern systems like Docker (http://www.docker.io/) work.
OpenVZ, which has turned 7 recently, is an implementation of lightweight virtualization technology for Linux, something which is also referred to as LXC or just containers. The talk gives an insight into 7 different problems with containers and how they were solved. While most of these problems and solutions belongs in the Linux kernel, kernel knowledge is not expected from the audience.
Internal presentation of Docker, Lightweight Virtualization, and linux Containers; at Spotify NYC offices, featuring engineers from Yandex, LinkedIn, Criteo, and NASA!
Docker storage drivers by Jérôme PetazzoniDocker, Inc.
The first release of Docker only supported AUFS, and AUFS was available (out of the box) only on Debian and Ubuntu kernel. Then Red Hat wanted Docker to run on its distros, and contributed the Device Mapper driver, and later the BTRFS driver, and recently the overlayfs driver.
Jérôme presents how those drivers compare from a high-level perspective, explaining their pros and cons.
Then he showed each driver in action, and look at low-level implementation details. We won't dive into the golang implementation code itself, but we will explain the concepts of each driver. This will help to better understand how they work, and give some hints when it comes to troubleshoot their behaviour.
Anatomy of a Container: Namespaces, cgroups & Some Filesystem Magic - LinuxConJérôme Petazzoni
Containers are everywhere. But what exactly is a container? What are they made from? What's the difference between LXC, butts-nspawn, Docker, and the other container systems out there? And why should we bother about specific filesystems?
In this talk, Jérôme will show the individual roles and behaviors of the components making up a container: namespaces, control groups, and copy-on-write systems. Then, he will use them to assemble a container from scratch, and highlight the differences (and likelinesses) with existing container systems.
Cgroups, namespaces, and beyond: what are containers made from? (DockerCon Eu...Jérôme Petazzoni
Linux containers are different from Solaris Zones or BSD Jails: they use discrete kernel features like cgroups, namespaces, SELinux, and more. We will describe those mechanisms in depth, as well as demo how to put them together to produce a container. We will also highlight how different container runtimes compare to each other.
This talk was delivered at DockerCon Europe 2015 in Barcelona.
Introduction to Docker at Glidewell Laboratories in Orange CountyJérôme Petazzoni
In this presentation we will introduce Docker, and how you can use it to build, ship, and run any application, anywhere. The presentation included short demos, links to further material, and of course Q&As. If you are already a seasoned Docker user, this presentation will probably be redundant; but if you started to use Docker and are still struggling with some of his facets, you'll learn some!
If you're not familiar with Docker yet, here is your chance to catch up: a quick overview of the Open Source Docker Engine, and its associated services delivered through the Docker Hub. It also includes Jérôme will also discuss the new features of Docker 1.0, and briefly explain how you can run and maintain Docker on Azure. In addition, an Azure team member will demonstrate how deploy docker to Azure. The presentation will be followed by a Q&A session!
Describes what is lightweight virtualization and containers, and the low-level mechanisms in the Linux kernel that it relies on: namespaces, cgroups. It also gives details on AUFS. Those component together are the key to understanding how modern systems like Docker (http://www.docker.io/) work.
OpenVZ, which has turned 7 recently, is an implementation of lightweight virtualization technology for Linux, something which is also referred to as LXC or just containers. The talk gives an insight into 7 different problems with containers and how they were solved. While most of these problems and solutions belongs in the Linux kernel, kernel knowledge is not expected from the audience.
Internal presentation of Docker, Lightweight Virtualization, and linux Containers; at Spotify NYC offices, featuring engineers from Yandex, LinkedIn, Criteo, and NASA!
Docker storage drivers by Jérôme PetazzoniDocker, Inc.
The first release of Docker only supported AUFS, and AUFS was available (out of the box) only on Debian and Ubuntu kernel. Then Red Hat wanted Docker to run on its distros, and contributed the Device Mapper driver, and later the BTRFS driver, and recently the overlayfs driver.
Jérôme presents how those drivers compare from a high-level perspective, explaining their pros and cons.
Then he showed each driver in action, and look at low-level implementation details. We won't dive into the golang implementation code itself, but we will explain the concepts of each driver. This will help to better understand how they work, and give some hints when it comes to troubleshoot their behaviour.
Anatomy of a Container: Namespaces, cgroups & Some Filesystem Magic - LinuxConJérôme Petazzoni
Containers are everywhere. But what exactly is a container? What are they made from? What's the difference between LXC, butts-nspawn, Docker, and the other container systems out there? And why should we bother about specific filesystems?
In this talk, Jérôme will show the individual roles and behaviors of the components making up a container: namespaces, control groups, and copy-on-write systems. Then, he will use them to assemble a container from scratch, and highlight the differences (and likelinesses) with existing container systems.
Cgroups, namespaces, and beyond: what are containers made from? (DockerCon Eu...Jérôme Petazzoni
Linux containers are different from Solaris Zones or BSD Jails: they use discrete kernel features like cgroups, namespaces, SELinux, and more. We will describe those mechanisms in depth, as well as demo how to put them together to produce a container. We will also highlight how different container runtimes compare to each other.
This talk was delivered at DockerCon Europe 2015 in Barcelona.
Introduction to Docker at Glidewell Laboratories in Orange CountyJérôme Petazzoni
In this presentation we will introduce Docker, and how you can use it to build, ship, and run any application, anywhere. The presentation included short demos, links to further material, and of course Q&As. If you are already a seasoned Docker user, this presentation will probably be redundant; but if you started to use Docker and are still struggling with some of his facets, you'll learn some!
If you're not familiar with Docker yet, here is your chance to catch up: a quick overview of the Open Source Docker Engine, and its associated services delivered through the Docker Hub. It also includes Jérôme will also discuss the new features of Docker 1.0, and briefly explain how you can run and maintain Docker on Azure. In addition, an Azure team member will demonstrate how deploy docker to Azure. The presentation will be followed by a Q&A session!
LXC, Docker, security: is it safe to run applications in Linux Containers?Jérôme Petazzoni
Linux Containers (or LXC) is now a popular choice for development and testing environments. As more and more people use them in production deployments, they face a common question: are Linux Containers secure enough? It is often claimed that containers have weaker isolation than virtual machines. We will explore whether this is true, if it matters, and what can be done about it.
Kernel Recipes 2013 - Viewing real time ltt trace using gtkwaveAnne Nicolas
This presentation will explain how to use some ltt to be viewd in gtkwave, a graphical visualization tool, developped by the Parrot team.
It will also explain why this tool was developed, review some of the problems that have been analyzed using these traces. It will finally end up on the ongoing integration with LTTng 2.x.
Introduction to Docker, December 2014 "Tour de France" Bordeaux Special EditionJérôme Petazzoni
Docker, the Open Source container Engine, lets you build, ship and run, any app, anywhere.
This is the presentation which was shown in December 2014 for the last stop of the "Tour de France" in Bordeaux. It is slightly different from the presentation which was shown in the other cities (http://www.slideshare.net/jpetazzo/introduction-to-docker-december-2014-tour-de-france-edition), and includes a detailed history of dotCloud and Docker and a few other differences.
Special thanks to https://twitter.com/LilliJane and https://twitter.com/zirkome, who gave me the necessary motivation to put together this slightly different presentation, since they had already seen the other presentation in Paris :-)
Kernel Recipes 2013 - Kernel for your deviceAnne Nicolas
Any industrial project based on Linux involves Longterm management of a Linux kernel and therefore a number of questions to ask about the choices to be made. BSP, Linux distribution, kernel.org? Which version?
These questions will be reviewed and best practices to facilitate this maintenance.
Docker, Linux Containers, and Security: Does It Add Up?Jérôme Petazzoni
Containers are becoming increasingly popular. They have many advantages over virtual machines: they boot faster, have less performance overhead, and use less resources. However, those advantages also stem from the fact that containers share the kernel of their host, instead of abstracting an new independent environment. This sharing has significant security implications, as kernel exploits can now lead to host-wide escalations.
In this presentation, we will:
- Review the actual security risks, in particular for multi-tenant environments running arbitrary applications and code
- Discuss how to mitigate those risks
- Focus on containers as implemented by Docker and the libcontainer project, but the discussion also stands for plain containers as implemented by LXC
[DockerCon 2019] Hardening Docker daemon with Rootless modeAkihiro Suda
https://dockercon19.smarteventscloud.com/connect/sessionDetail.ww?SESSION_ID=281879
Docker CE 19.03 is going to support "Rootless mode", which allows running the entire Docker daemon and its dependencies as a non-root user on the host, so as to protect the host from malicious containers in a simple but very strong way. Rootless mode is also attractive for users who cannot get `sudo` permission for installing Docker on shared computing machines. e.g. HPC users. In this talk, Akihiro Suda, the author of the Rootless mode (PR: moby#38050), will explain how users can get started with Rootless mode. He will also explain the implementation details of Rootless mode and planned enhancements such as LDAP integration.
Union FileSystem - A Building Blocks Of a ContainerKnoldus Inc.
Namespace, CGroup, and Union file-system are the basic building blocks of a container. Let’s have our focus on file-system. Why yet another file-system for the container? Is Conventional Linux file-systems like ext2, ext3, ext4, XFS, etc. not good enough to meet the purpose? In this blog post, I will try to answer these questions. Here we will be delving deeply into the Union File System and a few of its essential properties.
Agenda:
In this talk we will present various locking mechanisms implemented in the linux kernel.
From System V locks to raw spinlocks and the RT patch.
Speaker:
Mark Veltzer - CTO of Hinbit and a senior instructor at John Bryce. Mark is also a member of the Free Source Foundation and contributes to many free projects.
https://github.com/veltzer
Linux Containers(LXC) allow running multiple isolated Linux instances (containers) on the same host.
Containers share the same kernel with anything else that is running on it, but can be constrained to only use a defined amount of resources such as CPU, memory or I/O.
A container is a way to isolate a group of processes from the others on a running Linux system.
DOXLON November 2016: Facebook Engineering on cgroupv2Outlyer
Cgroupv1 (or just "cgroups") has helped revolutionize the way that we manage and use containers over the past 8 years. In kernel 4.5, a complete overhaul is coming -- cgroupv2. This talk will go into why a new control group system was needed, the changes from cgroupv1, and practical uses that you can apply to improve the level of control you have over the processes on your servers.
This presentation looks deep into the concept of containerization. What is containerization, how is it different from VMs, how containerization is achieved using Linux containers (LXC), control groups (cgroups) and copy on write file systems and current trends in containerization/docker are described.
LXC, Docker, security: is it safe to run applications in Linux Containers?Jérôme Petazzoni
Linux Containers (or LXC) is now a popular choice for development and testing environments. As more and more people use them in production deployments, they face a common question: are Linux Containers secure enough? It is often claimed that containers have weaker isolation than virtual machines. We will explore whether this is true, if it matters, and what can be done about it.
Kernel Recipes 2013 - Viewing real time ltt trace using gtkwaveAnne Nicolas
This presentation will explain how to use some ltt to be viewd in gtkwave, a graphical visualization tool, developped by the Parrot team.
It will also explain why this tool was developed, review some of the problems that have been analyzed using these traces. It will finally end up on the ongoing integration with LTTng 2.x.
Introduction to Docker, December 2014 "Tour de France" Bordeaux Special EditionJérôme Petazzoni
Docker, the Open Source container Engine, lets you build, ship and run, any app, anywhere.
This is the presentation which was shown in December 2014 for the last stop of the "Tour de France" in Bordeaux. It is slightly different from the presentation which was shown in the other cities (http://www.slideshare.net/jpetazzo/introduction-to-docker-december-2014-tour-de-france-edition), and includes a detailed history of dotCloud and Docker and a few other differences.
Special thanks to https://twitter.com/LilliJane and https://twitter.com/zirkome, who gave me the necessary motivation to put together this slightly different presentation, since they had already seen the other presentation in Paris :-)
Kernel Recipes 2013 - Kernel for your deviceAnne Nicolas
Any industrial project based on Linux involves Longterm management of a Linux kernel and therefore a number of questions to ask about the choices to be made. BSP, Linux distribution, kernel.org? Which version?
These questions will be reviewed and best practices to facilitate this maintenance.
Docker, Linux Containers, and Security: Does It Add Up?Jérôme Petazzoni
Containers are becoming increasingly popular. They have many advantages over virtual machines: they boot faster, have less performance overhead, and use less resources. However, those advantages also stem from the fact that containers share the kernel of their host, instead of abstracting an new independent environment. This sharing has significant security implications, as kernel exploits can now lead to host-wide escalations.
In this presentation, we will:
- Review the actual security risks, in particular for multi-tenant environments running arbitrary applications and code
- Discuss how to mitigate those risks
- Focus on containers as implemented by Docker and the libcontainer project, but the discussion also stands for plain containers as implemented by LXC
[DockerCon 2019] Hardening Docker daemon with Rootless modeAkihiro Suda
https://dockercon19.smarteventscloud.com/connect/sessionDetail.ww?SESSION_ID=281879
Docker CE 19.03 is going to support "Rootless mode", which allows running the entire Docker daemon and its dependencies as a non-root user on the host, so as to protect the host from malicious containers in a simple but very strong way. Rootless mode is also attractive for users who cannot get `sudo` permission for installing Docker on shared computing machines. e.g. HPC users. In this talk, Akihiro Suda, the author of the Rootless mode (PR: moby#38050), will explain how users can get started with Rootless mode. He will also explain the implementation details of Rootless mode and planned enhancements such as LDAP integration.
Union FileSystem - A Building Blocks Of a ContainerKnoldus Inc.
Namespace, CGroup, and Union file-system are the basic building blocks of a container. Let’s have our focus on file-system. Why yet another file-system for the container? Is Conventional Linux file-systems like ext2, ext3, ext4, XFS, etc. not good enough to meet the purpose? In this blog post, I will try to answer these questions. Here we will be delving deeply into the Union File System and a few of its essential properties.
Agenda:
In this talk we will present various locking mechanisms implemented in the linux kernel.
From System V locks to raw spinlocks and the RT patch.
Speaker:
Mark Veltzer - CTO of Hinbit and a senior instructor at John Bryce. Mark is also a member of the Free Source Foundation and contributes to many free projects.
https://github.com/veltzer
Linux Containers(LXC) allow running multiple isolated Linux instances (containers) on the same host.
Containers share the same kernel with anything else that is running on it, but can be constrained to only use a defined amount of resources such as CPU, memory or I/O.
A container is a way to isolate a group of processes from the others on a running Linux system.
DOXLON November 2016: Facebook Engineering on cgroupv2Outlyer
Cgroupv1 (or just "cgroups") has helped revolutionize the way that we manage and use containers over the past 8 years. In kernel 4.5, a complete overhaul is coming -- cgroupv2. This talk will go into why a new control group system was needed, the changes from cgroupv1, and practical uses that you can apply to improve the level of control you have over the processes on your servers.
This presentation looks deep into the concept of containerization. What is containerization, how is it different from VMs, how containerization is achieved using Linux containers (LXC), control groups (cgroups) and copy on write file systems and current trends in containerization/docker are described.
Historically, sharing a Linux server entailed all kinds of untenable compromises. In addition to the security concerns, there was simply no good way to keep one application from hogging resources and messing with the others. The classic “noisy neighbor” problem made shared systems the bargain-basement slums of the Internet, suitable only for small or throwaway projects.
Serious use-cases traditionally demanded dedicated systems. Over the past decade virtualization (in conjunction with Moore’s law) has democratized the availability of what amount to dedicated systems, and the result is hundreds of thousands of websites and applications deployed into VPS or cloud instances. It’s a step in the right direction, but still has glaring flaws.
Most of these websites are just piles of code sitting on a server somewhere. How did that code got there? How can it can be scaled? Secured? Maintained? It’s anybody’s guess. There simply isn’t enough SysAdmin talent in the world to meet the demands of managing all these apps with anything close to best practices without a better model.
Containers are a whole new ballgame. Unlike VMs, you skip the overhead of running an entire OS for every application environment. There’s also no need to provision a whole new machine to have a place to deploy, meaning you can spin up or scale your application with orders of magnitude more speed and accuracy.
An introduction to Linux Container, Namespace & Cgroup.
Virtual Machine, Linux operating principles. Application constraint execution environment. Isolate application working environment.
Linux container (LXC) seems to be preferred technology for deployment of Platform as a service (PaaS) in cloud. Partly because it's easy to install on top of existing visualization platforms (KVM, VMware, VirtualBox), partly because it is lightweight solution to provide separation and process allocations between separate containers running under single kernel.
In this talk we will take a look at LXC and try to explain how to combine it with mandatory access control (MAC) mechanisms within Linux kernel to provide secure separation between different users of applications.
A talk presented at the Automotive Grade Linux All-Members meeting on September 8, 2015. The focus on why AGL should adopt systemd, and highlights two of the more difficult integration issues that may arise while doing so. The embedded SVG image, courtesy Marko Hoyer of ADIT, is at http://she-devel.com/2015-07-23_amm_demo.svg
The Future of Security and Productivity in Our Newly Remote WorldDevOps.com
Andy has made mistakes. He's seen even more. And in this talk he details the best and the worst of the container and Kubernetes security problems he's experienced, exploited, and remediated.
This talk details low level exploitable issues with container and Kubernetes deployments. We focus on lessons learned, and show attendees how to ensure that they do not fall victim to avoidable attacks.
See how to bypass security controls and exploit insecure defaults in this technical appraisal of the container and cluster security landscape.
"Lightweight Virtualization with Linux Containers and Docker". Jerome Petazzo...Yandex
Lightweight virtualization", also called "OS-level virtualization", is not new. On Linux it evolved from VServer to OpenVZ, and, more recently, to Linux Containers (LXC). It is not Linux-specific; on FreeBSD it's called "Jails", while on Solaris it’s "Zones". Some of those have been available for a decade and are widely used to provide VPS (Virtual Private Servers), cheaper alternatives to virtual machines or physical servers. But containers have other purposes and are increasingly popular as the core components of public and private Platform-as-a-Service (PAAS), among others.
Just like a virtual machine, a Linux Container can run (almost) anywhere. But containers have many advantages over VMs: they are lightweight and easier to manage. After operating a large-scale PAAS for a few years, dotCloud realized that with those advantages, containers could become the perfect format for software delivery, since that is how dotCloud delivers from their build system to their hosts. To make it happen everywhere, dotCloud open-sourced Docker, the next generation of the containers engine powering its PAAS. Docker has been extremely successful so far, being adopted by many projects in various fields: PAAS, of course, but also continuous integration, testing, and more.
Docker and Containers for Development and Deployment — SCALE12XJérôme Petazzoni
Docker is an Open Source engine to build, run, and manage containers. We'll explain what are Linux Containers, what powers them (under the hood), and what extra value Docker brings to the table. Then we'll see what the typical Docker workflow looks like from a developer point of view. We'll also give an Ops perspective, including deployment options. If you already saw a "Docker 101", consider this presentation as the February 2014 update! :-)
Workflow story: Theory versus Practice in large enterprises by Marcin PiebiakNETWAYS
Uphill battle against large enterprise it environments and IT corporate culture. How those difficulties turned out opportunities and clever implementations. Interesting modules, integrations and workflow pieces.
Field Employee Tracking System| MiTrack App| Best Employee Tracking Solution|...informapgpstrackings
Keep tabs on your field staff effortlessly with Informap Technology Centre LLC. Real-time tracking, task assignment, and smart features for efficient management. Request a live demo today!
For more details, visit us : https://informapuae.com/field-staff-tracking/
We describe the deployment and use of Globus Compute for remote computation. This content is aimed at researchers who wish to compute on remote resources using a unified programming interface, as well as system administrators who will deploy and operate Globus Compute services on their research computing infrastructure.
Developing Distributed High-performance Computing Capabilities of an Open Sci...Globus
COVID-19 had an unprecedented impact on scientific collaboration. The pandemic and its broad response from the scientific community has forged new relationships among public health practitioners, mathematical modelers, and scientific computing specialists, while revealing critical gaps in exploiting advanced computing systems to support urgent decision making. Informed by our team’s work in applying high-performance computing in support of public health decision makers during the COVID-19 pandemic, we present how Globus technologies are enabling the development of an open science platform for robust epidemic analysis, with the goal of collaborative, secure, distributed, on-demand, and fast time-to-solution analyses to support public health.
May Marketo Masterclass, London MUG May 22 2024.pdfAdele Miller
Can't make Adobe Summit in Vegas? No sweat because the EMEA Marketo Engage Champions are coming to London to share their Summit sessions, insights and more!
This is a MUG with a twist you don't want to miss.
A Comprehensive Look at Generative AI in Retail App Testing.pdfkalichargn70th171
Traditional software testing methods are being challenged in retail, where customer expectations and technological advancements continually shape the landscape. Enter generative AI—a transformative subset of artificial intelligence technologies poised to revolutionize software testing.
Custom Healthcare Software for Managing Chronic Conditions and Remote Patient...Mind IT Systems
Healthcare providers often struggle with the complexities of chronic conditions and remote patient monitoring, as each patient requires personalized care and ongoing monitoring. Off-the-shelf solutions may not meet these diverse needs, leading to inefficiencies and gaps in care. It’s here, custom healthcare software offers a tailored solution, ensuring improved care and effectiveness.
Enhancing Project Management Efficiency_ Leveraging AI Tools like ChatGPT.pdfJay Das
With the advent of artificial intelligence or AI tools, project management processes are undergoing a transformative shift. By using tools like ChatGPT, and Bard organizations can empower their leaders and managers to plan, execute, and monitor projects more effectively.
Paketo Buildpacks : la meilleure façon de construire des images OCI? DevopsDa...Anthony Dahanne
Les Buildpacks existent depuis plus de 10 ans ! D’abord, ils étaient utilisés pour détecter et construire une application avant de la déployer sur certains PaaS. Ensuite, nous avons pu créer des images Docker (OCI) avec leur dernière génération, les Cloud Native Buildpacks (CNCF en incubation). Sont-ils une bonne alternative au Dockerfile ? Que sont les buildpacks Paketo ? Quelles communautés les soutiennent et comment ?
Venez le découvrir lors de cette session ignite
How to Position Your Globus Data Portal for Success Ten Good PracticesGlobus
Science gateways allow science and engineering communities to access shared data, software, computing services, and instruments. Science gateways have gained a lot of traction in the last twenty years, as evidenced by projects such as the Science Gateways Community Institute (SGCI) and the Center of Excellence on Science Gateways (SGX3) in the US, The Australian Research Data Commons (ARDC) and its platforms in Australia, and the projects around Virtual Research Environments in Europe. A few mature frameworks have evolved with their different strengths and foci and have been taken up by a larger community such as the Globus Data Portal, Hubzero, Tapis, and Galaxy. However, even when gateways are built on successful frameworks, they continue to face the challenges of ongoing maintenance costs and how to meet the ever-expanding needs of the community they serve with enhanced features. It is not uncommon that gateways with compelling use cases are nonetheless unable to get past the prototype phase and become a full production service, or if they do, they don't survive more than a couple of years. While there is no guaranteed pathway to success, it seems likely that for any gateway there is a need for a strong community and/or solid funding streams to create and sustain its success. With over twenty years of examples to draw from, this presentation goes into detail for ten factors common to successful and enduring gateways that effectively serve as best practices for any new or developing gateway.
top nidhi software solution freedownloadvrstrong314
This presentation emphasizes the importance of data security and legal compliance for Nidhi companies in India. It highlights how online Nidhi software solutions, like Vector Nidhi Software, offer advanced features tailored to these needs. Key aspects include encryption, access controls, and audit trails to ensure data security. The software complies with regulatory guidelines from the MCA and RBI and adheres to Nidhi Rules, 2014. With customizable, user-friendly interfaces and real-time features, these Nidhi software solutions enhance efficiency, support growth, and provide exceptional member services. The presentation concludes with contact information for further inquiries.
Experience our free, in-depth three-part Tendenci Platform Corporate Membership Management workshop series! In Session 1 on May 14th, 2024, we began with an Introduction and Setup, mastering the configuration of your Corporate Membership Module settings to establish membership types, applications, and more. Then, on May 16th, 2024, in Session 2, we focused on binding individual members to a Corporate Membership and Corporate Reps, teaching you how to add individual members and assign Corporate Representatives to manage dues, renewals, and associated members. Finally, on May 28th, 2024, in Session 3, we covered questions and concerns, addressing any queries or issues you may have.
For more Tendenci AMS events, check out www.tendenci.com/events
In software engineering, the right architecture is essential for robust, scalable platforms. Wix has undergone a pivotal shift from event sourcing to a CRUD-based model for its microservices. This talk will chart the course of this pivotal journey.
Event sourcing, which records state changes as immutable events, provided robust auditing and "time travel" debugging for Wix Stores' microservices. Despite its benefits, the complexity it introduced in state management slowed development. Wix responded by adopting a simpler, unified CRUD model. This talk will explore the challenges of event sourcing and the advantages of Wix's new "CRUD on steroids" approach, which streamlines API integration and domain event management while preserving data integrity and system resilience.
Participants will gain valuable insights into Wix's strategies for ensuring atomicity in database updates and event production, as well as caching, materialization, and performance optimization techniques within a distributed system.
Join us to discover how Wix has mastered the art of balancing simplicity and extensibility, and learn how the re-adoption of the modest CRUD has turbocharged their development velocity, resilience, and scalability in a high-growth environment.
Providing Globus Services to Users of JASMIN for Environmental Data AnalysisGlobus
JASMIN is the UK’s high-performance data analysis platform for environmental science, operated by STFC on behalf of the UK Natural Environment Research Council (NERC). In addition to its role in hosting the CEDA Archive (NERC’s long-term repository for climate, atmospheric science & Earth observation data in the UK), JASMIN provides a collaborative platform to a community of around 2,000 scientists in the UK and beyond, providing nearly 400 environmental science projects with working space, compute resources and tools to facilitate their work. High-performance data transfer into and out of JASMIN has always been a key feature, with many scientists bringing model outputs from supercomputers elsewhere in the UK, to analyse against observational or other model data in the CEDA Archive. A growing number of JASMIN users are now realising the benefits of using the Globus service to provide reliable and efficient data movement and other tasks in this and other contexts. Further use cases involve long-distance (intercontinental) transfers to and from JASMIN, and collecting results from a mobile atmospheric radar system, pushing data to JASMIN via a lightweight Globus deployment. We provide details of how Globus fits into our current infrastructure, our experience of the recent migration to GCSv5.4, and of our interest in developing use of the wider ecosystem of Globus services for the benefit of our user community.
Software Engineering, Software Consulting, Tech Lead.
Spring Boot, Spring Cloud, Spring Core, Spring JDBC, Spring Security,
Spring Transaction, Spring MVC,
Log4j, REST/SOAP WEB-SERVICES.
Quarkus Hidden and Forbidden ExtensionsMax Andersen
Quarkus has a vast extension ecosystem and is known for its subsonic and subatomic feature set. Some of these features are not as well known, and some extensions are less talked about, but that does not make them less interesting - quite the opposite.
Come join this talk to see some tips and tricks for using Quarkus and some of the lesser known features, extensions and development techniques.
Exploring Innovations in Data Repository Solutions - Insights from the U.S. G...Globus
The U.S. Geological Survey (USGS) has made substantial investments in meeting evolving scientific, technical, and policy driven demands on storing, managing, and delivering data. As these demands continue to grow in complexity and scale, the USGS must continue to explore innovative solutions to improve its management, curation, sharing, delivering, and preservation approaches for large-scale research data. Supporting these needs, the USGS has partnered with the University of Chicago-Globus to research and develop advanced repository components and workflows leveraging its current investment in Globus. The primary outcome of this partnership includes the development of a prototype enterprise repository, driven by USGS Data Release requirements, through exploration and implementation of the entire suite of the Globus platform offerings, including Globus Flow, Globus Auth, Globus Transfer, and Globus Search. This presentation will provide insights into this research partnership, introduce the unique requirements and challenges being addressed and provide relevant project progress.
AI Pilot Review: The World’s First Virtual Assistant Marketing SuiteGoogle
AI Pilot Review: The World’s First Virtual Assistant Marketing Suite
👉👉 Click Here To Get More Info 👇👇
https://sumonreview.com/ai-pilot-review/
AI Pilot Review: Key Features
✅Deploy AI expert bots in Any Niche With Just A Click
✅With one keyword, generate complete funnels, websites, landing pages, and more.
✅More than 85 AI features are included in the AI pilot.
✅No setup or configuration; use your voice (like Siri) to do whatever you want.
✅You Can Use AI Pilot To Create your version of AI Pilot And Charge People For It…
✅ZERO Manual Work With AI Pilot. Never write, Design, Or Code Again.
✅ZERO Limits On Features Or Usages
✅Use Our AI-powered Traffic To Get Hundreds Of Customers
✅No Complicated Setup: Get Up And Running In 2 Minutes
✅99.99% Up-Time Guaranteed
✅30 Days Money-Back Guarantee
✅ZERO Upfront Cost
See My Other Reviews Article:
(1) TubeTrivia AI Review: https://sumonreview.com/tubetrivia-ai-review
(2) SocioWave Review: https://sumonreview.com/sociowave-review
(3) AI Partner & Profit Review: https://sumonreview.com/ai-partner-profit-review
(4) AI Ebook Suite Review: https://sumonreview.com/ai-ebook-suite-review
7. whoami (1)
NAME
Anthony Wong - 黃彥邦
JOB
Engineering Manager, Hardware Enablement at Canonical
LINUX EXPERIENCE
First started Linux on Redhat 4.2 in 1997
Became Debian Developer in 1998
Works in Linux industry ever since
Contributed to lots of FOSS projects, e.g. Debian, Ubuntu
16. Linux is more secure because...
● Default user does not have admin privilege
● Linux is diverse
● Windows dominates desktop market, majority of viruses target Windows
● Linus Torvalds said "given enough eyeballs, all bugs are shallow."
18. 2 most severe security vulnerabilities in recent years
19. Heartbleed bug
● Disclosed in April 2014 in the OpenSSL.
● Due to improper input validation (missing bounds check) in the
implementation of the TLS heartbeat extension - buffer over-read.
● 17% (around half a million) of the Internet's secure web servers were
believed to be vulnerable to the attack. Some estimate 500 million
computers affected.
● Allow theft of the servers' private keys and users' session cookies and
passwords.
● Affected websites include Yahoo!, Stack Overflow, Github, Amazon Web
Services, Wikipedia.
20. Shellshock bug
● Disclosed on 24 September 2014
● A security hole in bash dating from version 1.03 (August 1989)
● Bash unintentionally executes commands when the commands are
concatenated to the end of function definitions stored in the values of
environment variables.
env x='() { :;}; echo vulnerable' bash -c "echo this is a test"
● Can be triggered through HTTP_USER_AGENT variable on CGI-based web
servers.
● Attackers exploited Shellshock within hours of the initial disclosure by
creating botnets to perform DDOS attacks and vulnerability scanning.
22. What problems do we have?
● OpenSSL, for a long time, was maintained by two guys named Steve. That
means that the internet for a long period of time was secured by those
two guys.
● OpenSSH was maintained by one guy working part time.
● Bash is maintained by just 1 guy.
● GnuPG author going broke
23. What problems do we have?
● From research data
○ 51% of active projects have only 1 contributor
○ 19% have 2
○ 9% have 3
○ 5% have 4
○ 3% have 5
○ Overall, 87% of projects have 5 or fewer committers per year.
○ Merely 1% of projects have 50 or more committers per year, and a
scant 0.1% have 200 or more
Source: http://redmonk.com/dberkholz/2013/04/22/the-size-of-open-source-communities-and-its-impact-upon-activity-licensing-and-
hosting/
24. But may be Linus’ Law still applies to linux kernel?
25. ● 21 million LOC in linux kernel
4.5.
● 3 million LOC (17%) in linux
kernel untouched for 10
years since 2005.
● 7.8 changes per hour!
● Linus’ Law applies to kernel
but not without its problems.
32. What is sandbox?
● A security mechanism for
separating running programs, so
that it won’t harm the host
machine.
● Implemented by executing the
software in a restricted
operating system environment,
thus controlling the resources
(for example, file descriptors,
memory, file system space, etc.)
that a process may use.
33. Sandbox related technologies
Virtual machine
Unix permissions
UID/GID
chroot
Linux Capabilities
cgroup
Namespaces
seccomp
SELinux & AppArmor
Container
34. Virtual machine
● Emulate another computer system.
● Processes are confined in the VM.
● Can act as a security boundary.
● Examples: VMware, virtualbox, KVM, Xen, OpenVZ, Java Virtual Machine,
.net runtime, Dalvik
● Fun fact (off-topic): there are non-general purpose virtual machine in
kernel, for BPF ("Berkeley packet filter") and ACPI.
35. UID separation
● Android assigns a unique user ID (UID) to each Android application and
runs it as that user in a separate process.
○ Unlike traditional Linux.
● On Android, the Dalvik VM is not a security boundary, so Dalvik can
interoperate with native code in the same application without any security
constraints.
36. chroot (2,8)
● Run command or interactive shell with special root directory.
● Commonly used for building software and packages.
● schroot allows normal user to chroot and more features.
● Only protects filesystem, but does not restrict the use of resources like
I/O, bandwidth, disk space or CPU time.
● chrooted programs with sufficient privileges may perform a second
chroot to break out.
● Can still create device nodes and mount the filesystems, can’t block low-
level access to system devices by privileged users.
37. Linux Capabilities (7)
● traditional UNIX distinguishes two categories of processes
○ privileged processes (effective user ID = 0)
○ unprivileged processes (effective UID ≠ 0)
● Linux divides the privileges traditionally associated with superuser into
capabilities.
● Provide fine-grained control over superuser permissions.
● Examples: CAP_SYS_ADMIN, CAP_NET_ADMIN, CAP_SYS_TIME,
CAP_NET_BIND_SERVICE
38. Linux Capabilities - demo
● Check your ping command is SUID root or not
● Check capabilities of /bin/ping
○ getcap /bin/ping
● Grant CAP_NET_RAW to /bin/ping
○ sudo setcap cap_net_raw+ep /bin/ping
● Remove capabilities from /bin/ping
○ sudo setcap -r /bin/ping
○ Can you still ping if /bin/ping is not SUID root and without capabilities set.
40. cgroup
● cgroups (control groups) limits system resource usage (CPU, memory,
disk I/O, network, etc.)
● ulimit can do some of these but not easy to manage.
● Resource limiting
○ groups can be set to not exceed a configured memory limit, which also includes the file
system cache
● Prioritization
○ some groups may get a larger share of CPU utilization or disk I/O throughput
● Accounting
○ measures a group's resource usage, which may be used, for example, for billing purposes
● Control
○ freezing groups of processes, their checkpointing and restarting
41. cgroup resource controllers (subsystems)
● memory: limit memory usage, OOM kicks in to kill process if limit reached.
● cpu: assign relative CPU share of a cgroup
● blkio: assign relative I/O access and upper limit for the number of I/O
operations performed by a specific device
● cpuset: assigns individual CPUs and memory nodes to cgroups
● devices: control read/write/mknod permission
● net_cls, net_prio: assigns class and priority to network traffic, does not set
limit.
● freezer cgroup: freeze/thaw group of processes. Better than
SIGSTOP/SIGCONT.
42. Using cgroup
● Imagine different cgroup subsystems (CPU, memory, block IO) are
different trees, and processes are nodes of the tree.
● Try “mount | grep cgroup” to see the cgroup sysfs is mounted.
● You can manipulate cgroup under /sys/fs/cgroup/<subsystem>/
● Check your current cgroup status: cat /proc/self/cgroup.
● You can try systemd-cgls and systemd-cgtop.
● Another tool is cgmanager.
● Can apply limits by systemd such as MemoryLimit.
44. Namespaces (7)
● Provides a process an isolated system view of the global system.
● Limits how much a process can see.
● Types of namespaces:
○ PID, isolates processes
○ Network, isolates network devices, stacks, ports, etc
○ Mount, isolates mount points
○ User, isolates User and Group IDs
○ UTS (Unix timesharing - host and domain name)
○ IPC (Inter-process communication)
○ Cgroup, isolates cgroup root directory
45. PID namespace
● Can only see processes in its
own namespace.
● Parent namespace can see all
child processes.
● The same processes will have
different PID in different PID
namespaces.
● Always start with PID 1.
46. Network namespace
● Process has its own network
stack
○ Network interfaces, including lo
○ Iptables
○ Routing tables
○ Sockets
47. Mount namespace
● Isolates filesystem mount points.
● Processes in different mount
namespaces have different views
of the filesystem hierarchy.
● Can be used like chroot.
● Can have private mount, e.g. can
have its own /tmp or /var/tmp
48. ● Does UID/GID mapping, so a
process's user and group IDs can
be different inside and outside a
user namespace.
● For example, process has
unprivileged user ID outside
namespace but have UID 0
inside the namespace
● That means the process has full
privileges inside the namespace,
but is unprivileged outside.
● Relatively new (since 3.8)
UID 0→5000 in namespace
maps to
UID 10000→15000 outside of
namespace
User namespace Example
49. More about namespaces
● Look into /proc/<PID>/ns for namespaces handles.
● Namespace API:
○ clone - create a new child process possibly with new namespace
■ Use instead of fork
○ setns - join an existing namespace
○ unshare - move a process to a new namespace
50. Namespaces demo
systemd-nspawn - Spawn a namespace container for debugging, testing and
building
$ debootstrap --arch=amd64 sid ~/debian-sid
# systemd-nspawn -D ~/debian-sid/
# systemd-nspawn --private-network --private-users=1000 -D ~/debian-sid/
● Run a command in the container and check its user.
● Check the network with ifconfig.
● Create some files and check its file ownership.
52. Seccomp
● Filters system calls unneeded by a process.
● Do you need all 300+ syscalls provided by kernel?
● Consider a number crunching application does not need bind(), accept()
or chroot().
● Can tremendously reduce the kernel attack surface.
● First version was merged to Linux 2.6.12 in 2005.
● Used in Chrome browser, OpenSSH, systemd, LXC, Docker, snapd
● Two modes: SECCOMP_SET_MODE_STRICT and
SECCOMP_SET_MODE_FILTER
53. Using seccomp in your code
● SECCOMP_SET_MODE_STRICT only allows to use read(2), write(2), _exit(2)
and sigreturn(2). You will get SIGKILL if you call other syscalls.
seccomp(SECCOMP_SET_MODE_STRICT, 0, NULL);
or
prctl(PR_SET_SECCOMP, SECCOMP_MODE_STRICT);
54. Using seccomp in your code
● SECCOMP_SET_MODE_FILTER: can control what system calls are allowed.
● Added to Linux 3.5 in 2010
seccomp(SECCOMP_SET_MODE_FILTER, types, args);
or
prctl(PR_SET_SECCOMP, SECCOMP_MODE_FILTER, args);
The system calls allowed are defined by a pointer to a Berkeley Packet Filter (BPF) passed via args.
55. How end-users use seccomp?
● What if we don’t trust the running code? We can’t trust it to use seccomp.
● Needs containers to help confine running programs.
● systemd: SystemCallFilter=<allowed syscalls>
● For snap packages, apps can be confined or unconfined (for
development). Confined apps can declare what extra capabilities it needs
(through “interfaces”).
56. {
"defaultAction": "SCMP_ACT_ERRNO",
"syscalls": [
{
"name": "accept",
"action": "SCMP_ACT_ALLOW",
"args": []
},
{
"name": "accept4",
"action": "SCMP_ACT_ALLOW",
"args": []
...
seccomp in Docker Example
● Pass a docker profile (in JSON
format) when running your
container.
● Docker's default seccomp
profile is a whitelist.
● Syscalls such as clone, ptrace,
reboot, umount are not in
whitelist.
58. SELinux & AppArmor
● Both are Linux kernel security module.
● Both implements mandatory access controls (MAC).
● Original primary developer of SELinux is NSA, do you trust it?
● SELinux policy is much more complex than AppArmor.
● SELinux is used in Fedora/RHEL and Android since 4.3 (permissive) and 4.4
(enforcing).
● Ubuntu uses AppArmor.
59. AppArmor example
● Both are Linux kernel security module.
● Both implements mandatory access controls (MAC).
● Original primary developer of SELinux is NSA, do you trust it?
● SELinux policy is much more complex than AppArmor. According to
research, SELinux scores 34.58 in usability while AppArmor scores 54.93.
● SELinux is used in Fedora/RHEL and Android since 4.3 (permissive) and 4.4
(enforcing).
● AppArmor is default in Ubuntu.
66. Container
Technologies I just talked about: userspace, cgroup, capabilities
are building blocks in many container runtimes.
67.
68. Container runtimes
● LXC
○ LXC being the runtime, LXD being the hypervisor
● systemd-nspawn (1)
○ Spawn a namespace container for debugging, testing and building
○ Not for serious production use
● Docker
○ use LXC at first, later libcontainer and now runc
○ DockerHub as ecosystem to share images
○ micro-service
● Rkt by CoreOS
● OpenVZ
○ Predates the container hype, does not use namespace or cgroup
○ Requires patched kernel for full feature
73. Snap
● Backed by Canonical, installed in 16.04 by default.
● Can be used in Fedora, Debian, Arch, Gentoo.
● Strive to be a universal application format.
● A minimal core OS to provide basic root filesystem.
● Secured by AppArmor and seccomp.
● Package is created by a tool called snapcraft.
● Common commands are: snap install, snap remove, snap find, snap list,
very easy to use.
● You can create unconfined snaps for development or local use.
● Interface is the mechanism for providing resource sharing and granting
permissions.
74. Snap sandbox
● Snaps are installed into the regular host filesystem in
/snap/$name/$version/
● When a snap is launched:
○ A slave mount namespace is created
○ A private /tmp directory is created
○ The ubuntu-core-launcher bind mounts /bin, /lib, /lib64, /sbin, /usr from the ubuntu-core
snap
○ The ubuntu-core-launcher applies the AppArmor/seccomp confinement
○ The application is launched: it can see the host's /dev, /proc/, /sys, /media and other
mount points, but that might be mitigated by AppArmor
● But X11 is insecure! Needs Mir/Wayland!
75. Snapcraft example
name: dash
version: "0.5.9"
summary: dash shell
description: |
The Debian Almquist Shell (dash) is a POSIX-compliant shell derived
from ash.
Since it executes scripts faster than bash, and has fewer library
dependencies (making it more robust against software or hardware
failures), it is used as the default system shell on Debian systems.
apps:
dash:
command: dash
plugs: [home, camera]
76. Snap interface
# Description: Can access non-hidden files in user's $HOME. This is restricted
# because it gives file access to all of the user's $HOME.
# Usage: reserved
# Note, @{HOME} is the user's $HOME, not the snap's $HOME
# Allow read access to toplevel $HOME for the user
owner @{HOME}/ r,
# Allow read/write access to all non-hidden files that aren't in ~/snap/
# allow creating a few files not caught above
owner @{HOME}/{s,sn,sna}{,/} rwk,
# allow access to gvfs mounts (only allow writes to files, not mount point)
owner /run/user/[0-9]*/gvfs/** r,
owner /run/user/[0-9]*/gvfs/*/** w,
77. Snap Demo
● Install dash_confined_0.5.9_amd64.snap and check access to /home.
● Write to /tmp see what happens.
● Check dmesg for apparmor errors.
● Install dash_home_0.5.9_amd64.snap and check home access.
● Install dash_home+camera_0.5.9_amd64.snap and check /dev/video0
access.
○ Check with getfacl /dev/video0 to make sure you have access.
○ cat /dev/video0 in dash
○ You still need snap connect dash:camera ubuntu-core:camera to grant
access.
78. Flatpak
● Originally called xdg-app, mainly contributed by a Red Hat engineer.
● Can be used in Fedora, Ubuntu, Debian, Arch, Gentoo.
● Strive to be a universal application format.
● Depends on systemd for cgroup, which makes it less universal.
● Works closely with the GNOME community.
● For desktop applications, need to open up "safe" ways for an application
to interact with the system, they called it Portals.
80. Flatpak sandbox
● All processes run as the user with no capabilities
● All processes run in a transient systemd user scope with the name
flatpak-$appid-$pid
● A filesystem namespace where:
○ / is a private tmpfs not visible anywhere else. This is pivot_root:ed into so it is the new /
and all other mounts from the host are unmounted from the namespace.
○ /usr is a bind mount of the runtime, /app is a bind mount of the application
○ /proc shows only the processes in the app sandbox
○ /sys is a read-only bind mount of the host /sys
○ /dev contains /dev/full, /dev/null, /dev/random, /dev/urandom, /dev/tty and /dev/zero
● Seccomp is used to disable unnecessary system calls
● A private pid namespace with a minimal init process that reaps zombies
81. Flatpak sandbox continues...
● A private user namespace
● A private ipc namespace
● A private network namespace with only an ipv4 loopback device
○ Optionally can use the host network namespace
● SELinux or AppArmor is NOT used.
● Need wayland compositor in the session and no access to the Xserver to
be properly sandboxed, because X is insecure.
85. Controversy
https://lkml.org/lkml/2014/4/2/420
On Wed, Apr 2, 2014 at 11:42 AM, Steven Rostedt <rostedt@goodmis.org> wrote:
>
> The response is:
>
> "Generic terms are generic, not the first user owns them."
And by "their" you mean Kay Sievers.
Key, I'm f*cking tired of the fact that you don't fix problems in the
code *you* write, so that the kernel then has to work around the
problems you cause.
Greg - just for your information, I will *not* be merging any code
from Kay into the kernel until this constant pattern is fixed.
[show linus’s middle finger photo]
87. systemd.slice
● encodes information about a slice which is a concept for hierarchically
managing resources of a group of processes.
● performed by creating a node in the Linux Control Group (cgroup) tree.
● For each slice, certain resource limits may be set that apply to all
processes of all units contained in that slice.
● Default slices:
○ -.slice (root)
○ system.slice
○ user.slice
○ machine.slice
88. Control group with systemd
● Shows control group
● systemd-cgls
● systemd-cgtop
89. Security features in systemd
● Can be used to sandbox traditional services.
● Makes use of existing technologies to protect system services.
90. Service unit configuration
PrivateTmp=yes|no
● Private instances of /var and /var/tmp.
● Lifecycle is bound to service runtime.
● Use filesystem namespace.
● Solves Tmp race , symlink race, insecure temp file.
91. Service unit configuration
CapabilityBoundingSet=
CAP_SYS_ADMIN, CAP_KILL, CAP_MKNOD, CAP_SYS_TIME,
CAP_NET_BIND_SERVICE
● Think about an ntpd daemon that no longer need to run as root.
● Example:
Network-manager.service:
CapabilityBoundingSet=CAP_NET_ADMIN CAP_DAC_OVERRIDE
CAP_NET_RAW CAP_NET_BIND_SERVICE CAP_SETGID CAP_SETUID
CAP_SYS_MODULE CAP_AUDIT_WRITE CAP_KILL CAP_SYS_CHROOT
92. Service unit configuration
PrivateDevices=yes|no
● Get rid of raw devices
● Only have must-have devices such as /dev/null, /dev/random, /dev/null,
/dev/zero.
● Examples:
systemd-bus-proxyd.service:PrivateDevices=yes
systemd-hostnamed.service:PrivateDevices=yes
systemd-localed.service:PrivateDevices=yes
systemd-timesyncd.service:PrivateDevices=yes
94. Service unit configuration
PrivateNetwork=yes|no
● ‘no’ means loopback and no access to network interfaces
● fwupd.service:
[Service]
Type=dbus
BusName=org.freedesktop.fwupd
ExecStart=/usr/lib/x86_64-linux-gnu/fwupd/fwupd
PrivateNetwork=yes
PrivateTmp=yes
RestrictAddressFamilies=AF_UNIX|AF_INET|AF_INET6
99. Service unit configuration
LimitNPROC=
● Limit number of processes a user can have.
● For fork() protection.
● Same as ulimit -u
● Example: bluetooth.service:LimitNPROC=1
104. Benefits of kernel live patching
● No need to reboot!
● System administrators are afraid of reboot.
● Need physical presence.
● Even more reluctant to reboot if the machine has been running for long
time.
● Keep the uptime record :)
105.
106. Summary
● The problems that the Linux ecosystem is facing.
● Reviewed sandboxing technologies in Linux
○ cgroup, namespace, seccomp, Linux capabilities
○ MAC mechanism such as AppArmor and SELinux
● We looked at containers.
● How systemd can be used to protect services.
● Snap vs Flatpak, how they make use of sandboxing.
● Kernel live patching for fixing kernel bugs without reboot.
● SSL certifications is now free thanks to Let’s Encrypt project.