Keep Your Data Safe in a Containerized Application

•Download as ODP, PDF•

0 likes•103 views

Containers are great. They make building and scaling applications a breeze. What they do not tell you is that containers are great for stateless applications. Managing and keeping your data is crucial for any long running and non-trivial application. In this talk, we will try to understand the challenges of persisting data in a containerized application and explore possible solutions using Docker as the container run-time. We will look at battle proven techniques and see how to adapt them to modern, scalable and containerized applications.

Software

Keep Your Data Safe
in a Containerized
Application
Hagai Barel

Talking Points
1
2
3
4
Introduction
Containers 101
The Challenges of a Distributed Application
Picking the Right Storage Solution
5 Use Case
6 Summery and Questions

Introduction - About Me
Senior DevOps Consultant at CodeValue
Infrastructure architect and container
specialist
Instructor, Lecturer and Trainer
Expert in distributed, containerized,
scalable applications and CI/CD
pipelines

Introduction - About CodeValue
High quality software projects and
managed services
Valuable training and mentoring
Advanced software technology and
methodology consulting
Visual Studio extension for c# debugging
Ton of cool features that will cut your debugging
time and improve the experience

Introduction – Containerized Applications

Containers 101 – What are Containers?
Method of OS level virtualization
Allows running applications in resource isolated processes
Isolation and encapsulation is achieved using cgroups and
namespaces

Containers 101 – Images and Containers
Images are the blueprints for containers
An Image is an immutable file that’s essentially a snapshot of a container
Several ways to build an image, using a text file is the most common
image container

Containers 101 – Benefits of Containers
Fast startup and shutdown times
Easily scalable
The image can contain just the stuff my code needs
Process isolation

The Challenges of a Distributed
Application

The Challenges of a Distributed Application
Network StorageCompute

The Challenges of a Distributed Application - Compute
Resource allocation
Scaling (horizontal / vertical)
Container placement
Etc...

The Challenges of a Distributed Application - Network
Service discovery
Routing (external and internal)
Security
Load balancing
Etc...

The Challenges of a Distributed Application - Storage
Availability
Durability
Scale
Etc...

Picking the Right Storage Solution – In Container
Pros:
“Cheap”
Fast
No special configuration
required
Cons:
No sharing of data
Very low durability
When to use:
Static files (that are part of the image)
Cached data
Clustered application

Picking the Right Storage Solution – on the Host
Pros:
Share volumes (but only
on that host)
Fast
Cons:
Data is bound to host life
Additional configuration
required
When to use:
Store and retrieve logs
Side car setup
Host

Picking the Right Storage Solution – External Storage
Pros:
Share volumes (on all
hosts)
Scalable
Durable
Cons:
Additional configuration
required
Latency can be an issue
Read / write locks
When to use:
Critical data
Long term storage
Host Host
S3, NFS etc..

Use Case – Running GeoServer in a Container
GeoServer is an open source server for sharing geospatial data
Java application that runs on top of tomcat web server
Web application receives REST api calls that are processed by the server
Map data and files are stored in file system

Solution – Part 1
Base the GeoServer image on official Tomcat image
Place the web application static files inside the GeoServer image during build

Solution – Part 2
Setup an NFS share to hold GeoServer’s data files
Mount that share to the container when it starts
docker run -d -v /nfs-share/geoserver_data:/opt/geoserver/data_dir hbarel/docker-
geoserver

Summery
Containers are short lived, critical data should be stored outside of them
Hosts fail, go down for maintenance or could become unavailable
If possible use cloud provided storage solutions
Consider performance and test it when deciding between storage options
Distributed system == distributed storage!

Kanister is an open source framework for application-level data management in Kubernetes. It allows developers to define data operations for applications through Blueprint definitions. These Blueprints capture tasks like backup and restore. Kanister then executes those tasks across Kubernetes using CustomResourceDefinitions and tools. It provides a way to perform complex data operations for stateful applications from the Kubernetes layer regardless of the underlying application architecture.

Enable stateful applications on aws with persistent storage for k8s.pptx

LibbySchulze

The document discusses using Amazon EFS for persistent storage with Amazon EKS. It provides examples of how customers like T-Mobile and Johnson & Johnson have used EFS to modernize applications with containers on EKS. It describes how EFS offers elastic, available, durable file storage that integrates seamlessly with EKS through the EFS CSI driver. It outlines the key concepts of using EFS with EKS including storage classes, persistent volumes, and volume claims. Finally, it shows an example of attaching an EFS file system to a pod.

Edge 2016 Session 1886 Building your own docker container cloud on ibm power...

Yong Feng

The material for IBM Edge 2016 session for a client use case of Spectrum Conductor for Containers https://www-01.ibm.com/events/global/edge/sessions/. Please refer to http://ibm.biz/ConductorForContainers for more details about Spectrum Conductor for Containers. Please refer to https://www.youtube.com/watch?v=7YMjP6EypqA and https://www.youtube.com/watch?v=d9oVPU3rwhE for the demo of Spectrum Conductor for Containers.

Veritas NetBackup benchmark comparison: Data protection in a large-scale virt...

Principled Technologies

In an enterprise environment, a data center VM footprint can grow quickly; large-scale deployments of thousands of virtual machines are becoming increasingly common. Risk of failure grows proportionally to the number of systems deployed and critical failures are unavoidable. Your ability to offer data protection from a backup solution is critical to business continuity. Elongated, inefficient protection windows can create resource contention with production environments, therefore, it is critical to execute system backup in a finite window of time. The Veritas NetBackup Integrated Appliance running NetBackup 7.6 offered application protection to 1,000 VMs in 80.3 percent less time in SAN testing and used NetApp array-based snapshots to create recovery points in 93.8 percent less time than Competitor “C.” In addition, the Veritas NetBackup Integrated Appliance with NetBackup 7.6 created backup images that offered granular recovery without additional steps and in a backup window 69.0 percent shorter than the backup window needed for Competitor “C.” These time savings can scale as your VM footprint grows, allowing you to execute both system protection and user-friendly, simplified recovery.

SQL PASS Taiwan 七月份聚會-1

SQLPASSTW

This document proposes a virtual heterogeneous database platform to address challenges with physical database servers like low utilization and high costs. It would provide a virtualization platform to host multiple database types and high availability solutions in virtual machines, improving efficiency through automated provisioning and management. The document discusses database server models, high availability solutions like Datakeeper and clustering, operations team concerns about flexibility and testing, and monitoring tools.

VMware vFabric Data Director for DB as a Service

VMware vFabric

vFabric Data Director is a software solution that enables enterprises to provide self-service database provisioning and management capabilities to users on demand. It allows developers to quickly provision fully configured databases with automated backup and restore. It also provides administrators with fine-grained access controls and tools to enforce IT policies and optimize resource utilization. The solution is built on VMware vSphere and uses a vSphere-optimized PostgreSQL database to deliver benefits like elastic resource allocation and automated high availability.

Data Agility for Devops - OSI 2018

OpenEBS

About the Talk: Cloud native ecosystem is bringing a huge change in the way of DevOps in every cloud native organisation. Developers and operators in cloud native organisations are using tools and platforms like Kubernetes to achieve the agility promised by DevOps and microservices. The tools and best practices for stateless applications have been well established and the results can be seen in the agility of teams using these stateless applications. However, stateful applications pose new challenges to DevOps teams in achieving the agility as the best practices around persistent storage management are still emerging. In this talk, first we discuss the challenges faced by DevOps while dealing with persistent storage handling in stateful applications. Then we discuss the open source tools and best practices for DevOps teams to achieve data agility of cloud native applications.

IBM Spectrum Scale Slidecast

inside-BigData.com

In this slidecast, Jim Gutowski and Scott Fadden from IBM describe the advantages IBM Spectrum Scale Storage brings to the enterprise. IBM Spectrum Scale is a proven, scalable, high-performance data and file management solution (based upon IBM General Parallel File System or GPFS, also formerly known as code name Elastic Storage). IBM Spectrum Scale provides world-class storage management with extreme scalability, flash accelerated performance, and automatic policy-based storage tiering from flash through disk to tape. IBM Spectrum Scale reduces storage costs up to 90% while improving security and management efficiency in cloud, big data & analytics environments." Watch the video presentation starting 4/8/2015: http://wp.me/p3RLHQ-e0a Learn more: http://ibm.co/1y1QIrj

For most federal agencies dealing with increased security threats, limiting machine-data collection is not an option. But faced with finite IT budgets, few agencies can continue to absorb the high costs of scaling high-end network attached storage (NAS) or moving to and expanding a block-based storage footprint. During this webcast, you’ll learn about more cost-effective solutions to support large-scale machine-data ingestion and fast data access for security analytics. You’ll learn about: - The common challenges organizations face when scaling security workflows - Why a high-performance cache works to solve these issues - How to integrate cloud into processing and storage for additional scalability and efficiencies

Continuity Software 4.3 Detailed Gaps

GilHecht

The document discusses challenges with high availability and disaster recovery configurations including configuration drift between production and standby systems. It introduces RecoverGuard as a solution to automatically detect configuration issues and vulnerabilities across HA/DR systems to prevent data loss and downtime. Key features of RecoverGuard include automated scanning, over 3,000 configuration signatures, analytics across availability, data protection, recoverability, and a dashboard. Examples of issues found include partial data replication, inconsistent access to storage, and configuration differences between production and disaster recovery systems.

Webinar: How Snapshots CAN be Backups

Storage Switzerland

Snapshots have been a key feature of primary storage infrastructures that IT professionals have relied on for years. But storage systems have traditionally been able to support only a limited number of active snapshots. And snapshots, being pointers and not actual data, are also susceptible to a primary storage system failure. As a result, most IT professionals use snapshots sparingly for protecting data. In this webinar Storage Switzerland and Nexenta show you how primary storage can be architected so that snapshots are able to meet almost all of the data protection requirements an organization has.

Flex Cloud - Conceptual Design - ver 0.2

David Pasek

The Energy ========= The cost of energy is increasing. A significant part of electrical energy cost is the cost of distribution. That's the reason why the popularity of small home solar systems increases. That's the way how to generate and consume electricity locally and be independent of the distribution network. However, we have a problem. "Green Energy" from solar, wind, and hydroelectric power stations is difficult to distribute via the electrical grid. Energy accumulation (batteries, pumped storage power plant, etc.) is costly and for the traditional electrical grid is very difficult to automatically manage the distribution of so many energy sources. The Cloud Computing ================= The demand for cloud (computing and storage) capacity is increasing year by year. Internet bandwidth increases and cost decreases every year. 5G Networks and SD-WANs are on the radar. Cloud Computing is operated on data centers. The significant part of data center costs is the cost of energy. The potential synergy between Energetics and Cloud Computing ================================================= The solution is to consume electricity in the proximity of green power generators. Excess electricity is accumulated into batteries but batteries capacity is limited. We should treat batteries like a cache or buffer to overcome times when green energy does not generate energy but we have local demand. However, when we have excess electricity and the battery (cache/buffer) is full, instead of providing the energy into the electrical grid, the excess electricity can be consumed by a computer system providing compute resources to cloud computing consumers over the internet. This is the form of Distributed Cloud Computing. Cloud-Native Applications ==================== So, let's assume we will have Distributed Cloud Computing with so-called Spot Compute Resource Pools". Spot Compute Resource Pools are computing resources that can appear or disappear within hours or minutes. This is not optimal IT infrastructure for traditional software applications which are not infrastructure aware. For such distributed cloud computing the software applications must be designed and developed with infrastructure resources ephemerality in mind. In other words, Cloud-Native Applications must be able to leverage ephemeral compute resource pools and know how to use "Spot Compute Resource Pools".

Logical Architecture for Protection

Sunita Shrivastava

This document discusses physical models and logical architecture for data protection. It proposes creating a unified protection namespace to provide a single view of protected elements across the enterprise and cloud. This would prevent fragmented solutions and allow easy management of protection policies and status. The document outlines existing assets, gaps, and new scenarios to consider in building out a new architecture. It discusses using the protection namespace to manage application migration and disaster recovery in a consistent way.

When HPC meet ML/DL: Manage HPC Data Center with Kubernetes

Yong Feng

When HPC Meet ML/DL Machine learning and deep learning (ML/DL) are becoming important workloads for high performance computing (HPC) as new algorithms are developed to solve business problems across many domains. Container technologies like Docker can help with the portability and scalability needs of ML/DL workloads on HPC systems. Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications that can help run MPI jobs and ML/DL pipelines on HPC systems, though it currently lacks some features important for HPC like advanced job scheduling capabilities. Running an HPC-specific job scheduler like IBM Spectrum LSF on top of Kubernetes is one approach to address current gaps in

Lessons Learned Running Hadoop and Spark in Docker Containers

BlueData, Inc.

Many initiatives for running applications inside containers have been scoped to run on a single host. Using Docker containers for large-scale production environments poses interesting challenges, especially when deploying distributed big data applications like Apache Hadoop and Apache Spark. This session at Strata + Hadoop World in New York City (September 2016) explores various solutions and tips to address the challenges encountered while deploying multi-node Hadoop and Spark production workloads using Docker containers. Some of these challenges include container life-cycle management, smart scheduling for optimal resource utilization, network configuration and security, and performance. BlueData is "all in” on Docker containers—with a specific focus on big data applications. BlueData has learned firsthand how to address these challenges for Fortune 500 enterprises and government organizations that want to deploy big data workloads using Docker. This session by Thomas Phelan, co-founder and chief architect at BlueData, discusses how to securely network Docker containers across multiple hosts and discusses ways to achieve high availability across distributed big data applications and hosts in your data center. Since we’re talking about very large volumes of data, performance is a key factor, so Thomas shares some of the storage options implemented at BlueData to achieve near bare-metal I/O performance for Hadoop and Spark using Docker as well as lessons learned and some tips and tricks on how to Dockerize your big data applications in a reliable, scalable, and high-performance environment. http://conferences.oreilly.com/strata/hadoop-big-data-ny/public/schedule/detail/52042

Highly scalable caching service on cloud - Redis

Krishna-Kumar

Dell emc back up solution in azure cloud

vipinvips

The document discusses backup solutions for Azure virtual machines and identifies limitations with native Azure backup. It recommends using Dell EMC Networker and DataDomain as a third-party enterprise backup solution that can meet tier 1 RPO and RTO requirements of less than an hour. It provides details on the proposed solution architecture with Networker and DataDomain instances in each region along with replication capabilities. The solution aims to address limitations of native Azure backup and provide application-aware backups, encryption, short RPOs, and support for workloads like Oracle databases.

From Containerized Application to Secure and Scaling With Kubernetes

Shikha Srivastava

Discuss following: What does it really take to make sure your application is production ready? With new privacy regulations being added, many aspects need to be taken into account when deciding when to deliver your final application is ready for production. Can your application handle multiple users with different levels of access? Can you extend your application to use existing authentication and authorization platforms? Have you invested in using Mutual TLS for communication between components? How do you manage the certificates and passwords used within your product? Is CICD your friend or your enemy when it comes to delivering your product? Have you considered the availability and scalability of the application?

Expert Panel Discussion + Case Study Review

ncceconnect

- Richland School District #400 has 11,000 students and 1,200 staff across 63 schools. - They migrated their servers from a distributed to centralized VMware environment between 2008-2009. Currently they have 55 VMs running on 3 ESXi hosts. - They selected Citrix XenDesktop for their VDI solution to provide Windows 2000 support through 2014 and refresh aging PCs and laptops. - For their VDI implementation they chose Cisco UCS server hardware for its ability to support 800 clients with room to expand and cost-effectiveness over HP and Dell solutions. Their architecture includes XenDesktop running on UCS blades with storage and networking.

Presentation v cat 3.0 - architecture to implementation

solarisyourep

The document introduces vCAT (VMware Cloud Architecture Toolkit), which provides guidance for architects on designing, implementing, operating and consuming a VMware cloud. It outlines a 5 step process for architecting a VMware cloud that includes assessing needs, defining services, designing resources, deploying examples, and managing the cloud lifecycle. The document then provides an example of using vCAT to design initial storage requirements for a private cloud being built for a software development company.

Techgate solution sets 2014

Techgate plc

Storage Architectures And Options

Alan McSweeney

The document discusses storage options and architectures for managing digital image data over long periods of time. It outlines objectives to provide high-level information on storage solutions and initiate discussions on configurations. Sample storage architectures are described, including multi-tier solutions using primary, secondary and tertiary storage, as well as hierarchical storage management. Hardware options like disk, tape, optical storage and virtual tape libraries are also summarized.

Big Data on Cloud Native Platform

Sunil Govindan

Nyoug delphix slideshare

Kyle Hailey

This presentation discusses how data constraints negatively impact IT and businesses. It introduces Delphix as a solution to virtualize data and eliminate the data constraint. The key points covered are: I. The problem of data constraints in IT which include strains on IT resources, huge costs, and lack of awareness of the problem by companies. II. Delphix addresses this by providing virtual data using file-level snapshots and compression to deliver fast, parallel environments without moving large amounts of data. III. The solution enables use cases such as accelerated development, QA, disaster recovery, real-time business intelligence, and application modernization.

Storage Integrations for Container Orchestrators

{code} by Dell EMC

Overcoming write availability challenges of PostgreSQL

EDB

There's no shortage of physical replication solutions for PostgreSQL, they scale horizontally and provide high read availability. But where they fall short is write availability, which leads many users to consider PostgreSQL logical replication. Existing solutions have a single point of failure or are dependent on a forked, vendor-provided PostgreSQL extension making reliable, enterprise-class logical replication hard to come by. Furthermore, these solutions put limits on scaling PostgreSQL. By combining Kafka, an open source event streaming system with PostgreSQL, customers can get a fault tolerant, scalable logical replication service. Learn how EDB Replicate leverages Kafka for high write availability needed for today's demanding consumers who expect their applications to be always available and won't tolerate latency.

Mesosphere and the Enterprise: Run Your Applications on Apache Mesos - Steve ...

{code} by Dell EMC

What do Apple, ADP, Netflix, eBay, Time Warner Cable, and UC Berkley have in common? All of them use Apache Mesos! Did you know {code} by Dell EMC contributed the first storage module to Apache Mesos? This session will examine Mesos architecture and will focus on how to use frameworks such as Marathon to deploy containerized applications. See how {code} by Dell EMC is integrating REX-Ray to provide stateful application support to the most battle-tested container platform.

Data Analytics Using Container Persistence Through SMACK - Manny Rodriguez-Pe...

{code} by Dell EMC

New digital business models facilitated by containers require collecting and analyzing device data. Apache Mesos removes the need to build separate stacks and combines optimized application containers and data analytics into a single platform. In this session, we will explore new approaches to data analytics using REX-Ray as a container persistence tool and the SMACK stack - Spark, Mesos, Akka, Cassandra, Kafka – a set of tools for building data and messaging layers for digital engagement apps.

Red Hat Storage Day Atlanta - Persistent Storage for Linux Containers

Red_Hat_Storage

This document discusses persistent storage options for Linux containers. It notes that while some containerized applications are stateless, most require persistence for storing application and configuration data. It evaluates options like NFS, GlusterFS, Ceph RBD, and block storage, noting that persistent storage needs to be scalable, resilient, flexible, software-defined, and open. It provides examples of using Gluster and Ceph storage with containers. The document concludes that most containerized apps will need persistent storage and that software-defined storage allows for hyperconverged applications and storage on premises or in hybrid clouds.

Evolving Your Distributed Cache In A Continuous Delivery World: Tyler Vangorder

Redis Labs

1. The document discusses the evolution of caching strategies at Build.com as their systems and traffic grew rapidly over time. They initially used a Java-based distributed cache and later switched to Redis which proved more effective. 2. As Build.com moved to a continuous delivery model with multiple environments, they needed a "shared" cache that both environments could use. They implemented a unified caching model where each version of code has its own bucket in the cache but objects can be promoted from older versions if they are compatible. 3. The key aspects of the unified caching model are using a serialization checksum to detect changes between versions, using a build number as the cache key so each version is separate, and attempting to promote

What's hot

Scaling Security Workflows in Government Agencies

Avere Systems

Continuity Software 4.3 Detailed Gaps

GilHecht

Webinar: How Snapshots CAN be Backups

Storage Switzerland

Flex Cloud - Conceptual Design - ver 0.2

David Pasek

Logical Architecture for Protection

Sunita Shrivastava

When HPC meet ML/DL: Manage HPC Data Center with Kubernetes

Yong Feng

Lessons Learned Running Hadoop and Spark in Docker Containers

BlueData, Inc.

Highly scalable caching service on cloud - Redis

Krishna-Kumar

Dell emc back up solution in azure cloud

vipinvips

From Containerized Application to Secure and Scaling With Kubernetes

Shikha Srivastava

Expert Panel Discussion + Case Study Review

ncceconnect

Presentation v cat 3.0 - architecture to implementation

solarisyourep

Techgate solution sets 2014

Techgate plc

Storage Architectures And Options

Alan McSweeney

Big Data on Cloud Native Platform

Sunil Govindan

Nyoug delphix slideshare

Kyle Hailey

Storage Integrations for Container Orchestrators

{code} by Dell EMC

Overcoming write availability challenges of PostgreSQL

EDB

Mesosphere and the Enterprise: Run Your Applications on Apache Mesos - Steve ...

{code} by Dell EMC

Data Analytics Using Container Persistence Through SMACK - Manny Rodriguez-Pe...

{code} by Dell EMC

What's hot (20)

Scaling Security Workflows in Government Agencies

Continuity Software 4.3 Detailed Gaps

Webinar: How Snapshots CAN be Backups

Flex Cloud - Conceptual Design - ver 0.2

Logical Architecture for Protection

When HPC meet ML/DL: Manage HPC Data Center with Kubernetes

Lessons Learned Running Hadoop and Spark in Docker Containers

Highly scalable caching service on cloud - Redis

Dell emc back up solution in azure cloud

From Containerized Application to Secure and Scaling With Kubernetes

Expert Panel Discussion + Case Study Review

Presentation v cat 3.0 - architecture to implementation

Techgate solution sets 2014

Storage Architectures And Options

Big Data on Cloud Native Platform

Nyoug delphix slideshare

Storage Integrations for Container Orchestrators

Overcoming write availability challenges of PostgreSQL

Mesosphere and the Enterprise: Run Your Applications on Apache Mesos - Steve ...

Data Analytics Using Container Persistence Through SMACK - Manny Rodriguez-Pe...

Similar to Keep Your Data Safe in a Containerized Application

Red Hat Storage Day Atlanta - Persistent Storage for Linux Containers

Red_Hat_Storage

Evolving Your Distributed Cache In A Continuous Delivery World: Tyler Vangorder

Redis Labs

Red Hat Storage Day LA - Persistent Storage for Linux Containers

Red_Hat_Storage

This document discusses persistent storage options for Linux containers and how Red Hat Storage addresses the storage needs of containerized applications. It begins by explaining how containers package applications and dependencies for portability and ease of management. Typical workloads for containers often require persistent storage. The document then evaluates options like NFS, GlusterFS, Ceph RBD, iSCSI/FC and public cloud storage, noting that Red Hat Storage provides scalable, resilient, flexible software-defined storage. It presents Red Hat Storage and OpenShift Enterprise as a solution that allows applications and storage to run together on servers for improved utilization and costs. The document closes with two customer case studies demonstrating how Red Hat Storage supports containerized workloads at CapitalOne and Ver

Webinar: Cloud Storage: The 5 Reasons IT Can Do it Better

Storage Switzerland

In this webinar learn the five reasons why a private cloud storage system may be more cost effective and deliver a higher quality of service than public cloud storage providers. In this webinar you will learn: 1. What Public Cloud Storage Architectures Look Like 2. Why Public Providers Chose These Architectures 3. The Problem With Traditional Data Center File Solutions 4. Bringing Cloud Lessons to Traditional IT 5. The Five Reasons IT can Do it Better

DCEU 18: Provisioning and Managing Storage for Docker Containers

Docker, Inc.

Anshul Pundir - Senior Software Engineer, Docker Anusha Ragunathan - Senior Software Engineer, Docker Inc In this talk, we will discuss storage concepts related to containers on the Docker platform with the perspective of what is important throughout the lifecycle of an application., We will focus on application provisioning: creating persistent volumes and policies for stateful data and management: replication and failover scenarios, backup/restore, monitoring etc. Through this talk, we will cover the latest storage features and also some of the current and future direction of container storage. Key concepts covered about running stateful applications: - Persistent Volumes - Provisioning (Static vs Topology-aware) - Data Availability (failover with scheduler policies) - Data Protection (using Backup/Restore) - Monitoring (using Prometheus/Grafana dashboards) We will look at each of the characteristics in detail with demos.

Demystifying Containerization Principles for Data Scientists

Dr Ganesh Iyer

Hands on kubernetes_container_orchestration

Amir Hossein Sorouri

Webinar : Docker in Production

Newt Global Consulting LLC

This document provides an overview and agenda for a webinar on Docker orchestration in production environments. It includes: - An introduction to Newt Global and their DevOps transformation services. - Details on two speakers - the DevOps practice leader and an AWS solutions specialist from Newt Global. - An outline of Newt Global's overall offerings including DevOps pipeline development, infrastructure automation, and architecture modernization. - Topics that will be covered in the webinar related to using Docker in production including quick deployment, resource management, backup strategies, security, monitoring, and image/storage management. - Instructions for participants on submitting questions and receiving follow-up responses.

Dockers and kubernetes

Dr Ganesh Iyer

BGOUG "Agile Data: revolutionizing database cloning'

Kyle Hailey

The document discusses using data virtualization to address the constraint of data in DevOps workflows. It describes how traditional database cloning methods are inefficient and consume significant resources. The solution presented uses thin cloning technology to take snapshots of production databases and provide virtual copies for development, QA, and other environments. This allows for unlimited, self-service virtual databases that reduce bottlenecks and waiting times compared to physical copies.

Cloud computing UNIT 2.1 presentation in

RahulBhole12

Cloud storage allows users to store files online through cloud storage providers like Apple iCloud, Dropbox, Google Drive, Amazon Cloud Drive, and Microsoft SkyDrive. These providers offer various amounts of free storage and options to purchase additional storage. They allow files to be securely uploaded, accessed, and synced across devices. The best cloud storage provider depends on individual needs and preferences regarding storage space requirements and features offered.

DCEU 18: Use Cases and Practical Solutions for Docker Container Storage on Sw...

Docker, Inc.

Mark Church - Product Manager, Docker Don Stewart - Solutions Architect, Docker Persistent storage has quickly advanced from something considered incompatible with containers to a mature set of solutions and patterns that have been thoroughly adopted by the industry. We’ll define the persistent characteristics of different use-cases and map these to some of the many solutions that exist for container storage. From this talk you’ll learn about the storage options available to users on Swarm, Kubernetes, on-premises, cloud, and how they work and compare to each other. You’ll also learn how to characterize different persistent application requirements and the solutions best for suited for them.

A Tight Ship: How Containers and SDS Optimize the Enterprise

Eric Kavanagh

The Briefing Room with Dez Blanchfield and Red Hat Think of containers as the drones of modern computing. They're small, agile, and can carry a significant payload. In many ways, they represent the fruition of the last two major paradigm shifts in enterprise software: SOA and virtualization. However, for companies to fully leverage this innovative approach, a persistent storage platform is needed that is as flexible and scalable as containers themselves. Register for this episode of The Briefing Room to hear Bloor Group Data Scientist Dez Blanchfield, who will explain the significance of container technology, and the relevance of software-defined storage (SDS) in a constantly evolving IT world. He'll be briefed by Steve Watt and Sayan Saha of Red Hat, who will demonstrate how open-source technology can help organizations take advantage of this brave new world of enterprise computing. They will explain how containers are the next step in the evolution of the operating system, and why SDS is now the optimal solution.

Webinar Docker Tri Series

Newt Global Consulting LLC

Newt Global provides DevOps transformation, cloud enablement, and test automation services. It was founded in 2004 and is headquartered in Dallas, Texas with locations in the US and India. The company is a leader in DevOps transformations and has been one of the top 100 fastest growing companies in Dallas twice. The document discusses an upcoming webinar on Docker 101 that will be presented by two Newt Global employees: Venkatnadhan Thirunalai, the DevOps Practice Leader, and Jayakarthi Dhanabalan, an AWS Solution Specialist.

How Globe Telecom does Primary Backups via StorReduce to the AWS Cloud

Amazon Web Services

Globe Telecom, a large telecommunications company in the Philippines with over 65 million subscribers, needed a cost-effective and scalable solution for storing primary data backups. Its previous on-premises data domain appliances created expensive hardware silos and a risk of data loss in the event of an ill-timed backup system outage, a vulnerability the company couldn’t afford. StorReduce’s scalable deduplication software solved Globe Telecom’s problem with throughputs and recovery speeds faster than leading backup appliances, enabling them to transfer primary and secondary backup data from backup appliances and tape to the Amazon Cloud. This resulted in a data storage savings of up to 80% and ensured best-of-breed scalability, durability, and recoverability for its data.

Hybrid Cloud Approach for Secure Authorized Deduplication

Prem Rao

This document proposes a hybrid cloud approach for secure authorized data deduplication. It discusses existing systems that use data deduplication to reduce storage usage but lack security features. The proposed system uses convergent encryption for data confidentiality while allowing deduplication. It also aims to support authorized duplicate checks by encrypting files with differential privilege keys. The system design involves data owner, encryption/decryption, private cloud, public cloud, and cloud server modules. Cryptographic techniques like hashing and encryption are used along with communication via HTTP. The development follows a waterfall model with phases for requirements analysis, design, implementation, testing, and maintenance.

Top 6 Practices to Harden Docker Images to Enhance Security

9 series

Denver devops : enabling DevOps with data virtualization

Kyle Hailey

This document discusses how data constraints can limit DevOps efforts and proposes a solution using virtual data and thin cloning. It notes that moving and copying production data is challenging due to storage, personnel and time requirements. This typically results in bottlenecks, long wait times for environments, code check-ins and production bugs. The solution presented is to use a data virtualization platform that can take thin clones of production data using file system snapshots, compress the data and share it across environments through a centralized cache. This allows self-service provisioning of database environments and accelerates DevOps processes.

EQR Reporting: Rails + Amazon EC2

jeperkins4

The document discusses the architecture of an online survey platform. It covers goals of performance, availability, and scalability. It describes using services like Amazon Web Services for storage, computing, and content delivery. It also discusses optimizing application design, database separation, coding best practices, and automated testing to achieve goals. Video hosting and online analytical reporting features are explained.

Analytics with unified file and object

Sandeep Patil

Similar to Keep Your Data Safe in a Containerized Application (20)

Red Hat Storage Day Atlanta - Persistent Storage for Linux Containers

Evolving Your Distributed Cache In A Continuous Delivery World: Tyler Vangorder

Red Hat Storage Day LA - Persistent Storage for Linux Containers

Webinar: Cloud Storage: The 5 Reasons IT Can Do it Better

DCEU 18: Provisioning and Managing Storage for Docker Containers

Demystifying Containerization Principles for Data Scientists

Hands on kubernetes_container_orchestration

Webinar : Docker in Production

Dockers and kubernetes

BGOUG "Agile Data: revolutionizing database cloning'

Cloud computing UNIT 2.1 presentation in

DCEU 18: Use Cases and Practical Solutions for Docker Container Storage on Sw...

A Tight Ship: How Containers and SDS Optimize the Enterprise

Webinar Docker Tri Series

How Globe Telecom does Primary Backups via StorReduce to the AWS Cloud

Hybrid Cloud Approach for Secure Authorized Deduplication

Top 6 Practices to Harden Docker Images to Enhance Security

Denver devops : enabling DevOps with data virtualization

EQR Reporting: Rails + Amazon EC2

Analytics with unified file and object

Recently uploaded

Atelier - Innover avec l’IA Générative et les graphes de connaissances

Neo4j

Atelier - Innover avec l’IA Générative et les graphes de connaissances Allez au-delà du battage médiatique autour de l’IA et découvrez des techniques pratiques pour utiliser l’IA de manière responsable à travers les données de votre organisation. Explorez comment utiliser les graphes de connaissances pour augmenter la précision, la transparence et la capacité d’explication dans les systèmes d’IA générative. Vous partirez avec une expérience pratique combinant les relations entre les données et les LLM pour apporter du contexte spécifique à votre domaine et améliorer votre raisonnement. Amenez votre ordinateur portable et nous vous guiderons sur la mise en place de votre propre pile d’IA générative, en vous fournissant des exemples pratiques et codés pour démarrer en quelques minutes.

E-commerce Application Development Company.pdf

Hornet Dynamics

Revolutionizing Visual Effects Mastering AI Face Swaps.pdf

Undress Baby

The quest for the best AI face swap solution is marked by an amalgamation of technological prowess and artistic finesse, where cutting-edge algorithms seamlessly replace faces in images or videos with striking realism. Leveraging advanced deep learning techniques, the best AI face swap tools meticulously analyze facial features, lighting conditions, and expressions to execute flawless transformations, ensuring natural-looking results that blur the line between reality and illusion, captivating users with their ingenuity and sophistication. Web:- https://undressbaby.com/

LORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOM

lorraineandreiamcidl

DDS-Security 1.2 - What's New? Stronger security for long-running systems

Gerardo Pardo-Castellote

LORRAINE ANDREI_LEQUIGAN_HOW TO USE WHATSAPP.pptx

lorraineandreiamcidl

WhatsApp offers simple, reliable, and private messaging and calling services for free worldwide. With end-to-end encryption, your personal messages and calls are secure, ensuring only you and the recipient can access them. Enjoy voice and video calls to stay connected with loved ones or colleagues. Express yourself using stickers, GIFs, or by sharing moments on Status. WhatsApp Business enables global customer outreach, facilitating sales growth and relationship building through showcasing products and services. Stay connected effortlessly with group chats for planning outings with friends or staying updated on family conversations.

OpenMetadata Community Meeting - 5th June 2024

OpenMetadata

The OpenMetadata Community Meeting was held on June 5th, 2024. In this meeting, we discussed about the data quality capabilities that are integrated with the Incident Manager, providing a complete solution to handle your data observability needs. Watch the end-to-end demo of the data quality features. * How to run your own data quality framework * What is the performance impact of running data quality frameworks * How to run the test cases in your own ETL pipelines * How the Incident Manager is integrated * Get notified with alerts when test cases fail Watch the meeting recording here - https://www.youtube.com/watch?v=UbNOje0kf6E

What is Augmented Reality Image Tracking

pavan998932

Hand Rolled Applicative User ValidationCode Kata

Philip Schwarz

Could you use a simple piece of Scala validation code (granted, a very simplistic one too!) that you can rewrite, now and again, to refresh your basic understanding of Applicative operators <*>, <*, *>? The goal is not to write perfect code showcasing validation, but rather, to provide a small, rough-and ready exercise to reinforce your muscle-memory. Despite its grandiose-sounding title, this deck consists of just three slides showing the Scala 3 code to be rewritten whenever the details of the operators begin to fade away. The code is my rough and ready translation of a Haskell user-validation program found in a book called Finding Success (and Failure) in Haskell - Fall in love with applicative functors.

Graspan: A Big Data System for Big Code Analysis

Aftab Hussain

We built a disk-based parallel graph system, Graspan, that uses a novel edge-pair centric computation model to compute dynamic transitive closures on very large program graphs. We implement context-sensitive pointer/alias and dataflow analyses on Graspan. An evaluation of these analyses on large codebases such as Linux shows that their Graspan implementations scale to millions of lines of code and are much simpler than their original implementations. These analyses were used to augment the existing checkers; these augmented checkers found 132 new NULL pointer bugs and 1308 unnecessary NULL tests in Linux 4.4.0-rc5, PostgreSQL 8.3.9, and Apache httpd 2.2.18. - Accepted in ASPLOS ‘17, Xi’an, China. - Featured in the tutorial, Systemized Program Analyses: A Big Data Perspective on Static Analysis Scalability, ASPLOS ‘17. - Invited for presentation at SoCal PLS ‘16. - Invited for poster presentation at PLDI SRC ‘16.

GraphSummit Paris - The art of the possible with Graph Technology

Neo4j

GreenCode-A-VSCode-Plugin--Dario-Jurisic

Green Software Development

A Study of Variable-Role-based Feature Enrichment in Neural Models of Code

Aftab Hussain

Understanding variable roles in code has been found to be helpful by students in learning programming -- could variable roles help deep neural models in performing coding tasks? We do an exploratory study. - These are slides of the talk given at InteNSE'23: The 1st International Workshop on Interpretability and Robustness in Neural Software Engineering, co-located with the 45th International Conference on Software Engineering, ICSE 2023, Melbourne Australia

GOING AOT WITH GRAALVM FOR SPRING BOOT (SPRING IO)

Alina Yurenko

openEuler Case Study - The Journey to Supply Chain Security

Shane Coughlan

Oracle Database 19c New Features for DBAs and Developers.pptx

Remote DBA Services

Mobile App Development Company In Noida | Drona Infotech

Drona Infotech

Utilocate provides Smarter, Better, Faster, Safer Locate Ticket Management

Utilocate

Utilocate offers a comprehensive solution for locate ticket management by automating and streamlining the entire process. By integrating with Geospatial Information Systems (GIS), it provides accurate mapping and visualization of utility locations, enhancing decision-making and reducing the risk of errors. The system's advanced data analytics tools help identify trends, predict potential issues, and optimize resource allocation, making the locate ticket management process smarter and more efficient. Additionally, automated ticket management ensures consistency and reduces human error, while real-time notifications keep all relevant personnel informed and ready to respond promptly. The system's ability to streamline workflows and automate ticket routing significantly reduces the time taken to process each ticket, making the process faster and more efficient. Mobile access allows field technicians to update ticket information on the go, ensuring that the latest information is always available and accelerating the locate process. Overall, Utilocate not only enhances the efficiency and accuracy of locate ticket management but also improves safety by minimizing the risk of utility damage through precise and timely locates.

Need for Speed: Removing speed bumps from your Symfony projects ⚡️

Łukasz Chruściel

No one wants their application to drag like a car stuck in the slow lane! Yet it’s all too common to encounter bumpy, pothole-filled solutions that slow the speed of any application. Symfony apps are not an exception. In this talk, I will take you for a spin around the performance racetrack. We’ll explore common pitfalls - those hidden potholes on your application that can cause unexpected slowdowns. Learn how to spot these performance bumps early, and more importantly, how to navigate around them to keep your application running at top speed. We will focus in particular on tuning your engine at the application level, making the right adjustments to ensure that your system responds like a well-oiled, high-performance race car.

Neo4j - Product Vision and Knowledge Graphs - GraphSummit Paris

Neo4j

Recently uploaded (20)

Atelier - Innover avec l’IA Générative et les graphes de connaissances

E-commerce Application Development Company.pdf

Revolutionizing Visual Effects Mastering AI Face Swaps.pdf

LORRAINE ANDREI_LEQUIGAN_HOW TO USE ZOOM

DDS-Security 1.2 - What's New? Stronger security for long-running systems

LORRAINE ANDREI_LEQUIGAN_HOW TO USE WHATSAPP.pptx

OpenMetadata Community Meeting - 5th June 2024

What is Augmented Reality Image Tracking

Hand Rolled Applicative User ValidationCode Kata

Graspan: A Big Data System for Big Code Analysis

GraphSummit Paris - The art of the possible with Graph Technology

GreenCode-A-VSCode-Plugin--Dario-Jurisic

A Study of Variable-Role-based Feature Enrichment in Neural Models of Code

GOING AOT WITH GRAALVM FOR SPRING BOOT (SPRING IO)

openEuler Case Study - The Journey to Supply Chain Security

Oracle Database 19c New Features for DBAs and Developers.pptx

Mobile App Development Company In Noida | Drona Infotech

Utilocate provides Smarter, Better, Faster, Safer Locate Ticket Management

Need for Speed: Removing speed bumps from your Symfony projects ⚡️

Neo4j - Product Vision and Knowledge Graphs - GraphSummit Paris

Keep Your Data Safe in a Containerized Application

1. Keep Your Data Safe in a Containerized Application Hagai Barel

2. Talking Points 1 2 3 4 Introduction Containers 101 The Challenges of a Distributed Application Picking the Right Storage Solution 5 Use Case 6 Summery and Questions

3. Introduction - About Me Senior DevOps Consultant at CodeValue Infrastructure architect and container specialist Instructor, Lecturer and Trainer Expert in distributed, containerized, scalable applications and CI/CD pipelines

4. Introduction - About CodeValue High quality software projects and managed services Valuable training and mentoring Advanced software technology and methodology consulting Visual Studio extension for c# debugging Ton of cool features that will cut your debugging time and improve the experience

5. Introduction – Containerized Applications

6. Containers 101

7. Containers 101 – What are Containers? Method of OS level virtualization Allows running applications in resource isolated processes Isolation and encapsulation is achieved using cgroups and namespaces

8. Containers 101 – Images and Containers Images are the blueprints for containers An Image is an immutable file that’s essentially a snapshot of a container Several ways to build an image, using a text file is the most common image container

9. Containers 101 – Benefits of Containers Fast startup and shutdown times Easily scalable The image can contain just the stuff my code needs Process isolation

10. The Challenges of a Distributed Application

11. The Challenges of a Distributed Application Network StorageCompute

12. The Challenges of a Distributed Application - Compute Resource allocation Scaling (horizontal / vertical) Container placement Etc...

13. The Challenges of a Distributed Application - Network Service discovery Routing (external and internal) Security Load balancing Etc...

14. The Challenges of a Distributed Application - Storage Availability Durability Scale Etc...

15. Picking the Right Storage Solution

16. Picking the Right Storage Solution – In Container Pros: “Cheap” Fast No special configuration required Cons: No sharing of data Very low durability When to use: Static files (that are part of the image) Cached data Clustered application

17. Picking the Right Storage Solution – on the Host Pros: Share volumes (but only on that host) Fast Cons: Data is bound to host life Additional configuration required When to use: Store and retrieve logs Side car setup Host

18. Picking the Right Storage Solution – External Storage Pros: Share volumes (on all hosts) Scalable Durable Cons: Additional configuration required Latency can be an issue Read / write locks When to use: Critical data Long term storage Host Host S3, NFS etc..

19. Use Case

20. Use Case – Running GeoServer in a Container GeoServer is an open source server for sharing geospatial data Java application that runs on top of tomcat web server Web application receives REST api calls that are processed by the server Map data and files are stored in file system

21. Solution – Part 1 Base the GeoServer image on official Tomcat image Place the web application static files inside the GeoServer image during build

22. Solution – Part 2 Setup an NFS share to hold GeoServer’s data files Mount that share to the container when it starts docker run -d -v /nfs-share/geoserver_data:/opt/geoserver/data_dir hbarel/docker- geoserver

23. Summery

24. Summery Containers are short lived, critical data should be stored outside of them Hosts fail, go down for maintenance or could become unavailable If possible use cloud provided storage solutions Consider performance and test it when deciding between storage options Distributed system == distributed storage!

25. Q&A

26. Thank You!

Keep Your Data Safe in a Containerized Application

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Keep Your Data Safe in a Containerized Application

Similar to Keep Your Data Safe in a Containerized Application (20)

Recently uploaded

Recently uploaded (20)

Keep Your Data Safe in a Containerized Application