This was presented @ vbrownbag aka tech talk event happening along the sides of openstack paris nov. 2014. This presentation is recorded and posted in youtube @ http://youtu.be/Rxay08XzMZ0
GlusterFS Native driver for Openstack Manila at GlusterNight Paris @ Openstac...Deepak Shetty
Manila is OpenStack's shared file system service that provisions file shares to Nova instances. It supports multiple protocols and storage backends, including NFS and CIFS primarily as well as GlusterFS. There are two approaches to integrating GlusterFS with Manila - a GlusterFS native driver and using NFS-Ganesha with GlusterFS. The GlusterFS native driver supports certificate-based access with Manila and allows instances to communicate directly with the GlusterFS backend securely without a service VM. It provisions shares using the 'glusterfs' protocol and is available upstream with some TODO items around documentation, snapshots, and dynamic volume management.
NATS in action - A Real time Microservices Architecture handled by NATSRaül Pérez
The document describes an architecture for managing infrastructure and platforms using microservices that communicate over NATS. Key points:
- Ernest is an IAAS+PAAS hybrid cloud platform that uses microservices and NATS to manage infrastructure resources, deploy applications, and automate scaling across multiple cloud providers.
- NATS is used as the central communication system between Ernest microservices to process user-defined workflows for building environments.
- Workflows define things like networks, virtual machine instances, configuration, and can deploy and provision applications. This allows Ernest to automate the creation and management of environments.
Hitch TLS is a small and fast TLS terminator that is bundled with Varnish Plus. It allows client-side TLS termination with Varnish Cache Plus handling encryption and decryption. TLS can also be used to encrypt connections to backends by adding ".ssl = 1" to backend definitions in Varnish. Hitch TLS supports features like OCSP stapling, PROXY protocol, and run-time reloads for updating certificates without interrupting service. Performance testing shows it can handle high throughput workloads with good scalability on commodity hardware.
Logs/Metrics Gathering With OpenShift EFK StackJosef Karásek
This document summarizes a presentation about logs and metrics gathering with the OpenShift EFK stack. It introduces the OpenShift logging team and their objectives of collecting distributed logs in a common data model with security and scalability. It describes the main components of Fluendt for collection and normalization and Elasticsearch for storage. It provides examples of using the logging stack with OpenShift, OpenStack, and oVirt and advice for custom application logging.
Linux server penetration testing projectEmad Soltani
This document outlines a Linux server penetration testing project conducted by Emad Soltani Nezhad and Bardia Izadpanahi from Islamic Azad University Pardis Branch. It describes gathering information on a target system using tools like Nmap, penetrating the system using Hydra for password cracking, escalating privileges by exploiting a setuid file, and embedding backdoors by opening a remote port to maintain access without logging in.
GlusterFS Native driver for Openstack Manila at GlusterNight Paris @ Openstac...Deepak Shetty
Manila is OpenStack's shared file system service that provisions file shares to Nova instances. It supports multiple protocols and storage backends, including NFS and CIFS primarily as well as GlusterFS. There are two approaches to integrating GlusterFS with Manila - a GlusterFS native driver and using NFS-Ganesha with GlusterFS. The GlusterFS native driver supports certificate-based access with Manila and allows instances to communicate directly with the GlusterFS backend securely without a service VM. It provisions shares using the 'glusterfs' protocol and is available upstream with some TODO items around documentation, snapshots, and dynamic volume management.
NATS in action - A Real time Microservices Architecture handled by NATSRaül Pérez
The document describes an architecture for managing infrastructure and platforms using microservices that communicate over NATS. Key points:
- Ernest is an IAAS+PAAS hybrid cloud platform that uses microservices and NATS to manage infrastructure resources, deploy applications, and automate scaling across multiple cloud providers.
- NATS is used as the central communication system between Ernest microservices to process user-defined workflows for building environments.
- Workflows define things like networks, virtual machine instances, configuration, and can deploy and provision applications. This allows Ernest to automate the creation and management of environments.
Hitch TLS is a small and fast TLS terminator that is bundled with Varnish Plus. It allows client-side TLS termination with Varnish Cache Plus handling encryption and decryption. TLS can also be used to encrypt connections to backends by adding ".ssl = 1" to backend definitions in Varnish. Hitch TLS supports features like OCSP stapling, PROXY protocol, and run-time reloads for updating certificates without interrupting service. Performance testing shows it can handle high throughput workloads with good scalability on commodity hardware.
Logs/Metrics Gathering With OpenShift EFK StackJosef Karásek
This document summarizes a presentation about logs and metrics gathering with the OpenShift EFK stack. It introduces the OpenShift logging team and their objectives of collecting distributed logs in a common data model with security and scalability. It describes the main components of Fluendt for collection and normalization and Elasticsearch for storage. It provides examples of using the logging stack with OpenShift, OpenStack, and oVirt and advice for custom application logging.
Linux server penetration testing projectEmad Soltani
This document outlines a Linux server penetration testing project conducted by Emad Soltani Nezhad and Bardia Izadpanahi from Islamic Azad University Pardis Branch. It describes gathering information on a target system using tools like Nmap, penetrating the system using Hydra for password cracking, escalating privileges by exploiting a setuid file, and embedding backdoors by opening a remote port to maintain access without logging in.
Log aggregation: using Elasticsearch, Fluentd/Fluentbit and Kibana (EFK)Lee Myring
A quick introduction to log aggregation in a local Docker development environment using Fluentd followed by a demonstration using a publicly available GitHub repo.
The document provides an overview of Container Network Interface (CNI) which is a standard for configuring network interfaces for Linux containers. It discusses how CNI uses plugins to configure networking for containers and provides examples of plugins like ptp, bridge, and IPvlan. It also covers overlay networking techniques like VXLAN and OVN which can be used to provide networking between containers on different hosts.
Securing the Software Supply Chain with TUF and Docker - Justin Cappos and Sa...Docker, Inc.
If you want to compromise millions of machines and users, software distribution and software updates are an excellent attack vector. Using public cryptography to sign your packages is a good starting point, but as we will see, it still leaves you open to a variety of attacks. This is why we designed TUF, a secure software update framework. TUF helps to handle key revocation securely, limits the impact a man-in-the-middle attacker may have, and reduces the impact of repository compromise. We will discuss TUF's protections and integration into Docker's Notary software, and demonstrate new techniques that could be added to verify other parts of the software supply chain, including the development, build, and quality assurance processes.
OSMC 2018 | Tailored SNMP monitoring – Your own SNMP MIB and sub-agent with P...NETWAYS
SNMP continues to be a essential component in monitoring where the information being made available is structured in so-called Management Information (MIB) modules. The standard net-snmp distribution comes a with a variety of standard MIBs implemented by its snmpd, but sometimes there is the need to make your own information available via SNMP. Luckily snmpd can be dynamically extended by so-called subagents implementing the AgentX protocol (RFC2747). The net-snmp API however pretty much focusses on the C programming language only, laying the entrance barrier especially for non-developers rather high. In this talk Pieter will not only demonstrate the creation of a MIB, which, being a text file, is the easier part, but also how easy it is to implement a simple subagent in Python using his python-netsnmpagent module. python-netsnmpagent is a shim wrapper over the net-snmp C API trying to implement just enough abstraction. Licensed under the GPL it is available at https://github.com/pief/python-netsnmpagent as well as PyPI.
The document outlines Gluster's roadmap, including recent improvements to versions 3.5-3.7 like bitrot detection and sharding, and plans for upcoming releases 3.8 and 4.0 such as tiering support, REST APIs, new style replication, and improving the distributed hashing translator to scale to 1000 servers. It also provides an overview of Gluster's architecture and quick start instructions.
NGINX Plus is often deployed in a cluster, and the new features in R16 help our customers working in a clustered environment. New features include global rate limiting, a cluster-aware key-value store, Random with Two Choices load-balancing algorithm, and more.
Join this webinar to learn:
- About the new cluster-aware features in NGINX Plus R16: global rate limiting, key-value store, and Random with Two Choices load balancing
- How to use key-value stores in use cases such as DDoS mitigation and dynamic bandwidth limiting
- About enhanced UDP load balancing, AWS PrivateLink support, and additional new features
- How the NGINX Plus R16 features behave in action, in a live demo
https://www.nginx.com/resources/webinars/whats-new-nginx-plus-r16-emea/
Containerd internals: building a core container runtimeDocker, Inc.
In this talk, we’ll briefly overview of the OpenWhisk serverless (function-as-a-service) framework that initially used the full Docker container engine as the execution vehicle for invoking user functions via containers. After several performance and stability challenges, this project decided to assess the various layers of the Docker engine (containerd and runC) as potential options for the function invoker. Out of that work came an open source project, bucketbench, that can be used to generate benchmarks of container lifecycle operations (e.g., start, stop, kill, remove, pause, unpause) and compare multithreaded operation throughput and stability of each optional engine.
This talk will provide details on the bucketbench project, explain how it has been used to generate performance data for these container runtimes, and shares lessons learned along the way that greatly impact container runtime performance, including bottlenecks in the Linux kernel.
In this talk you’ll learn how you can use bucketbench for your own performance tuning or assessment of container runtimes and how you can collaborate on improvements to the bucketbench project.
Networking-odl and ODL Neutron Northbound are the key components for integrating OpenStack Neutron and OpenDaylight. They are actively developed open source projects. The document encourages giving the integration a try, providing feedback, and contributing to help further the integration of OpenStack and OpenDaylight networking.
This document discusses security considerations for cloud computing instances, networking, and storage. It recommends using bastion servers for CLI access with key-based authentication over secure protocols. For instances, it suggests using LTS OS releases, security patches, and HIDS for critical hosts. Networking advice includes using subnets for applications, public subnets only for public-facing systems, and private subnets with NAT for non-public systems. The document also recommends encrypting sensitive stored data with role-based access controls and logging, and storing authentication materials securely.
OSMC 2018 | Distributed Tracing FAQ by Gianluca ArbezzanoNETWAYS
Microservices, containers and more in general distributed systems have opened a different point of view on our system and applications. We need to understand how a single event or requests cross our app jumping over networks, containers, virtual machines and sometime clod provider. There is a specific practice called distributed tracing to increase observability of systems like that. After this talk, you will have a solid idea around what tracing means, how you can instrument your applications and you will be ready to trace your application across many languages using open source technologies like OpenTracing, OpenCensus, Zipkin, Jaeger, InfluxDB. You will ask yourself how you survived until today!
Couchbase Server is a NoSQL database that allows developers to build applications with agility and scale them to any size. It provides a flexible JSON schema, fast document storage and retrieval using document IDs, and various data access methods like views, global secondary indexes, and the N1QL query language. Couchbase supports many development frameworks and platforms, and can be deployed in various environments including Docker. It provides features like auto-sharding of data across nodes, replication, and cross data center replication for high availability and disaster recovery.
Smart contracts and NFTs call for a revised approach to store data. In these slides, 3 options for distributed and fault-tolerant data storage are presented:
IPFS
Filecoin
Arweave
Control Plane: Security Rationale for Istio (DevSecOps - London Gathering, Ja...Michael Man
Security Rationale For Istio
An introduction to Istio security, looking at how Istio helps to keeps your security team happy by satisfying Kubernetes security requirements for multi-tenancy, and your developers happy by reducing implementation effort. Istio is still an evolving technology, and outstanding issues and impending improvements will be discussed.
NATS & Docker Meetup in Toronto - August 2016
Implementing Microservices with NATS, Diogo Monteiro
-How Aytra uses NATS
-Benefits of using NATS for inter service communication
-Lessons learned adopting NATS
-Overview of Houston NATS library
-Demo of Aytra
You can learn more about NATS at http://www.nats.io
Cortex: Horizontally Scalable, Highly Available PrometheusGrafana Labs
In this talk we present Cortex - a horizontally scalable, highly available Prometheus implementation. Like Prometheus, Cortex is a CNCF (sandbox) project.
Cortex turns a lot of the Prometheus architectural assumptions on its head, by marrying a scale-out PromQL query engine with a storage layer based on NOSQL databases such as Bigtable, DynamoDB and Cassandra. We have disaggregated the Prometheus binary into a microservices-style architecture, with separate services for query, ingest, alerting and recording rules. By designing all these services as fungible replicas, this solution can be scaled out with ease and failure of any individual replica can be dealt with gracefully.
Deep dive into highly available open stack architecture openstack summit va...Arthur Berezin
This document summarizes a presentation on highly available OpenStack architecture. It discusses using Pacemaker and HAProxy for high availability enabling services. Shared databases like MariaDB Galera and message queues like RabbitMQ are made highly available. Individual OpenStack services like Keystone, Glance, Cinder, Nova, Neutron, and Horizon are made highly available through active-active clustering, load balancing, and fencing. The presentation covers topologies for controller, compute, network, and storage nodes. It provides examples of making individual services highly available and discusses ongoing work and future plans to improve high availability in OpenStack.
Load Balancing Apps in Docker Swarm with NGINXNGINX, Inc.
On-demand webinar recording: http://bit.ly/2mRjk2g
Docker and other container technologies continue to gain in popularity. We recently surveyed the broad community of NGINX and NGINX Plus users and found that two-thirds of organizations are either investigating containers, using them in development, or using them in production. Why? Because abstracting your applications from the underlying infrastructure makes developing, distributing, and running software simpler, faster, and more robust than ever before.
But when you move from running your app in a development environment to deploying containers in production, you face new challenges – such as how to effectively run and scale an application across multiple hosts with the performance and uptime that your customers demand.
The latest Docker release, 1.12, supports multihost container orchestration, which simplifies deployment and management of containers across a cluster of Docker hosts. In a complex environment like this, load balancing plays an essential part in delivering your container-based application with reliability and high performance.
Join us in this webinar to learn:
* The basic built-in load balancing options available in Docker Swarm Mode
* The pros and cons of moving to an advanced load balancer like NGINX
* How to integrate NGINX and NGINX Plus with Swarm Mode to provide an advanced load-balancing solution for a cluster with orchestration
* How to scale your Docker-based application with Swarm Mode and NGINX Plus
The document discusses various ways that GlusterFS can integrate with OpenStack components including Swift, Cinder, Manila, and as hyper-converged storage. Current integration includes using GlusterFS as ephemeral storage for Nova instances, as the backend for Swift, Cinder, Glance, and Manila. Work is ongoing to provide a hyper-converged solution with GlusterFS providing both compute and storage on the same nodes.
GlusterFS Cinder integration presented at GlusterNight Paris event @ Openstac...Deepak Shetty
This was a brief presentation talking about the current state of affairs on the GlusterFS Cinder Integration in the GlusterNight Paris event organised by Red Hat for GlusterFS community members, as part of the Openstack Paris Nov. 2014
This presentation talks about how to use GlusterFS in Openshift to provide Storage for application pods. If you need more details please refer http://humblec.com/persistent-volume-and-persistent-volume-claim-in-openshift-and-kubernetes-using-glusterfs-volume-plugin/
The document discusses Linux User Management (LUM) and various file access protocols and proxy user configurations available in Novell Open Enterprise Server 2, including NCP, CIFS, AFP, FTP, and HTTP. It provides an overview of features in OES2 SP2 and SP3, recommendations for deployment and troubleshooting, and how multiple protocols can be deployed for data integrity and performance.
Log aggregation: using Elasticsearch, Fluentd/Fluentbit and Kibana (EFK)Lee Myring
A quick introduction to log aggregation in a local Docker development environment using Fluentd followed by a demonstration using a publicly available GitHub repo.
The document provides an overview of Container Network Interface (CNI) which is a standard for configuring network interfaces for Linux containers. It discusses how CNI uses plugins to configure networking for containers and provides examples of plugins like ptp, bridge, and IPvlan. It also covers overlay networking techniques like VXLAN and OVN which can be used to provide networking between containers on different hosts.
Securing the Software Supply Chain with TUF and Docker - Justin Cappos and Sa...Docker, Inc.
If you want to compromise millions of machines and users, software distribution and software updates are an excellent attack vector. Using public cryptography to sign your packages is a good starting point, but as we will see, it still leaves you open to a variety of attacks. This is why we designed TUF, a secure software update framework. TUF helps to handle key revocation securely, limits the impact a man-in-the-middle attacker may have, and reduces the impact of repository compromise. We will discuss TUF's protections and integration into Docker's Notary software, and demonstrate new techniques that could be added to verify other parts of the software supply chain, including the development, build, and quality assurance processes.
OSMC 2018 | Tailored SNMP monitoring – Your own SNMP MIB and sub-agent with P...NETWAYS
SNMP continues to be a essential component in monitoring where the information being made available is structured in so-called Management Information (MIB) modules. The standard net-snmp distribution comes a with a variety of standard MIBs implemented by its snmpd, but sometimes there is the need to make your own information available via SNMP. Luckily snmpd can be dynamically extended by so-called subagents implementing the AgentX protocol (RFC2747). The net-snmp API however pretty much focusses on the C programming language only, laying the entrance barrier especially for non-developers rather high. In this talk Pieter will not only demonstrate the creation of a MIB, which, being a text file, is the easier part, but also how easy it is to implement a simple subagent in Python using his python-netsnmpagent module. python-netsnmpagent is a shim wrapper over the net-snmp C API trying to implement just enough abstraction. Licensed under the GPL it is available at https://github.com/pief/python-netsnmpagent as well as PyPI.
The document outlines Gluster's roadmap, including recent improvements to versions 3.5-3.7 like bitrot detection and sharding, and plans for upcoming releases 3.8 and 4.0 such as tiering support, REST APIs, new style replication, and improving the distributed hashing translator to scale to 1000 servers. It also provides an overview of Gluster's architecture and quick start instructions.
NGINX Plus is often deployed in a cluster, and the new features in R16 help our customers working in a clustered environment. New features include global rate limiting, a cluster-aware key-value store, Random with Two Choices load-balancing algorithm, and more.
Join this webinar to learn:
- About the new cluster-aware features in NGINX Plus R16: global rate limiting, key-value store, and Random with Two Choices load balancing
- How to use key-value stores in use cases such as DDoS mitigation and dynamic bandwidth limiting
- About enhanced UDP load balancing, AWS PrivateLink support, and additional new features
- How the NGINX Plus R16 features behave in action, in a live demo
https://www.nginx.com/resources/webinars/whats-new-nginx-plus-r16-emea/
Containerd internals: building a core container runtimeDocker, Inc.
In this talk, we’ll briefly overview of the OpenWhisk serverless (function-as-a-service) framework that initially used the full Docker container engine as the execution vehicle for invoking user functions via containers. After several performance and stability challenges, this project decided to assess the various layers of the Docker engine (containerd and runC) as potential options for the function invoker. Out of that work came an open source project, bucketbench, that can be used to generate benchmarks of container lifecycle operations (e.g., start, stop, kill, remove, pause, unpause) and compare multithreaded operation throughput and stability of each optional engine.
This talk will provide details on the bucketbench project, explain how it has been used to generate performance data for these container runtimes, and shares lessons learned along the way that greatly impact container runtime performance, including bottlenecks in the Linux kernel.
In this talk you’ll learn how you can use bucketbench for your own performance tuning or assessment of container runtimes and how you can collaborate on improvements to the bucketbench project.
Networking-odl and ODL Neutron Northbound are the key components for integrating OpenStack Neutron and OpenDaylight. They are actively developed open source projects. The document encourages giving the integration a try, providing feedback, and contributing to help further the integration of OpenStack and OpenDaylight networking.
This document discusses security considerations for cloud computing instances, networking, and storage. It recommends using bastion servers for CLI access with key-based authentication over secure protocols. For instances, it suggests using LTS OS releases, security patches, and HIDS for critical hosts. Networking advice includes using subnets for applications, public subnets only for public-facing systems, and private subnets with NAT for non-public systems. The document also recommends encrypting sensitive stored data with role-based access controls and logging, and storing authentication materials securely.
OSMC 2018 | Distributed Tracing FAQ by Gianluca ArbezzanoNETWAYS
Microservices, containers and more in general distributed systems have opened a different point of view on our system and applications. We need to understand how a single event or requests cross our app jumping over networks, containers, virtual machines and sometime clod provider. There is a specific practice called distributed tracing to increase observability of systems like that. After this talk, you will have a solid idea around what tracing means, how you can instrument your applications and you will be ready to trace your application across many languages using open source technologies like OpenTracing, OpenCensus, Zipkin, Jaeger, InfluxDB. You will ask yourself how you survived until today!
Couchbase Server is a NoSQL database that allows developers to build applications with agility and scale them to any size. It provides a flexible JSON schema, fast document storage and retrieval using document IDs, and various data access methods like views, global secondary indexes, and the N1QL query language. Couchbase supports many development frameworks and platforms, and can be deployed in various environments including Docker. It provides features like auto-sharding of data across nodes, replication, and cross data center replication for high availability and disaster recovery.
Smart contracts and NFTs call for a revised approach to store data. In these slides, 3 options for distributed and fault-tolerant data storage are presented:
IPFS
Filecoin
Arweave
Control Plane: Security Rationale for Istio (DevSecOps - London Gathering, Ja...Michael Man
Security Rationale For Istio
An introduction to Istio security, looking at how Istio helps to keeps your security team happy by satisfying Kubernetes security requirements for multi-tenancy, and your developers happy by reducing implementation effort. Istio is still an evolving technology, and outstanding issues and impending improvements will be discussed.
NATS & Docker Meetup in Toronto - August 2016
Implementing Microservices with NATS, Diogo Monteiro
-How Aytra uses NATS
-Benefits of using NATS for inter service communication
-Lessons learned adopting NATS
-Overview of Houston NATS library
-Demo of Aytra
You can learn more about NATS at http://www.nats.io
Cortex: Horizontally Scalable, Highly Available PrometheusGrafana Labs
In this talk we present Cortex - a horizontally scalable, highly available Prometheus implementation. Like Prometheus, Cortex is a CNCF (sandbox) project.
Cortex turns a lot of the Prometheus architectural assumptions on its head, by marrying a scale-out PromQL query engine with a storage layer based on NOSQL databases such as Bigtable, DynamoDB and Cassandra. We have disaggregated the Prometheus binary into a microservices-style architecture, with separate services for query, ingest, alerting and recording rules. By designing all these services as fungible replicas, this solution can be scaled out with ease and failure of any individual replica can be dealt with gracefully.
Deep dive into highly available open stack architecture openstack summit va...Arthur Berezin
This document summarizes a presentation on highly available OpenStack architecture. It discusses using Pacemaker and HAProxy for high availability enabling services. Shared databases like MariaDB Galera and message queues like RabbitMQ are made highly available. Individual OpenStack services like Keystone, Glance, Cinder, Nova, Neutron, and Horizon are made highly available through active-active clustering, load balancing, and fencing. The presentation covers topologies for controller, compute, network, and storage nodes. It provides examples of making individual services highly available and discusses ongoing work and future plans to improve high availability in OpenStack.
Load Balancing Apps in Docker Swarm with NGINXNGINX, Inc.
On-demand webinar recording: http://bit.ly/2mRjk2g
Docker and other container technologies continue to gain in popularity. We recently surveyed the broad community of NGINX and NGINX Plus users and found that two-thirds of organizations are either investigating containers, using them in development, or using them in production. Why? Because abstracting your applications from the underlying infrastructure makes developing, distributing, and running software simpler, faster, and more robust than ever before.
But when you move from running your app in a development environment to deploying containers in production, you face new challenges – such as how to effectively run and scale an application across multiple hosts with the performance and uptime that your customers demand.
The latest Docker release, 1.12, supports multihost container orchestration, which simplifies deployment and management of containers across a cluster of Docker hosts. In a complex environment like this, load balancing plays an essential part in delivering your container-based application with reliability and high performance.
Join us in this webinar to learn:
* The basic built-in load balancing options available in Docker Swarm Mode
* The pros and cons of moving to an advanced load balancer like NGINX
* How to integrate NGINX and NGINX Plus with Swarm Mode to provide an advanced load-balancing solution for a cluster with orchestration
* How to scale your Docker-based application with Swarm Mode and NGINX Plus
The document discusses various ways that GlusterFS can integrate with OpenStack components including Swift, Cinder, Manila, and as hyper-converged storage. Current integration includes using GlusterFS as ephemeral storage for Nova instances, as the backend for Swift, Cinder, Glance, and Manila. Work is ongoing to provide a hyper-converged solution with GlusterFS providing both compute and storage on the same nodes.
GlusterFS Cinder integration presented at GlusterNight Paris event @ Openstac...Deepak Shetty
This was a brief presentation talking about the current state of affairs on the GlusterFS Cinder Integration in the GlusterNight Paris event organised by Red Hat for GlusterFS community members, as part of the Openstack Paris Nov. 2014
This presentation talks about how to use GlusterFS in Openshift to provide Storage for application pods. If you need more details please refer http://humblec.com/persistent-volume-and-persistent-volume-claim-in-openshift-and-kubernetes-using-glusterfs-volume-plugin/
The document discusses Linux User Management (LUM) and various file access protocols and proxy user configurations available in Novell Open Enterprise Server 2, including NCP, CIFS, AFP, FTP, and HTTP. It provides an overview of features in OES2 SP2 and SP3, recommendations for deployment and troubleshooting, and how multiple protocols can be deployed for data integrity and performance.
Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications. It groups containers that make up an application into logical units for easy management and discovery called Pods. ReplicaSets ensure that a specified number of pod replicas are running at any given time. Key components include Pods, Services for enabling network access to applications, and Deployments to update Pods and manage releases.
The document outlines Gluster's roadmap, including recent improvements to versions 3.5-3.7 like bitrot detection and sharding, and plans for upcoming releases 3.8 and 4.0 such as tiering support, REST APIs, new style replication, and improving the distributed hashing translator. It also provides an overview of Gluster's architecture and quick start instructions.
The document discusses GlusterFS, an open source distributed file system. It provides an overview of GlusterFS, describes current features in version 3.7 including bitrot support, sharding, and NFS-Ganesha support. It outlines planned features for upcoming releases 3.8 and 4.0 such as tiering support, REST APIs, and improved distributed hashing translator and replication.
You’re ready to make your applications more responsive, scalable, fast and secure. Then it’s time to get started with NGINX. In this webinar, you will learn how to install NGINX from a package or from source onto a Linux host. We’ll then look at some common operating system tunings you could make to ensure your NGINX install is ready for prime time.
View full webinar on demand at http://nginx.com/resources/webinars/installing-tuning-nginx/
Slides from the talk on lessons on running Kafka on Kubernetes by Pavan Keshavamurthy and Avinash Upadhyaya of Platformatory at the Apache Kafka Mumbai July 2023 meetup.
Look at various tooling around running Apache Kafka on Kubernetes and cover best practices for running a distributed system such as Kafka on Kubernetes.
Building a Messaging Solutions for OVHcloud with Apache Pulsar_Pierre ZembStreamNative
OVHcloud is the biggest European cloud provider. From dedicated servers to Managed Kubernetes, from VMware® based Hosted Private Cloud to OpenStack-based Public Cloud, we have over 1.4 million customers worldwide.
Internally, we have been running Apache Kafka for years, and despite all the skills obtained operating multiples clusters with millions of messages per second, we decided to shift and build the foundation of our 'topic-as-a-service' product called ioStream on Apache Pulsar.
In this talk, you will have the insights of why we decided to use Apache Pulsar instead of Apache Kafka as the core of ioStream. We will tell you our journey to use Apache Pulsar, from our deployments to the management, what did work and what did not.
Scenic City Summit (2021): Real-Time Streaming in any and all clouds, hybrid...Timothy Spann
Scenic city summit real-time streaming in any and all clouds, hybrid and beyond
24-September-2021. Scenic City Summit. Virtual. Real-Time Streaming in Any and All Clouds, Hybrid and Beyond
Apache Pulsar, Apache NiFi, Apache Flink
StreamNative
Tim Spann
https://sceniccitysummit.com/
A Primer Towards Running Kafka on Top of Kubernetes.pdfAvinashUpadhyaya3
Slides from the talk on Running Kafka on Kubernetes by Avinash Upadhyaya and Ashwin Venkatesan of Platformatory at the Apache Kafka Bengaluru July 2023 meetup.
This talk will provide an introduction to concerns around running Apache Kafka on top of K8S and the operator pattern. It will cover a comparative view of operators available as well as experiential guidance around operations at scale
AIDevWorld 23 Apache NiFi 101 Introduction and Best Practices
https://sched.co/1RoAO
Timothy Spann, Cloudera, Principal Developer Advocate
In this talk, we will walk step by step through Apache NiFi from the first load to first application. I will include slides, articles and examples to take away as a Quick Start to utilizing Apache NiFi in your real-time dataflows. I will help you get up and running locally on your laptop, Docker or in CDP Public Cloud.
Wednesday November 1, 2023 12:00pm - 12:25pm PDT
VIRTUAL AI DevWorld -- Main Stage https://app.hopin.com/events/api-world-2023-ai-devworld/stages
Retail & E-Commerce AI (Industry AI Conference)
Session Type OPEN TALK
Track or Conference Retail & E-Commerce AI (Industry AI Conference), Industry AI Conference, VIRTUAL, Tensorflow & PyTorch & Open Source Frameworks (AI/ML Engineering Conference), AI/ML Engineering Conference, AI DevWorld
In-Person/Virtual Virtual, Virtual Exclusive
apache nifi
Timothy Spann
Cloudera
Principal Developer Advocate for Data in Motion
Tim Spann is the Principal Developer Advocate for Data in Motion @ Cloudera where he works with Apache Kafka, Apache Flink, Apache NiFi, Apache Iceberg, TensorFlow, Apache Spark, big data, the IoT, machine learning, and deep learning. Tim has over a decade of experience with the IoT, big data, distributed computing, streaming technologies, and Java programming. Previously, he was a Developer Advocate at StreamNative, Principal Field Engineer at Cloudera, a Senior Solutions Architect at AirisData and a senior field engineer at Pivotal. He blogs for DZone, where he is the Big Data Zone leader, and runs a popular meetup in Princeton on big data, the IoT, deep learning, streaming, NiFi, the blockchain, and Spark. Tim is a frequent speaker at conferences such as IoT Fusion, Strata, ApacheCon, Data Works Summit Berlin, DataWorks Summit Sydney, and Oracle Code NYC. He holds a BS and MS in computer science.
cloudera dataflow
Implementing Flux for Scale with Soft Multi-tenancyWeaveworks
Soft multi-tenancy can be hard to achieve and secure. Multiple tenants sharing the same cluster means there are global objects, like Custom Resource Definitions (CRDs), namespaces, and so on, that you don’t want tenants controlling. Platform admins, cluster admins, and tenants, should be separated, with dedicated namespaces, role bindings, node groups, taints and tolerations, etc.
With Flux, tenant isolation is enforced by default, so you don’t have to worry about accidental tenant cross-over / cross-contamination.
In this session, Priyanka “Pinky” Ravi, Developer Experience Engineer at Weaveworks, will walk you through how to set up multi-tenancy on an existing Kubernetes cluster and manage several tenants within the cluster.
Take advantage of the benefits that come with infrastructure as code.
3-2-1 Action! Running OpenStack Shared File System Service in ProductionSean Cohen
As OpenStack’s Shared File System Service is getting more and more adoption as one of top leading emerging projects in OpenStack deployments (according to the last OpenStack foundation user survey), we would like to share some of the key customers use cases such as DevOps, Containers and Enterprise Applications as well review the latest Newton release project updates towards delivering a production-grade deployments.
Slides from OpenStack Summit Barcelona,, October 25, 2016
Session video: https://www.youtube.com/watch?v=F5o-EbESNr8
NGINX Plus is often deployed in a cluster, and the new features in R16 help our customers working in a clustered environment. New features include global rate limiting, a cluster-aware key-value store, Random with Two Choices load-balancing algorithm, and more.
Join this webinar to learn:
- About the new cluster-aware features in NGINX Plus R16: global rate limiting, key-value store, and Random with Two Choices load balancing
- How to use key-value stores in use cases such as DDoS mitigation and dynamic bandwidth limiting
- About enhanced UDP load balancing, AWS PrivateLink support, and additional new features
- How the NGINX Plus R16 features behave in action, in a live demo
https://www.nginx.com/resources/webinars/whats-new-nginx-plus-r16/
Big data analytics and docker the thrilla in manilaDean Hildebrand
This document discusses using Big Data analytics with Docker containers in a university cloud environment. It describes using OpenStack Manila to provide shared file systems across containers via NFS. IBM Spectrum Scale is used as the back-end storage for its high performance, scale-out capabilities. OpenStack Heat orchestrates the deployment of Docker instances, subnets, and data folders upon user requests. Manila shares are mounted within containers to enable big data analytics access to shared data. Challenges in integrating storage with Docker and ensuring proper resource cleanup are also outlined.
This document provides an overview of the CloudStack architecture and its evolution from a developer's perspective. It describes the key components of CloudStack including hosts, primary storage, clusters, pods, networks, secondary storage, and zones. It also outlines the general architecture abstractions used in CloudStack like resource agents, message bus, and asynchronous job execution. Finally, it details some of the core CloudStack subsystems including the compute subsystem and management server deployment architecture.
This document outlines an agenda for a talk on automating infrastructure and security scans using Cloudflare and other tools. It discusses:
1. The goals of automation like reducing administration tasks, improving inventory and scaling/deployment.
2. A overview of configuration management tools like Chef, Puppet, SaltStack and Ansible comparing their architectures, licensing, and support for containers and clouds.
3. Security principles for content delivery networks like Cloudflare focusing on hiding real server IPs.
4. The planned rollout of the Cloudflare recon tool including preparations, building virtual infrastructure, software requirements, and performing/collecting scan results.
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdfPaige Cruz
Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack.
While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack.
I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:
Building RAG with self-deployed Milvus vector database and Snowpark Container...Zilliz
This talk will give hands-on advice on building RAG applications with an open-source Milvus database deployed as a docker container. We will also introduce the integration of Milvus with Snowpark Container Services.
Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...SOFTTECHHUB
The choice of an operating system plays a pivotal role in shaping our computing experience. For decades, Microsoft's Windows has dominated the market, offering a familiar and widely adopted platform for personal and professional use. However, as technological advancements continue to push the boundaries of innovation, alternative operating systems have emerged, challenging the status quo and offering users a fresh perspective on computing.
One such alternative that has garnered significant attention and acclaim is Nitrux Linux 3.5.0, a sleek, powerful, and user-friendly Linux distribution that promises to redefine the way we interact with our devices. With its focus on performance, security, and customization, Nitrux Linux presents a compelling case for those seeking to break free from the constraints of proprietary software and embrace the freedom and flexibility of open-source computing.
Generative AI Deep Dive: Advancing from Proof of Concept to ProductionAggregage
Join Maher Hanafi, VP of Engineering at Betterworks, in this new session where he'll share a practical framework to transform Gen AI prototypes into impactful products! He'll delve into the complexities of data collection and management, model selection and optimization, and ensuring security, scalability, and responsible use.
UiPath Test Automation using UiPath Test Suite series, part 5DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 5. In this session, we will cover CI/CD with devops.
Topics covered:
CI/CD with in UiPath
End-to-end overview of CI/CD pipeline with Azure devops
Speaker:
Lyndsey Byblow, Test Suite Sales Engineer @ UiPath, Inc.
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...Neo4j
Leonard Jayamohan, Partner & Generative AI Lead, Deloitte
This keynote will reveal how Deloitte leverages Neo4j’s graph power for groundbreaking digital twin solutions, achieving a staggering 100x performance boost. Discover the essential role knowledge graphs play in successful generative AI implementations. Plus, get an exclusive look at an innovative Neo4j + Generative AI solution Deloitte is developing in-house.
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slackshyamraj55
Discover the seamless integration of RPA (Robotic Process Automation), COMPOSER, and APM with AWS IDP enhanced with Slack notifications. Explore how these technologies converge to streamline workflows, optimize performance, and ensure secure access, all while leveraging the power of AWS IDP and real-time communication via Slack notifications.
In his public lecture, Christian Timmerer provides insights into the fascinating history of video streaming, starting from its humble beginnings before YouTube to the groundbreaking technologies that now dominate platforms like Netflix and ORF ON. Timmerer also presents provocative contributions of his own that have significantly influenced the industry. He concludes by looking at future challenges and invites the audience to join in a discussion.
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
Introducing Milvus Lite: Easy-to-Install, Easy-to-Use vector database for you...Zilliz
Join us to introduce Milvus Lite, a vector database that can run on notebooks and laptops, share the same API with Milvus, and integrate with every popular GenAI framework. This webinar is perfect for developers seeking easy-to-use, well-integrated vector databases for their GenAI apps.
Full-RAG: A modern architecture for hyper-personalizationZilliz
Mike Del Balso, CEO & Co-Founder at Tecton, presents "Full RAG," a novel approach to AI recommendation systems, aiming to push beyond the limitations of traditional models through a deep integration of contextual insights and real-time data, leveraging the Retrieval-Augmented Generation architecture. This talk will outline Full RAG's potential to significantly enhance personalization, address engineering challenges such as data management and model training, and introduce data enrichment with reranking as a key solution. Attendees will gain crucial insights into the importance of hyperpersonalization in AI, the capabilities of Full RAG for advanced personalization, and strategies for managing complex data integrations for deploying cutting-edge AI solutions.
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!SOFTTECHHUB
As the digital landscape continually evolves, operating systems play a critical role in shaping user experiences and productivity. The launch of Nitrux Linux 3.5.0 marks a significant milestone, offering a robust alternative to traditional systems such as Windows 11. This article delves into the essence of Nitrux Linux 3.5.0, exploring its unique features, advantages, and how it stands as a compelling choice for both casual users and tech enthusiasts.
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!
Certificate based access type in openstack Manila @ openstack paris nov. 2014
1. Certificate based access type in Manila
vbrownbag-techtalk@openstack-paris
Deepak C Shetty
deepakcs@redhat.com
IRC: deepakcs
2. What is Manila
● Shared FileSystem as a service
● Incubated since openstack Juno
● Provision file shares to Nova (openstack compute) instance(s)
● Strives to provide an API for management of shared filesystems
with support for multiple protocols and backend implementations
– NFS and CIFS primarily supported
– Other protocols are encouraged too (eg: glusterfs)
Openstack Paris Summit 2
● Supports Multi-tenancy
– Enables public cloud usecase
– Has framework to support storage backends that don't support
multi-tenancy natively
4. Manila access types
Openstack Paris Summit 4
● IP
– Access control using IP address
– Takes IP as an argument
– Typically used in controlling access to NFS shares
● User
– Access control using user name
– Takes user name as argument
– Typically used in controlling access to CIFS shares
● Cert
– Access control using SSL certificates
– Takes SSL Certificate's CN (common name) as argument
– Certificate setup (aka trust setup) between client and server is out of band
– Currently implemented by GlusterFS native driver ('glusterfs' protocol)
5. GlusterFS Native Driver
● Supports Certificate based access type of Manila
● Provision shares that use the 'glusterfs' protocol
● Instances directly talk with GlusterFS storage backend
Openstack Paris Summit 5
– No service VM needed
● Secure access
– Only tenants with the right certificate will be able to access the share
● Multi-tenant
– Separation using tenant specific certificates
● Supports certificate chaining and cipher lists
6. GlusterFS Native Driver contd.
Openstack Paris Summit 6
● Available upstream
– 1 Manila share == 1 GlusterFS volume
– Pre-requisites
● GlusterFS volume(s) setup with Cert based access enabled
● Instance should have server signed client certificates pre-loaded
● Manila.conf – Provide list of glusterfs volume(s) to work with
● TODOs
– Add documentation
– Snapshot support
– Dynamic creation of glusterfs volumes
– Data shredding as part of gluster volume delete
– Create share from snapshot
7. GlusterFS Native Driver contd.
Openstack Paris Summit 6
● Available upstream
– 1 Manila share == 1 GlusterFS volume
– Pre-requisites
● GlusterFS volume(s) setup with Cert based access enabled
● Instance should have server signed client certificates pre-loaded
● Manila.conf – Provide list of glusterfs volume(s) to work with
● TODOs
– Add documentation
– Snapshot support
– Dynamic creation of glusterfs volumes
– Data shredding as part of gluster volume delete
– Create share from snapshot