This document discusses using the open-source virtualization platform Ovirt together with the distributed storage platform Ceph. It evaluates three options for combining Ovirt and Ceph that provide high availability and scalability without single points of failure. The recommended option uses CephFS to provide POSIX-compliant storage for virtual machines hosted on an Ovirt cluster, along with GlusterFS for the Ovirt hosted engine and Ceph monitor nodes. This allows up to seven hosts without any single points of failure.
Improving Hadoop Cluster Performance via Linux ConfigurationAlex Moundalexis
Â
Administering a Hadoop cluster isn't easy. Many Hadoop clusters suffer from Linux configuration problems that can negatively impact performance. With vast and sometimes confusing config/tuning options, it can can tempting (and scary) for a cluster administrator to make changes to Hadoop when cluster performance isn't as expected. Learn how to improve Hadoop cluster performance and eliminate common problem areas, applicable across use cases, using a handful of simple Linux configuration changes.
While Go is the language-of-choice in the cloud-native world, Python has a huge community and makes it really easy to extend Kubernetes in only a few lines of code.
This talk shows examples on how to use Python to query the Kubernetes API, how to write simple controllers in only 10 lines of Python, how to build complete web UIs, and how to test everything with py.test and Kind.
Some of the open-source projects which will be covered: pykube-ng, Kubernetes Web View, kube-janitor, and Kopf (Kubernetes Operator Pythonic Framework).
Talk held in Prague on 2019-09-05:
https://www.meetup.com/Cloud-Native-Prague/events/263802447/
In this PowerPoint, learn how a security policy can be your first line of defense. Servers running AIX and other operating systems are frequent targets of cyberattacks, according to the Data Breach Investigations Report. From DoS attacks to malware, attackers have a variety of strategies at their disposal. Having a security policy in place makes it easier to ensure you have appropriate controls in place to protect mission-critical data.
Improving Hadoop Cluster Performance via Linux ConfigurationAlex Moundalexis
Â
Administering a Hadoop cluster isn't easy. Many Hadoop clusters suffer from Linux configuration problems that can negatively impact performance. With vast and sometimes confusing config/tuning options, it can can tempting (and scary) for a cluster administrator to make changes to Hadoop when cluster performance isn't as expected. Learn how to improve Hadoop cluster performance and eliminate common problem areas, applicable across use cases, using a handful of simple Linux configuration changes.
While Go is the language-of-choice in the cloud-native world, Python has a huge community and makes it really easy to extend Kubernetes in only a few lines of code.
This talk shows examples on how to use Python to query the Kubernetes API, how to write simple controllers in only 10 lines of Python, how to build complete web UIs, and how to test everything with py.test and Kind.
Some of the open-source projects which will be covered: pykube-ng, Kubernetes Web View, kube-janitor, and Kopf (Kubernetes Operator Pythonic Framework).
Talk held in Prague on 2019-09-05:
https://www.meetup.com/Cloud-Native-Prague/events/263802447/
In this PowerPoint, learn how a security policy can be your first line of defense. Servers running AIX and other operating systems are frequent targets of cyberattacks, according to the Data Breach Investigations Report. From DoS attacks to malware, attackers have a variety of strategies at their disposal. Having a security policy in place makes it easier to ensure you have appropriate controls in place to protect mission-critical data.
Overview of HBase cluster replication feature, covering implementation details as well as monitoring tools and tips for troubleshooting and support of Replication deployments.
Broken benchmarks, misleading metrics, and terrible tools. This talk will help you navigate the treacherous waters of Linux performance tools, touring common problems with system tools, metrics, statistics, visualizations, measurement overhead, and benchmarks. You might discover that tools you have been using for years, are in fact, misleading, dangerous, or broken.
The speaker, Brendan Gregg, has given many talks on tools that work, including giving the Linux PerformanceTools talk originally at SCALE. This is an anti-version of that talk, to focus on broken tools and metrics instead of the working ones. Metrics can be misleading, and counters can be counter-intuitive! This talk will include advice for verifying new performance tools, understanding how they work, and using them successfully.
It goes without saying that DNS is only as secure as its servers. To ensure the successful and secure operation of a DNS server, secure configuration is paramount.
The new BIND 9 version 9.11 is a major version of the popular DNS server, released in August by ISC.
In this webinar Mr. Carsten Strotmann will demonstrate new features such as:
- Catalog Zones,
- dnssec-keymgr, new *rndc* functions
- CDS/CDNSKEY auto generation
- Negative Trust Anchor
- DNS cookies
-Refuse âanyâ
-and more.
Real Time Analytics for Big Data a Twitter Case StudyNati Shalom
Â
Hadoop's batch-oriented processing is sufficient for many use cases, especially where the frequency of data reporting doesn't need to be up-to-the-minute. However, batch processing isn't always adequate, particularly when serving online needs such as mobile and web clients, or markets with real-time changing conditions such as finance and advertising.
In the same way that Hadoop was born out of large-scale web applications, a new class of scalable frameworks and platforms for handling real time streaming processing or real time analysis is born to handle the needs of large-scale location-aware mobile, social and sensor use.
Facebook, Twitter and Google have been pioneers in that arena and recently launched new analytics services designed to meet the real time needs.
In this session we will review the common patterns and architectures that drive these platforms and learn how to build a Twitter-like analytics system in a simple way using frameworks such as Spring Social, Active In-Memory Data Grid for Big Data event processing, and NoSQL database such as Cassandra or HBase for handling the managing the historical data.
Participants in this session will also receive a hands-on tutorial for trying out these patterns on their own environment.
A detailed post covering the topic including a reference to a code example illustrating the reference architecture is available below:
http://horovits.wordpress.com/2012/01/27/analytics-for-big-data-venturing-with-the-twitter-use-case/
Hadoop Meetup Jan 2019 - Dynamometer and a Case Study in NameNode GCErik Krogen
Â
Erik Krogen of LinkedIn presents regarding Dynamometer, a system open sourced by LinkedIn for scale- and performance-testing HDFS. He discusses one major use case for Dynamometer, tuning NameNode GC, and discusses characteristics of NameNode GC such as why it is important, and how it interacts with various current and future GC algorithms.
This is taken from the Apache Hadoop Contributors Meetup on January 30, hosted by LinkedIn in Mountain View.
The demand for managing a large amount of data in a scalable yet reliable and cost-effective way has became more and more relevant in this day and age. Ceph, a software-defined storage, provides an original solution for this problem and guarantees a resilient and self-healing way for managing large amount of data up to the Exabyte level. In this session I will talk about a new feature introduced in oVirt 3.6 which provides the ability to integrate with Red Hat Ceph storage using Cinder, a storage service used mainly for OpenStack. This integration reveals new opportunities and tools for storage management in a scalable and virtualized way and also opens the door for interesting future integrations with other storage providers.
In this session I will describe how oVirt, an open source virtualization management platform, has extended and elevated its storage virtualization management capabilities by integrating with Cinder, a storage service, to manage resources from the Ceph Storage. oVirt 3.6 revolutionize the way it manages virtualized storage to be much more scalable and flexible, and opens the door for future integrations with well known storage providers such as NetApp, EMC, HP and more.
Overview of HBase cluster replication feature, covering implementation details as well as monitoring tools and tips for troubleshooting and support of Replication deployments.
Broken benchmarks, misleading metrics, and terrible tools. This talk will help you navigate the treacherous waters of Linux performance tools, touring common problems with system tools, metrics, statistics, visualizations, measurement overhead, and benchmarks. You might discover that tools you have been using for years, are in fact, misleading, dangerous, or broken.
The speaker, Brendan Gregg, has given many talks on tools that work, including giving the Linux PerformanceTools talk originally at SCALE. This is an anti-version of that talk, to focus on broken tools and metrics instead of the working ones. Metrics can be misleading, and counters can be counter-intuitive! This talk will include advice for verifying new performance tools, understanding how they work, and using them successfully.
It goes without saying that DNS is only as secure as its servers. To ensure the successful and secure operation of a DNS server, secure configuration is paramount.
The new BIND 9 version 9.11 is a major version of the popular DNS server, released in August by ISC.
In this webinar Mr. Carsten Strotmann will demonstrate new features such as:
- Catalog Zones,
- dnssec-keymgr, new *rndc* functions
- CDS/CDNSKEY auto generation
- Negative Trust Anchor
- DNS cookies
-Refuse âanyâ
-and more.
Real Time Analytics for Big Data a Twitter Case StudyNati Shalom
Â
Hadoop's batch-oriented processing is sufficient for many use cases, especially where the frequency of data reporting doesn't need to be up-to-the-minute. However, batch processing isn't always adequate, particularly when serving online needs such as mobile and web clients, or markets with real-time changing conditions such as finance and advertising.
In the same way that Hadoop was born out of large-scale web applications, a new class of scalable frameworks and platforms for handling real time streaming processing or real time analysis is born to handle the needs of large-scale location-aware mobile, social and sensor use.
Facebook, Twitter and Google have been pioneers in that arena and recently launched new analytics services designed to meet the real time needs.
In this session we will review the common patterns and architectures that drive these platforms and learn how to build a Twitter-like analytics system in a simple way using frameworks such as Spring Social, Active In-Memory Data Grid for Big Data event processing, and NoSQL database such as Cassandra or HBase for handling the managing the historical data.
Participants in this session will also receive a hands-on tutorial for trying out these patterns on their own environment.
A detailed post covering the topic including a reference to a code example illustrating the reference architecture is available below:
http://horovits.wordpress.com/2012/01/27/analytics-for-big-data-venturing-with-the-twitter-use-case/
Hadoop Meetup Jan 2019 - Dynamometer and a Case Study in NameNode GCErik Krogen
Â
Erik Krogen of LinkedIn presents regarding Dynamometer, a system open sourced by LinkedIn for scale- and performance-testing HDFS. He discusses one major use case for Dynamometer, tuning NameNode GC, and discusses characteristics of NameNode GC such as why it is important, and how it interacts with various current and future GC algorithms.
This is taken from the Apache Hadoop Contributors Meetup on January 30, hosted by LinkedIn in Mountain View.
The demand for managing a large amount of data in a scalable yet reliable and cost-effective way has became more and more relevant in this day and age. Ceph, a software-defined storage, provides an original solution for this problem and guarantees a resilient and self-healing way for managing large amount of data up to the Exabyte level. In this session I will talk about a new feature introduced in oVirt 3.6 which provides the ability to integrate with Red Hat Ceph storage using Cinder, a storage service used mainly for OpenStack. This integration reveals new opportunities and tools for storage management in a scalable and virtualized way and also opens the door for interesting future integrations with other storage providers.
In this session I will describe how oVirt, an open source virtualization management platform, has extended and elevated its storage virtualization management capabilities by integrating with Cinder, a storage service, to manage resources from the Ceph Storage. oVirt 3.6 revolutionize the way it manages virtualized storage to be much more scalable and flexible, and opens the door for future integrations with well known storage providers such as NetApp, EMC, HP and more.
I invite you to come and listen to my presentation about how Openstack and Gluster are integrating together in both Cinder and Swift.
I will give a brief description about Openstack storage components (Cinder, Swift and Glance) , followed by an intro to Gluster, and then present the integration points and some preferred topology and configuration between gluster and openstack.
Kubernetes is awesome! But what does it takes for a Java developer to design, implement and run Cloud Native applications? In this session, we will look at Kubernetes from a user point of view and demonstrate how to consume it effectively. We will discover which concerns Kubernetes addresses and how it helps to develop highly scalable and resilient Java applications.
FOSDEM TALK: https://fosdem.org/2017/schedule/event/cnjavadev/
CoreOS automated MySQL Cluster Failover using Galera ClusterYazz Atlas
Â
CoreOS Fleet and Etcd provide a simple and eloquent framework for application clusters to both auto-configure and recover from node failure. Galera Cluster is a multi-master, open solution for clustering MySQL. Mix the two, sprinkle in a bit of âglueâ and you have a Docker based MySQL cluster that will react automatically to container failure. This presentation will cover the nuts and bolts of automating a Galera Cluster, built from Docker Images and deployed in a distributed fashion using etcd, confd, and fleet for both initial and failure recovery configuration.
[Pass the SALT 2020] Understand password policy in OpenLDAP and discover tool...Worteks
Â
We introduce the password policy standard (IETF Behera draft) and its implementation in OpenLDAP. We then present LDAP Tool Box Service Desk, a simple web application do display user account status and providing functions to check and reset password, lock/unlock account.
LemonLDAP::NG is a well known WebSSO software. The 2.0 version was released in 2018 and brings a lot of new features, like multi-factor authentication (TOTP, U2F, ...), WebService and API protection, Plugin system...
FusionIAM is based on the following softwares:
* OpenLDAP
* Fusion Directory
* LemonLDAP::NG
* LDAP Tool Box
* LSC
It is hosted by OW2: https://gitlab.ow2.org/fusioniam/fusioniam
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Â
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
Â
As AI technology is pushing into IT I was wondering myself, as an âinfrastructure container kubernetes guyâ, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefitâs both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
Â
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Â
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Â
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as âpredictable inferenceâ.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Â
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
Â
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Generating a custom Ruby SDK for your web service or Rails API using Smithyg2nightmarescribd
Â
Have you ever wanted a Ruby client API to communicate with your web service? Smithy is a protocol-agnostic language for defining services and SDKs. Smithy Ruby is an implementation of Smithy that generates a Ruby SDK using a Smithy model. In this talk, we will explore Smithy and Smithy Ruby to learn how to generate custom feature-rich SDKs that can communicate with any web service, such as a Rails JSON API.
2. 2
(v .t ks)ÉÊ É
Services
Heterogeneous and complex
infrastructures, cloud, mail,
authentication, security
î
Studies, audit and consulting
î
Technical expertise
î
Technical support
î
Training
î
R&D
Edition
Collaboration and
application portal
Mutualized platform
for development
Identity and Access
Management
Partnership
6. 6
Ovirt - What is it ?
â
Virtualization management platform
â
Based on KVM and Libvirt
â
Apache License 2.0
â
Version 4.3.6
â
Main Contributor : Redhat
9. 9
Ceph - What is it ?
â
Storage Platform
â
Different usages :
â Block Storage
â Object Storage
â FileSystem
â
High Scalability
â
No SPOF
â
Version Mimic
12. 12
Ovirt â Default Storage Options
â
You can use :
â NFS
â ISCSI
â GlusterFS
â POSIX Compliant
â
Hyperconverged Mode :
â Based on glusterFS
13. 13
Ovirt â Why not use default options ?
â
NFS :
â Really Simple
â
But :
â Huge SPOF
â Donât Use NFS 4.1 : pNFS
14. 14
Ovirt â Why not use default options ?
â
ISCSI :
â High Availability
â Multi Path
â
But :
â No SAN
15. 15
Ovirt â Default Storage Options
â
GlusterFS :
â High Availability
â Block Storage
â FileSystem
â
But :
â Configuration Sync
â Limit :
â
Max Host
â
Max Volume
16. 16
Ovirt â Default Storage Options
â
Hyperconverged Mode :
â Based on glusterFS âŠ
â Max 12 Hosts
17. 17
Ovirt â Default Storage Options
â
Best Option :
â POSIX Compliant ??
â
After the deployment :
â Use external Cinder :
â
Use Cinder in Openstack
â
Use Cinder Standalone
20. 20
Option : 1
â
Use GlusterFS
â Ovirt Hosted Engine
â Ceph MON
â Cinder Standalone
â
Use Cinder with backend Ceph RBD
â For all other VMs
â
Number of Host : 11
â
Number of SPOF : 1
21. 21
Option : 2
â
Use GlusterFS
â Ovirt Hosted Engine
â Ceph MON
â
Use POSIX Compliant with CephFS
â For all other Vms
â
Number of Host : 10
â
Number of SPOF : 0
22. 22
Option : 3
â
Use Ceph ISCSI
â Ovirt Hosted Engine
â
Use CephFS
â For all other VMs
â
Number of Host : 12
â
Number of SPOF : 0
23. 23
My opinion
â
Option 2
â
Hyperconverged :
â
Ovirt
â
GlusterFS
â
Ceph OS
â
GlusterFS use :
â
Hosted Engine
â
3 Ceph MON
â
Number of Host : 7
â
Number of SPOF : 0
28. 28
Update
â
Ovirt :
â Minor update without Downtime
â Update 4.2 to 4.3 : no issue
â
Ceph :
â Minor update without Downtime
â Update Mimic to Nautillus : In progress