An Operator is an application that encodes the domain knowledge of the application and extends the Kubernetes API through custom resources. They enable users to create, configure, and manage their applications. Operators have been around for a while now, and that has allowed for patterns and best practices to be developed.
In this talk, Lili will explain what operators are in the context of Kubernetes and present the different tools out there to create and maintain operators over time. She will end by demoing the building of an operator from scratch, and also using the helper tools available out there.
Disaster Recovery Options Running Apache Kafka in Kubernetes with Rema Subra...HostedbyConfluent
Active-Active, Active-Passive, and stretch clusters are hallmark patterns that have been the gold standard in Apache Kafka® disaster recovery architectures for years. Moving to Kubernetes requires unpacking these patterns and choosing a configuration that allows you to meet the same RTO and RPO requirements.
In this talk, we will cover how Active-Active/Active-Passive modes for disaster recovery have worked in the past and how the architecture evolves with deploying Apache Kafka on Kubernetes. We'll also look at how stretch clusters sitting on this architecture give a disaster recovery solution that's built-in!
Armed with this information, you will be able to architect your new Apache Kafka Kubernetes deployment (or retool your existing one) to achieve the resilience you require.
Kubernetes Webinar - Using ConfigMaps & Secrets Janakiram MSV
Many applications require configuration using some combination of configuration files, command line arguments, and environment variables. ConfigMaps in Kubernetes provide mechanisms to inject containers with configuration data while keeping them portable. Secrets decouple sensitive content from the pods using a volume plug-in. This webinar will discuss the use cases and scenarios for using ConfigMaps and Secrets.
Timelines at Scale (Raffi Krikorian - VP of Engineering at Twitter)Chris Bolman
Presentation by Raffi Krikorian, VP of Engineering at Twitter, on scaling Twitter to over 150 million active users with redis and other architectural approaches
데브시스터즈의 Cookie Run: OvenBreak 에 적용된 Kubernetes 기반 다중 개발 서버 환경 구축 시스템에 대한 발표입니다.
Container orchestration 기반 개발 환경 구축 시스템의 필요성과, 왜 Kubernetes를 선택했는지, Kubernetes의 개념과 유용한 기능들을 다룹니다. 아울러 구축한 시스템에 대한 데모와, 작업했던 항목들에 대해 리뷰합니다.
*NDC17 발표에서는 데모 동영상을 사용했으나, 슬라이드 캡쳐로 대신합니다.
A Comprehensive Introduction to Kubernetes. This slide deck serves as the lecture portion of a full-day Workshop covering the architecture, concepts and components of Kubernetes. For the interactive portion, please see the tutorials here:
https://github.com/mrbobbytables/k8s-intro-tutorials
An Operator is an application that encodes the domain knowledge of the application and extends the Kubernetes API through custom resources. They enable users to create, configure, and manage their applications. Operators have been around for a while now, and that has allowed for patterns and best practices to be developed.
In this talk, Lili will explain what operators are in the context of Kubernetes and present the different tools out there to create and maintain operators over time. She will end by demoing the building of an operator from scratch, and also using the helper tools available out there.
Disaster Recovery Options Running Apache Kafka in Kubernetes with Rema Subra...HostedbyConfluent
Active-Active, Active-Passive, and stretch clusters are hallmark patterns that have been the gold standard in Apache Kafka® disaster recovery architectures for years. Moving to Kubernetes requires unpacking these patterns and choosing a configuration that allows you to meet the same RTO and RPO requirements.
In this talk, we will cover how Active-Active/Active-Passive modes for disaster recovery have worked in the past and how the architecture evolves with deploying Apache Kafka on Kubernetes. We'll also look at how stretch clusters sitting on this architecture give a disaster recovery solution that's built-in!
Armed with this information, you will be able to architect your new Apache Kafka Kubernetes deployment (or retool your existing one) to achieve the resilience you require.
Kubernetes Webinar - Using ConfigMaps & Secrets Janakiram MSV
Many applications require configuration using some combination of configuration files, command line arguments, and environment variables. ConfigMaps in Kubernetes provide mechanisms to inject containers with configuration data while keeping them portable. Secrets decouple sensitive content from the pods using a volume plug-in. This webinar will discuss the use cases and scenarios for using ConfigMaps and Secrets.
Timelines at Scale (Raffi Krikorian - VP of Engineering at Twitter)Chris Bolman
Presentation by Raffi Krikorian, VP of Engineering at Twitter, on scaling Twitter to over 150 million active users with redis and other architectural approaches
데브시스터즈의 Cookie Run: OvenBreak 에 적용된 Kubernetes 기반 다중 개발 서버 환경 구축 시스템에 대한 발표입니다.
Container orchestration 기반 개발 환경 구축 시스템의 필요성과, 왜 Kubernetes를 선택했는지, Kubernetes의 개념과 유용한 기능들을 다룹니다. 아울러 구축한 시스템에 대한 데모와, 작업했던 항목들에 대해 리뷰합니다.
*NDC17 발표에서는 데모 동영상을 사용했으나, 슬라이드 캡쳐로 대신합니다.
A Comprehensive Introduction to Kubernetes. This slide deck serves as the lecture portion of a full-day Workshop covering the architecture, concepts and components of Kubernetes. For the interactive portion, please see the tutorials here:
https://github.com/mrbobbytables/k8s-intro-tutorials
K8s in 3h - Kubernetes Fundamentals TrainingPiotr Perzyna
Kubernetes (K8s) is an open-source system for automating deployment, scaling, and management of containerized applications. This training helps you understand key concepts within 3 hours.
Optimizing Kubernetes Resource Requests/Limits for Cost-Efficiency and Latenc...Henning Jacobs
Kubernetes has the concept of resource requests and limits. Pods get scheduled on the nodes based on their requests and optionally limited in how much of the resource they can consume. Understanding and optimizing resource requests/limits is crucial both for reducing resource "slack" and ensuring application performance/low-latency. This talk shows our approach to monitoring and optimizing Kubernetes resources for 80+ clusters to achieve cost-efficiency and reducing impact for latency-critical applications. All shown tools are Open Source and can be applied to most Kubernetes deployments.
Running Containers at Scale at Netflix. An update on the usage of containers at Netflix. Technical discussions on new features and concepts we've added across container scheduling and execution.
Extending kubernetes with CustomResourceDefinitionsStefan Schimanski
The Kubernetes API provides a number of proven patterns to build distributed systems. More and more 3rd-party components are built on-top of Kubernetes and these patterns, providing their own resources stored in the cluster. In this presentation we will discuss CustomResourcesDefinitions and how they can extend the Kubernetes API in a quasi-native way. We look at the features, limits and their future.
Cloud Native ClickHouse at Scale--Using the Altinity Kubernetes Operator-2022...Altinity Ltd
Over the last few years Kubernetes has transitioned from an object of curiosity and fear to a robust platform for big data. Watch this webinar and you will learn how the Altinity Kubernetes Operator for ClickHouse enables users to run high performance analytics on ClickHouse. You will see a simple installation and teach you how to scale it into a cluster that can analyze 100s of terabytes of data. Along the way we’ll share our lessons for ClickHouse on Kubernetes in Altinity.Cloud. We built it on Kubernetes using the Altinity Operator and now run hundreds of clusters in the cloud. You can too!
Scaling your Data Pipelines with Apache Spark on KubernetesDatabricks
There is no doubt Kubernetes has emerged as the next generation of cloud native infrastructure to support a wide variety of distributed workloads. Apache Spark has evolved to run both Machine Learning and large scale analytics workloads. There is growing interest in running Apache Spark natively on Kubernetes. By combining the flexibility of Kubernetes and scalable data processing with Apache Spark, you can run any data and machine pipelines on this infrastructure while effectively utilizing resources at disposal.
In this talk, Rajesh Thallam and Sougata Biswas will share how to effectively run your Apache Spark applications on Google Kubernetes Engine (GKE) and Google Cloud Dataproc, orchestrate the data and machine learning pipelines with managed Apache Airflow on GKE (Google Cloud Composer). Following topics will be covered: – Understanding key traits of Apache Spark on Kubernetes- Things to know when running Apache Spark on Kubernetes such as autoscaling- Demonstrate running analytics pipelines on Apache Spark orchestrated with Apache Airflow on Kubernetes cluster.
This is a talk on how you can monitor your microservices architecture using Prometheus and Grafana. This has easy to execute steps to get a local monitoring stack running on your local machine using docker.
Kubernetes Architecture - beyond a black box - Part 1Hao H. Zhang
This is part 1 of my Kubernetes architecture deep-dive slide series.
I have been working with Kubernetes for more than a year, from v1.3.6 to v1.6.7, and I am a CNCF certified Kubernetes administrator. Before I move on to something else, I would like to summarize and share my knowledges and take-aways about Kubernetes, from a software engineer perspective.
This set of slides is a humble dig into one level below your running application in production, revealing how different components of Kubernetes work together to orchestrate containers and present your applications to the rest of the world.
The slides contains 80+ external links to Kubernetes documentations, blog posts, Github issues, discussions, design proposals, pull requests, papers, source code files I went through when I was working with Kubernetes - which I think are valuable for people to understand how Kubernetes works, Kubernetes design philosophies and why these design came into places.
Helm - Application deployment management for KubernetesAlexei Ledenev
Use Helm to package and deploy a composed application to any Kubernetes cluster. Manage your releases easily over time and across multiple K8s clusters.
Presented at GDG Devfest Ukraine 2018.
Prometheus has become the defacto monitoring system for cloud native applications, with systems like Kubernetes and Etcd natively exposing Prometheus metrics. In this talk Tom will explore all the moving part for a working Prometheus-on-Kubernetes monitoring system, including kube-state-metrics, node-exporter, cAdvisor and Grafana. You will learn about the various methods for getting to a working setup: the manual approach, using CoreOS’s Prometheus Operator, or using Prometheus Ksonnet Mixin. Tom will also share some little tips and tricks for getting the most out of your Prometheus monitoring, including the common pitfalls and what you should be alerting on.
Slide deck from my "OpenStack and MySQL" presentation at Oracle OpenWorld 2015:
"This session details exactly how MySQL fits in throughout OpenStack, takes a deeper look at the database-as-a-service (DBaaS) offering with OpenStack Trove with MySQL, and discusses how Oracle supports this thriving ecosystem."
Apache Spark on K8S Best Practice and Performance in the CloudDatabricks
Kubernetes As of Spark 2.3, Spark can run on clusters managed by Kubernetes. we will describes the best practices about running Spark SQL on Kubernetes upon Tencent cloud includes how to deploy Kubernetes against public cloud platform to maximum resource utilization and how to tune configurations of Spark to take advantage of Kubernetes resource manager to achieve best performance. To evaluate performance, the TPC-DS benchmarking tool will be used to analysis performance impact of queries between configurations set.
Speakers: Junjie Chen, Junping Du
Traditional virtualization technologies have been used by cloud infrastructure providers for many years in providing isolated environments for hosting applications. These technologies make use of full-blown operating system images for creating virtual machines (VMs). According to this architecture, each VM needs its own guest operating system to run application processes. More recently, with the introduction of the Docker project, the Linux Container (LXC) virtualization technology became popular and attracted the attention. Unlike VMs, containers do not need a dedicated guest operating system for providing OS-level isolation, rather they can provide the same level of isolation on top of a single operating system instance.
An enterprise application may need to run a server cluster to handle high request volumes. Running an entire server cluster on Docker containers, on a single Docker host could introduce the risk of single point of failure. Google started a project called Kubernetes to solve this problem. Kubernetes provides a cluster of Docker hosts for managing Docker containers in a clustered environment. It provides an API on top of Docker API for managing docker containers on multiple Docker hosts with many more features.
Rook turns distributed storage systems into self-managing, self-scaling, self-healing storage services. It automates the tasks of a storage administrator: deployment, bootstrapping, configuration, provisioning, scaling, upgrading, migration, disaster recovery, monitoring, and resource management.
Rook uses the power of the Kubernetes platform to deliver its services via a Kubernetes Operator for each storage provider.
Oleg Chunikhin, Co-Founder and CTO @ Kublr.com, will present an introduction to storage management on k8s using Rook and Ceph.
20180503 kube con eu kubernetes metrics deep diveBob Cotton
Kubernetes generates a wealth of metrics. Some explicitly within the Kubernetes API server, the Kublet, and cAdvisor or implicitly by observing events such as the kube-state-metrics project. A subset of these metrics are used within Kubernetes itself to make scheduling decisions, however, other metrics can be used to determine the overall health of the system or for capacity planning purposes.
Kubernetes exposes metrics from several places, some available internally, others through add-on projects. In this session you will learn about:
- Node level metrics, as exposed from the node_exporter
- Kublet metrics
- API server metrics
- etcd metrics
- cAdvisor metrics
- Metrics exposed from kube-state-metrics
K8s in 3h - Kubernetes Fundamentals TrainingPiotr Perzyna
Kubernetes (K8s) is an open-source system for automating deployment, scaling, and management of containerized applications. This training helps you understand key concepts within 3 hours.
Optimizing Kubernetes Resource Requests/Limits for Cost-Efficiency and Latenc...Henning Jacobs
Kubernetes has the concept of resource requests and limits. Pods get scheduled on the nodes based on their requests and optionally limited in how much of the resource they can consume. Understanding and optimizing resource requests/limits is crucial both for reducing resource "slack" and ensuring application performance/low-latency. This talk shows our approach to monitoring and optimizing Kubernetes resources for 80+ clusters to achieve cost-efficiency and reducing impact for latency-critical applications. All shown tools are Open Source and can be applied to most Kubernetes deployments.
Running Containers at Scale at Netflix. An update on the usage of containers at Netflix. Technical discussions on new features and concepts we've added across container scheduling and execution.
Extending kubernetes with CustomResourceDefinitionsStefan Schimanski
The Kubernetes API provides a number of proven patterns to build distributed systems. More and more 3rd-party components are built on-top of Kubernetes and these patterns, providing their own resources stored in the cluster. In this presentation we will discuss CustomResourcesDefinitions and how they can extend the Kubernetes API in a quasi-native way. We look at the features, limits and their future.
Cloud Native ClickHouse at Scale--Using the Altinity Kubernetes Operator-2022...Altinity Ltd
Over the last few years Kubernetes has transitioned from an object of curiosity and fear to a robust platform for big data. Watch this webinar and you will learn how the Altinity Kubernetes Operator for ClickHouse enables users to run high performance analytics on ClickHouse. You will see a simple installation and teach you how to scale it into a cluster that can analyze 100s of terabytes of data. Along the way we’ll share our lessons for ClickHouse on Kubernetes in Altinity.Cloud. We built it on Kubernetes using the Altinity Operator and now run hundreds of clusters in the cloud. You can too!
Scaling your Data Pipelines with Apache Spark on KubernetesDatabricks
There is no doubt Kubernetes has emerged as the next generation of cloud native infrastructure to support a wide variety of distributed workloads. Apache Spark has evolved to run both Machine Learning and large scale analytics workloads. There is growing interest in running Apache Spark natively on Kubernetes. By combining the flexibility of Kubernetes and scalable data processing with Apache Spark, you can run any data and machine pipelines on this infrastructure while effectively utilizing resources at disposal.
In this talk, Rajesh Thallam and Sougata Biswas will share how to effectively run your Apache Spark applications on Google Kubernetes Engine (GKE) and Google Cloud Dataproc, orchestrate the data and machine learning pipelines with managed Apache Airflow on GKE (Google Cloud Composer). Following topics will be covered: – Understanding key traits of Apache Spark on Kubernetes- Things to know when running Apache Spark on Kubernetes such as autoscaling- Demonstrate running analytics pipelines on Apache Spark orchestrated with Apache Airflow on Kubernetes cluster.
This is a talk on how you can monitor your microservices architecture using Prometheus and Grafana. This has easy to execute steps to get a local monitoring stack running on your local machine using docker.
Kubernetes Architecture - beyond a black box - Part 1Hao H. Zhang
This is part 1 of my Kubernetes architecture deep-dive slide series.
I have been working with Kubernetes for more than a year, from v1.3.6 to v1.6.7, and I am a CNCF certified Kubernetes administrator. Before I move on to something else, I would like to summarize and share my knowledges and take-aways about Kubernetes, from a software engineer perspective.
This set of slides is a humble dig into one level below your running application in production, revealing how different components of Kubernetes work together to orchestrate containers and present your applications to the rest of the world.
The slides contains 80+ external links to Kubernetes documentations, blog posts, Github issues, discussions, design proposals, pull requests, papers, source code files I went through when I was working with Kubernetes - which I think are valuable for people to understand how Kubernetes works, Kubernetes design philosophies and why these design came into places.
Helm - Application deployment management for KubernetesAlexei Ledenev
Use Helm to package and deploy a composed application to any Kubernetes cluster. Manage your releases easily over time and across multiple K8s clusters.
Presented at GDG Devfest Ukraine 2018.
Prometheus has become the defacto monitoring system for cloud native applications, with systems like Kubernetes and Etcd natively exposing Prometheus metrics. In this talk Tom will explore all the moving part for a working Prometheus-on-Kubernetes monitoring system, including kube-state-metrics, node-exporter, cAdvisor and Grafana. You will learn about the various methods for getting to a working setup: the manual approach, using CoreOS’s Prometheus Operator, or using Prometheus Ksonnet Mixin. Tom will also share some little tips and tricks for getting the most out of your Prometheus monitoring, including the common pitfalls and what you should be alerting on.
Slide deck from my "OpenStack and MySQL" presentation at Oracle OpenWorld 2015:
"This session details exactly how MySQL fits in throughout OpenStack, takes a deeper look at the database-as-a-service (DBaaS) offering with OpenStack Trove with MySQL, and discusses how Oracle supports this thriving ecosystem."
Apache Spark on K8S Best Practice and Performance in the CloudDatabricks
Kubernetes As of Spark 2.3, Spark can run on clusters managed by Kubernetes. we will describes the best practices about running Spark SQL on Kubernetes upon Tencent cloud includes how to deploy Kubernetes against public cloud platform to maximum resource utilization and how to tune configurations of Spark to take advantage of Kubernetes resource manager to achieve best performance. To evaluate performance, the TPC-DS benchmarking tool will be used to analysis performance impact of queries between configurations set.
Speakers: Junjie Chen, Junping Du
Traditional virtualization technologies have been used by cloud infrastructure providers for many years in providing isolated environments for hosting applications. These technologies make use of full-blown operating system images for creating virtual machines (VMs). According to this architecture, each VM needs its own guest operating system to run application processes. More recently, with the introduction of the Docker project, the Linux Container (LXC) virtualization technology became popular and attracted the attention. Unlike VMs, containers do not need a dedicated guest operating system for providing OS-level isolation, rather they can provide the same level of isolation on top of a single operating system instance.
An enterprise application may need to run a server cluster to handle high request volumes. Running an entire server cluster on Docker containers, on a single Docker host could introduce the risk of single point of failure. Google started a project called Kubernetes to solve this problem. Kubernetes provides a cluster of Docker hosts for managing Docker containers in a clustered environment. It provides an API on top of Docker API for managing docker containers on multiple Docker hosts with many more features.
Rook turns distributed storage systems into self-managing, self-scaling, self-healing storage services. It automates the tasks of a storage administrator: deployment, bootstrapping, configuration, provisioning, scaling, upgrading, migration, disaster recovery, monitoring, and resource management.
Rook uses the power of the Kubernetes platform to deliver its services via a Kubernetes Operator for each storage provider.
Oleg Chunikhin, Co-Founder and CTO @ Kublr.com, will present an introduction to storage management on k8s using Rook and Ceph.
20180503 kube con eu kubernetes metrics deep diveBob Cotton
Kubernetes generates a wealth of metrics. Some explicitly within the Kubernetes API server, the Kublet, and cAdvisor or implicitly by observing events such as the kube-state-metrics project. A subset of these metrics are used within Kubernetes itself to make scheduling decisions, however, other metrics can be used to determine the overall health of the system or for capacity planning purposes.
Kubernetes exposes metrics from several places, some available internally, others through add-on projects. In this session you will learn about:
- Node level metrics, as exposed from the node_exporter
- Kublet metrics
- API server metrics
- etcd metrics
- cAdvisor metrics
- Metrics exposed from kube-state-metrics
KubeCon Prometheus Salon -- Kubernetes metrics deep diveBob Cotton
Kubernetes generates a wealth of metrics. Some explicitly within the Kubernetes API server, the Kublet, and cAdvisor or implicitly by observing events such as the kube-state-metrics project. A subset of these metrics are used within Kubernetes itself to make scheduling decisions, however other metrics can be used to determine the overall health of the system or for capacity planning purposes.
Kubernetes exposes metrics from several places, some available internally, others through add-on projects. In this session you will learn about:
- Node level metrics, as exposed from the node_exporter
- Kublet metrics
- API server metrics
- etcd metrics
- cAdvisor metrics
- Metrics exposed from kube-state-metrics
Join this session to learn about how these metrics are calculated, their use within Kubernetes scheduling decisions and application in monitoring, alerting and capacity planning. This session will also cover the new metrics implementation/proposals that are to replace the cAdvisor metrics in Kubernetes 1.8.
Kubernetes provides a powerful framework and great tooling to control hundreds of heterogenous workloads on thousands of machines. In a production environment, however, the collection of metrics to automatically detect and act on issues in such a cluster is essential. Prometheus was created to meet such needs: highly dynamic scheduling, automatic service discovery, and reliable operations.
Prometheus has become the defacto monitoring system for cloud native applications, with systems like Kubernetes and Etcd natively exposing Prometheus metrics. In this talk Tom will explore all the moving part for a working Prometheus-on-Kubernetes monitoring system, including kube-state-metrics, node-exporter, cAdvisor and Grafana. You will learn about the various methods for getting to a working setup: the manual approach, using CoreOSs Prometheus Operator, or using Prometheus Ksonnet Mixin. Tom will also share some little tips and tricks for getting the most out of your Prometheus monitoring, including the common pitfalls and what you should be alerting on.
OSMC 2019 | Monitoring Cockpit for Kubernetes Clusters by Ulrike KlusikNETWAYS
Monitoring Kubernetes Clusters with Prometheus is state of the art. The difficulty is to find the significant metrics from the vast amount of available metrics. This talk shows a Monitoring Cockpit defined to get a quick overview of the cluster health and usage. It uses the Standard Metrics available for Kubernetes/OpenShift Clusters and their standard services. The monitoring solution is based on Prometheus, using InfluxDB for central long term storage and Grafana.
Jacopo Nardiello - Monitoring Cloud-Native applications with Prometheus - Cod...Codemotion
We are going to talk about Prometheus and how to use to monitor micro-services "Cloud-Native" application s. We are going to dive deep into the Prometheus monitoring model, we will see what are the components be hind this system and how they integrate with each others to provide an efficient and modern monitoring sy stem. We will also have a glance on Prometheus native integrations for cloud-native environments such as Kubernetes.
This presentation on "Monitoring on Kubernetes using Prometheus" was made by Chandresh Pancholi on 9th June Cloud Native meetup in Bridgei2i Analytics in Bangalore
DevoxxUK: Optimizating Application Performance on KubernetesDinakar Guniguntala
Now that you have your apps running on K8s, wondering how to get the response time that you need ? Tuning a polyglot set of microservices to get the performance that you need can be challenging in Kubernetes. The key to overcoming this is observability. Luckily there are a number of tools such as Prometheus that can provide all the metrics you need, but here is the catch, there is so much of data and metrics that is difficult make sense of it all. This is where Hyperparameter tuning can come to the rescue to help build the right models.
This talk covers best practices that will help attendees
1. To understand and avoid common performance related problems.
2. Discuss observability tools and how they can help identify perf issues.
3. Look closer into Kruize Autotune which is a Open Source Autonomous Performance Tuning Tool for Kubernetes and where it can help.
Cosmetic shop management system project report.pdfKamal Acharya
Buying new cosmetic products is difficult. It can even be scary for those who have sensitive skin and are prone to skin trouble. The information needed to alleviate this problem is on the back of each product, but it's thought to interpret those ingredient lists unless you have a background in chemistry.
Instead of buying and hoping for the best, we can use data science to help us predict which products may be good fits for us. It includes various function programs to do the above mentioned tasks.
Data file handling has been effectively used in the program.
The automated cosmetic shop management system should deal with the automation of general workflow and administration process of the shop. The main processes of the system focus on customer's request where the system is able to search the most appropriate products and deliver it to the customers. It should help the employees to quickly identify the list of cosmetic product that have reached the minimum quantity and also keep a track of expired date for each cosmetic product. It should help the employees to find the rack number in which the product is placed.It is also Faster and more efficient way.
Saudi Arabia stands as a titan in the global energy landscape, renowned for its abundant oil and gas resources. It's the largest exporter of petroleum and holds some of the world's most significant reserves. Let's delve into the top 10 oil and gas projects shaping Saudi Arabia's energy future in 2024.
COLLEGE BUS MANAGEMENT SYSTEM PROJECT REPORT.pdfKamal Acharya
The College Bus Management system is completely developed by Visual Basic .NET Version. The application is connect with most secured database language MS SQL Server. The application is develop by using best combination of front-end and back-end languages. The application is totally design like flat user interface. This flat user interface is more attractive user interface in 2017. The application is gives more important to the system functionality. The application is to manage the student’s details, driver’s details, bus details, bus route details, bus fees details and more. The application has only one unit for admin. The admin can manage the entire application. The admin can login into the application by using username and password of the admin. The application is develop for big and small colleges. It is more user friendly for non-computer person. Even they can easily learn how to manage the application within hours. The application is more secure by the admin. The system will give an effective output for the VB.Net and SQL Server given as input to the system. The compiled java program given as input to the system, after scanning the program will generate different reports. The application generates the report for users. The admin can view and download the report of the data. The application deliver the excel format reports. Because, excel formatted reports is very easy to understand the income and expense of the college bus. This application is mainly develop for windows operating system users. In 2017, 73% of people enterprises are using windows operating system. So the application will easily install for all the windows operating system users. The application-developed size is very low. The application consumes very low space in disk. Therefore, the user can allocate very minimum local disk space for this application.
Industrial Training at Shahjalal Fertilizer Company Limited (SFCL)MdTanvirMahtab2
This presentation is about the working procedure of Shahjalal Fertilizer Company Limited (SFCL). A Govt. owned Company of Bangladesh Chemical Industries Corporation under Ministry of Industries.
Event Management System Vb Net Project Report.pdfKamal Acharya
In present era, the scopes of information technology growing with a very fast .We do not see any are untouched from this industry. The scope of information technology has become wider includes: Business and industry. Household Business, Communication, Education, Entertainment, Science, Medicine, Engineering, Distance Learning, Weather Forecasting. Carrier Searching and so on.
My project named “Event Management System” is software that store and maintained all events coordinated in college. It also helpful to print related reports. My project will help to record the events coordinated by faculties with their Name, Event subject, date & details in an efficient & effective ways.
In my system we have to make a system by which a user can record all events coordinated by a particular faculty. In our proposed system some more featured are added which differs it from the existing system such as security.
Water scarcity is the lack of fresh water resources to meet the standard water demand. There are two type of water scarcity. One is physical. The other is economic water scarcity.
Immunizing Image Classifiers Against Localized Adversary Attacksgerogepatton
This paper addresses the vulnerability of deep learning models, particularly convolutional neural networks
(CNN)s, to adversarial attacks and presents a proactive training technique designed to counter them. We
introduce a novel volumization algorithm, which transforms 2D images into 3D volumetric representations.
When combined with 3D convolution and deep curriculum learning optimization (CLO), itsignificantly improves
the immunity of models against localized universal attacks by up to 40%. We evaluate our proposed approach
using contemporary CNN architectures and the modified Canadian Institute for Advanced Research (CIFAR-10
and CIFAR-100) and ImageNet Large Scale Visual Recognition Challenge (ILSVRC12) datasets, showcasing
accuracy improvements over previous techniques. The results indicate that the combination of the volumetric
input and curriculum learning holds significant promise for mitigating adversarial attacks without necessitating
adversary training.
Final project report on grocery store management system..pdfKamal Acharya
In today’s fast-changing business environment, it’s extremely important to be able to respond to client needs in the most effective and timely manner. If your customers wish to see your business online and have instant access to your products or services.
Online Grocery Store is an e-commerce website, which retails various grocery products. This project allows viewing various products available enables registered users to purchase desired products instantly using Paytm, UPI payment processor (Instant Pay) and also can place order by using Cash on Delivery (Pay Later) option. This project provides an easy access to Administrators and Managers to view orders placed using Pay Later and Instant Pay options.
In order to develop an e-commerce website, a number of Technologies must be studied and understood. These include multi-tiered architecture, server and client-side scripting techniques, implementation technologies, programming language (such as PHP, HTML, CSS, JavaScript) and MySQL relational databases. This is a project with the objective to develop a basic website where a consumer is provided with a shopping cart website and also to know about the technologies used to develop such a website.
This document will discuss each of the underlying technologies to create and implement an e- commerce website.
Forklift Classes Overview by Intella PartsIntella Parts
Discover the different forklift classes and their specific applications. Learn how to choose the right forklift for your needs to ensure safety, efficiency, and compliance in your operations.
For more technical information, visit our website https://intellaparts.com
Explore the innovative world of trenchless pipe repair with our comprehensive guide, "The Benefits and Techniques of Trenchless Pipe Repair." This document delves into the modern methods of repairing underground pipes without the need for extensive excavation, highlighting the numerous advantages and the latest techniques used in the industry.
Learn about the cost savings, reduced environmental impact, and minimal disruption associated with trenchless technology. Discover detailed explanations of popular techniques such as pipe bursting, cured-in-place pipe (CIPP) lining, and directional drilling. Understand how these methods can be applied to various types of infrastructure, from residential plumbing to large-scale municipal systems.
Ideal for homeowners, contractors, engineers, and anyone interested in modern plumbing solutions, this guide provides valuable insights into why trenchless pipe repair is becoming the preferred choice for pipe rehabilitation. Stay informed about the latest advancements and best practices in the field.
Vaccine management system project report documentation..pdfKamal Acharya
The Division of Vaccine and Immunization is facing increasing difficulty monitoring vaccines and other commodities distribution once they have been distributed from the national stores. With the introduction of new vaccines, more challenges have been anticipated with this additions posing serious threat to the already over strained vaccine supply chain system in Kenya.
2. 본 세션에서는...
• Introduction to Prometheus
ü Prometheus overview
ü How to monitoring with k8s
• k8s 주요 Metric 소개
ü Node metric
ü Pod/Container metric
ü Kubernetes metric
• 활용기
ü HPA(Horizon Pod Autoscaler)
ü Istio + Prometheus
4. • CNCF Monitoring Project
• 시계열 데이터를 저장하고 활용에 특화된 형태의 Database
ü Pull(Default), Push(Push Gateway)
• Service Discovery
ü Dynamic or Cloud 환경에 적합
• Graphing
• Alerting (E-mail, Webhook, Hipchat, OpsGenie ..)
• Performance
ü Millions of Time series
ü Thousands of targets
Introduction to Prometheus
Time Series
Database
• High write Performance
• Quick to process
• Easy Range Query
• Data Compaction
• Cost Efficient
Metric Collector + Metric Database
Monitoring
Target
Discover & Pull Metric
5. Prometheus component diagram
Prometheus
Server
+
Local Storage
Email
Slack
…
Alertmanager Hipch
at
Webho
ok
push
Alert
Service Discovery
DNS, File
Kubernetes
Openstack
EC2, GCE, Azure
② Dynamic
Target
① Discover Target
Application
(/metrics)
MySQL
Kafka
Pushgateway
Jobs
mysql exporter
kafka_exporter
Metric Push
Web UI
Grafana
API Clients
Prom Query
Static Target
Prometheus Architecture
6. Prometheus Metric Collections
Prometheus scrape metric
• Pulling time series data (Scrape)
• Metric Source
ü Directly : prometheus metric endpoint
ü Exporter : official & 3rd-party exporter
Directly
Scrape
Not Directly
Scrape
Exporter
/metrics
/metrics
HTTP
Scrape
HTTP
Scrape
7. Prometheus Metric Collections
Prometheus scrape example
• Node Exporter
ü Hardware and OS metrics 수집 / 노출
ü Host CPU, Memory, Disk, Filesystem, vmstat, netstat, iostat, /proc/~
Server
Instance
Exporter
# HELP node_cpu_seconds_total Seconds the cpus spent in each mode.
# TYPE node_cpu_seconds_total counter
node_cpu_seconds_total{cpu="0",mode="idle"} 710442.62
node_cpu_seconds_total{cpu="0",mode="nice"} 0
node_cpu_seconds_total{cpu="0",mode="system"} 57773.05
node_cpu_seconds_total{cpu="0",mode="user"} 59689.81
# HELP node_load1 1m load average.
# TYPE node_load1 gauge
node_load1 2.158203125
# HELP node_load5 5m load average.
# TYPE node_load5 gauge
node_load5 2.14599609375
...
Scrape Interval
/metrics
Baremetal
VM
/metrics
Node
exporter
Node
exporter
8. Prometheus Metric Target
Prometheus scrape target
Prometheus
Server
Instance Instance
Service Discovery
DNS, File
Kubernetes
OpenStack
EC2, GCE, Azure
Exporter
192.168.1.2:9090 192.168.1.3:9100
Instance Instance
Exporter
Discover Target
• Service Discovery
ü Static Target
ü Service Discovery
• File-based Discovery
• Automated Discovery (DNS, Consul, Cloud Provider, Kubernetes...)
9. Prometheus Alerting & Alertmanager
Prometheus & Alertmanager 간 Alert Rule & Notification 연계 기능 제공
Alertmanager
Prometheus
Alert Rules
Prometheus
Alert Rules
• Alert Rules setting
• Alert Trigger
Prometheus
• Notification Channel Integration
• Send to Notification Channel
• Alert De-Duplication
• Alert Routing
• Silence
Alertmanager
- name: node.rules
rules:
- alert: NodeCPUUsage
expr: (100 - avg by (instance,mode) (irate(node_cpu_seconds_total{mode="idle"}[1m])) * 100) > 75
for: 2m
labels:
severity: warning
annotations:
message: CPU Usage HIGH
ü node_alerts.rules
11. Native Monitoring
1) Kubernetes Components Metrics
• Host/Kubernetes/Container/Application 까지다양한Metric 수집
ü All Components Expose Metrics (/metrics) + kube-state-metrics(exporter)
ü Ready to Monitoring with Prometheus
etcd
cluster
K-V Store
K8S Master
Kube-apiserver
Kube-scheduler
…
Kube-contoller-manager
K8S Master
kube-apiserver
kube-scheduler
…
kube-contoller-manager
K8S Worker Node
kubelet
kube-proxy
node exporter cAdvisor
POD
POD
kubelet
kube-proxy
node exporter cAdvisor
POD
POD
kubelet
kube-proxy
node exporter cAdvisor
POD
POD
Prometheus
kube-state-metri
cs
Grafana
Alertmanager
① k8s Service Discovery
② Pull component metric
Target Endpoint
kube-apiservers https://[Master]:443/metrics
kube-contoller-manager https://[Master]:10252/metrics
kube-scheduler https://[Master]:10251/metrics
kubelet https://[Master]:10250/metrics
etcd https://[Master]:2379/metrics
cadvisor https://[ALL]:4194/metrics
Node Exporter https://[ALL]:9100/metrics
kube-state-metric
(exporter)
https://[kube-state-metric-
pod]:8080/metrics
[Target Config][Metric Scrape]
12. Monitoring Kubernetes with Prometheus
2) Prometheus Service Discovery - Kubernetes
• Discovery all Node, Pods, Service, Endpoint, Ingress
ü Metric 수집 설정 자동화 (Register, Unregister, Update)
ü Node의 추가/삭제, POD과 Service의 동적인 변경에 유연하게 동작
13. 별도의 복잡한 모니터링 시스템 필요 없음
3) Running Prometheus on Kubernetes
• Kubernetes 상에Container 형태로 배포
ü 빠르고 작은 규모로 모니터링 시스템 구축/활용
ü Kubernetes Resource를 통한 내부 연계, 설정 자동화
Component DeployManifest Files
• Grafana
• Prometheus
• Alertmanager
• Exporter
ü Kube-state-metrics
ü Node-exporter
ü Blackbox-exporter
ü …
KubernetesObjects
• Deployment
• DeamonSet
• ConfigMap
• Service
• Ingress
• PVC, PV
• RBAC
15. Prometheus Type of Metrics
• Gauges
ü current state : snapshot of a specific measurement
ü Memory, Disk Usage 등 실시간 형태로 Metric 측정 Type
Gauges, Counter, Histogram
16. Prometheus Type of Metrics
• Counter
ü cumulative Metric Type, suffix : [xxx]_total, reset to zero on restart
ü rate()/irate() 함수를 통해 변화량/추이 분석에 유리
ü CPU, Request Count, Error Count, Network Usage 등
Gauges, Counter, Histogram
17. Prometheus Type of Metrics
• Histogram
ü 구간 별 데이터의 분포도 파악(Cumulative)
ü 데이터를 버킷으로 그룹화 - suffix : [xxx]_bucket
ü histogram_quantile() 함수를 통해 백분위 별 평균 집계에 용이
Gauges, Counter, Histogram
18. Core Metrics: kubernetes node
• Physical machines (or VMs)
• Node-Exporter + DaemonSet 조합
ü prefix : node_[xxx]
Kubernetes All Node Level Metric 노출
구분 Metric Type Description
CPU node_cpu_seconds_total Count CPU Mode 별 점유 시간(per processor)
Load node_load1 / node_load5 /node_load15 Gauge System Load Average 1/5/15
Memory
node_memory_MemTotal
node_memory_MemeAvailable
node_memory_MemFree
node_memory_Buffers
node_memory_Cached
Gauge System Memory Information
Disk
node_filesystem_size
node_filesystem_avail
Gauge File System Info
node_disk_read_time_ms
node_disk_write_time_ms
node_disk_reads_completed
node_disk_writes_completed
Count Disk Latency & R/W Available
Network
node_network_receive_bytes
node_network_transmit_bytes
Gauge Network Information
System
node_time
node_boot_time
Gauge System Time
node_filefd_allocated
node_filed_maximum
Gauge File discriptor
20. Core Metrics: kubernetes Container
• Container Resource Usage 중심 (CPU,Mem, Network, Disk..)
• Pod, Deployment, StatefulSet, DaemonSet 등 Replica controller 기준으로 조합 활용
ü prefix : container_[xxx]
cAdvisor : Docker Daemon 내 Running Container Metric 노출
구분 Metric Type Description
CPU
container_cpu_usage_seconds_total Counter Container CPU Usage (per processer)
container_cpu_cfs_throttled_seconds_total Counter Container CPU throttled Time
Memory
container_memory_usage_bytes Gauge
Current memory usage in bytes(all memory
regardless of when it was accessed)
container_memory_working_set_bytes Gauge Current working set in bytes
Network
container_network_receive_bytes_total Counter Cumulative count of bytes received.
container_network_transmit_bytes_total Counter Cumulative count of bytes transmitted.
System container_start_time_seconds Gauge Container start time
21. Core Metrics: kubernetes metric
Kube-state-metric : Generate metrics about k8s API
• Exporter로 Container 구동되어 Kubernetes Metric 수집
• kubectl get ~ : node, service, deployment, replicasets, pods, pv, pvc, configmap, quotas,
secret, etc…
ü prefix : kube_[xxx]
ü https://github.com/kubernetes/kube-state-metrics/tree/master/Documentation#exposed-
metrics
Node Metric
Pod Metric
Deployment
Metric
- Node Info IP (Name, IP, Version)
- Node Condition
- Node Capacity
- Pod / Container Info (Pod IP,
Namespace, Node IP)
- Pod / Container status
- Container Resource (Requests,limit)
Service Metric
- Service Info (Cluster IP, Name)
- Service Type
- Deployment Replica Count
- Replica Availables
Quota Metric
- Resource Quota 할당 정보
22. Core Metrics: kubernetes metric
Node Status Condition
• 전반적인 k8s cluster Node의 알람 설정과 Node Status 파악에 용이
구분 Metric Type Description
Node Info
kube_node_created Gauge Node Create Time
kube_node_labels Gauge Node Label 정보
kube_node_info Gauge OS Image, kernel, Conatiner Runtime, kubelet
kube_node_spec_unschedulable Gauge Cordon 정보
kube_node_taint Gauge Taint 정보
Condition kube_node_status_condition Gauge
Condition : Ready, MemoryPressure, DiskPressure,
DIskPressure, OutofDisk
23. Core Metrics: kubernetes metric
Node Capacity
• 전반적인 Cluster Node 용량 관리와 증설 시점 판단
• Node Overcommit 관리
• Pod QOS 에 따른 Node 부하 분산&Eviction 관리 (Guaranteed > Burstable > Best-Effort)
구분 Metric Type Description
Node Capacity
kube_node_status_capacity_cpu_cores Gauge Node Capacity (CPU)
kube_node_status_capacity_memory_bytes Gauge Node Capacity (Mem)
kube_node_status_capacity_pods Gauge Node Capacity (Pod), Default 110
kube_node_status_allocatable_cpu_cores Gauge Node Allocatable (CPU)
kube_node_status_allocatable_memory_bytes Gauge Node Allocatable (Mem)
kube_node_status_allocatable_pods Gauge Node Allocatable (Pod)
24. Core Metrics: kubernetes metric
Pod/Container Summary
• Pod 배포에 따른 Node 위치 정보
• 분산 scheduling 상태 파악
• Namespace, Node 별 Pod 추이/Count
구분 Metric Type Description
Pod/Container
Info
kube_pod_info Gauge Pod Info(Pod-name, namespace, Host-IP, Pod-IP)
kube_pod_start_time Gauge Pod 시작 시간 (UnixTime)
kube_pod_created Gauge Container 생성 시간 (UnixTime)
kube_pod_container_info Gauge
Container Info(Container-name, Pod-name,
namespace, image, image-id, container-id)
25. Core Metrics: kubernetes metric
Pod/Container Summary
• Container 별 Resource Current/Reqeust/Limit 비교
ü QOS 설정 파악 (Guaranteed > Burstable > Best-Effort)
ü Replica count 조정 또는 HPA, VPA 참조
구분 Metric Type Description
Pod/Container
Resource
kube_pod_container_resource_request_cpu_cores Gauge Container Request CPU
kube_pod_container_resource_request_memory_bytes Gauge Container Request Mem
kube_pod_container_resource_limits_cpu_cores Gauge Container Limit CPU
kube_pod_container_resource_limits_memory_bytes Gauge Container Limit Mem
26. Core Metrics: kubernetes metric
Pod, Container Condition
• Pod Restart, Pod Status Phase 상태 모니터링 지표
구분 Metric Type Description
Condition
kube_pod_status_phase Gauge
Pod Lifecycle 상태
<Pending, Running, Succeeded, Failed, Unknown>
kube_pod_container_status_restart_total Counter Pod Restart Count
kube_pod_status_ready Gauge Pod Condition 상태 (true, false, Unknown)
kube_pod_status_scheduled Gauge Pod Scheduling 상태 (true, false, Unknown)
kube_pod_container_status_waiting_reason Gauge
Container waiting status
<ContainerCreating|CrashLoopBackOff|ErrImagePull | I
magePullBackOff>
kube_pod_container_status_terminated_reason Gauge
Container terminated status
<OOMKilled|Error|Completed|ContainerCannotRun>
27. Core Metrics: kubernetes metric
APIServer Request & Latency 모니터링 설정
• APIServer Latency
• HTTP Response code 5xx 비율
구분 Metric Type Description
Apiserver
apiserver_request_count Counter APIServer Request 요청 수
apiserver_request_latencies_sum Gauge API 작업의 총 지연 시간
apiserver_request_latencies_count Gauge API 대한 요청 수
apiserver_request_latencies_bucket Histogram Latencies bucket
28. Core Metrics: kubernetes metric
kubelet metric 모니터링
• Pod from pending to running Latency
• Node 별 Pod/Container Create Rate
구분 Metric Type Description
kubelet
kubelet_running_pod_count Counter kubelet running Pod
kubelet_running_container_count Counter kubelet running Container
kubelet_pod_start_latency_microseconds_sum Counter kubelet Pod 기동 횟수
kubelet_pod_start_latency_microseconds_count Counter kubelet Pod 기동 시 latency 합계
30. Kubernetes Auto Scaler (Horizon Pod Autoscaler)
• Heapster Collect cAdvisor Metric (CPU, Memory, File system, Network)
• Only Support HPA v1 Pod CPU Utilization
Heapster is DEPRECATED k8s 1.11
31. Metric Server - successor of Heapster
• Using memory-efficient API (kubernetes.summary_api)
• HPA v2 with Metric Server
• Support Pod CPU/Memory Utilization
• metrics.k8s.io/v1beta1
Collect Cluster Resource – (Node, Pod)
$ kubectl get --raw "/apis/metrics.k8s.io/v1beta1/nodes" | jq .
$ kubectl get --raw "/apis/metrics.k8s.io/v1beta1/pods" | jq .
32. Custom Metrics Adapter for Prometheus
• Need to Custom Metrics API Server (adaptor)
• HPA V2 API를 통해 Custom Metrics 기반의 HPA 적용 가능
• prometheus-hpa-adapter
ü https://github.com/DirectXMan12/k8s-prometheus-adapter
Custom Metrics API Support Kubernetes 1.6
ü container_cpu_usage_seconds_total -> cpu_usage (Counter)
ü container_memory_working_set_bytes -> memory_working_set_bytes (Gauge)
ü container_cpu_load_average_10s -> cpu_load_average_10s (Gauge)
ü container_network_tcp_usage_total -> network_tcp_usage (Counter)
ü tomcat_requestcount_total -> tomcat_requestcount (Counter)
apiVersion: autoscaling/v2beta1
kind: HorizontalPodAutoscaler
metadata:
name: spring-music-hpa
namespace: default
spec:
scaleTargetRef:
apiVersion: extensions/v1beta1
kind: Deployment
name: spring-music
minReplicas: 1
maxReplicas: 10
metrics:
- type: Object
object:
target:
kind: Pod
name: spring-music-854b4f6884-87lbf
metricName: tomcat_requestcount
targetValue: 100
33. Istio + Prometheus
• Istio Prometheus format Metric 노출
• Service-Discovery + Directly Scrape 형태로 Metric 수집
• Rich Metric
Mixer comes with a built-in Prometheus
34. Istio + Prometheus Demo
1) Istio + prometheus + prometheus+hpa-adapter 배포
2) Sample App + nginx Ingress
3) Injection
4) App restart & Envoy 확인
5) Gateway & VirtualService 생성 & istio ingress 확인
6) Istio Custom Metric HPA 설정
7) 부하 발생 & HPA 확인
Istio + Prometheus-hpa-adapter