This document summarizes a presentation about performance management tools. It discusses the different viewpoints of management, engineering, QA testing, and operations when it comes to performance monitoring. It also provides overviews of various free network monitoring, system monitoring, load generation, and data collection tools that can be used for capacity planning and performance management, including SNMP tools like Nagios, OpenNMS, Zenoss, and ntop as well as the RRDtool, SE Toolkit, and Cacti.
Introduction to OpenDaylight and Hydrogen, Learnings from the Year, What's Ne...David Meyer
OpenDaylight is an open source SDN platform developed under the Linux Foundation. The presentation discusses OpenDaylight and its first release (Hydrogen). Key points:
- OpenDaylight aims to create a common, extensible SDN platform to further adoption and innovation.
- The Hydrogen release included 14 projects and delivered core SDN functions. It faced various issues during development.
- Looking ahead, the speaker discusses continuing to build the community, improving code quality, and releasing new projects to address areas like security and distributed systems. The overall goal is to advance the OpenDaylight platform.
A tour of scalability improvements between Havana and Juno.
The presentation discusses results from an experimental campaign and the various features that enable the scalability improvements
Presentation from Aaron Rose and Salvatore Orlando.
All Things Open SDN, NFV and Open Daylight Mark Hinkle
The document provides an overview of SDN and OpenDaylight. It discusses how SDN enables programmable networks through the separation of the control plane from the data plane. It then summarizes what OpenDaylight is and its role as an open source SDN platform. The document also provides examples of how various organizations such as AT&T, Comcast, Orange, and Tencent are leveraging OpenDaylight in their networks.
Cacti is an open source software that uses RRDTool to graph and store time-series data from data sources like SNMP. It stores data in a MySQL database and uses PHP to provide a frontend interface for creating graphs, templates, and managing users. Cacti supports unlimited graph items, auto-padding, custom data gathering scripts, and SNMP to monitor network traffic and system metrics over time through graphs. It also provides features like data source templates, host templates, and user management to scale monitoring of large networks.
Cacti is a front-end GUI for RRDtool that provides graphing and monitoring of data gathered by various plugins. It stores configuration and user data in MySQL and stores monitored device data in RRD files. Cacti includes polling capabilities via PHP or C-based plugins, and allows additional applications and data sources to be monitored through an extensible framework. It also has alerting functionality via the thold plugin. The MySQL plugin provides predefined monitoring templates for databases, servers, and applications.
Agile Tool Hacking - Taking Your Agile Development Tools To The Next LevelCraig Smith
Agile Tool Hacking - Taking Your Agile Development Tools To The Next Level, presented by Craig Smith & Paul King at the Agile 2009 conference in Chicago
Kubernetes is an open-source system for managing containerized applications across multiple hosts. It includes key components like Pods, Services, ReplicationControllers, and a master node for managing the cluster. The master maintains state using etcd and schedules containers on worker nodes, while nodes run the kubelet daemon to manage Pods and their containers. Kubernetes handles tasks like replication, rollouts, and health checking through its API objects.
Introduction to OpenDaylight and Hydrogen, Learnings from the Year, What's Ne...David Meyer
OpenDaylight is an open source SDN platform developed under the Linux Foundation. The presentation discusses OpenDaylight and its first release (Hydrogen). Key points:
- OpenDaylight aims to create a common, extensible SDN platform to further adoption and innovation.
- The Hydrogen release included 14 projects and delivered core SDN functions. It faced various issues during development.
- Looking ahead, the speaker discusses continuing to build the community, improving code quality, and releasing new projects to address areas like security and distributed systems. The overall goal is to advance the OpenDaylight platform.
A tour of scalability improvements between Havana and Juno.
The presentation discusses results from an experimental campaign and the various features that enable the scalability improvements
Presentation from Aaron Rose and Salvatore Orlando.
All Things Open SDN, NFV and Open Daylight Mark Hinkle
The document provides an overview of SDN and OpenDaylight. It discusses how SDN enables programmable networks through the separation of the control plane from the data plane. It then summarizes what OpenDaylight is and its role as an open source SDN platform. The document also provides examples of how various organizations such as AT&T, Comcast, Orange, and Tencent are leveraging OpenDaylight in their networks.
Cacti is an open source software that uses RRDTool to graph and store time-series data from data sources like SNMP. It stores data in a MySQL database and uses PHP to provide a frontend interface for creating graphs, templates, and managing users. Cacti supports unlimited graph items, auto-padding, custom data gathering scripts, and SNMP to monitor network traffic and system metrics over time through graphs. It also provides features like data source templates, host templates, and user management to scale monitoring of large networks.
Cacti is a front-end GUI for RRDtool that provides graphing and monitoring of data gathered by various plugins. It stores configuration and user data in MySQL and stores monitored device data in RRD files. Cacti includes polling capabilities via PHP or C-based plugins, and allows additional applications and data sources to be monitored through an extensible framework. It also has alerting functionality via the thold plugin. The MySQL plugin provides predefined monitoring templates for databases, servers, and applications.
Agile Tool Hacking - Taking Your Agile Development Tools To The Next LevelCraig Smith
Agile Tool Hacking - Taking Your Agile Development Tools To The Next Level, presented by Craig Smith & Paul King at the Agile 2009 conference in Chicago
Kubernetes is an open-source system for managing containerized applications across multiple hosts. It includes key components like Pods, Services, ReplicationControllers, and a master node for managing the cluster. The master maintains state using etcd and schedules containers on worker nodes, while nodes run the kubelet daemon to manage Pods and their containers. Kubernetes handles tasks like replication, rollouts, and health checking through its API objects.
Half day workshop slides that have been presented at Computer Measurement Group for the last few years, and at Usenix 08 and LISA 08. This version is what will be presented at Usenix 09, San Diego, June 16th, along with the Solaris/Linux Performance slide deck.
The prpl foundation is an open-source, community-driven, collaborative, organization. It mainly targets and supports the MIPS architecture – but it is open to all –, with a focus on enabling next-generation datacenter-to-device portable software and virtualized architectures.
This document discusses network softwarization and the role of open source. It begins by defining network softwarization as the trend toward more programmable, software-defined networks achieved through SDN and NFV. It then discusses how open source has played a role in networking, particularly through projects that enable network softwarization like OpenDaylight, OpenStack, and OPNFV. The document concludes by discussing some experiences with open source networking projects at UNICAMP like RouteFlow, libfluid, and Mininet-WiFi, which aim to advance the state of SDN and network programmability through open collaboration.
This document discusses network softwarization and the role of open source. It begins by defining network softwarization as the trend toward more programmable, software-defined networks achieved through SDN and NFV. It then discusses how open source has played a role in networking, particularly through management plane and software appliance projects. Several ongoing open source SDN/NFV projects and standards organizations are presented. Experiences with open source projects like RouteFlow, softswitch13, and Mini-CCNx are summarized. The role of open source in accelerating standardization is also discussed.
This document provides an overview of network refactoring and offloading trends, including fluid network planes. It discusses the evolution of SDN from 2009 to 2019 and concepts like network softwarization. Instances of fluid network planes are described, such as RouteFlow, NFV layers, and VNF offloading to hardware or multi-vendor P4 fabrics. The document also covers slicing for IoT analytics and references recent works on in-network computing, fast connectivity recovery, and scaling distributed machine learning with in-network aggregation.
Presentation of the status of my PhD in 2012 done to ABLE group at Carnegie Mellon.
Years later from that appeared
https://github.com/iTransformers/netTransformer
"In love with Open Source : Past, Present and Future" : Keynote OSDConf 2014Piyush Kumar
OPEN SOURCE DEVELOPERS CONFERENCE http://osdconf.in/
★ April 26-27th, Noida ★
Keynote Session By Piyush Kumar (Lead of Infrastructure and Website Operations at MakeMyTrip)
This document discusses the challenges of monitoring dynamic containerized infrastructure and provides an overview of how Datadog addresses these challenges. It describes how Datadog collects metrics from containers, applications, and hosts using agents, APIs, and files to provide monitoring of things like CPU usage, memory usage, requests per second and error rates. It also allows tagging and querying metrics for improved visibility.
Next Generation Vulnerability Assessment Using Datadog and SnykDevOps.com
Vulnerability assessment for teams can often be overwhelming. The dependency graph could be thousands of packages depending on the application. Triaging vulnerability data and prioritizing actions has historically been a very manual process, until now. With Datadog and Snyk, learn how to trace security and performance issues by leveraging continuous profiling capabilities for actionable insight that help developers remediate problems.
Join us on Thursday, January 21 for a unique opportunity to learn more about continuous profiling, vulnerability management, and the benefit to customers from using both of these products. In this webinar, you will:
Bust some myths around continuous profiling and learn how Datadog differentiates itself
See decorated traces in action for sample Java applications and understand how Snyk + Datadog reduce time to triage supply chain vulnerabilities
Learn roadmap information for upcoming public announcements from both partners
This document describes SuperVessel, a public cloud built on IBM POWER servers using OpenStack. It provides Spark as a Service using Docker containers to deploy Spark clusters on OpenStack. The architecture utilizes OpenStack services like Heat, Nova, Neutron, and Manila along with Docker to provision and manage Spark clusters in a scalable and customizable manner.
This document summarizes a presentation on machine learning and fluid network planes. It begins with an agenda and introduction to fluid network planes and instances. It then discusses the role of machine learning in fluid network planes, including applications such as optimization, virtual network embedding problems, run-time operations, and intent-based closed-loop automation. Recent research is presented on machine learning-based YouTube QoE estimation using real 4G/5G network traces to predict video quality and inform control actions. Results are shown comparing 4G and 5G networks in terms of radio parameters, stalling events, handovers, and video resolutions under different mobility conditions.
This document discusses VPN types, vulnerabilities, and solutions. It begins by introducing VPNs and their purpose of maintaining privacy and security when communicating over public networks. It then outlines the research objectives to prove that VPN networks are more secure and reliable than WAN networks, but also still need updates. The document discusses why organizations use VPNs over WANs for flexibility, scalability, and outsourcing. It also covers how VPNs work and the main types of VPN connections. The proposed research method is to use vulnerability scanning tools to quantitatively compare the security of VPNs versus WANs.
Juan Vazquez & Julián Vilas – Tú a Barcelona y yo a Tejas, a patadas con mi S...RootedCON
1. The document discusses a presentation given by Juan Vazquez and Julian Vilas analyzing vulnerabilities in the Yokogawa CENTUM CS 3000 R3 SCADA system.
2. They discovered several buffer overflow vulnerabilities through fuzzing protocols and applications used by the system. Exploits were developed and disclosed to the vendor through a responsible disclosure process.
3. The presentation demonstrates exploits through Metasploit and discusses how an attacker could potentially tamper with process variables to manipulate control processes after gaining remote code execution on the system through the buffer overflows.
This document provides a summary of Christian Esteve Rothenberg, a professor researching network functions virtualization and software defined infrastructures. It outlines his professional experience which includes positions at University of Campinas and CPqD R&D Center in Telecommunication. It also lists his research interests such as SDN, NFV, ICN and various open source projects he has led like Mininet-WiFi and libfluid. The document discusses some of his research questions around NFV/SDN including VNF benchmarking and multi-domain orchestration.
How Comcast Turns Big Data into Real Time Operational Insights: Winter Olympi...Brett Sheppard
2014 O'Reilly Strata conference presentation by Patrick Shumate and Brett Sheppard, about the Comcast technology stack delivering content from the 2014 Winter Olympics in Sochi. The session and slides are presented with approval from NBC and its parent company Comcast.
David Grizzanti and Bill Jones of Sungard Availability Services presented on building a CloudStack UI for enterprises. They discussed Sungard's current managed cloud offering and why they chose CloudStack as their future platform. They described key features of their custom UI such as real-time notifications, elevated user permissions, and a customer-centric workplace concept. The presentation concluded with a demo of the UI and discussions around continuous integration, testing challenges, and tools they have created to support their work.
SRv6 experience and future perspectives
1) SRv6 and SRv6 Network Programming model
2) ROSE : Research on Open source SRv6 Ecosystem
3) SRv6 for SD-WAN & our EveryWAN solution
4) User Controlled SD-WAN Services (UCSS) project
5) Conclusions & next steps
This document presents a case study on using mashups for network management in a software defined networking (SDN) environment. It introduces SDN and how it addresses issues around network ossification. It then discusses how mashups could be used to create composite applications for managing heterogeneous and virtualized SDNs. Specifically, it proposes a "SDN mashup system" that would allow network administrators to integrate and visualize data from different SDN resources to facilitate network monitoring and management. As a proof of concept, it also describes a "slice monitoring mashup" developed within the proposed system to monitor a heterogeneous virtual SDN slice.
Recording and media manipulation of WebRTC streamsLuis Lopez
This presentation introduces Kurento technologies to developers at the WebRTC Conference & Expo 2014 in San Jose. It focuses on Kurento Client APIs and on its capabilities for recording and manipulating the audio and video streams in WebRTC sessions.
The future of multimedia communications and services: Kurento and it's roleLuis Lopez
This is a presentation specifically created for the GSMA interest group on WebRTC. This presentations introduces Kurento from the perspective of operators. Kurento is a multimedia development framework. It has been created to ease the life of multimedia application developers. Using multimedia capabilities such as embedding a video onto your app or establishing a video conferencing link between two clients may be tricky, but there is no rocket science there. However, for applications requiring more advanced features things quickly get unmanageable. If you have been involved in multimedia projects, you probably know that features such as interoperable group communications, different communication roles (e. g.. publishers/viewers), video transforming and transcoding, video storage and tagging, integration into legacy video/voice infrastructures, computer vision, augmented reality, integration with external systems and databases and many others, pose quite a complex challenge, which usually requires huge expertise and effort. Specially when real-time communications are involved. If this is your case, Kurento will help you.
Este documento apresenta os principais conceitos e requisitos para a gestão da segurança da informação de acordo com a norma ABNT NBR ISO/IEC 27002:2005. Ele discute o que é segurança da informação, por que é necessária, e como estabelecer requisitos através da análise de riscos considerando fontes legais, de negócios e ameaças. Também fornece diretrizes gerais para o ponto de partida e fatores críticos de sucesso na implementação da segurança da informação.
1) O documento discute os conceitos e processos da estrutura de gerenciamento de serviços de TI ITIL V3;
2) Apresenta os principais processos da fase de Estratégia de Serviços como geração da estratégia, gestão financeira, gerência de portfólio e gestão de demandas;
3) Também descreve os processos e atividades da fase de Desenho de Serviços, como identificação de requisitos, desenvolvimento de soluções e métricas.
Half day workshop slides that have been presented at Computer Measurement Group for the last few years, and at Usenix 08 and LISA 08. This version is what will be presented at Usenix 09, San Diego, June 16th, along with the Solaris/Linux Performance slide deck.
The prpl foundation is an open-source, community-driven, collaborative, organization. It mainly targets and supports the MIPS architecture – but it is open to all –, with a focus on enabling next-generation datacenter-to-device portable software and virtualized architectures.
This document discusses network softwarization and the role of open source. It begins by defining network softwarization as the trend toward more programmable, software-defined networks achieved through SDN and NFV. It then discusses how open source has played a role in networking, particularly through projects that enable network softwarization like OpenDaylight, OpenStack, and OPNFV. The document concludes by discussing some experiences with open source networking projects at UNICAMP like RouteFlow, libfluid, and Mininet-WiFi, which aim to advance the state of SDN and network programmability through open collaboration.
This document discusses network softwarization and the role of open source. It begins by defining network softwarization as the trend toward more programmable, software-defined networks achieved through SDN and NFV. It then discusses how open source has played a role in networking, particularly through management plane and software appliance projects. Several ongoing open source SDN/NFV projects and standards organizations are presented. Experiences with open source projects like RouteFlow, softswitch13, and Mini-CCNx are summarized. The role of open source in accelerating standardization is also discussed.
This document provides an overview of network refactoring and offloading trends, including fluid network planes. It discusses the evolution of SDN from 2009 to 2019 and concepts like network softwarization. Instances of fluid network planes are described, such as RouteFlow, NFV layers, and VNF offloading to hardware or multi-vendor P4 fabrics. The document also covers slicing for IoT analytics and references recent works on in-network computing, fast connectivity recovery, and scaling distributed machine learning with in-network aggregation.
Presentation of the status of my PhD in 2012 done to ABLE group at Carnegie Mellon.
Years later from that appeared
https://github.com/iTransformers/netTransformer
"In love with Open Source : Past, Present and Future" : Keynote OSDConf 2014Piyush Kumar
OPEN SOURCE DEVELOPERS CONFERENCE http://osdconf.in/
★ April 26-27th, Noida ★
Keynote Session By Piyush Kumar (Lead of Infrastructure and Website Operations at MakeMyTrip)
This document discusses the challenges of monitoring dynamic containerized infrastructure and provides an overview of how Datadog addresses these challenges. It describes how Datadog collects metrics from containers, applications, and hosts using agents, APIs, and files to provide monitoring of things like CPU usage, memory usage, requests per second and error rates. It also allows tagging and querying metrics for improved visibility.
Next Generation Vulnerability Assessment Using Datadog and SnykDevOps.com
Vulnerability assessment for teams can often be overwhelming. The dependency graph could be thousands of packages depending on the application. Triaging vulnerability data and prioritizing actions has historically been a very manual process, until now. With Datadog and Snyk, learn how to trace security and performance issues by leveraging continuous profiling capabilities for actionable insight that help developers remediate problems.
Join us on Thursday, January 21 for a unique opportunity to learn more about continuous profiling, vulnerability management, and the benefit to customers from using both of these products. In this webinar, you will:
Bust some myths around continuous profiling and learn how Datadog differentiates itself
See decorated traces in action for sample Java applications and understand how Snyk + Datadog reduce time to triage supply chain vulnerabilities
Learn roadmap information for upcoming public announcements from both partners
This document describes SuperVessel, a public cloud built on IBM POWER servers using OpenStack. It provides Spark as a Service using Docker containers to deploy Spark clusters on OpenStack. The architecture utilizes OpenStack services like Heat, Nova, Neutron, and Manila along with Docker to provision and manage Spark clusters in a scalable and customizable manner.
This document summarizes a presentation on machine learning and fluid network planes. It begins with an agenda and introduction to fluid network planes and instances. It then discusses the role of machine learning in fluid network planes, including applications such as optimization, virtual network embedding problems, run-time operations, and intent-based closed-loop automation. Recent research is presented on machine learning-based YouTube QoE estimation using real 4G/5G network traces to predict video quality and inform control actions. Results are shown comparing 4G and 5G networks in terms of radio parameters, stalling events, handovers, and video resolutions under different mobility conditions.
This document discusses VPN types, vulnerabilities, and solutions. It begins by introducing VPNs and their purpose of maintaining privacy and security when communicating over public networks. It then outlines the research objectives to prove that VPN networks are more secure and reliable than WAN networks, but also still need updates. The document discusses why organizations use VPNs over WANs for flexibility, scalability, and outsourcing. It also covers how VPNs work and the main types of VPN connections. The proposed research method is to use vulnerability scanning tools to quantitatively compare the security of VPNs versus WANs.
Juan Vazquez & Julián Vilas – Tú a Barcelona y yo a Tejas, a patadas con mi S...RootedCON
1. The document discusses a presentation given by Juan Vazquez and Julian Vilas analyzing vulnerabilities in the Yokogawa CENTUM CS 3000 R3 SCADA system.
2. They discovered several buffer overflow vulnerabilities through fuzzing protocols and applications used by the system. Exploits were developed and disclosed to the vendor through a responsible disclosure process.
3. The presentation demonstrates exploits through Metasploit and discusses how an attacker could potentially tamper with process variables to manipulate control processes after gaining remote code execution on the system through the buffer overflows.
This document provides a summary of Christian Esteve Rothenberg, a professor researching network functions virtualization and software defined infrastructures. It outlines his professional experience which includes positions at University of Campinas and CPqD R&D Center in Telecommunication. It also lists his research interests such as SDN, NFV, ICN and various open source projects he has led like Mininet-WiFi and libfluid. The document discusses some of his research questions around NFV/SDN including VNF benchmarking and multi-domain orchestration.
How Comcast Turns Big Data into Real Time Operational Insights: Winter Olympi...Brett Sheppard
2014 O'Reilly Strata conference presentation by Patrick Shumate and Brett Sheppard, about the Comcast technology stack delivering content from the 2014 Winter Olympics in Sochi. The session and slides are presented with approval from NBC and its parent company Comcast.
David Grizzanti and Bill Jones of Sungard Availability Services presented on building a CloudStack UI for enterprises. They discussed Sungard's current managed cloud offering and why they chose CloudStack as their future platform. They described key features of their custom UI such as real-time notifications, elevated user permissions, and a customer-centric workplace concept. The presentation concluded with a demo of the UI and discussions around continuous integration, testing challenges, and tools they have created to support their work.
SRv6 experience and future perspectives
1) SRv6 and SRv6 Network Programming model
2) ROSE : Research on Open source SRv6 Ecosystem
3) SRv6 for SD-WAN & our EveryWAN solution
4) User Controlled SD-WAN Services (UCSS) project
5) Conclusions & next steps
This document presents a case study on using mashups for network management in a software defined networking (SDN) environment. It introduces SDN and how it addresses issues around network ossification. It then discusses how mashups could be used to create composite applications for managing heterogeneous and virtualized SDNs. Specifically, it proposes a "SDN mashup system" that would allow network administrators to integrate and visualize data from different SDN resources to facilitate network monitoring and management. As a proof of concept, it also describes a "slice monitoring mashup" developed within the proposed system to monitor a heterogeneous virtual SDN slice.
Recording and media manipulation of WebRTC streamsLuis Lopez
This presentation introduces Kurento technologies to developers at the WebRTC Conference & Expo 2014 in San Jose. It focuses on Kurento Client APIs and on its capabilities for recording and manipulating the audio and video streams in WebRTC sessions.
The future of multimedia communications and services: Kurento and it's roleLuis Lopez
This is a presentation specifically created for the GSMA interest group on WebRTC. This presentations introduces Kurento from the perspective of operators. Kurento is a multimedia development framework. It has been created to ease the life of multimedia application developers. Using multimedia capabilities such as embedding a video onto your app or establishing a video conferencing link between two clients may be tricky, but there is no rocket science there. However, for applications requiring more advanced features things quickly get unmanageable. If you have been involved in multimedia projects, you probably know that features such as interoperable group communications, different communication roles (e. g.. publishers/viewers), video transforming and transcoding, video storage and tagging, integration into legacy video/voice infrastructures, computer vision, augmented reality, integration with external systems and databases and many others, pose quite a complex challenge, which usually requires huge expertise and effort. Specially when real-time communications are involved. If this is your case, Kurento will help you.
Este documento apresenta os principais conceitos e requisitos para a gestão da segurança da informação de acordo com a norma ABNT NBR ISO/IEC 27002:2005. Ele discute o que é segurança da informação, por que é necessária, e como estabelecer requisitos através da análise de riscos considerando fontes legais, de negócios e ameaças. Também fornece diretrizes gerais para o ponto de partida e fatores críticos de sucesso na implementação da segurança da informação.
1) O documento discute os conceitos e processos da estrutura de gerenciamento de serviços de TI ITIL V3;
2) Apresenta os principais processos da fase de Estratégia de Serviços como geração da estratégia, gestão financeira, gerência de portfólio e gestão de demandas;
3) Também descreve os processos e atividades da fase de Desenho de Serviços, como identificação de requisitos, desenvolvimento de soluções e métricas.
Este documento apresenta um exemplo de exame de Fundamentos da Segurança da Informação baseado na norma ISO/IEC 27002. O exame contém 40 questões de múltipla escolha com uma única resposta correta cada. O objetivo é avaliar os conhecimentos dos candidatos sobre conceitos e práticas de segurança da informação.
Este documento apresenta os objetivos e conceitos fundamentais de sistemas de informação. Discute a importância dos sistemas de informação para as empresas e identifica as cinco áreas principais: conceitos básicos, tecnologias da informação, aplicações empresariais, processos de desenvolvimento e desafios gerenciais. Também fornece exemplos de componentes de sistemas de informação e tipos de sistemas usados em empresas.
Conceito de analise de desenvolvivento de sistemasluanrjesus
1. O documento discute os conceitos e processos fundamentais da análise de sistemas, incluindo definições de sistemas, análise e modelagem. 2. Aborda as etapas do ciclo de vida de desenvolvimento de software, como definição, desenvolvimento e manutenção. 3. Destaca a importância da análise para identificar requisitos, reduzir erros e facilitar a manutenção do sistema.
Planejamento de Capacidade Técnicas e Ferramentasluanrjesus
O documento discute técnicas e ferramentas para planejamento de capacidade. Apresenta métricas de desempenho comuns, modelos de filas de espera e ferramentas open source populares como Nagios, Zabbix, Collectd e Cacti para monitoramento de sistemas e geração de métricas.
Planejamento e Gerenciamento de Capacidade para Sistemas Distribuídosluanrjesus
O documento discute técnicas e desafios do planejamento e gerenciamento de capacidade para sistemas distribuídos. Ele aborda motivações históricas como o alto custo do mainframe e o crescimento desordenado com PCs, e técnicas como aquisição de métricas, consolidação, visualização e previsão. Também discute os impactos da virtualização, nuvem e métodos ágeis no planejamento de capacidade.
UiPath Test Automation using UiPath Test Suite series, part 6DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 6. In this session, we will cover Test Automation with generative AI and Open AI.
UiPath Test Automation with generative AI and Open AI webinar offers an in-depth exploration of leveraging cutting-edge technologies for test automation within the UiPath platform. Attendees will delve into the integration of generative AI, a test automation solution, with Open AI advanced natural language processing capabilities.
Throughout the session, participants will discover how this synergy empowers testers to automate repetitive tasks, enhance testing accuracy, and expedite the software testing life cycle. Topics covered include the seamless integration process, practical use cases, and the benefits of harnessing AI-driven automation for UiPath testing initiatives. By attending this webinar, testers, and automation professionals can gain valuable insights into harnessing the power of AI to optimize their test automation workflows within the UiPath ecosystem, ultimately driving efficiency and quality in software development processes.
What will you get from this session?
1. Insights into integrating generative AI.
2. Understanding how this integration enhances test automation within the UiPath platform
3. Practical demonstrations
4. Exploration of real-world use cases illustrating the benefits of AI-driven test automation for UiPath
Topics covered:
What is generative AI
Test Automation with generative AI and Open AI.
UiPath integration with generative AI
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Full-RAG: A modern architecture for hyper-personalizationZilliz
Mike Del Balso, CEO & Co-Founder at Tecton, presents "Full RAG," a novel approach to AI recommendation systems, aiming to push beyond the limitations of traditional models through a deep integration of contextual insights and real-time data, leveraging the Retrieval-Augmented Generation architecture. This talk will outline Full RAG's potential to significantly enhance personalization, address engineering challenges such as data management and model training, and introduce data enrichment with reranking as a key solution. Attendees will gain crucial insights into the importance of hyperpersonalization in AI, the capabilities of Full RAG for advanced personalization, and strategies for managing complex data integrations for deploying cutting-edge AI solutions.
Maruthi Prithivirajan, Head of ASEAN & IN Solution Architecture, Neo4j
Get an inside look at the latest Neo4j innovations that enable relationship-driven intelligence at scale. Learn more about the newest cloud integrations and product enhancements that make Neo4j an essential choice for developers building apps with interconnected data and generative AI.
Sudheer Mechineni, Head of Application Frameworks, Standard Chartered Bank
Discover how Standard Chartered Bank harnessed the power of Neo4j to transform complex data access challenges into a dynamic, scalable graph database solution. This keynote will cover their journey from initial adoption to deploying a fully automated, enterprise-grade causal cluster, highlighting key strategies for modelling organisational changes and ensuring robust disaster recovery. Learn how these innovations have not only enhanced Standard Chartered Bank’s data infrastructure but also positioned them as pioneers in the banking sector’s adoption of graph technology.
zkStudyClub - Reef: Fast Succinct Non-Interactive Zero-Knowledge Regex ProofsAlex Pruden
This paper presents Reef, a system for generating publicly verifiable succinct non-interactive zero-knowledge proofs that a committed document matches or does not match a regular expression. We describe applications such as proving the strength of passwords, the provenance of email despite redactions, the validity of oblivious DNS queries, and the existence of mutations in DNA. Reef supports the Perl Compatible Regular Expression syntax, including wildcards, alternation, ranges, capture groups, Kleene star, negations, and lookarounds. Reef introduces a new type of automata, Skipping Alternating Finite Automata (SAFA), that skips irrelevant parts of a document when producing proofs without undermining soundness, and instantiates SAFA with a lookup argument. Our experimental evaluation confirms that Reef can generate proofs for documents with 32M characters; the proofs are small and cheap to verify (under a second).
Paper: https://eprint.iacr.org/2023/1886
A tale of scale & speed: How the US Navy is enabling software delivery from l...sonjaschweigert1
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Dr. Sean Tan, Head of Data Science, Changi Airport Group
Discover how Changi Airport Group (CAG) leverages graph technologies and generative AI to revolutionize their search capabilities. This session delves into the unique search needs of CAG’s diverse passengers and customers, showcasing how graph data structures enhance the accuracy and relevance of AI-generated search results, mitigating the risk of “hallucinations” and improving the overall customer journey.
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
Alt. GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using ...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
UiPath Test Automation using UiPath Test Suite series, part 5DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 5. In this session, we will cover CI/CD with devops.
Topics covered:
CI/CD with in UiPath
End-to-end overview of CI/CD pipeline with Azure devops
Speaker:
Lyndsey Byblow, Test Suite Sales Engineer @ UiPath, Inc.
1. The Performance People
Performance Management with
Free and Bundled Tools
Adrian Cockcroft
Netflix Inc.
acockcroft@netflix.com
(Co-authored with Mario Jauvin
MFJ Associates
mario@mfjassociates.net)
14 May 2014
2. May 14, 2014 Adrian Cockcroft and Mario Jauvin
Agenda
Overview of Capacity Planning Requirements
and Data Sources
Performance Data Collection
Free Network Monitoring Tools
Free System Monitoring Tools
Free Load Generation and Modelling Tools
Licences and References
3. May 14, 2014 Adrian Cockcroft and Mario Jauvin
What are we talking about?
Network
monitoring with
WireShark, MRTG,
BigSister, Cacti,
Nagios, OpenNMS,
Zenoss, Openxtra,
ntop
Database Tier monitoring
With SEtoolkit, Orca,
XEtoolkit
Application Tier
monitoring with Orca,
Cacti, BigSister, Ganglia,
XEtoolkit
QA Load generation with
Grinder or SLAMD,
modelling with PDQ and R
4. May 14, 2014 Adrian Cockcroft and Mario Jauvin
Capacity Planning
Requirements and Data
Sources
5. May 14, 2014 Adrian Cockcroft and Mario Jauvin
Definitions
Capacity
– Resource utilization and headroom
Planning
– Predicting future needs by analyzing historical data
and modeling future scenarios
Performance Monitoring
– Collecting and reporting on performance data
Free Tools
– Bundled with the OS or available for no $$$
6. May 14, 2014 Adrian Cockcroft and Mario Jauvin
Capacity Planning Requirements
We care about CPU, Memory, Network and Disk
resources, and Application response times
We need to know how much of each resource we
are using now, and will use in the future
We need to know how much headroom we have to
handle higher loads
We want to understand how headroom varies, and
how it relates to application response times and
throughput
7. May 14, 2014 Adrian Cockcroft and Mario Jauvin
CPU Capacity Measurements
CPU Capacity is defined by CPU type and
clock rate, or a benchmark rating like
SPECrateInt2000
CPU utilization is defined as busy time
divided by elapsed time for each CPU
CPU load average measures the average
number of jobs running and ready to run
8. May 14, 2014 Adrian Cockcroft and Mario Jauvin
Memory Capacity Measurements
Physical Memory Capacity Utilization and Limits
– Kernel memory
– Shared Memory segment
– Executable code, stack and heap
– File system cache usage
– Unused free memory
Virtual Memory Capacity - Swap Space
Memory Throughput
– Page in and page out rates
9. May 14, 2014 Adrian Cockcroft and Mario Jauvin
Network Capacity Measurements
Network Interface Throughput
– Byte and packet rates input and output
TCP Protocol Specific Throughput
– TCP connection count and connection rates
– TCP byte rates input and output
NFS/SMB Protocol Specific Throughput
– Byte rates read and write
– NFS/SMB service response times
HTTP Protocol Specific Throughput
– HTTP operation rates
– Get and put payload byte rates and size distribution
10. May 14, 2014 Adrian Cockcroft and Mario Jauvin
Disk Capacity Measurements
Detailed metrics vary by platform
Easy for the simple disk cases
Hard for cached RAID subsystems
Almost Impossible for shared disk
subsystems and SANs
– Another system or volume can be sharing a
backend spindle, when it gets busy your own
volume can saturate, even though you did not
change your own workload
11. May 14, 2014 Adrian Cockcroft and Mario Jauvin
Capacity Planning Challenges
Constantly changing infrastructure
Limited attention span from staff
Horizontally scaled commodity systems
Per node software licencing costs too much
Too many tools, too many agents per node
Too much data, not enough analysis
Non-linear and non-intuitive scalability
Lack of tools and metrics for virtualized resources
12. May 14, 2014 Adrian Cockcroft and Mario Jauvin
Observability
Four different viewpoints
– Management
– Engineering
– QA Testing
– Operations
Each needs very different information
Ideal would be different views of the same
performance database
Reality is a mess of disjoint tools
13. May 14, 2014 Adrian Cockcroft and Mario Jauvin
Management Viewpoint
Daily summary of status and problems
Business oriented metrics
Future scenario planning
Marketing and management input
Concise report with dashboard style status
indicators
Free tools: R, Spreadsheet and Web based
displays, no good summarization tools
14. May 14, 2014 Adrian Cockcroft and Mario Jauvin
Engineering Viewpoint
Large volumes of detailed data at several different
time scales
Input to tuning, reconfiguring and future product
development
Low level problem diagnosis
Detailed reports with drill down and correlation
analysis
Free tools: XE/SE Toolkit, Orca, Ganglia, Cacti, R
15. May 14, 2014 Adrian Cockcroft and Mario Jauvin
QA Test Viewpoint
Workload specification tools
Load generation frameworks
Testing for functionality and performance
Regression tools to compare releases
Modelling difference between test configuration
and production configuration
Free Tools: The Grinder, SLAMD, R, PDQ
16. May 14, 2014 Adrian Cockcroft and Mario Jauvin
Operations Viewpoint
Immediate timeframe
Real time display, updated in seconds
Alert based monitoring
High level problem diagnosis
Simple high level graphs and views
Free tools: BigSister, Nagios, OpenNMS,
MRTG, Cacti, Ganglia, WireShark, ntop
17. May 14, 2014 Adrian Cockcroft and Mario Jauvin
Measurement Data Interfaces
Several generic raw access methods
– Read the kernel directly (not a good idea)
– Structured system data (Solaris kstat, Linux /proc)
– Process data
– Network data
– Accounting data
– Application data
Command based data interfaces
– Scrape data from vmstat, iostat, netstat, sar, ps
– Higher overhead, lower resolution, missing metrics
Data available is platform specific either way
Much more detail on this topic in the Solaris/Linux Performance
Measurement and Tuning Class
18. May 14, 2014 Adrian Cockcroft and Mario Jauvin
Free Network
Monitoring Tools
19. May 14, 2014 Adrian Cockcroft and Mario Jauvin
SNMP
Simple network management protocol
UDP protocol based on port 161
Client/server like
– Client is called management application entity
– Server is called an agent entity
Agent entity is designed to be implemented
on network hardware, router, switches, etc
20. May 14, 2014 Adrian Cockcroft and Mario Jauvin
SNMP – MIBs
Management information base
Defines the structure and the semantic of the
information that can be reported on
Most commonly used is MIB-II which defines a set of
standard networking attributes
– Interface tables
– System level information
– Routing tables
Specified using ASN.1 (abstract syntax notation 1)
21. May 14, 2014 Adrian Cockcroft and Mario Jauvin
SNMP – commands
Called PDU (protocol data units)
GET
GETNEXT
GETBULK
SET
Encoded using BER (basic encoding rules)
22. May 14, 2014 Adrian Cockcroft and Mario Jauvin
Versions
Version 1, original version done in May 1991
Version 2, around 1993. Failed because the
IETF credo of “rough consensus and running
code” could not be met on securing SNMP
Turned into V2c for community string security
(like V1)
Version 3, added security and complexity in
1998
23. May 14, 2014 Adrian Cockcroft and Mario Jauvin
SNMP tools
Too numerous to name all but…
OpenNMS
Nagios
Cacti
MRTG
Net-snmp
– See www.snmplink.org
24. May 14, 2014 Adrian Cockcroft and Mario Jauvin
SNMP tools
Snmpwalk – will report all data in a specified
MIB
getIf – will report data about interfaces and
includes built-in MIB browser
Snmptable – will report tabular data from MIB
tables
25. May 14, 2014 Adrian Cockcroft and Mario Jauvin
OpenNMS
Well…. it’s not that portable
– 95% java is not 100% java
– Requires about 20-30 different platform specific
packages (PostgreSQL, Perl, RRD tool, Tomcat 4
etc…)
– Difficult to install
– Easy auto discovery
– Web-based interface
26. May 14, 2014 Adrian Cockcroft and Mario Jauvin
OpenNMS
Main screen shot
27. May 14, 2014 Adrian Cockcroft and Mario Jauvin
OpenNMS
Node screen shot
28. May 14, 2014 Adrian Cockcroft and Mario Jauvin
Nagios
Easy to build/compile (on Solaris 10)
Easy to install
Quick response from CGI
Configuration is manual and a pain
– 13 configuration files with all kinds of interrelated
entries
– Tedious and error prone
Requires plugins to do anything
29. May 14, 2014 Adrian Cockcroft and Mario Jauvin
Nagios
Main screen shot
30. May 14, 2014 Adrian Cockcroft and Mario Jauvin
Nagios
Host detail screen shot
32. May 14, 2014 Adrian Cockcroft and Mario Jauvin
ntop
Similar to familiar UNIX top tool for
processes but used for network
Provide huge selection of real-time data
Can be found at http://www.openxtra.co.uk/
33. May 14, 2014 Adrian Cockcroft and Mario Jauvin
ntop – Active Sessions
34. May 14, 2014 Adrian Cockcroft and Mario Jauvin
ntop Hosts
35. May 14, 2014 Adrian Cockcroft and Mario Jauvin
ntop Network Load
36. May 14, 2014 Adrian Cockcroft and Mario Jauvin
ntop_Network_Thruput
37. May 14, 2014 Adrian Cockcroft and Mario Jauvin
ntop Port Dist
38. May 14, 2014 Adrian Cockcroft and Mario Jauvin
ntop_Protocol_Dist
39. May 14, 2014 Adrian Cockcroft and Mario Jauvin
ntop Protocols
40. May 14, 2014 Adrian Cockcroft and Mario Jauvin
Zenoss
Open source monitoring and management of
IT infrastructure
Zenoss core is free
Other editions are for a fee
Get it from http://www.zenoss.com/download/
41. May 14, 2014 Adrian Cockcroft and Mario Jauvin
zenoss Architecture
42. May 14, 2014 Adrian Cockcroft and Mario Jauvin
zenoss Dash Config
43. May 14, 2014 Adrian Cockcroft and Mario Jauvin
zenoss Google
44. May 14, 2014 Adrian Cockcroft and Mario Jauvin
zenoss Google Alerts
45. May 14, 2014 Adrian Cockcroft and Mario Jauvin
Zenoss Graphs
46. May 14, 2014 Adrian Cockcroft and Mario Jauvin
zenoss Topology
47. May 14, 2014 Adrian Cockcroft and Mario Jauvin
MRTG
Really simple to install and configure
Require manual config file creation
Only for MIB-II interface plotting out of the
box
Graphing not flexible, axis, time etc
48. May 14, 2014 Adrian Cockcroft and Mario Jauvin
MRTG
Interface screen shot
49. May 14, 2014 Adrian Cockcroft and Mario Jauvin
MRTG
Other CPU screen shot
50. May 14, 2014 Adrian Cockcroft and Mario Jauvin
RRD tool
Software to store, retrieve and graph
numerical time series data
Use a round robin algorithm
Data files are a fixed size
– Don’t grow
– Don’t require maintenance
51. May 14, 2014 Adrian Cockcroft and Mario Jauvin
RRD tool
Compiles on most platforms
Used by many SNMP based tools
– OpenNMS
– Cacti
– BigSister
– WeatherMap4RRD
– MailGraph
52. May 14, 2014 Adrian Cockcroft and Mario Jauvin
RRD tool
14all CGI script that plots data similar to
MRTG
Configurable to collect data at different
interval (unlike MRTG)
Flexible and variable in what data can be
collected
53. May 14, 2014 Adrian Cockcroft and Mario Jauvin
RRD tool
Sample screen shot
54. May 14, 2014 Adrian Cockcroft and Mario Jauvin
RRD tool
Screen shot
55. May 14, 2014 Adrian Cockcroft and Mario Jauvin
RRD tool
Create a RRD database
rrdtool create test.rrd
--start 920804400
DS:speed:COUNTER:600:U:U
RRA:AVERAGE:0.5:1:24
RRA:AVERAGE:0.5:6:10
56. May 14, 2014 Adrian Cockcroft and Mario Jauvin
RRD tool
Create a graph
rrdtool graph speed.png
--start 920804400 --end 920808000
DEF:myspeed=test.rrd:speed:AVERAGE
LINE2:myspeed#FF0000
57. May 14, 2014 Adrian Cockcroft and Mario Jauvin
Free Performance Data
Collection and Rules
Toolkits
58. May 14, 2014 Adrian Cockcroft and Mario Jauvin
SE toolkit Example Tools
A free performance toolkit for rapidly creating custom data sources
Makes all the very extensive Solaris metrics easily available
Very system specific and not enough metrics exist to port to Linux
Written by Rich Pettit with contributions from Adrian Cockcroft
Get SE3.4 from http://sourceforge.net/projects/setoolkit/
Open source with support for SPARC & x86 Solaris 8, 9, 10
Function Example SE Programs
Rule Monitors cpg.se monlog.se mon_cm.se live_test.se percollator.se
zoom.se virtual_adrian.se virtual_adrian_lite.se
Disk Monitors siostat.se xio.se xiostat.se iomonitor.se iost.se xit.se disks.se
CPU Monitors cpu_meter.se vmmonitor.se mpvmstat.se
Process Monitors msacct.se pea.se ps-ax.se ps-p.se pwatch.se pw.se
Network Monitors net.se tcp_monitor.se netmonitor.se netstatx.se nfsmonitor.se nx.se
Clones iostat.se uname.se vmstat.se nfsstat-m.se perfmeter.se xload.se
Data browsers aw.se infotool.se multi_meter.se
Contributed Code anasa dfstats kview systune watch orcollator.se
Test Programs syslog.se cpus.se pure_test.se collisions.se uptime.se dumpkstats.se
net_example nproc.se kvmname.se
59. May 14, 2014 Adrian Cockcroft and Mario Jauvin
SE language features
SE is a 64bit interpreted dialect of C
– Not a new language to learn from scratch!
– Standard C /usr/ccs/bin/cpp used at runtime to preprocess SE scripts
– Main omissions - pointer types and goto
– Main additions - classes and “string” type
– powerful ways to handle dynamically allocated data
– built-in fast balanced tree routines for storing key indexed data
Dynamic linking to all existing C libraries
– Built-in classes access kernel data
– Supplied class code hides details, provides the data you want
Example scripts improve on basic utilities e.g. siostat.se, nx.se, pea.se
Example rule based monitors e.g. virtual_adrian.se, orcallator.se
60. Creating Rules
Based on real experiences of all the things that go
wrong
Capture an approximation to intuition
Test and calibrate rules on as many systems as
possible
Easy??
May 14, 2014 Adrian Cockcroft and Mario Jauvin
61. May 14, 2014 Adrian Cockcroft and Mario Jauvin
Configuring Rules
Thresholds should be configured
Very application dependent
Capture the operating envelope
– Measure the underlying values
– Measure peaks in normal operation
– Note values during problems
– Set thresholds to capture the difference
This applies to any tool
– SE Toolkit, Cacti, Ganglia, Nagios, OpenNMS etc.
62. May 14, 2014 Adrian Cockcroft and Mario Jauvin
Rules as Objects
Define only the input and output information
Hide implementation details
Make high level rule objects trivial to use and
reuse
SE Toolkit does it in three lines of code:
– #include <rules file>
– Declare rule object as a typed variable
– Read and use or print object status
63. May 14, 2014 Adrian Cockcroft and Mario Jauvin
"virtual adrian" rules summary
Disk Rule for all disks at once
– Looks for slow disks and unbalanced usage
Network Rule for all networks at once
– Looks for slow nets and unbalanced usage
Swap Rule - Looks for lack of available swap space
RAM Rule - Looks for short page residence times
CPU Power Rule
– Scales on MP systems
– Looks for long run queue delays
Mutex Rule - Looks for kernel lock contention and high sys CPU time
TCP Rule
– Looks for listen queue problems
– Reports on connection attempt failures
64. May 14, 2014 Adrian Cockcroft and Mario Jauvin
XE Toolkit - www.xetoolkit.com
Complete re-write of SE Toolkit by Rich Pettit
– Extensible Java collector, customize with jar files
– Release 1.2 available April 2008
– Multi-platform support Solaris, Linux/x86, Windows, BSD,
OSX, HP-UX, AIX, Linux/s390, Linux/Power
Licencing
– Free GPL version for standard use and shared derivations
– Open source, hosted at http://sourceforge.net/projects/xe-toolkit/
– Commercial support available if needed
– Commercial product license for custom in-house derivations
Addresses all the issues people had with SE toolkit !
65. May 14, 2014 Adrian Cockcroft and Mario Jauvin
Captive Metrics / XE Toolkit
Architecture
66. May 14, 2014 Adrian Cockcroft and Mario Jauvin
Free System Monitoring
Tools
67. May 14, 2014 Adrian Cockcroft and Mario Jauvin
Collated Performance Data - Orca
Problems with time sync when collecting data from multiple tools
– No timestamp at all for vmstat, netstat, df...
– No timestamp by default for iostat and ps...
– No way to collect realtime stats from an http logfile
Use SE Toolkit to generate one timestamped row containing all the data
– First version of percollator.se written by Adrian Cockcroft in 1996
– Extended orcallator.se written by Blair Zajac a few years later
– Graphs generated by orca batch job feeding rrdtool based web pages
– Active community developing tool at http://www.orcaware.com
– Extended to collect much more data, including process workloads
– Basic data collection ported to Linux, HP-UX and Windows
Orca is basically MRTG for System metrics rather than Network
See http://www.orcaware.com/orca/docs/Orca_Understanding_Performance_Data.ppt
68. May 14, 2014 Adrian Cockcroft and Mario Jauvin
Orca data collections
Collected using “procollator” reading info from /proc on Linux
[Uptime] [Average # Processes in Run Queue (Load Average)] [CPU Usage]
[New Process Spawn Rate] [Number of System & Running Processes]
[Context Switches & Interrupts Rate] [Interface Input Bits Per Second]
[Interface Output Bits Per Second] [Interface Input Packets Per Second]
[Interface Output Packets Per Second] [Interface Input Errors Per Second]
[Interface Output Errors Per Second] [Interface Input Dropped Per Second]
[Interface Output Dropped Per Second] [Interface Output Collisions]
[Interface Output Carrier Losses] [TCP Current Connections] [IP Statistics]
[TCP Statistics] [ICMP Statistics] [UDP Statistics]
[Disk System Wide Reads/Writes Per Second] [Disk System Wide Transfer Rate]
[Disk Reads/Writes Per Second] [Disk Transfer Rate] [Disk Space Percent
Usage] [Physical Memory Usage] [Swap Usage] [Page Ins & Outs Rate]
[Swap Ins & Outs Rate]
Orca on Solaris collects many more metrics than shown above
Strength of Orca is lots of detailed metrics with low overhead for collection
Easily customized to add more system metrics or application metrics
Orca can already track HTTP traffic and parse log files
69. May 14, 2014 Adrian Cockcroft and Mario Jauvin
All metrics are stored in
“round robin database” format
using RRDtool to generate
displays over different time
spans
Web page is simple collection
of plots with drill down by
metric or by time
Suitable for monitoring a
relatively small number of
systems in great detail, e.g.
backend database servers
70. May 14, 2014 Adrian Cockcroft and Mario Jauvin
Cacti – www.cacti.net
Web based user interface based on RRDtool
More sophisticated GUI than Orca or MRTG
Less sophisticated system metric collection,
but more coverage of networking
Better management of groups of systems
and devices than Orca, useful for tens to
hundreds of nodes
Access control and personalization for users
73. May 14, 2014 Adrian Cockcroft and Mario Jauvin
Ganglia – www.ganglia.info
Web based RRDtool GUI somewhat similar to Cacti
Better management of clusters of systems and
devices than Cacti, useful for hundreds to thousands
of nodes in a hierarchy of clusters
Provides many summary statistic plots at cluster
level and collects detailed configuration data
XML based data representation
Uses low overhead network protocol
In common use at hundreds of large HPC Grid sites,
less visibly in use at some large commercial sites
77. May 14, 2014 Adrian Cockcroft and Mario Jauvin
BigBrother and BigSister
Network and system dashboard alert monitor
Widely used at internet sites
Bigbrother is at http://www.bb4.com
Bigsister is at http://bigsister.graeff.com
Bigsister seems to have more features, alert
logging, better portability and more efficient
data collection. Compatible update to BB4.
81. May 14, 2014 Adrian Cockcroft and Mario Jauvin
Free QA Test and
Modelling Tools
82. May 14, 2014 Adrian Cockcroft and Mario Jauvin
QA Test Requirements
Generate test workload
– SLAMD, Grinder
Collect performance metrics
– Any of the tools already mentioned
Report regression against baseline
Predict capacity needed for production system
– Use spreadsheets for simple linear prediction
– Use modelling tools such as PDQ for queuing models
83. May 14, 2014 Adrian Cockcroft and Mario Jauvin
Grinder 3 - Powerful New Features
100% Pure Java - works on any hardware platform and any
operating system that supports J2SE 1.3 and above.
Java and Jython based load testing framework
– Web Browsers: simulate web browsers using HTTP, and HTTPS.
– Web Services: test interfaces using SOAP and XML-RPC.
– Database: test databases using JDBC.
– Middleware: RPC and MOM based systems using IIOP, RMI/IIOP,
RMI/JRMP, and JMS.
– Other Internet protocols: POP3, SMTP, FTP, and LDAP.
See http://grinder.sourceforge.net/g3/features.html
J2EE Performance Testing with BEA WebLogic Server by Peter
Zadrozny, Philip Aston and Ted Osborne, originally published
by Expert Press and now by APress uses Grinder 2 throughout.
84. May 14, 2014 Adrian Cockcroft and Mario Jauvin
SLAMD
Load generation framework, written in Java
Originally built to test LDAP servers by Sun
Extended to be very generic and published
as open source. Actively being developed.
Sophisticated functions and user interface
See http://www.slamd.com
Latest Release 2.0 has better usability focus
88. May 14, 2014 Adrian Cockcroft and Mario Jauvin
PDQ Modelling Tool
Dr Neil Gunther’s toolkit at
http://www.perfdynamics.com
Library used from C or Perl provides MVA queueing
models
Use to calibrate in QA and predict in production
PDQ modelling tool details:
– The Practical Performance Analyst Dr. Neil Gunther -
McGraw-Hill, 1998 ISBN 0-07-912946-3
– Analyzing Computer System Performance with Perl:PDQ
2004, ISBN 3-54-020865-8
89. May 14, 2014 Adrian Cockcroft and Mario Jauvin
References and
Conclusion
90. May 14, 2014 Adrian Cockcroft and Mario Jauvin
Licences for Free Tools
Open Source Initiative
– “OSI Approved licences”
– http://opensource.org/licenses/category
Comparisons of Common Licences
– http://zooko.com/license_quick_ref.html
91. May 14, 2014 Adrian Cockcroft and Mario Jauvin
Web Pages and Books
Adrian’s Performance and other topics blog
– http://perfcap.blogspot.com
MFJ Associates performance tools link page
– http://www.mfjassociates.net/perf_links.html
More free tools compiled by John Sellens
– http://www.generalconcepts.com/resources/monitoring/
More tools compiled by Openxtra
– http://www.openxtra.co.uk/resource-center/open_source_network_monitor_tools.php
SE toolkit info: Sun Performance and Tuning - Java and the Internet - Adrian
Cockcroft and Richard Pettit - Sun Press/Prentice Hall, 2nd
Edition, 1998 ISBN 0-13-
095249-4
Solaris 8 and Linux: System Performance Tuning 2nd
Edition – Gian-Paolo Musumeci,
O’Reilly 2002 ISBN: 0-596-00284-X
Solaris Internals http://www.solarisinternals.com
– Richard McDougall and James Mauro - new 2nd edition and new performance book by
Richard McDougall and Brendan Gregg
92. May 14, 2014 Adrian Cockcroft and Mario Jauvin
Concluding Remarks
Many large installations depend on free tools
A full suite of functionality is available
Several tools are needed to cover the bases
Tradeoff between function and ease of use
Support may be available, but typically
Google is the best support tool
Functionality is increasing….
93. May 14, 2014 Adrian Cockcroft and Mario Jauvin
Questions?
acockcroft@netflix.com
mario@mfjassociates.net
Editor's Notes
&quot;We reject kings, presidents, and voting;
we believe in rough consensus and running code.&quot;
-- David Clark, IAB chair, 1992
I added the SNMP-Informant and it started appearing automagically.
920804400 – Noon 7th of March, 1999
DS – data source SPEED as counter, collected every 300 seconds (defaults)
600 is heartbeat – maximum time to wait after which data is unknown
U:U means unknown minimum and maximum
RRA – round robin archive
0.5 – xfiles factor - % of unknown after which whole archive is unknown
1:24 average every 1 interval (no average) and keep 24 (2 hours worth)
6:10 everage every 6 values and keep 10
Start at noon, end at 13:00, average RRA called SPEED using 2 pixel thickness and color red #FF0000