PostgreSQL Configuration for Humans / Alvaro Hernandez (OnGres)Ontico
HighLoad++ 2017
Зал «Кейптаун», 8 ноября, 17:00
Тезисы:
http://www.highload.ru/2017/abstracts/3096.html
PostgreSQL is the world’s most advanced open source database. Indeed! With around 270 configuration parameters in postgresql.conf, plus all the knobs in pg_hba.conf, it is definitely ADVANCED!
How many parameters do you tune? 1? 8? 32? Anyone ever tuned more than 64?
No tuning means below par performance. But how to start? Which parameters to tune? What are the appropriate values? Is there a tool --not just an editor like vim or emacs-- to help users manage the 700-line postgresql.conf file?
Join this talk to understand the performance advantages of appropriately tuning your postgresql.conf file, showcase a new free tool to make PostgreSQL configuration possible for HUMANS, and learn the best practices for tuning several relevant postgresql.conf parameters.
Doker jest projektem opensource pomagającym zautomatyzować wdrażanie i uruchamianie aplikacji przy wykorzystaniu kontenerów linuksowych. Dzięki swojej wydajności oraz praktycznie zerowemu narzutowi jest coraz częściej wybierany jako alternatywa dla tradycyjnych rozwiązań wirtualizacji systemów.
Podczas tego wykładu dowiecie się czym jest docker, jak nim zarządzać oraz jak stworzyć swoje pierwsze kontenery aplikacyjne.
Spotkanie poprowadził Kamil Grabowski z Rebased
PostgreSQL Configuration for Humans / Alvaro Hernandez (OnGres)Ontico
HighLoad++ 2017
Зал «Кейптаун», 8 ноября, 17:00
Тезисы:
http://www.highload.ru/2017/abstracts/3096.html
PostgreSQL is the world’s most advanced open source database. Indeed! With around 270 configuration parameters in postgresql.conf, plus all the knobs in pg_hba.conf, it is definitely ADVANCED!
How many parameters do you tune? 1? 8? 32? Anyone ever tuned more than 64?
No tuning means below par performance. But how to start? Which parameters to tune? What are the appropriate values? Is there a tool --not just an editor like vim or emacs-- to help users manage the 700-line postgresql.conf file?
Join this talk to understand the performance advantages of appropriately tuning your postgresql.conf file, showcase a new free tool to make PostgreSQL configuration possible for HUMANS, and learn the best practices for tuning several relevant postgresql.conf parameters.
Doker jest projektem opensource pomagającym zautomatyzować wdrażanie i uruchamianie aplikacji przy wykorzystaniu kontenerów linuksowych. Dzięki swojej wydajności oraz praktycznie zerowemu narzutowi jest coraz częściej wybierany jako alternatywa dla tradycyjnych rozwiązań wirtualizacji systemów.
Podczas tego wykładu dowiecie się czym jest docker, jak nim zarządzać oraz jak stworzyć swoje pierwsze kontenery aplikacyjne.
Spotkanie poprowadził Kamil Grabowski z Rebased
OSNoise Tracer: Who Is Stealing My CPU Time?ScyllaDB
In the context of high-performance computing (HPC), the Operating System Noise (osnoise) refers to the interference experienced by an application due to activities inside the operating system. In the context of Linux, NMIs, IRQs, softirqs, and any other system thread can cause noise to the application. Moreover, hardware-related jobs can also cause noise, for example, via SMIs.
HPC users and developers that care about every microsecond stolen by the OS need not only a precise way to measure the osnoise but mainly to figure out who is stealing cpu time so that they can pursue the perfect tune of the system. These users and developers are the inspiration of Linux's osnoise tracer.
The osnoise tracer runs an in-kernel loop measuring how much time is available. It does it with preemption, softirq and IRQs enabled, thus allowing all the sources of osnoise during its execution. The osnoise tracer takes note of the entry and exit point of any source of interferences. When the noise happens without any interference from the operating system level, the tracer can safely point to a hardware-related noise. In this way, osnoise can account for any source of interference. The osnoise tracer also adds new kernel tracepoints that auxiliaries the user to point to the culprits of the noise in a precise and intuitive way.
At the end of a period, the osnoise tracer prints the sum of all noise, the max single noise, the percentage of CPU available for the thread, and the counters for the noise sources, serving as a benchmark tool.
Node.js: Whats the Big Deal? Presented and JS Meetup ChicagoAziz Ali
Presentation on Node.js. Whats the big deal with Node.js.
Talk about Event Driven, Non-Blocking, and Single Threaded Nature of Node.js
I also show the opportunity that Node.js opens up to make ALL kinds of apps on any platform including desktop applications.
Puppet Camp LA 2015 talk covering: packages, package managers, puppet, and tips, tricks, and puppet modules for setting up secure package repositories.
High-Performance Networking Using eBPF, XDP, and io_uringScyllaDB
In the networking world there are a number of ways to increase performance over naive use of basic Berkeley sockets. These techniques have ranged from polling blocking sockets, non-blocking sockets controlled by Epoll, all the way through completely bypassing the Linux kernel for maximum network performance where you talk directly to the network interface card by using something like DPDK or Netmap. All these tools have their place, and generally occupy a space from convenience to performance. But in recent years, that landscape has changed massively.. The tools available to the average Linux systems developer have improved from the creation of io_uring, to the expansion of bpf from a simple filtering language to a full-on programming environment embedded directly in the kernel. Along with that came something called XDP (express datapath). This was Linux kernel's answer to kernel-bypass networking. AF_XDP is the new socket type created by this feature, and generally works very similarly to something like DPDK. History lessons out of the way, this talk will look into, and discuss the merits of this technology, it's place in the broader ecosystem and how it can be used to attain the highest level of performance possible. This talk will dive into crucial details, such as how AF_XDP works, how it can be integrated into a larger system and finally more advanced topics such as request sharding/load balancing. There will be detailed look at the design of AF_XDP, the eBpf code used, as well as the userspace code required to drive it all. It will also include performance numbers from this setup compared to regular kernel networking. And most importantly how to put all this together to handle as much data as possible on a single modern multi-core system.
Building a REST API with Node.js and MongoDBVivochaLabs
Get the code at:
github.com/0xfede/lunch-o-clock
A quick introduction to MongoDB and Node.js and a sample application on how to use a rest API
by Federico Pinna
at Vivocha Tech Evenings 2014
"Lightweight Virtualization with Linux Containers and Docker". Jerome Petazzo...Yandex
Lightweight virtualization", also called "OS-level virtualization", is not new. On Linux it evolved from VServer to OpenVZ, and, more recently, to Linux Containers (LXC). It is not Linux-specific; on FreeBSD it's called "Jails", while on Solaris it’s "Zones". Some of those have been available for a decade and are widely used to provide VPS (Virtual Private Servers), cheaper alternatives to virtual machines or physical servers. But containers have other purposes and are increasingly popular as the core components of public and private Platform-as-a-Service (PAAS), among others.
Just like a virtual machine, a Linux Container can run (almost) anywhere. But containers have many advantages over VMs: they are lightweight and easier to manage. After operating a large-scale PAAS for a few years, dotCloud realized that with those advantages, containers could become the perfect format for software delivery, since that is how dotCloud delivers from their build system to their hosts. To make it happen everywhere, dotCloud open-sourced Docker, the next generation of the containers engine powering its PAAS. Docker has been extremely successful so far, being adopted by many projects in various fields: PAAS, of course, but also continuous integration, testing, and more.
OSNoise Tracer: Who Is Stealing My CPU Time?ScyllaDB
In the context of high-performance computing (HPC), the Operating System Noise (osnoise) refers to the interference experienced by an application due to activities inside the operating system. In the context of Linux, NMIs, IRQs, softirqs, and any other system thread can cause noise to the application. Moreover, hardware-related jobs can also cause noise, for example, via SMIs.
HPC users and developers that care about every microsecond stolen by the OS need not only a precise way to measure the osnoise but mainly to figure out who is stealing cpu time so that they can pursue the perfect tune of the system. These users and developers are the inspiration of Linux's osnoise tracer.
The osnoise tracer runs an in-kernel loop measuring how much time is available. It does it with preemption, softirq and IRQs enabled, thus allowing all the sources of osnoise during its execution. The osnoise tracer takes note of the entry and exit point of any source of interferences. When the noise happens without any interference from the operating system level, the tracer can safely point to a hardware-related noise. In this way, osnoise can account for any source of interference. The osnoise tracer also adds new kernel tracepoints that auxiliaries the user to point to the culprits of the noise in a precise and intuitive way.
At the end of a period, the osnoise tracer prints the sum of all noise, the max single noise, the percentage of CPU available for the thread, and the counters for the noise sources, serving as a benchmark tool.
Node.js: Whats the Big Deal? Presented and JS Meetup ChicagoAziz Ali
Presentation on Node.js. Whats the big deal with Node.js.
Talk about Event Driven, Non-Blocking, and Single Threaded Nature of Node.js
I also show the opportunity that Node.js opens up to make ALL kinds of apps on any platform including desktop applications.
Puppet Camp LA 2015 talk covering: packages, package managers, puppet, and tips, tricks, and puppet modules for setting up secure package repositories.
High-Performance Networking Using eBPF, XDP, and io_uringScyllaDB
In the networking world there are a number of ways to increase performance over naive use of basic Berkeley sockets. These techniques have ranged from polling blocking sockets, non-blocking sockets controlled by Epoll, all the way through completely bypassing the Linux kernel for maximum network performance where you talk directly to the network interface card by using something like DPDK or Netmap. All these tools have their place, and generally occupy a space from convenience to performance. But in recent years, that landscape has changed massively.. The tools available to the average Linux systems developer have improved from the creation of io_uring, to the expansion of bpf from a simple filtering language to a full-on programming environment embedded directly in the kernel. Along with that came something called XDP (express datapath). This was Linux kernel's answer to kernel-bypass networking. AF_XDP is the new socket type created by this feature, and generally works very similarly to something like DPDK. History lessons out of the way, this talk will look into, and discuss the merits of this technology, it's place in the broader ecosystem and how it can be used to attain the highest level of performance possible. This talk will dive into crucial details, such as how AF_XDP works, how it can be integrated into a larger system and finally more advanced topics such as request sharding/load balancing. There will be detailed look at the design of AF_XDP, the eBpf code used, as well as the userspace code required to drive it all. It will also include performance numbers from this setup compared to regular kernel networking. And most importantly how to put all this together to handle as much data as possible on a single modern multi-core system.
Building a REST API with Node.js and MongoDBVivochaLabs
Get the code at:
github.com/0xfede/lunch-o-clock
A quick introduction to MongoDB and Node.js and a sample application on how to use a rest API
by Federico Pinna
at Vivocha Tech Evenings 2014
"Lightweight Virtualization with Linux Containers and Docker". Jerome Petazzo...Yandex
Lightweight virtualization", also called "OS-level virtualization", is not new. On Linux it evolved from VServer to OpenVZ, and, more recently, to Linux Containers (LXC). It is not Linux-specific; on FreeBSD it's called "Jails", while on Solaris it’s "Zones". Some of those have been available for a decade and are widely used to provide VPS (Virtual Private Servers), cheaper alternatives to virtual machines or physical servers. But containers have other purposes and are increasingly popular as the core components of public and private Platform-as-a-Service (PAAS), among others.
Just like a virtual machine, a Linux Container can run (almost) anywhere. But containers have many advantages over VMs: they are lightweight and easier to manage. After operating a large-scale PAAS for a few years, dotCloud realized that with those advantages, containers could become the perfect format for software delivery, since that is how dotCloud delivers from their build system to their hosts. To make it happen everywhere, dotCloud open-sourced Docker, the next generation of the containers engine powering its PAAS. Docker has been extremely successful so far, being adopted by many projects in various fields: PAAS, of course, but also continuous integration, testing, and more.
NoSQL — это слово громко "жужжит".
К сожалению, оно при этом ничего не означает. Это не продукт, не технология, и даже не концепция. Это даже не подход к проектированию. Это, скорее, декларация отказа от некоторых паттернов проектирования, господствовавших в разработке клиент-серверных систем долгие годы.
На этом доклад можно было бы и закончить. Если бы мы не знали достоверно, что на свете есть люди, которые умудряются извлекать прибыль, используя NoSQL в своих проектах. Ну или сокращать убытки, по крайней мере.
Попробуем еще раз.
NoSQL — это именно декларация отказа от некоторых паттернов.
- От чего именно придется отказаться? Упомянутые паттерны так живучи совсем не случайно.
- Как это ударит по проекту? Не сомневайтесь, оно ударит, в этом мире нет ни серебряных пуль, ни бесплатного сыра.
- Какими свойствами должен обладать проект, чтобы внедрение NoSQL СУБД принесло ему пользу? Избегать NoSQL — это не трусость, это осторожность.
- Каковы сильные стороны NoSQL СУБД, и в чем профит? Выбор NoSQL — это всегда выбор в пользу меньшего зла.
- Как выбрать NoSQL СУБД под свою задачу? На http://nosql-database.org/ есть список LIST OF NOSQL DATABASES [currently >225], и даже просто прочесть его — тяжелая работа.
- Почему реальный выбор NoSQL СУБД — это выбор между Aerospike и Cassandra? Да, это провокационный вопрос, но на него есть not-so-provocative ответ.
- С какими проблемами сталкиваются разработчики и администраторы при эксплуатации "тяжелой" NoSQL базы? К сожалению, большая часть этих проблем создается именно присутствием NoSQL.
- Что можно делать с NoSQL СУБД и чего нельзя? На какие параметры производительности и отказоустойчивости можно рассчитывать? В чем особенности выбора "железа" для NoSQL?
- И в чем, все-таки, profit?
На эти и некоторые другие, связанные с этими, вопросы автор намерен дать ответ в своем докладе.
My talk on Hadoop stack operations engineering at OSPConAlex Chistyakov
My talk on Hadoop stack operations engineering at OSPCon Nov 2015 (http://www.ospcon.ru/event/prakticheskaya-konferentsiya-tekhnologii-bolshikh-dannykh_130.html)
This talk will try to take you into thinking about your technical reasoning for scaling on the first 18 months of your startup, some things are hard to get right and we hope you learn from our experience!
All Your IOPS Are Belong To Us - A Pinteresting Case Study in MySQL Performan...Ernie Souhrada
Have you ever thought that your SSD storage just doesn't seem anywhere near as fast as advertised? What if I told you that a couple of small changes to your Linux servers had the potential to more than double the amount of throughput your MySQL servers could handle while simultaneously reducing query response time by around 50 percent? It's not a dream, Neo.
If you're still in a traditional data center and/or your MySQL footprint lives on spinning rust, then you might want to take the blue pill, attend a different session, and believe whatever you want to believe.
However, if your server footprint runs inside Amazon Web Services or if you use SSDs in any fashion, I invite you to take the red pill and come with me on a journey down the rabbit hole, where we'll discuss such things as the Linux block IO driver, CPU starvation, and interrupt handling, using real-world query performance data from the primary MySQL data stores at Pinterest to illustrate the effect.
Kernel Recipes 2016 - Speeding up development by setting up a kernel build farmAnne Nicolas
Building a full kernel takes time but is often necessary during development or when backporting patches. The nature of the kernel makes it easy to distribute its build on multiple cheap machines. This presentation will explain how to set up a build farm based on cost, size, and performance.
Willy Tarreau, HaProxy
What I learned building a parallel processor from scratchAndreas Olofsson
Title: What I learned building a parallel processor from scratch
Presenter: Andreas Olofsson
Location: ISPASS 2015 Keynote
Date: March
Abstract:
In 2008 I left my long time employer (Analog Devices) to start a company with a mission to build a new type of parallel computer. This talk will present the inspiration, design philosophy, mistakes, success, iterations and surprises encountered in designing the Epiphany computer architecture, four generations of Epiphany chips, and the $99 Parallella credit card sized “supercomputer” project.
strangeloop 2012 apache cassandra anti patternsMatthew Dennis
random list of Apache Cassndra Anti Patterns. There is a lot of info on what to use Cassandra for and how, but not a lot of information on what not to do. This presentation works towards filling that gap.
Working with software means working with bugs. Bugs in software, bugs in hardware; bugs in Open Source code, bugs in proprietary code. If software is eating the world, bugs might end up taking the first bite.
We will present a few typical bugs, some of them famous, some of them infamous (including bugs that actually killed people). Since one can never be too well-prepared to fend off the next infestation, we will give tools, tips, and best practices to fix bugs in Open Source software. We will give real world examples of Really Mysterious Bugs (sometimes nicknamed "Heisenbugs" because they tend to disappear when you try to observe them), and how they were fixed, in Node.js, Docker, and the Linux Kernel.
Kernel Recipes 2014 - Performance Does MatterAnne Nicolas
Deploying clouds is in everybody’s mind but how to make an efficient deployment ?
After setting up the hardware, it’s mandatory to make a deep inspection of server’s performance.
In a farm of supposed identical servers, many miss-{installation|configuration} could seriously degrade performance. If you want to discovery such counter-performance before users complains of their VMs, you have to be detect them before installing any software. Another performance metric to know is “how many VMs could I load on top of my servers ?”. By using the same methodology it is possible the compare how a set of VMs performs regarding the bare metal capabilities.
The challenge is here: How do detect automatically servers that under perform ? How to insure that a new server entering a farm will not degrade it ? How to measure the overhead of all the virtualization layers from the VM point of view ?
Erwan Velu – Performance Engineer @eNovance
AWS Big Data Demystified #1: Big data architecture lessons learned Omid Vahdaty
AWS Big Data Demystified #1: Big data architecture lessons learned . a quick overview of a big data techonoligies, which were selected and disregard in our company
The video: https://youtu.be/l5KmaZNQxaU
dont forget to subcribe to the youtube channel
The website: https://amazon-aws-big-data-demystified.ninja/
The meetup : https://www.meetup.com/AWS-Big-Data-Demystified/
The facebook group : https://www.facebook.com/Amazon-AWS-Big-Data-Demystified-1832900280345700/
Video and slides synchronized, mp3 and slide download available at URL http://bit.ly/2lGNybu.
Stefan Krawczyk discusses how his team at StitchFix use the cloud to enable over 80 data scientists to be productive. He also talks about prototyping ideas, algorithms and analyses, how they set up & keep schemas in sync between Hive, Presto, Redshift & Spark and make access easy for their data scientists, etc. Filmed at qconsf.com..
Stefan Krawczyk is Algo Dev Platform Lead at StitchFix, where he’s leading development of the algorithm development platform. He spent formative years at Stanford, LinkedIn, Nextdoor & Idibon, working on everything from growth engineering, product engineering, data engineering, to recommendation systems, NLP, data science and business intelligence.
The Dark Side Of Go -- Go runtime related problems in TiDB in productionPingCAP
Ed Huang, CTO of PingCAP, talked at Go System Conference about dealing with the typical and profound issues related to Go’s runtime as your systems become more complex. Taking TiDB as an example, he demonstrated how these problems can be reproduced, located, and analyzed in production.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
Generating a custom Ruby SDK for your web service or Rails API using Smithyg2nightmarescribd
Have you ever wanted a Ruby client API to communicate with your web service? Smithy is a protocol-agnostic language for defining services and SDKs. Smithy Ruby is an implementation of Smithy that generates a Ruby SDK using a Smithy model. In this talk, we will explore Smithy and Smithy Ruby to learn how to generate custom feature-rich SDKs that can communicate with any web service, such as a Rails JSON API.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
Securing your Kubernetes cluster_ a step-by-step guide to success !KatiaHIMEUR1
Today, after several years of existence, an extremely active community and an ultra-dynamic ecosystem, Kubernetes has established itself as the de facto standard in container orchestration. Thanks to a wide range of managed services, it has never been so easy to set up a ready-to-use Kubernetes cluster.
However, this ease of use means that the subject of security in Kubernetes is often left for later, or even neglected. This exposes companies to significant risks.
In this talk, I'll show you step-by-step how to secure your Kubernetes cluster for greater peace of mind and reliability.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
UiPath Test Automation using UiPath Test Suite series, part 3DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 3. In this session, we will cover desktop automation along with UI automation.
Topics covered:
UI automation Introduction,
UI automation Sample
Desktop automation flow
Pradeep Chinnala, Senior Consultant Automation Developer @WonderBotz and UiPath MVP
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...
My talk at Linux Piter 2015
1. Performance optimization in Linux
Tales from the trenches
Alex Chistyakov, Principal Engineer, Git in Sky
Linux Piter 2015
2. Who are we?
●
A small consulting company based in SPb.
●
Web operations
●
Automation
●
Performance tuning
●
Sponsors of local DevOps meetup
3. Who are you?
●
Linux fans?
●
Developers?
●
Web developers, maybe?
●
System architects?
●
Performance engineers?
4. Okay, why Linux?
●
Is there anything else?
●
According to W3Cook stats, Linux
serves 95.8% of public web sites
●
And it's on the desktop too!
●
(At least on my desktop)
5. Linux in the perfect world
●
Linux fans?
●
Developers?
●
Web developers, maybe?
●
System architects?
●
Performance engineers?
6. Linux in the real world
●
Linux fans?
●
Developers?
●
Web developers, maybe?
●
System architects?
●
Performance engineers?
7. 504 on the main page!
●
A customer is stressed extremely
●
Reaction should be quick and effective
●
The obvious plan does not work
●
We should be prepared!
8. The obvious plan
●
1) Change something
●
2) Expect the situation to become better
●
3) Wait anxiously
●
4) ????
●
5) PROFIT!!!
●
This plan is quite popular in fact
for some reason (simplicity?)
9. The proper plan (top secret!)
●
1) Gather metrics (you have them already, don't
you?)
●
2) Analyze metrics
●
3) Elaborate a hypothesis
●
4) Plan and make a single change
●
5) Repeat until success
●
If you were proficient in physics at high
school, this plan should sound extremely familiar
11. How do we collect/analyze?
●
atop (30 sec intervals)
12. How do we collect/analyze?
●
atop (30 sec intervals)
●
Ex-graphite stack (Grafana, InfluxDB or OpenTSDB
or Cyanite/Cassandra, but NOT Whisper, collectd)
13. How do we collect/analyze?
●
atop (30 sec intervals)
●
Ex-graphite stack (Grafana, InfluxDB or OpenTSDB
or Cyanite/Cassandra, but NOT Whisper, collectd)
●
NewRelic
14. How do we collect/analyze?
●
atop (30 sec intervals)
●
Ex-graphite stack (Grafana, InfluxDB or OpenTSDB
or Cyanite/Cassandra, but NOT Whisper, collectd)
●
NewRelic
●
pidstat (not iotop)
15. How do we collect/analyze?
●
atop (30 sec intervals)
●
Ex-graphite stack (Grafana, InfluxDB or OpenTSDB
or Cyanite/Cassandra, but NOT Whisper, collectd)
●
NewRelic
●
pidstat (not iotop)
●
perf top and perf record
16. How do we collect/analyze?
●
atop (30 sec intervals)
●
Ex-graphite stack (Grafana, InfluxDB or OpenTSDB
or Cyanite/Cassandra, but NOT Whisper, collectd)
●
NewRelic
●
pidstat (not iotop)
●
perf top and perf record
●
sysdig
17. How do we collect/analyze?
●
atop (30 sec intervals)
●
Ex-graphite stack (Grafana, InfluxDB or OpenTSDB
or Cyanite/Cassandra, but NOT Whisper, collectd)
●
NewRelic
●
pidstat (not iotop)
●
perf top and perf record
●
sysdig
●
iostat -x 1
18. Most common case (a no-brainer)
●
CPU time is too high (a lucky customer got an SSD)
●
Disk saturation is above 60% (a not-so-lucky customer)
19. Most common case (a no-brainer)
●
CPU time is too high (a lucky customer got an SSD)
●
Disk saturation is above 60% (a not-so-lucky customer)
●
PHP and, of course, MySQL
20. Most common case (a no-brainer)
●
CPU time is too high (a lucky customer got an SSD)
●
Disk saturation is above 60% (a not-so-lucky customer)
●
PHP and, of course, MySQL
●
InnoDB buffers are too low
●
Synchronous commit is 'on'
21. Most common case (a no-brainer)
●
CPU time is too high (a lucky customer got an SSD)
●
Disk saturation is above 60% (a not-so-lucky customer)
●
PHP and, of course, MySQL
●
InnoDB buffers are too low
●
Synchronous commit is 'on'
●
Too many slow queries
●
Queries with 'filesort' in execution plan
22. How to solve
●
Install Anemometer, turn on slow queries log
●
Range queries based on their cumulative exec time
●
Read and understand execution plans
●
Blame developers
●
Cry in vain
23. Story time!
●
Case #1: measuring the measurer
●
It seems that everybody still uses Graphite/Whisper
24. Story time!
●
Case #1: measuring the measurer
●
It seems that everybody still uses Graphite/Whisper
●
Even the big guys like Mail.Ru
25. Story time!
●
Case #1: measuring the measurer
●
It seems that everybody still uses Graphite/Whisper
●
Even the big guys like Mail.Ru
●
Well, because SAS HDDs are cheap...
26. Story time!
●
Case #1: measuring the measurer
●
It seems that everybody still uses Graphite/Whisper
●
Even the big guys like Mail.Ru
●
Well, because SAS HDDs are cheap…
●
But...
27. A challenger appears!
●
InfluxDB vs. Whisper, July 2015
●
The same set of metrics (carbon-relay-ng in the middle)
●
And the winner is...
28. Okay, we hate magic
●
Whisper is just a set of RRD-like files on a plain old FS
●
20000 metrics lead you to 20000 files
●
Accessing 20000 files every 10 secs is a major pain
●
InfluxDB is a time series database based on an LSM-tree
●
InfluxDB is much more write-optimized than 20000 separate
files on your ext4/XFS/you-name-it
●
But, of course, SAS drives are quite cheap...
29. In case you are not scared
●
Case #2: the site got a SUDS (sudden unexpected death
syndrome)
30. In case you are not scared
●
Case #2: the site got a SUDS (sudden unexpected death
syndrome)
●
Symptoms: everything slows down to a crawl (sounds familiar)
31. In case you are not scared
●
Case #2: the site got a SUDS (sudden unexpected death
syndrome)
●
Symptoms: everything slows down to a crawl (sounds familiar)
●
NewRelic shows nothing unusual
32. In case you are not scared
●
Case #2: the site got a SUDS (sudden unexpected death
syndrome)
●
Symptoms: everything slows down to a crawl (sounds familiar)
●
NewRelic shows nothing unusual
●
Well, if parameters more suitable for a busy site that for a very
low traffic one can be called “nothing unusual”
●
And this site is not busy at all
33. Diagnostic card
●
PHP is OK
●
MySQL does not sort anything
●
Top queries in MySQL sorted by total exec time are all indexed
●
Every MySQL query runs very slow when there is even
moderate load
34. But how did we solve it?
●
Even a modern rig w/decent Xeons and SATA HDDs can be
turned into a slug
●
As simple as disabling AHCI in BIOS and staying on plain IDE
●
Well this one was not that hard but was quite unusual
●
Rented servers do not suffer from problems like this because
they are configured uniformly
●
I can't easily explain how I came upon this solution, pure
intuition seemed to be involved
37. Diagnostic card
●
NewRelic blames PHP code
●
Even the SSH console is slow
●
Nothing unusual or unexpected in daily CPU load graphs
●
CPU flamegraph shows nothing
39. How did we solve it
●
Analyzed atop recorded stats for outage periods
●
atop is quite smart in fact and color suspected values in red or
blue
●
IRQ % is over 50%
40. How did we solve it
●
Analyzed atop recorded stats for outage periods
●
atop is quite smart in fact and color suspected values in red or
blue
●
IRQ % is over 50%
●
But what is “IRQ %” anyway?
●
Oh, who cares, let's install Munin and get per-interrupt graphs
41. A blast from the past
●
Analyzed atop recorded stats for outage periods
●
atop is quite smart in fact and color suspected values in red or
blue
●
IRQ % is over 50%
●
But what is “IRQ %” anyway?
●
Oh, who cares, let's install Munin and get per-interrupt graphs
42. How did we solve it
●
Well we have not solved it yet
●
The graph from previous slide is for past two days
●
But at least we have a plan!
●
https://help.ubuntu.com/community/ReschedulingInterrupts
44. Thank you!
●
Questions?
●
Oh, BTW you can hire us!
●
http://gitinsky.com
●
alex@gitinsky.com
●
Please do not forget to attend our meetups:
●
http://meetup.com/Docker-Spb, http://meetup.com/Ansible-Spb,
http://meetup.com/DevOps-40