This document discusses PostgreSQL and Solaris as a low-cost platform for medium to large scale critical scenarios. It provides an overview of PostgreSQL, highlighting features like MVCC, PITR, and ACID compliance. It describes how Solaris and PostgreSQL integrate well, with benefits like DTrace support, scalability on multicore/multiprocessor systems, and Solaris Cluster support. Examples are given for installing PostgreSQL on Solaris using different methods, configuring zones for isolation, using ZFS for storage, and monitoring performance with DTrace scripts.
Migrating KSM page causes the VM lock up as the KSM page merging list is too ...Gavin Guo
Topic: Migrating KSM page causes the VM lock up as the KSM page merging list is too large - 2019 OSS China Shanghai
https://sched.co/NruN
- Description
A classic example of kdump related to KSM/khugepaged/NUMA balance/KVM in server/cloud environment.
- Youtube Chinese Presentation
https://youtu.be/oEtkEntScd0
- Speaker: Gavin Guo, Canonical
Technical Lead - Sustaining Engineering
Taipei, Taiwan
Gavin Guo is a Linux kernel developer in the Ubuntu community. He is the speaker of Spectre v2 Internal in 2018 China L3C and KASan debugging in 2016 China Linux Kernel Conference. He is now working for Canonical in the Customer Success division. He is responsible for the kernel stability and performance tuning of the OpenStack platform, especially in the NUMA(Nonuniform Memory Access), Page Reclaim, and SLUB allocator. He is also the one who introduces KASAN into the team to investigate kernel issues on OpenStack platform and that ended a lot of nightmares.
Migrating KSM page causes the VM lock up as the KSM page merging list is too ...Gavin Guo
Topic: Migrating KSM page causes the VM lock up as the KSM page merging list is too large - 2019 OSS China Shanghai
https://sched.co/NruN
- Description
A classic example of kdump related to KSM/khugepaged/NUMA balance/KVM in server/cloud environment.
- Youtube Chinese Presentation
https://youtu.be/oEtkEntScd0
- Speaker: Gavin Guo, Canonical
Technical Lead - Sustaining Engineering
Taipei, Taiwan
Gavin Guo is a Linux kernel developer in the Ubuntu community. He is the speaker of Spectre v2 Internal in 2018 China L3C and KASan debugging in 2016 China Linux Kernel Conference. He is now working for Canonical in the Customer Success division. He is responsible for the kernel stability and performance tuning of the OpenStack platform, especially in the NUMA(Nonuniform Memory Access), Page Reclaim, and SLUB allocator. He is also the one who introduces KASAN into the team to investigate kernel issues on OpenStack platform and that ended a lot of nightmares.
seccomp is a computer security facility in the Linux kernel, pledge is a similar security facility in the OpenBSD kernel. In this presentation Giovanni Bechis will review the development story and progress of both kernel interfaces and will analyze the main differences. There will be some examples of implementations of security patches made for some important open source projects.
Presentation from SCALE 17x (https://www.socallinuxexpo.org/scale/17x/presentations/fast-http-string-processing-algorithms):
There are binary optimizations in HTTP/2, so the protocol becomes less about string processing. However, strings, sometimes quite large like URI or Cookie, stil exists in HTTP. A typical program working with HTTP, must perform various string operations, e.g. tokenization, string matching, searching for a pattern etc. Classic computer science describe many string processing algorithms, but HTTP strings are special and specialized algorithms can improve performance of the strings processing in several times.
This talk describes:
* How HTTP flood may make you HTTP parser the bottleneck x86-64 issues with branch mispredictions, caching and unaligned memory access
* C compiler optimizations for multi-branch statements and autovectorization
* switch-driven finite state machines (FSM) versus direct jumps (e.g. Ragel)
* what makes HTTP strings special and why LIBC functions aren't good
* strspn()- and strcasecmp()-like algorithms for HTTP strings using SSE and AVX
* efficient custom filtering to prevent injection attacks using AVX
* the cost of FPU context switch and how the Linux kernel works with SIMD
* all the topics are illustrated with microbenchmarks
Практический опыт профайлинга и оптимизации производительности Ruby-приложенийOlga Lavrentieva
Алексей Туля, Senior Software Developer в Sam Solutions
«Практический опыт профайлинга и оптимизации производительности Ruby-приложений»
В своем докладе Алексей сделает краткий обзор различных реализаций Ruby, попытается найти причины, почему Ruby медленный. Рассмотрит вопрос сборки мусора в Ruby и вызова методов – почему в Ruby это дорого. Расскажет и покажет, что делать, чтобы поднять производительность, проведет обзор утилит для поиска проблемных мест, обзор профайлеров и расскажет, как интерпретировать результаты.
Доклад в основном нацелен на практический подход по поиску проблем. Материал предназначен для пользователей Linux, поэтому все практические советы будут для ОС Linux.
seccomp is a computer security facility in the Linux kernel, pledge is a similar security facility in the OpenBSD kernel. In this presentation Giovanni Bechis will review the development story and progress of both kernel interfaces and will analyze the main differences. There will be some examples of implementations of security patches made for some important open source projects.
Presentation from SCALE 17x (https://www.socallinuxexpo.org/scale/17x/presentations/fast-http-string-processing-algorithms):
There are binary optimizations in HTTP/2, so the protocol becomes less about string processing. However, strings, sometimes quite large like URI or Cookie, stil exists in HTTP. A typical program working with HTTP, must perform various string operations, e.g. tokenization, string matching, searching for a pattern etc. Classic computer science describe many string processing algorithms, but HTTP strings are special and specialized algorithms can improve performance of the strings processing in several times.
This talk describes:
* How HTTP flood may make you HTTP parser the bottleneck x86-64 issues with branch mispredictions, caching and unaligned memory access
* C compiler optimizations for multi-branch statements and autovectorization
* switch-driven finite state machines (FSM) versus direct jumps (e.g. Ragel)
* what makes HTTP strings special and why LIBC functions aren't good
* strspn()- and strcasecmp()-like algorithms for HTTP strings using SSE and AVX
* efficient custom filtering to prevent injection attacks using AVX
* the cost of FPU context switch and how the Linux kernel works with SIMD
* all the topics are illustrated with microbenchmarks
Практический опыт профайлинга и оптимизации производительности Ruby-приложенийOlga Lavrentieva
Алексей Туля, Senior Software Developer в Sam Solutions
«Практический опыт профайлинга и оптимизации производительности Ruby-приложений»
В своем докладе Алексей сделает краткий обзор различных реализаций Ruby, попытается найти причины, почему Ruby медленный. Рассмотрит вопрос сборки мусора в Ruby и вызова методов – почему в Ruby это дорого. Расскажет и покажет, что делать, чтобы поднять производительность, проведет обзор утилит для поиска проблемных мест, обзор профайлеров и расскажет, как интерпретировать результаты.
Доклад в основном нацелен на практический подход по поиску проблем. Материал предназначен для пользователей Linux, поэтому все практические советы будут для ОС Linux.
Artificial intelligence (AI) is everywhere, promising self-driving cars, medical breakthroughs, and new ways of working. But how do you separate hype from reality? How can your company apply AI to solve real business problems?
Here’s what AI learnings your business should keep in mind for 2017.
Study: The Future of VR, AR and Self-Driving CarsLinkedIn
We asked LinkedIn members worldwide about their levels of interest in the latest wave of technology: whether they’re using wearables, and whether they intend to buy self-driving cars and VR headsets as they become available. We asked them too about their attitudes to technology and to the growing role of Artificial Intelligence (AI) in the devices that they use. The answers were fascinating – and in many cases, surprising.
This SlideShare explores the full results of this study, including detailed market-by-market breakdowns of intention levels for each technology – and how attitudes change with age, location and seniority level. If you’re marketing a tech brand – or planning to use VR and wearables to reach a professional audience – then these are insights you won’t want to miss.
CONFidence 2015: DTrace + OSX = Fun - Andrzej Dyjak PROIDEA
Speaker: Andrzej Dyjak
Language: English
In recent years security industry started to grow fond of Apple’s iOS and OS X platforms. This talk will cover one of XNU's flagship debugging utilities: DTrace, a dynamic tracing framework for troubleshooting kernel and application problems on production systems in real time. It will be shown how it can be used in order to ease various tasks within the realm of dynamic binary analysis and beyond.
CONFidence: http://confidence.org.pl/
Brief introduction to DTrace technologies within OpenSolaris/Solaris 10 and DTrace probes within Apache, PHP and MySQL can provide end to end dynamic tracing of your Drupal based web site..
Como analisar planos de execução e estatísticas no PostgreSQL.
- Rastreamento de consultas lentas
- Uso do EXPLAIN
- Métodos de acesso
- Junções
- Parâmetros relevantes para o otimizador
fog or: How I Learned to Stop Worrying and Love the Cloud (OpenStack Edition)Wesley Beary
Cloud computing scared the crap out of me - the quirks and nightmares of provisioning cloud computing, dns, storage, ... on AWS, Terremark, Rackspace, ... - I mean, where do you even start?
Since I couldn't find a good answer, I undertook the (probably insane) task of creating one. fog gives you a place to start by creating abstractions that work across many different providers, greatly reducing the barrier to entry (and the cost of switching later). The abstractions are built on top of solid wrappers for each api. So if the high level stuff doesn't cut it you can dig in and get the job done. On top of that, mocks are available to simulate what clouds will do for development and testing (saving you time and money).
You'll get a whirlwind tour of basic through advanced as we create the building blocks of a highly distributed (multi-cloud) system with some simple Ruby scripts that work nearly verbatim from provider to provider. Get your feet wet working with cloud resources or just make it easier on yourself as your usage gets more complex, either way fog makes it easy to get what you need from the cloud.
The OpenStack Edition adds my concerns about OpenStack API development, including things that have already been fixed and things that we haven't yet encountered. Hopefully this consumer perspective can help shed light on some rough spots.
Salesforce at Stacki Atlanta Meetup February 2016StackIQ
Dave Peterson's presentation on how Salesforce uses Stacki and Chef to provision and manage thousands of servers. Stacki Atlanta kickoff Meetup on 2/23/16 at the Microsoft Innovation Center. Dave is a Lead Systems Engineer at Salesforce.
fog or: How I Learned to Stop Worrying and Love the CloudWesley Beary
Learn how to easily get started on cloud computing with fog. If you can control your infrastructure choices, you’ll make better choices in development and get what you need in production. You'll get an overview of fog and concrete examples to give you a head start on your provisioning workflow.
Nowadays, scaling and auto-scaling have become relatively easy tasks. Everyone knows how to set up auto-scaling environments - Auto-Scaling groups, Swarm, Kubernetes, etc.
But when we try to scale I/O Bound workloads:
- Message queues (Kafka, Rabbit, NATS)
- Distributed databases (Hadoop, Cassandra)
- Storage subsystems (CEPH, GlusterFS, HDFS),
the traditional auto-scaling mechanisms are just not enough.
Heavy calculations must be performed to determine the I/O bottlenecks. Rebalancing the data after a scaling event can take up to hours depending on your data & could, resulting in data loss if not properly designed.
We will deep dive into this type of workload and walk you through code samples you can apply in your own environment.
Video games are written as a main loop: process player input, update the state of the game, render a new frame to the screen, repeat. They do this 60 times a second, with millisecond timing. Most monitoring tools are also written as loops: send a probe, wait for the response, update a data store, sleep. Often this is done pretty slowly, maybe once a second! In video games if you can’t update fast enough, you skip the rendering step and the frame rate drops. With monitoring tools if your loop takes to long you also stop logging data as often, and instead of choppy gameplay you get gaps in your graphs, often when you need that data the most!
Let’s use ping as an example and see how we can rewrite its main loop to function more like a video game, keeping a high frame rate.
DevOps and Testing slides at DASA ConnectKari Kakkonen
My and Rik Marselis slides at 30.5.2024 DASA Connect conference. We discuss about what is testing, then what is agile testing and finally what is Testing in DevOps. Finally we had lovely workshop with the participants trying to find out different ways to think about quality and testing in different parts of the DevOps infinity loop.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
Key Trends Shaping the Future of Infrastructure.pdfCheryl Hung
Keynote at DIGIT West Expo, Glasgow on 29 May 2024.
Cheryl Hung, ochery.com
Sr Director, Infrastructure Ecosystem, Arm.
The key trends across hardware, cloud and open-source; exploring how these areas are likely to mature and develop over the short and long-term, and then considering how organisations can position themselves to adapt and thrive.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
Neuro-symbolic is not enough, we need neuro-*semantic*Frank van Harmelen
Neuro-symbolic (NeSy) AI is on the rise. However, simply machine learning on just any symbolic structure is not sufficient to really harvest the gains of NeSy. These will only be gained when the symbolic structures have an actual semantics. I give an operational definition of semantics as “predictable inference”.
All of this illustrated with link prediction over knowledge graphs, but the argument is general.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
JMeter webinar - integration with InfluxDB and GrafanaRTTS
Watch this recorded webinar about real-time monitoring of application performance. See how to integrate Apache JMeter, the open-source leader in performance testing, with InfluxDB, the open-source time-series database, and Grafana, the open-source analytics and visualization application.
In this webinar, we will review the benefits of leveraging InfluxDB and Grafana when executing load tests and demonstrate how these tools are used to visualize performance metrics.
Length: 30 minutes
Session Overview
-------------------------------------------
During this webinar, we will cover the following topics while demonstrating the integrations of JMeter, InfluxDB and Grafana:
- What out-of-the-box solutions are available for real-time monitoring JMeter tests?
- What are the benefits of integrating InfluxDB and Grafana into the load testing stack?
- Which features are provided by Grafana?
- Demonstration of InfluxDB and Grafana using a practice web application
To view the webinar recording, go to:
https://www.rttsweb.com/jmeter-integration-webinar
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
AI for Every Business: Unlocking Your Product's Universal Potential by VP of ...
Osol Pgsql
1. ¿Open Solaris y
Postgresql?
Por lo menos, tenés que escucharlos...
Emanuel Calvo Franco @ AOSUG
2. “Postgresql y Solaris conforman una
plataforma de bajo costo para escenarios
críticos de mediana a gran escala.”
3. ¿ Que es Postgresql ?
Base de Datos Objeto Relacional.
Arquitectura Cliente -Servidor.
Estable, capaz de correr con escasos recursos,
segura y extensible.
5. Solaris + Pgsql
Soporte Dtrace.
Escalabilidad para multinucleos y procesadores.
Solaris Cluster support and the PostgreSQL HA Agent
Solaris Service Manager integration.
11. Más beneficios con Solaris
• Sun Cluster support
> Sun Cluster agent available
> Fully supported
• ZFS + Zones for Hot Standby
> ZFS cloning & replication lets you “clone” your database
> Zones allows you to “clone” your DB server
> OmniTI is using this technique to support a 6TB data warehouse
on PostgreSQL
12. Zonas ... + seguridad
• Zones are Perfect for “hosted” PostgreSQL
> allows 100% isolation of PostgreSQL instances per
customer/application
> set CPU & RAM limits for each Zone (in SXDE)
> PostgreSQL can't do this natively
> no per-user limits, no ability to hide the system tables
> Being used by Joyent.com to host 100's of customers.
15. Instalar II
emanuel@opensolaris_test:~$ pfexec su - postgres
Sun Microsystems Inc. SunOS 5.11 snv_101b November 2008
-bash-3.2$ ls
bin data84 include lib share
-bash-3.2$ bin/post
postgres postmaster
-bash-3.2$ bin/pg_ctl -D data84/ start
server starting
-bash-3.2$ LOG: database system was shut down at 2009-05-18 18:18:51 ART
LOG: autovacuum launcher started
LOG: database system is ready to accept connections
16. Instalar (usando Sun Studio -
Homo Paquetus ® )
# pkg install SUNWpostgr83server && pkg install SUNWpostgr83client
(ultimo en el repo SUNWpostgr-83-server@8.3.4-0.101)
- Levantar el servicio:
# svcadm enable svc:/application/database/postgresql_83:default_32bit
# /usr/sbin/svcadm enable postgresql:version_82
20. Very simple tricks
# zfs create -b 8192 rpool/data
# zfs list
# zfs set mountpoint=data84 rpool/data84
# zfs set mountpoint=$PWD/data84 rpool/data84
# zfs set compression=on rpool/data84
# zfs list
# zfs set quota=1g rpool/data84
# zfs set reservation=100m rpool/data84
# zfs set sharenfs=rw rpool/data84
# zpool list
# zfs list
25. Salida
******** Transactions Per Second *********
Txn Type Count
==========================================
New 192
Commit 192
Abort 1
******** Transactions Per Second *********
Txn Type Count
==========================================
New 175
Commit 172
Abort 0
DELTA: select now(),sum(n_tup_ins) as n_tup_ins,sum(n_tup_upd) as n_tup_upd,sum(n_tup_del) as n_tup_del from
pg_stat_user_tables;