Want to make rsyslog talk to some not-yet-supported output? With v8 it's extremly simple.This presentation contains everything you need to know about writing great plugins in ANY language (like perl or pyton).
Spencer Christensen
There are many aspects to managing an RDBMS. Some of these are handled by an experienced DBA, but there are a good many things that any sys admin should be able to take care of if they know what to look for.
This presentation will cover basics of managing Postgres, including creating database clusters, overview of configuration, and logging. We will also look at tools to help monitor Postgres and keep an eye on what is going on. Some of the tools we will review are:
* pgtop
* pg_top
* pgfouine
* check_postgres.pl.
Check_postgres.pl is a great tool that can plug into your Nagios or Cacti monitoring systems, giving you even better visibility into your databases.
Learn about structured logging with rsyslog and how it can be used to do actual format conversions. Include config samples for Linux and Windows log sources.
Meta/Facebook's database serving social workloads is running on top of MyRocks (MySQL on RocksDB). This means our performance and reliability depends a lot on RocksDB. Not just MyRocks, but also we have other important systems running on top of RocksDB. We have learned many lessons from operating and debugging RocksDB at scale.
In this session, we will offer an overview of RocksDB, key differences from InnoDB, and share a few interesting lessons learned from production.
ELK (Elasticsearch, Logstash, Kibana) is an open source toolset for centralized logging, where Logstash collects, parses, and filters logs, Elasticsearch stores and indexes logs for search, and Kibana visualizes logs. Logstash processes logs through an input, filter, output pipeline using plugins. It can interpret various log formats and event types. Elasticsearch allows real-time search and scaling through replication/sharding. Kibana provides browser-based dashboards and visualization of Elasticsearch query results.
Log Analytics with ELK Stack describes optimizing an ELK stack implementation for a mobile gaming company to reduce costs and scale data ingestion. Key optimizations included moving to spot instances, separating logs into different indexes based on type and retention needs, tuning Elasticsearch and Logstash configurations, and implementing a hot-warm architecture across different EBS volume types. These changes reduced overall costs by an estimated 80% while maintaining high availability and scalability.
Want to make rsyslog talk to some not-yet-supported output? With v8 it's extremly simple.This presentation contains everything you need to know about writing great plugins in ANY language (like perl or pyton).
Spencer Christensen
There are many aspects to managing an RDBMS. Some of these are handled by an experienced DBA, but there are a good many things that any sys admin should be able to take care of if they know what to look for.
This presentation will cover basics of managing Postgres, including creating database clusters, overview of configuration, and logging. We will also look at tools to help monitor Postgres and keep an eye on what is going on. Some of the tools we will review are:
* pgtop
* pg_top
* pgfouine
* check_postgres.pl.
Check_postgres.pl is a great tool that can plug into your Nagios or Cacti monitoring systems, giving you even better visibility into your databases.
Learn about structured logging with rsyslog and how it can be used to do actual format conversions. Include config samples for Linux and Windows log sources.
Meta/Facebook's database serving social workloads is running on top of MyRocks (MySQL on RocksDB). This means our performance and reliability depends a lot on RocksDB. Not just MyRocks, but also we have other important systems running on top of RocksDB. We have learned many lessons from operating and debugging RocksDB at scale.
In this session, we will offer an overview of RocksDB, key differences from InnoDB, and share a few interesting lessons learned from production.
ELK (Elasticsearch, Logstash, Kibana) is an open source toolset for centralized logging, where Logstash collects, parses, and filters logs, Elasticsearch stores and indexes logs for search, and Kibana visualizes logs. Logstash processes logs through an input, filter, output pipeline using plugins. It can interpret various log formats and event types. Elasticsearch allows real-time search and scaling through replication/sharding. Kibana provides browser-based dashboards and visualization of Elasticsearch query results.
Log Analytics with ELK Stack describes optimizing an ELK stack implementation for a mobile gaming company to reduce costs and scale data ingestion. Key optimizations included moving to spot instances, separating logs into different indexes based on type and retention needs, tuning Elasticsearch and Logstash configurations, and implementing a hot-warm architecture across different EBS volume types. These changes reduced overall costs by an estimated 80% while maintaining high availability and scalability.
Traditionally database systems were optimized either for OLAP either for OLTP workloads. Such mainstream DBMSes like Postgres,MySQL,... are mostly used for OLTP, while Greenplum, Vertica, Clickhouse, SparkSQL,... are oriented on analytic queries. But right now many companies do not want to have two different data stores for OLAP/OLTP and need to perform analytic queries on most recent data. I want to discuss which features should be added to Postgres to efficiently handle HTAP workload.
On-demand recording: nginx.com/resources/webinars/nginx-basics-best-practices
You’ve heard of NGINX and the benefits it can provide to your web application, but maybe you’re not sure how to get started. There are a lot of tutorials online, but they can be outdated and contradict each other, making things more challenging. In this webinar we’ll cover the basics of NGINX to help you effectively begin using it as part of your existing or new web app.
This webinar covers how to:
* Install NGINX and verify it's properly running
* Create NGINX configurations for reverse proxy, load balancer, etc.
* Improve performance using keepalives and other NGINX directives
* Debug and troubleshoot using NGINX logs
NGINX: Basics & Best Practices - EMEA BroadcastNGINX, Inc.
This document provides an overview of installing and configuring the NGINX web server. It discusses installing NGINX from official repositories or from source on Linux systems like Ubuntu, Debian, CentOS and Red Hat. It also covers verifying the installation, basic configurations for web serving, reverse proxying, load balancing and caching. The document discusses modifications that can be made to the main nginx.conf file to improve performance and reliability. It also covers monitoring NGINX using status pages and logs, and summarizes key documentation resources.
Amazon Elastic Block Store (Amazon EBS) provides persistent block level storage volumes for use with Amazon EC2 instances. In this technical session, we conduct a detailed analysis of the differences among the three types of Amazon EBS block storage: General Purpose (SSD), Provisioned IOPS (SSD), and Magnetic. We discuss how to maximize Amazon EBS performance, with a special eye towards low-latency, high-throughput applications like databases. We discuss Amazon EBS encryption and share best practices for Amazon EBS snapshot management. Throughout, we share tips for success.
Nginx is an open-source, lightweight web server that can serve static files, act as a reverse proxy, load balancer, and HTTP cache. It is fast, scalable, and improves performance and security for large websites. Some key companies that use Nginx include Google, IBM, LinkedIn, and Facebook. Nginx follows a master-slave architecture with an event-driven, asynchronous, and non-blocking model. The master process manages worker processes that handle requests in a single-threaded manner, improving concurrency.
Everyday I'm Shuffling - Tips for Writing Better Spark Programs, Strata San J...Databricks
Watch video at: http://youtu.be/Wg2boMqLjCg
Want to learn how to write faster and more efficient programs for Apache Spark? Two Spark experts from Databricks, Vida Ha and Holden Karau, provide some performance tuning and testing tips for your Spark applications
There are many ways to run high availability with PostgreSQL. Here, we present a template for you to create your own customized, high-availability solution using Python and for maximum accessibility, a distributed configuration store like ZooKeeper or etcd.
The Parquet Format and Performance Optimization OpportunitiesDatabricks
The Parquet format is one of the most widely used columnar storage formats in the Spark ecosystem. Given that I/O is expensive and that the storage layer is the entry point for any query execution, understanding the intricacies of your storage format is important for optimizing your workloads.
As an introduction, we will provide context around the format, covering the basics of structured data formats and the underlying physical data storage model alternatives (row-wise, columnar and hybrid). Given this context, we will dive deeper into specifics of the Parquet format: representation on disk, physical data organization (row-groups, column-chunks and pages) and encoding schemes. Now equipped with sufficient background knowledge, we will discuss several performance optimization opportunities with respect to the format: dictionary encoding, page compression, predicate pushdown (min/max skipping), dictionary filtering and partitioning schemes. We will learn how to combat the evil that is ‘many small files’, and will discuss the open-source Delta Lake format in relation to this and Parquet in general.
This talk serves both as an approachable refresher on columnar storage as well as a guide on how to leverage the Parquet format for speeding up analytical workloads in Spark using tangible tips and tricks.
This document provides an overview of IT automation using Ansible. It discusses using Ansible to automate tasks across multiple servers like installing packages and copying files without needing to login to each server individually. It also covers Ansible concepts like playbooks, variables, modules, and vault for securely storing passwords. Playbooks allow defining automation jobs as code that can be run on multiple servers simultaneously in a consistent and repeatable way.
This talk explores PostgreSQL 15 enhancements (along with some history) and looks at how they improve developer experience (MERGE and SQL/JSON), optimize support for backups and compression, logical replication improvements, enhanced security and performance, and more.
The document provides an overview of PostgreSQL performance tuning. It discusses caching, query processing internals, and optimization of storage and memory usage. Specific topics covered include the PostgreSQL configuration parameters for tuning shared buffers, work memory, and free space map settings.
5 things you didn't know nginx could dosarahnovotny
NGINX is a well kept secret of high performance web service. Many people know NGINX as an Open Source web server that delivers static content blazingly fast. But, it has many more features to help accelerate delivery of bits to your end users even in more complicated application environments. In this talk we'll cover several things that most developers or administrators could implement to further delight their end users.
Top 10 Mistakes When Migrating From Oracle to PostgreSQLJim Mlodgenski
As more and more people are moving to PostgreSQL from Oracle, a pattern of mistakes is emerging. They can be caused by the tools being used or just not understanding how PostgreSQL is different than Oracle. In this talk we will discuss the top mistakes people generally make when moving to PostgreSQL from Oracle and what the correct course of action.
MySQL uses different storage engines to store, retrieve and index data. The major storage engines are MyISAM, InnoDB, MEMORY, and ARCHIVE. MyISAM uses table-level locking and supports full-text searching but not transactions. InnoDB supports transactions, row-level locking and foreign keys but with more overhead than MyISAM. MEMORY stores data in memory for very fast access but data is lost on server restart. ARCHIVE is for read-only tables to improve performance and reduce storage requirements.
PostgreSQL is a very popular and feature-rich DBMS. At the same time, PostgreSQL has a set of annoying wicked problems, which haven't been resolved in decades. Miraculously, with just a small patch to PostgreSQL core extending this API, it appears possible to solve wicked PostgreSQL problems in a new engine made within an extension.
The document introduces the ELK stack, which consists of Elasticsearch, Logstash, Kibana, and Beats. Beats ship log and operational data to Elasticsearch. Logstash ingests, transforms, and sends data to Elasticsearch. Elasticsearch stores and indexes the data. Kibana allows users to visualize and interact with data stored in Elasticsearch. The document provides descriptions of each component and their roles. It also includes configuration examples and demonstrates how to access Elasticsearch via REST.
Gerenciamento de Backups PostgreSQL com pgbarmanJuliano Atanazio
O documento descreve o uso da ferramenta pgbarman para gerenciamento de backups do PostgreSQL. O pgbarman permite fazer backups remotos de múltiplos servidores PostgreSQL, auxiliando DBAs na recuperação de dados. É apresentada a instalação, configuração e uso básico do pgbarman, incluindo a criação de servidores, listagem de configurações e backups.
This is the talk I have given on Fedora Developer's Conference 2014 in Brno. It provides insight into the security features we added to rsyslog v7, integration into systemd journal, enhancements of the v8 engine and a glimpse at how to write rsyslog plugins in languages other than C.
Fluentd processes logs and events from multiple sources by assigning each a tag based on its input source. It then matches these tags to specific outputs like databases or search engines. Fluentd can also re-process events by assigning a new tag to allow for filtering of data to different destinations.
Traditionally database systems were optimized either for OLAP either for OLTP workloads. Such mainstream DBMSes like Postgres,MySQL,... are mostly used for OLTP, while Greenplum, Vertica, Clickhouse, SparkSQL,... are oriented on analytic queries. But right now many companies do not want to have two different data stores for OLAP/OLTP and need to perform analytic queries on most recent data. I want to discuss which features should be added to Postgres to efficiently handle HTAP workload.
On-demand recording: nginx.com/resources/webinars/nginx-basics-best-practices
You’ve heard of NGINX and the benefits it can provide to your web application, but maybe you’re not sure how to get started. There are a lot of tutorials online, but they can be outdated and contradict each other, making things more challenging. In this webinar we’ll cover the basics of NGINX to help you effectively begin using it as part of your existing or new web app.
This webinar covers how to:
* Install NGINX and verify it's properly running
* Create NGINX configurations for reverse proxy, load balancer, etc.
* Improve performance using keepalives and other NGINX directives
* Debug and troubleshoot using NGINX logs
NGINX: Basics & Best Practices - EMEA BroadcastNGINX, Inc.
This document provides an overview of installing and configuring the NGINX web server. It discusses installing NGINX from official repositories or from source on Linux systems like Ubuntu, Debian, CentOS and Red Hat. It also covers verifying the installation, basic configurations for web serving, reverse proxying, load balancing and caching. The document discusses modifications that can be made to the main nginx.conf file to improve performance and reliability. It also covers monitoring NGINX using status pages and logs, and summarizes key documentation resources.
Amazon Elastic Block Store (Amazon EBS) provides persistent block level storage volumes for use with Amazon EC2 instances. In this technical session, we conduct a detailed analysis of the differences among the three types of Amazon EBS block storage: General Purpose (SSD), Provisioned IOPS (SSD), and Magnetic. We discuss how to maximize Amazon EBS performance, with a special eye towards low-latency, high-throughput applications like databases. We discuss Amazon EBS encryption and share best practices for Amazon EBS snapshot management. Throughout, we share tips for success.
Nginx is an open-source, lightweight web server that can serve static files, act as a reverse proxy, load balancer, and HTTP cache. It is fast, scalable, and improves performance and security for large websites. Some key companies that use Nginx include Google, IBM, LinkedIn, and Facebook. Nginx follows a master-slave architecture with an event-driven, asynchronous, and non-blocking model. The master process manages worker processes that handle requests in a single-threaded manner, improving concurrency.
Everyday I'm Shuffling - Tips for Writing Better Spark Programs, Strata San J...Databricks
Watch video at: http://youtu.be/Wg2boMqLjCg
Want to learn how to write faster and more efficient programs for Apache Spark? Two Spark experts from Databricks, Vida Ha and Holden Karau, provide some performance tuning and testing tips for your Spark applications
There are many ways to run high availability with PostgreSQL. Here, we present a template for you to create your own customized, high-availability solution using Python and for maximum accessibility, a distributed configuration store like ZooKeeper or etcd.
The Parquet Format and Performance Optimization OpportunitiesDatabricks
The Parquet format is one of the most widely used columnar storage formats in the Spark ecosystem. Given that I/O is expensive and that the storage layer is the entry point for any query execution, understanding the intricacies of your storage format is important for optimizing your workloads.
As an introduction, we will provide context around the format, covering the basics of structured data formats and the underlying physical data storage model alternatives (row-wise, columnar and hybrid). Given this context, we will dive deeper into specifics of the Parquet format: representation on disk, physical data organization (row-groups, column-chunks and pages) and encoding schemes. Now equipped with sufficient background knowledge, we will discuss several performance optimization opportunities with respect to the format: dictionary encoding, page compression, predicate pushdown (min/max skipping), dictionary filtering and partitioning schemes. We will learn how to combat the evil that is ‘many small files’, and will discuss the open-source Delta Lake format in relation to this and Parquet in general.
This talk serves both as an approachable refresher on columnar storage as well as a guide on how to leverage the Parquet format for speeding up analytical workloads in Spark using tangible tips and tricks.
This document provides an overview of IT automation using Ansible. It discusses using Ansible to automate tasks across multiple servers like installing packages and copying files without needing to login to each server individually. It also covers Ansible concepts like playbooks, variables, modules, and vault for securely storing passwords. Playbooks allow defining automation jobs as code that can be run on multiple servers simultaneously in a consistent and repeatable way.
This talk explores PostgreSQL 15 enhancements (along with some history) and looks at how they improve developer experience (MERGE and SQL/JSON), optimize support for backups and compression, logical replication improvements, enhanced security and performance, and more.
The document provides an overview of PostgreSQL performance tuning. It discusses caching, query processing internals, and optimization of storage and memory usage. Specific topics covered include the PostgreSQL configuration parameters for tuning shared buffers, work memory, and free space map settings.
5 things you didn't know nginx could dosarahnovotny
NGINX is a well kept secret of high performance web service. Many people know NGINX as an Open Source web server that delivers static content blazingly fast. But, it has many more features to help accelerate delivery of bits to your end users even in more complicated application environments. In this talk we'll cover several things that most developers or administrators could implement to further delight their end users.
Top 10 Mistakes When Migrating From Oracle to PostgreSQLJim Mlodgenski
As more and more people are moving to PostgreSQL from Oracle, a pattern of mistakes is emerging. They can be caused by the tools being used or just not understanding how PostgreSQL is different than Oracle. In this talk we will discuss the top mistakes people generally make when moving to PostgreSQL from Oracle and what the correct course of action.
MySQL uses different storage engines to store, retrieve and index data. The major storage engines are MyISAM, InnoDB, MEMORY, and ARCHIVE. MyISAM uses table-level locking and supports full-text searching but not transactions. InnoDB supports transactions, row-level locking and foreign keys but with more overhead than MyISAM. MEMORY stores data in memory for very fast access but data is lost on server restart. ARCHIVE is for read-only tables to improve performance and reduce storage requirements.
PostgreSQL is a very popular and feature-rich DBMS. At the same time, PostgreSQL has a set of annoying wicked problems, which haven't been resolved in decades. Miraculously, with just a small patch to PostgreSQL core extending this API, it appears possible to solve wicked PostgreSQL problems in a new engine made within an extension.
The document introduces the ELK stack, which consists of Elasticsearch, Logstash, Kibana, and Beats. Beats ship log and operational data to Elasticsearch. Logstash ingests, transforms, and sends data to Elasticsearch. Elasticsearch stores and indexes the data. Kibana allows users to visualize and interact with data stored in Elasticsearch. The document provides descriptions of each component and their roles. It also includes configuration examples and demonstrates how to access Elasticsearch via REST.
Gerenciamento de Backups PostgreSQL com pgbarmanJuliano Atanazio
O documento descreve o uso da ferramenta pgbarman para gerenciamento de backups do PostgreSQL. O pgbarman permite fazer backups remotos de múltiplos servidores PostgreSQL, auxiliando DBAs na recuperação de dados. É apresentada a instalação, configuração e uso básico do pgbarman, incluindo a criação de servidores, listagem de configurações e backups.
This is the talk I have given on Fedora Developer's Conference 2014 in Brno. It provides insight into the security features we added to rsyslog v7, integration into systemd journal, enhancements of the v8 engine and a glimpse at how to write rsyslog plugins in languages other than C.
Fluentd processes logs and events from multiple sources by assigning each a tag based on its input source. It then matches these tags to specific outputs like databases or search engines. Fluentd can also re-process events by assigning a new tag to allow for filtering of data to different destinations.
This document summarizes techniques for optimizing Logstash and Rsyslog for high volume log ingestion into Elasticsearch. It discusses using Logstash and Rsyslog to ingest logs via TCP and JSON parsing, applying filters like grok and mutate, and outputting to Elasticsearch. It also covers Elasticsearch tuning including refresh rate, doc values, indexing performance, and using time-based indices on hot and cold nodes. Benchmark results show Logstash and Rsyslog can handle thousands of events per second with appropriate configuration.
The document discusses configuring various services on a Red Hat Enterprise Linux system. It includes instructions for setting up a Yum repository, configuring SELinux, modifying cron access, adding a boot parameter, mounting an ISO file, setting up an FTP server allowing anonymous access only from the local network, creating an email alias, and more.
This document discusses security and integrity in Linux filesystems. It covers topics such as filesystem sources in the Linux kernel, the virtual filesystem (VFS) layer, common filesystems like ext2/3 and XFS, atomic operations and journaling to improve robustness, cryptography and encryption, and Linux Security Modules (LSM) for access control and security policy enforcement. The goal is to provide an overview of how Linux maintains security and data integrity across its various filesystem implementations.
Continuous Infrastructure: Modern Puppet for the Jenkins Project - PuppetConf...Puppet
This document summarizes Tyler Croy's presentation on managing the Jenkins infrastructure using Puppet. It describes how the infrastructure evolved from an unmanaged setup at Sun/Oracle to using masterless Puppet and eventually Puppet Enterprise. Key aspects covered include managing services, hardware, code layout, testing, and deployment process. Special thanks are given to Puppet Labs for their support of the project.
Assets, Files, and Data Parsing
Android offers a few structured ways to store data, notably SharedPreferences and local SQLite databases.
And, of course, you are welcome to store your data “in the cloud” by using an Internet-based service.
Beyond that, though, Android allows you to work with plain old ordinary files, either one baked into your app (“assets”) or ones on so-called internal or external storage.
To make those files work — and to consume data off of the Internet — you will likely need to employ a parser. Android ships with several choices for XML and JSON parsing, in addition to third-party libraries you can attempt to use.
This session focuses on Assets, Raw, and Files.
Native Android Userspace part of the Embedded Android Workshop at Linaro Conn...Opersys inc.
This document provides an overview of the native Android user-space environment, including:
1) The filesystem layout and key directories like /system, /data, and /vendor.
2) How the build system determines where files are placed.
3) The adb debugging tool and its capabilities.
4) Common command line tools and properties.
5) The init process and ueventd daemon.
6) Libraries like Bionic and integration options for legacy systems.
This document discusses ways to optimize logging by centralizing and proactively using log data. It recommends using Monolog to log from application code in a standardized format. Rsyslog can then collect logs centrally from applications and systems. Logstash can further process logs with filters and output them to destinations like Elasticsearch. Graylog2 provides a web interface for powerful log searching, analytics, and alerting. Centralizing, standardizing, and proactively analyzing logs with these open source tools allows for improved monitoring and troubleshooting.
This document discusses how to extract source code from websites that have exposed their .git directories without authorization. It describes finding repositories through tools like Nmap scripts and DVCS-Pillage, but notes limitations in completeness. A new tool called DVCS-rip is presented that can fully clone exposed git repositories over various protocols, including branches, with the goal of getting the full source code when it is not otherwise open source. The talk encourages feedback and contributions to improve upon existing tools.
Nagios Conference 2014 - Andy Brist - Nagios XI Failover and HA SolutionsNagios
Andy Brist's presentation on High Availability and Failover Solutions for Nagios XI. The presentation was given during the Nagios World Conference North America held Oct 13th - Oct 16th, 2014 in Saint Paul, MN. For more information on the conference (including photos and videos), visit: http://go.nagios.com/conference
The Linux directory structure is organized with / as the root directory. Key directories include /bin and /sbin for essential system binaries, /boot for boot files, /dev for device files, /etc for configuration files, /home for user home directories, /lib for shared libraries, /media and /mnt for mounting removable media, /opt for optional application software, /proc for process information, /root for the root user's home, /tmp for temporary files, /usr for secondary hierarchy data and binaries, and /var for variable data.
This document discusses techniques for path traversal attacks after gaining initial access to a system. It provides examples of important files and directories to access outside the web root, including password files, SSH keys, log files, and system information files. It also discusses common locations for these types of files on Linux, Windows, and Coldfusion systems. The goal of path traversal after initial access is to elevate privileges and gather additional sensitive information to fully compromise the target system.
The document provides an overview of diagnosing performance and other issues with the InnoDB storage engine in MySQL. It discusses various sources of information for troubleshooting like SHOW ENGINE INNODB STATUS and OS tools. Common problems covered include the InnoDB data dictionary getting out of sync, crashes/segmentation faults, locking issues, and performance problems related to disk I/O, buffer pool hit rates, high CPU usage from row operations or thread thrashing. Interpreting diagnostic output and potential solutions are also outlined.
Harrison fisk masteringinnodb-diagnosticsguest8212a5
This document provides an overview of techniques for diagnosing and troubleshooting performance and other issues with the InnoDB storage engine in MySQL. It discusses sources of diagnostic information like SHOW ENGINE INNODB STATUS and various status variables. Common problems covered include data dictionary issues, crashing, locking, and performance problems related to disk I/O, tablespace usage, CPU usage, and thread thrashing. Interpreting diagnostic information and potential solutions are provided for each type of issue.
This document summarizes key concepts about file systems from the textbook "Operating System Concepts" by Silberschatz, Galvin and Gagne. It discusses file concepts like attributes and operations. It describes different access methods for files including sequential, direct, and indexed access. It also covers directory structures from single-level to tree-structured and graph-based designs. File system interfaces like mounting, locking, and protection mechanisms are also overviewed.
This document discusses syslog and log files. It describes what events should be logged, such as activities in the accounting system and kernel. It discusses different logging policies like rotating log files daily and archiving older files. Syslog is introduced as the system logging utility that routes log messages to files or terminals based on configuration rules. Key syslog components and how software uses the syslog API to generate log entries are outlined.
Ansible is an open source tool that uses SSH to automate and simplify configuration management across servers. It is agentless, uses Python for its language, and has a pull-based model. Ansible uses YAML files called playbooks to define automation jobs and roles. Playbooks contain tasks that are executed sequentially on managed nodes. Ansible supports variables, facts, conditionals, loops and templates to customize automation. It has features like idempotency, vault for secrets, and roles for reusability. Galaxy is a library of community roles.
This document provides an introduction and overview of key concepts related to the Windows operating system internals. It discusses basic OS concepts, how Windows is implemented, and a simplified view of its inner workings. The document outlines user mode vs kernel mode, processes and threads, virtual memory, objects and handles, Windows design goals, core system files, the executive and kernel, and important system processes like CSRSS and Winlogon. It also covers symmetric multiprocessing, subsystems, and the Windows in Windows 64 (Wow64) subsystem that allows 32-bit apps to run on 64-bit Windows.
This document discusses files and storage in Windows Phone 8. It covers the different storage APIs available, including the Windows Phone Runtime storage APIs and the legacy WP7.1 isolated storage APIs. It also discusses best practices for data serialization, accessing the local storage folder and removable SD cards, and tools for exploring isolated storage like the Isolated Storage Explorer.
Apache Pig is a platform for analyzing large datasets that sits on top of Hadoop. It allows users to write scripts in Pig Latin to transform and analyze their data without needing to write Java code. Pig Latin scripts are compiled into sequences of MapReduce jobs that process the data in parallel across large clusters. Pig is useful for data summarization, querying, reporting, and analysis of large datasets.
PigHive presentation and hive impor.pptxRahul Borate
Pig is a platform for analyzing large datasets that sits on top of Hadoop. It allows users to write scripts in Pig Latin, a language similar to SQL, to transform and analyze their data without needing to write Java code. Pig scripts are compiled into sequences of MapReduce jobs that process data in parallel across a Hadoop cluster. Key features of Pig include data filtering, joining, grouping, and the ability to extend it with custom user-defined functions.
The document discusses storage options for Windows Phone 8 applications. It covers using the isolated storage APIs from Windows Phone 7.1 as well as the new storage APIs introduced in Windows Phone 8. Special folders like Shared/Media and Shared/ShellContent are mentioned. Serializing and deserializing data for storage is also covered. Tools like Isolated Storage Explorer for exploring application data storage are presented. Best practices around storage like quota management and serialization performance are also highlighted.
Talk given on BalCCon 2013 by Vlatko Kosturjak: Wonderful world of (distributed) SCM or VCS. Ripping and extracting useful info from CVS, Subversion (SVN) and GIT repositories publicly exposed on the web.
Similar to Using Wildcards with rsyslog's File Monitor imfile (20)
Sicherheit im Internet - Wie kann man sich schützen?Rainer Gerhards
Einfache Methoden mit denen sich auch "Normalos" im Internet schützen können. Leicht erklärt. Mit Detailinfos unter https://www.rainer-gerhards.de/sicherheit-im-internet/
Slides for GUUG FFG2018 talk on rsyslog and containers. Describes the initial steps the rsyslog project took towards containers, uses cases seen by the team, problems we have seen and use of docker inside rsyslog's CI.
The rsyslog project has adopted a new versioning and release cycle scheme to provide features to users more quickly while maintaining stability. The major changes are:
1) Stable releases will now occur every 6 weeks rather than distinguishing between stable and development releases.
2) The minor version number will increment with each new release rather than distinguishing between odd and even numbers.
3) Development versions are now identified by their git commit hash rather than a version number.
This allows new features to reach users more rapidly without compromising stability through more frequent testing.
RSYSLOG v8 improvements and how to write plugins in any language.Rainer Gerhards
RSYSLOG is a next generation log processing tool. In the frist part, we will explain the new RSYSLOG v8 engine, its motivation and its benefits. Learn, for example, why writing to Elasticsearch is much faster with the new engine. We will describe the tuning parameters vital for making best use of the new features.
In the second part we will explain how to write RSYSLOG plugins in any language. Traditionally, writing rsyslog plugins has been considered quite hard, with at least C knowledge necessary. In v8, we have introduced new interfaces which make it possible to write plugins in any language - be it Python, Perl or Java. Even bash will do. In essence, this is a great tool for any admin to add special needs with just a bit of scripting. We will proivde concrete instructions on how to write a plugin, point to read-to-copy samples and tell how to integrate this into rsyslog.
NOTE: This is my LinuxTag Berlin 2014 talk.
Which enhancements does the rsyslog v8 engine contain? This short presentation focuses on those changes in the output engine that greatly helps to improve performance. We describe what exactly was changed, how this looks in real code and what developers need to know before upgrading a plugin to v8.
Wetterbeobachtung - Ein Vortrag für die GrundschuleRainer Gerhards
Der Vortrag erklärt in einfachen Worten und anhand von Beispielbildern die Grundlagen der Wetterbeobachtung und eignet sich für den Sachkunde-Unterricht.
Do rsyslog and the journal cooperate? If so, how? This is the presentation from the LinuxTag 2013 conference. It details the rsyslog team's current position on the journal, how it affected rsyslog, what is being done for integration and some notes about how to configure rsyslog to do things that the journal announcement claimed to be impossible.
Do rsyslog and the journal cooperate? If so, how? This is the paper from the LinuxTag 2013 conference. It details the rsyslog team's current position on the journal, how it affected rsyslog, what is being done for integration and some notes about how to configure rsyslog to do things that the journal announcemnt claimed to be impossible.
CEE Log Integrity and the "Counterpane Paper"Rainer Gerhards
What are the problems in signing log data while it traverses the network? This paper was originaly written to support CEE discussion on log integrity but also clearly describes both the problem and a partial, but practical solution to it. While it uses some CEE terms, it should be easy to follow without CEE knowledge.
The paper was written in December 2010 and uploaded in May 2013 to make it easier to obtain it.
This presentation covers the state of the syslog protocol and its standardization as of 2005. It was created for and held at Linuxtag in Germany (and as such is in German).
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceIndexBug
Imagine a world where machines not only perform tasks but also learn, adapt, and make decisions. This is the promise of Artificial Intelligence (AI), a technology that's not just enhancing our lives but revolutionizing entire industries.
UiPath Test Automation using UiPath Test Suite series, part 6DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 6. In this session, we will cover Test Automation with generative AI and Open AI.
UiPath Test Automation with generative AI and Open AI webinar offers an in-depth exploration of leveraging cutting-edge technologies for test automation within the UiPath platform. Attendees will delve into the integration of generative AI, a test automation solution, with Open AI advanced natural language processing capabilities.
Throughout the session, participants will discover how this synergy empowers testers to automate repetitive tasks, enhance testing accuracy, and expedite the software testing life cycle. Topics covered include the seamless integration process, practical use cases, and the benefits of harnessing AI-driven automation for UiPath testing initiatives. By attending this webinar, testers, and automation professionals can gain valuable insights into harnessing the power of AI to optimize their test automation workflows within the UiPath ecosystem, ultimately driving efficiency and quality in software development processes.
What will you get from this session?
1. Insights into integrating generative AI.
2. Understanding how this integration enhances test automation within the UiPath platform
3. Practical demonstrations
4. Exploration of real-world use cases illustrating the benefits of AI-driven test automation for UiPath
Topics covered:
What is generative AI
Test Automation with generative AI and Open AI.
UiPath integration with generative AI
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Cosa hanno in comune un mattoncino Lego e la backdoor XZ?Speck&Tech
ABSTRACT: A prima vista, un mattoncino Lego e la backdoor XZ potrebbero avere in comune il fatto di essere entrambi blocchi di costruzione, o dipendenze di progetti creativi e software. La realtà è che un mattoncino Lego e il caso della backdoor XZ hanno molto di più di tutto ciò in comune.
Partecipate alla presentazione per immergervi in una storia di interoperabilità, standard e formati aperti, per poi discutere del ruolo importante che i contributori hanno in una comunità open source sostenibile.
BIO: Sostenitrice del software libero e dei formati standard e aperti. È stata un membro attivo dei progetti Fedora e openSUSE e ha co-fondato l'Associazione LibreItalia dove è stata coinvolta in diversi eventi, migrazioni e formazione relativi a LibreOffice. In precedenza ha lavorato a migrazioni e corsi di formazione su LibreOffice per diverse amministrazioni pubbliche e privati. Da gennaio 2020 lavora in SUSE come Software Release Engineer per Uyuni e SUSE Manager e quando non segue la sua passione per i computer e per Geeko coltiva la sua curiosità per l'astronomia (da cui deriva il suo nickname deneb_alpha).
“An Outlook of the Ongoing and Future Relationship between Blockchain Technologies and Process-aware Information Systems.” Invited talk at the joint workshop on Blockchain for Information Systems (BC4IS) and Blockchain for Trusted Data Sharing (B4TDS), co-located with with the 36th International Conference on Advanced Information Systems Engineering (CAiSE), 3 June 2024, Limassol, Cyprus.
Programming Foundation Models with DSPy - Meetup SlidesZilliz
Prompting language models is hard, while programming language models is easy. In this talk, I will discuss the state-of-the-art framework DSPy for programming foundation models with its powerful optimizers and runtime constraint system.
Driving Business Innovation: Latest Generative AI Advancements & Success StorySafe Software
Are you ready to revolutionize how you handle data? Join us for a webinar where we’ll bring you up to speed with the latest advancements in Generative AI technology and discover how leveraging FME with tools from giants like Google Gemini, Amazon, and Microsoft OpenAI can supercharge your workflow efficiency.
During the hour, we’ll take you through:
Guest Speaker Segment with Hannah Barrington: Dive into the world of dynamic real estate marketing with Hannah, the Marketing Manager at Workspace Group. Hear firsthand how their team generates engaging descriptions for thousands of office units by integrating diverse data sources—from PDF floorplans to web pages—using FME transformers, like OpenAIVisionConnector and AnthropicVisionConnector. This use case will show you how GenAI can streamline content creation for marketing across the board.
Ollama Use Case: Learn how Scenario Specialist Dmitri Bagh has utilized Ollama within FME to input data, create custom models, and enhance security protocols. This segment will include demos to illustrate the full capabilities of FME in AI-driven processes.
Custom AI Models: Discover how to leverage FME to build personalized AI models using your data. Whether it’s populating a model with local data for added security or integrating public AI tools, find out how FME facilitates a versatile and secure approach to AI.
We’ll wrap up with a live Q&A session where you can engage with our experts on your specific use cases, and learn more about optimizing your data workflows with AI.
This webinar is ideal for professionals seeking to harness the power of AI within their data management systems while ensuring high levels of customization and security. Whether you're a novice or an expert, gain actionable insights and strategies to elevate your data processes. Join us to see how FME and AI can revolutionize how you work with data!
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc
How does your privacy program stack up against your peers? What challenges are privacy teams tackling and prioritizing in 2024?
In the fifth annual Global Privacy Benchmarks Survey, we asked over 1,800 global privacy professionals and business executives to share their perspectives on the current state of privacy inside and outside of their organizations. This year’s report focused on emerging areas of importance for privacy and compliance professionals, including considerations and implications of Artificial Intelligence (AI) technologies, building brand trust, and different approaches for achieving higher privacy competence scores.
See how organizational priorities and strategic approaches to data security and privacy are evolving around the globe.
This webinar will review:
- The top 10 privacy insights from the fifth annual Global Privacy Benchmarks Survey
- The top challenges for privacy leaders, practitioners, and organizations in 2024
- Key themes to consider in developing and maintaining your privacy program
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
Best 20 SEO Techniques To Improve Website Visibility In SERPPixlogix Infotech
Boost your website's visibility with proven SEO techniques! Our latest blog dives into essential strategies to enhance your online presence, increase traffic, and rank higher on search engines. From keyword optimization to quality content creation, learn how to make your site stand out in the crowded digital landscape. Discover actionable tips and expert insights to elevate your SEO game.
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!SOFTTECHHUB
As the digital landscape continually evolves, operating systems play a critical role in shaping user experiences and productivity. The launch of Nitrux Linux 3.5.0 marks a significant milestone, offering a robust alternative to traditional systems such as Windows 11. This article delves into the essence of Nitrux Linux 3.5.0, exploring its unique features, advantages, and how it stands as a compelling choice for both casual users and tech enthusiasts.
GraphSummit Singapore | The Future of Agility: Supercharging Digital Transfor...Neo4j
Leonard Jayamohan, Partner & Generative AI Lead, Deloitte
This keynote will reveal how Deloitte leverages Neo4j’s graph power for groundbreaking digital twin solutions, achieving a staggering 100x performance boost. Discover the essential role knowledge graphs play in successful generative AI implementations. Plus, get an exclusive look at an innovative Neo4j + Generative AI solution Deloitte is developing in-house.
Maruthi Prithivirajan, Head of ASEAN & IN Solution Architecture, Neo4j
Get an inside look at the latest Neo4j innovations that enable relationship-driven intelligence at scale. Learn more about the newest cloud integrations and product enhancements that make Neo4j an essential choice for developers building apps with interconnected data and generative AI.
Using Wildcards with rsyslog's File Monitor imfile
1. Using Wildcards
with rsyslog’s
File Monitor
Rainer Gerhards, rsyslog project lead
2. Prerequisites
● kernel with inotify support
● at least rsyslog v8.5.0
● if not available in your distro
o use rsyslog package repository (recommended)
o build from source
● imfile module (usually in base package)
3. State Files
● rsyslog needs to know how much of a file it
already processed
● upon shutdown a “state file” is created with
this information
● stored in rsyslog work directory
● let rsyslog generate the state file name
automatically!
4. Restrictions
● wildcards are support at the file level, not at
the directory level
o /var/log/applog*.log is valid
o /var/applog*/logfile.log is invalid
● subdirectories that match the wildcard are
not processed
o if /var/log/applog-dir.log is a directory, it will not be
processed
● wildcards do not work in polling mode
5. Base Config Sample
global(workDirectory=”/home/rsyslog/spool”)m
odule(load=”imfile”)
input(type=”imfile” tag=”applog”
file=”/var/log/applog*.log”)
7. Notes on Remote Forwarding Conf
● forwarding happens totally independent from
rest of logging configuration due to use of
ruleset
● module() statement must occur only once
● workDirectory
o is used for all rsyslog work and state files
o must be set only once (usually at top of top level
rsyslog.conf)