The document discusses PostgreSQL database administration. It describes how users connect and authenticate to the database using methods defined in pg_hba.conf. It lists some common PostgreSQL configuration files and authentication methods. It also covers the roles of database administration like defining users, roles, and managing access privileges.
The paperback version is available on lulu.com there http://goo.gl/fraa8o
This is the first volume of the postgresql database administration book. The book covers the steps for installing, configuring and administering a PostgreSQL 9.3 on Linux debian. The book covers the logical and physical aspect of PostgreSQL. Two chapters are dedicated to the backup/restore topic.
This presentation covers all aspects of PostgreSQL administration, including installation, security, file structure, configuration, reporting, backup, daily maintenance, monitoring activity, disk space computations, and disaster recovery. It shows how to control host connectivity, configure the server, find the query being run by each session, and find the disk space used by each database.
This ppt was used by Devrim at pgDay Asia 2017. He talked about some important facts about WAL - Transaction Logs or xlogs in PostgreSQL. Some of these can really come handy on a bad day
The paperback version is available on lulu.com there http://goo.gl/fraa8o
This is the first volume of the postgresql database administration book. The book covers the steps for installing, configuring and administering a PostgreSQL 9.3 on Linux debian. The book covers the logical and physical aspect of PostgreSQL. Two chapters are dedicated to the backup/restore topic.
This presentation covers all aspects of PostgreSQL administration, including installation, security, file structure, configuration, reporting, backup, daily maintenance, monitoring activity, disk space computations, and disaster recovery. It shows how to control host connectivity, configure the server, find the query being run by each session, and find the disk space used by each database.
This ppt was used by Devrim at pgDay Asia 2017. He talked about some important facts about WAL - Transaction Logs or xlogs in PostgreSQL. Some of these can really come handy on a bad day
Spencer Christensen
There are many aspects to managing an RDBMS. Some of these are handled by an experienced DBA, but there are a good many things that any sys admin should be able to take care of if they know what to look for.
This presentation will cover basics of managing Postgres, including creating database clusters, overview of configuration, and logging. We will also look at tools to help monitor Postgres and keep an eye on what is going on. Some of the tools we will review are:
* pgtop
* pg_top
* pgfouine
* check_postgres.pl.
Check_postgres.pl is a great tool that can plug into your Nagios or Cacti monitoring systems, giving you even better visibility into your databases.
MySQL and MariaDB though they share the same roots for replication .They support parallel replication , but they diverge the way the parallel replication is implemented.
In 40 minutes the audience will learn a variety of ways to make postgresql database suddenly go out of memory on a box with half a terabyte of RAM.
Developer's and DBA's best practices for preventing this will also be discussed, as well as a bit of Postgres and Linux memory management internals.
There are many ways to run high availability with PostgreSQL. Here, we present a template for you to create your own customized, high-availability solution using Python and for maximum accessibility, a distributed configuration store like ZooKeeper or etcd.
Josh Berkus
You've heard that PostgreSQL is the highest-performance transactional open source database, but you're not seeing it on YOUR server. In fact, your PostgreSQL application is kind of poky. What should you do? While doing advanced performance engineering for really high-end systems takes years to learn, you can learn the basics to solve performance issues for 80% of PostgreSQL installations in less than an hour. In this session, you will learn: -- The parts of database application performance -- The performance setup procedure -- Basic troubleshooting tools -- The 13 postgresql.conf settings you need to know -- Where to look for more information.
MySQL Administrator
Basic course
- MySQL 개요
- MySQL 설치 / 설정
- MySQL 아키텍처 - MySQL 스토리지 엔진
- MySQL 관리
- MySQL 백업 / 복구
- MySQL 모니터링
Advanced course
- MySQL Optimization
- MariaDB / Percona
- MySQL HA (High Availability)
- MySQL troubleshooting
네오클로바
http://neoclova.co.kr/
Optimizing MariaDB for maximum performanceMariaDB plc
When it comes to optimizing the performance of a database, DBAs have to look at everything from the OS to the network. In this session, MariaDB Enterprise Architect Manjot Singh shares best practices for getting the most out of MariaDB. He highlights recommended OS settings, important configuration and tuning parameters, options for improving replication and clustering performance and features such as query result caching.
MongoDB Ops Manager is the easiest way to manage/monitor/operationalize your MongoDB footprint across your enterprise. Ops Manager automates key operations such as deployments, scaling, upgrades, and backups, all with the click of a button and integration with your favorite tools. It also provide the ability to monitor and alert on dozens of platform specific metrics. In this webinar, we'll cover the components of Ops Manager, as well as how it integrates and accelerates your use of MongoDB.
PostgreSQL is one of the most advanced relational databases. It offers superb replication capabilities. The most important features are: Streaming replication, Point-In-Time-Recovery, advanced monitoring, etc.
Covered:
1. Databases and Schemas
2. Tablespaces
3. Data Type
4. Exploring Databases
5. Locating the database server's message log
6. Locating the database's system identifier
7. Listing databases on this database server
8. How much disk space does a table use?
9. Which are my biggest tables?
10. How many rows are there in a table?
11. Quickly estimating the number of rows in a table
12. Understanding object dependencies
Spencer Christensen
There are many aspects to managing an RDBMS. Some of these are handled by an experienced DBA, but there are a good many things that any sys admin should be able to take care of if they know what to look for.
This presentation will cover basics of managing Postgres, including creating database clusters, overview of configuration, and logging. We will also look at tools to help monitor Postgres and keep an eye on what is going on. Some of the tools we will review are:
* pgtop
* pg_top
* pgfouine
* check_postgres.pl.
Check_postgres.pl is a great tool that can plug into your Nagios or Cacti monitoring systems, giving you even better visibility into your databases.
MySQL and MariaDB though they share the same roots for replication .They support parallel replication , but they diverge the way the parallel replication is implemented.
In 40 minutes the audience will learn a variety of ways to make postgresql database suddenly go out of memory on a box with half a terabyte of RAM.
Developer's and DBA's best practices for preventing this will also be discussed, as well as a bit of Postgres and Linux memory management internals.
There are many ways to run high availability with PostgreSQL. Here, we present a template for you to create your own customized, high-availability solution using Python and for maximum accessibility, a distributed configuration store like ZooKeeper or etcd.
Josh Berkus
You've heard that PostgreSQL is the highest-performance transactional open source database, but you're not seeing it on YOUR server. In fact, your PostgreSQL application is kind of poky. What should you do? While doing advanced performance engineering for really high-end systems takes years to learn, you can learn the basics to solve performance issues for 80% of PostgreSQL installations in less than an hour. In this session, you will learn: -- The parts of database application performance -- The performance setup procedure -- Basic troubleshooting tools -- The 13 postgresql.conf settings you need to know -- Where to look for more information.
MySQL Administrator
Basic course
- MySQL 개요
- MySQL 설치 / 설정
- MySQL 아키텍처 - MySQL 스토리지 엔진
- MySQL 관리
- MySQL 백업 / 복구
- MySQL 모니터링
Advanced course
- MySQL Optimization
- MariaDB / Percona
- MySQL HA (High Availability)
- MySQL troubleshooting
네오클로바
http://neoclova.co.kr/
Optimizing MariaDB for maximum performanceMariaDB plc
When it comes to optimizing the performance of a database, DBAs have to look at everything from the OS to the network. In this session, MariaDB Enterprise Architect Manjot Singh shares best practices for getting the most out of MariaDB. He highlights recommended OS settings, important configuration and tuning parameters, options for improving replication and clustering performance and features such as query result caching.
MongoDB Ops Manager is the easiest way to manage/monitor/operationalize your MongoDB footprint across your enterprise. Ops Manager automates key operations such as deployments, scaling, upgrades, and backups, all with the click of a button and integration with your favorite tools. It also provide the ability to monitor and alert on dozens of platform specific metrics. In this webinar, we'll cover the components of Ops Manager, as well as how it integrates and accelerates your use of MongoDB.
PostgreSQL is one of the most advanced relational databases. It offers superb replication capabilities. The most important features are: Streaming replication, Point-In-Time-Recovery, advanced monitoring, etc.
Covered:
1. Databases and Schemas
2. Tablespaces
3. Data Type
4. Exploring Databases
5. Locating the database server's message log
6. Locating the database's system identifier
7. Listing databases on this database server
8. How much disk space does a table use?
9. Which are my biggest tables?
10. How many rows are there in a table?
11. Quickly estimating the number of rows in a table
12. Understanding object dependencies
PostgreSQL Performance Problems: Monitoring and AlertingGrant Fritchey
PostgreSQL can be difficult to troubleshoot when the pressure is on without the right knowledge and tools. Knowing where to find the information you need to improve performance is central to your ability to act quickly and solve problems. In this training, we'll discuss the various query statistic views and log information that's available in PostgreSQL so that you can solve problems quickly. Along the way, we'll highlight a handful of open-source and paid tools that can help you track data over time and provide better alerting capabilities so that you know about problems before they become critical.
PostgreSQL (or Postgres) began its life in 1986 as POSTGRES, a research project of the University of California at Berkeley.
PostgreSQL isn't just relational, it's object-relational.it's object-relational. This gives it some advantages over other open source SQL databases like MySQL, MariaDB and Firebird.
Peeking into the Black Hole Called PL/PGSQL - the New PL Profiler / Jan Wieck...Ontico
The new PL profiler allows you to easily get through the dark barrier, PL/pgSQL puts between tools like pgbadger and the queries, you are looking for.
Query and schema tuning is tough enough by itself. But queries, buried many call levels deep in PL/pgSQL functions, make it torture. The reason is that the default monitoring tools like logs, pg_stat_activity and pg_stat_statements cannot penetrate into PL/pgSQL. All they report is that your query calling function X is slow. That is useful if function X has 20 lines of simple code. Not so useful if it calls other functions and the actual problem query is many call levels down in a dungeon of 100,000 lines of PL code.
Learn from the original author of PL/pgSQL and current maintainer of the plprofiler extension how you can easily analyze, what is going on inside your PL code.
sqlmap is an open source penetration testing tool that automates the process of detecting and exploiting SQL injection flaws and taking over of database servers.
Covered Database Maintenance & Performance and Concurrency :
1. PostgreSQL Tuning and Performance
2. Find and Tune Slow Running Queries
3. Collecting regular statistics from pg_stat* views
4. Finding out what makes SQL slow
5. Speeding up queries without rewriting them
6. Discovering why a query is not using an index
7. Forcing a query to use an index
8. EXPLAIN and SQL Execution
9. Workload Analysis
Sure, here's a comprehensive overview of PostgreSQL in about 3000 words:
---
**Introduction to PostgreSQL:**
PostgreSQL is a powerful, open-source object-relational database system known for its reliability, robustness, and extensibility. It was developed by the PostgreSQL Global Development Group and is released under the PostgreSQL License, a free and open-source software license.
**Key Features of PostgreSQL:**
1. **SQL Compliance:** PostgreSQL follows the SQL standards closely, making it compatible with most SQL-based applications. It supports a wide range of SQL features, including complex queries, transactions, and various data types.
2. **Data Integrity:** PostgreSQL ensures data integrity through constraints, such as unique, primary key, foreign key, and check constraints. These constraints help maintain the consistency and accuracy of the data stored in the database.
3. **Advanced Data Types:** PostgreSQL offers a rich set of data types, including primitive types (e.g., integer, text, boolean) and advanced types (e.g., arrays, JSON, XML, geometric types). This flexibility allows users to store and manipulate diverse types of data efficiently.
4. **Extensibility:** PostgreSQL is highly extensible, allowing users to define custom data types, functions, and procedural languages. Additionally, users can create extensions to add extra functionalities to the database system.
5. **Concurrency Control:** PostgreSQL employs multiversion concurrency control (MVCC) to manage concurrent transactions effectively. This mechanism ensures that multiple transactions can access and modify data concurrently without interfering with each other.
6. **Scalability:** PostgreSQL supports both vertical and horizontal scalability. Vertical scalability involves adding more resources (e.g., CPU, memory) to a single server, while horizontal scalability involves distributing data across multiple servers using techniques like sharding and replication.
7. **High Availability:** PostgreSQL provides built-in features for achieving high availability, such as streaming replication, automatic failover, and online backups. These features help minimize downtime and ensure continuous access to data.
8. **Security:** PostgreSQL offers various security features, including role-based access control (RBAC), SSL/TLS encryption, and data masking. Administrators can define fine-grained access controls to restrict user privileges and protect sensitive data.
9. **Full Text Search:** PostgreSQL includes a powerful full-text search engine, which supports advanced search features like stemming, ranking, and phrase matching. This feature is especially useful for applications that require complex text search capabilities.
10. **Rich Ecosystem:** PostgreSQL has a vibrant ecosystem with a large community of developers, contributors, and third-party vendors. It is supported by a wide range of tools, libraries, and frameworks, making it suitable for diverse use cases.
**Architect
As the popularity of PostgreSQL continues to soar, many companies are exploring ways of migrating their application database over. At Redgate Software, we recently added PostgreSQL as an optional data store for SQL Monitor, our flagship monitoring application, after nearly 18 years of being backed exclusively by SQL Server. Knowing that others will be taking this journey in the near future, we'd like to discuss what we learned. In this training, we'll discuss the planning that needs to take place before a migration begins, including datatype changes, PostgreSQL configuration modifications, and query differences. This will be a mix of slides and demo from our own learnings, as well as those of some clients we've helped along the way.
SQLMAP is an open source penetration testing tool that automates the process of detecting and exploiting SQL injection flaws and taking over of database servers.
Review the latest features released in Neo4j version 4.1 including Cypher, database drivers, clustering, security, and extension libraries like APOC and Spring Data Neo4j!
Description of some of the elements that go in to creating a PostgreSQL-as-a-Service for organizations with many teams and a diverse ecosystem of applications and teams.
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Elevating Tactical DDD Patterns Through Object CalisthenicsDorra BARTAGUIZ
After immersing yourself in the blue book and its red counterpart, attending DDD-focused conferences, and applying tactical patterns, you're left with a crucial question: How do I ensure my design is effective? Tactical patterns within Domain-Driven Design (DDD) serve as guiding principles for creating clear and manageable domain models. However, achieving success with these patterns requires additional guidance. Interestingly, we've observed that a set of constraints initially designed for training purposes remarkably aligns with effective pattern implementation, offering a more ‘mechanical’ approach. Let's explore together how Object Calisthenics can elevate the design of your tactical DDD patterns, offering concrete help for those venturing into DDD for the first time!
Generating a custom Ruby SDK for your web service or Rails API using Smithyg2nightmarescribd
Have you ever wanted a Ruby client API to communicate with your web service? Smithy is a protocol-agnostic language for defining services and SDKs. Smithy Ruby is an implementation of Smithy that generates a Ruby SDK using a Smithy model. In this talk, we will explore Smithy and Smithy Ruby to learn how to generate custom feature-rich SDKs that can communicate with any web service, such as a Rails JSON API.
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
UiPath Test Automation using UiPath Test Suite series, part 4DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 4. In this session, we will cover Test Manager overview along with SAP heatmap.
The UiPath Test Manager overview with SAP heatmap webinar offers a concise yet comprehensive exploration of the role of a Test Manager within SAP environments, coupled with the utilization of heatmaps for effective testing strategies.
Participants will gain insights into the responsibilities, challenges, and best practices associated with test management in SAP projects. Additionally, the webinar delves into the significance of heatmaps as a visual aid for identifying testing priorities, areas of risk, and resource allocation within SAP landscapes. Through this session, attendees can expect to enhance their understanding of test management principles while learning practical approaches to optimize testing processes in SAP environments using heatmap visualization techniques
What will you get from this session?
1. Insights into SAP testing best practices
2. Heatmap utilization for testing
3. Optimization of testing processes
4. Demo
Topics covered:
Execution from the test manager
Orchestrator execution result
Defect reporting
SAP heatmap example with demo
Speaker:
Deepak Rai, Automation Practice Lead, Boundaryless Group and UiPath MVP
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
Essentials of Automations: Optimizing FME Workflows with ParametersSafe Software
Are you looking to streamline your workflows and boost your projects’ efficiency? Do you find yourself searching for ways to add flexibility and control over your FME workflows? If so, you’re in the right place.
Join us for an insightful dive into the world of FME parameters, a critical element in optimizing workflow efficiency. This webinar marks the beginning of our three-part “Essentials of Automation” series. This first webinar is designed to equip you with the knowledge and skills to utilize parameters effectively: enhancing the flexibility, maintainability, and user control of your FME projects.
Here’s what you’ll gain:
- Essentials of FME Parameters: Understand the pivotal role of parameters, including Reader/Writer, Transformer, User, and FME Flow categories. Discover how they are the key to unlocking automation and optimization within your workflows.
- Practical Applications in FME Form: Delve into key user parameter types including choice, connections, and file URLs. Allow users to control how a workflow runs, making your workflows more reusable. Learn to import values and deliver the best user experience for your workflows while enhancing accuracy.
- Optimization Strategies in FME Flow: Explore the creation and strategic deployment of parameters in FME Flow, including the use of deployment and geometry parameters, to maximize workflow efficiency.
- Pro Tips for Success: Gain insights on parameterizing connections and leveraging new features like Conditional Visibility for clarity and simplicity.
We’ll wrap up with a glimpse into future webinars, followed by a Q&A session to address your specific questions surrounding this topic.
Don’t miss this opportunity to elevate your FME expertise and drive your projects to new heights of efficiency.
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
Software Delivery At the Speed of AI: Inflectra Invests In AI-Powered QualityInflectra
In this insightful webinar, Inflectra explores how artificial intelligence (AI) is transforming software development and testing. Discover how AI-powered tools are revolutionizing every stage of the software development lifecycle (SDLC), from design and prototyping to testing, deployment, and monitoring.
Learn about:
• The Future of Testing: How AI is shifting testing towards verification, analysis, and higher-level skills, while reducing repetitive tasks.
• Test Automation: How AI-powered test case generation, optimization, and self-healing tests are making testing more efficient and effective.
• Visual Testing: Explore the emerging capabilities of AI in visual testing and how it's set to revolutionize UI verification.
• Inflectra's AI Solutions: See demonstrations of Inflectra's cutting-edge AI tools like the ChatGPT plugin and Azure Open AI platform, designed to streamline your testing process.
Whether you're a developer, tester, or QA professional, this webinar will give you valuable insights into how AI is shaping the future of software delivery.
2. How Does PostgreSQL Administration Work?
• We can access the PostgreSQL database from localhost or any host. We need to define the IP address of that host in pg_hba.conf file.
• The user is connected to the database using authentication methods. If authentication is successful then the user connected to a specific database otherwise it will disconnect with
an error.
• After authentication user have access to the database and execute the query on which the user has access.
• If a user does not have access to execute the specified query it will throw an error that “Permission Denied”.
• The user also connects through peer authentication in PostgreSQL, it will access through OS user authentication. But in this case, we need both the user to have the same name.
• Below is the PostgreSQL configuration files are as follows.
• Postgresql.conf
• Pg_hba.conf
• postgresql.auto.auto.conf
• pg_ident.conf
• There are multiple methods of authentication used in PostgreSQL like trust, MD5, LDAP, password, SSPI, Kerberos, ident, peer radius, certificate, PAM.
• These methods have different authentication algorithms for each, basically, we have used MD5 authentication methods to authenticate the database from users in PostgreSQL
3. Roles of PostgreSQL Administration
Below are the roles of PostgreSQL administration.
• Roles and users are very important in PostgreSQL to administer the database. User has default login privileges to the
database. Roles do not have default login privileges to the database.
• We can consider the role as a group in PostgreSQL.
• PostgreSQL roles are very important to administer the PostgreSQL database.
• Roles and users are used in PostgreSQL administration to authenticate with the database.
• There are multiple methods of authentication that have used to authenticate the database from the user.
• PostgreSQL administration is used to authenticate the database from unauthenticated access, PostgreSQL
administration is very important to give appropriate access to the database, tables, and all the objects.
• We can see PostgreSQL users and roles by using the following command. The default administrative user is postgres.
• Postgres is a default administrative user which has full access and grant on the database.
8. PostgreSQL - How to grant access to users?
1.How to grant access to users in PostgreSQL?
Here are some common statement to grant access to a PostgreSQL user:
1. Grant CONNECT to the database:
GRANT CONNECT ON DATABASE database_name TO username;
2. Grant USAGE on schema:
GRANT USAGE ON SCHEMA schema_name TO username;
3. Grant on all tables for DML statements: SELECT, INSERT, UPDATE, DELETE:
GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA schema_name TO username;
4. Grant all privileges on all tables in the schema:
GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA schema_name TO username;
5. Grant all privileges on all sequences in the schema:
GRANT ALL PRIVILEGES ON ALL SEQUENCES IN SCHEMA schema_name TO username;
6. Grant all privileges on the database:
GRANT ALL PRIVILEGES ON DATABASE database_name TO username;
7. Grant permission to create database:
ALTER USER username CREATEDB;
8. Make a user superuser:
ALTER USER myuser WITH SUPERUSER;
9. Remove superuser status:
ALTER USER username WITH NOSUPERUSER;
Those statements above only affect the current existing tables. To apply to newly created tables, you need to use alter default. For example:
ALTER DEFAULT PRIVILEGESFOR USER usernameIN SCHEMA schema_nameGRANT SELECT, INSERT, UPDATE, DELETE ON TABLES TO username;
12. Postgres
Monitoring /
Audit tools:- 1.PgBadger: A fast PostgreSQL log analyzer
2.PgCluu: PostgreSQL and system performances monitoring and auditing tool
3.Powa: PostgreSQL Workload Analyzer. Gathers performance stats and provides real-time charts
and graphs to help monitor and tune your PostgreSQL servers. Similar to Oracle AWR.
4.PgObserver: monitor performance metrics of different PostgreSQL clusters.
5.OPM: Open PostgreSQL Monitoring. Gather stats, display dashboards and send warnings
when something goes wrong. Tend to be similar to Oracle Grid Control.
6.check_postgres: script for monitoring various attributes of your database. It is designed
to work with Nagios, MRTG, or in standalone scripts.
7.Pgwatch: monitor PostgreSQL databases and provides a fast and efficient overview
of what is really going on.
8.pgAgent:pgAgent is a job scheduler for PostgreSQL which may be managed using pgAdmin. Prior
to pgAdmin v1.9, pgAgent shipped as part of pgAdmin. From pgAdmin v1.9 onwards, pgAgent is
shipped as a separate application.
9.pgbouncer:PgBouncer is a lightweight connection pooler for PostgreSQL.It contains the
connection pooler and it is used to establish connection between application and database
13. Table Info
select table_catalog, table_schema, table_name, table_type from information_schema.tables where table_schema not in ('pg_catalog', 'information_schema');
select * from information_schema.tables where table_schema not in ('pg_catalog', 'information_schema') and table_schema not like 'pg_toast%';
Table Activity SELECT * FROM pg_stat_all_tables; Table Block Activity SELECT * FROM pg_statio_all_tables;
How to find the largest table in the postgreSQL database?
SELECT relname, relpages FROM pg_class ORDER BY relpages DESC;
SELECT relname, relpages FROM pg_class ORDER BY relpages DESC limit 1;
Object List
select nsp.nspname as object_schema,
cls.relname as object_name, rol.rolname as owner, case cls.relkind
when 'r' then 'TABLE'
when 'm' then 'MATERIALIZED_VIEW'
when 'i' then 'INDEX'
when 'S' then 'SEQUENCE'
when 'v' then 'VIEW'
when 'c' then 'TYPE'
else cls.relkind::text
end as object_type from pg_class cls join pg_roles rol on rol.oid = cls.relowner join pg_namespace nsp on nsp.oid = cls.relnamespace where nsp.nspname not in
('information_schema', 'pg_catalog’) and nsp.nspname not like 'pg_toast%’ order by nsp.nspname, cls.relname;
--and rol.rolname = current_user --- remove this if you want to see all objects
Current User objects
SELECT * FROM pg_tables t WHERE t.tableowner = current_user;
SELECT relname as "Table", pg_size_pretty(pg_total_relation_size(relid)) As "Size", pg_size_pretty(pg_total_relation_size(relid) - pg_relation_size(relid)) as "External Size“ FROM
pg_catalog.pg_statio_user_tables ORDER BY pg_total_relation_size(relid) DESC;
Size of the Objects
SELECT relname AS objectname, relkind AS objecttype, reltuples AS "#entries", pg_size_pretty(relpages::bigint*8*1024) AS size FROM pg_class WHERE relpages >= 8
ORDER BY relpages DESC;
14. Handy queries
List procedure/function
SELECT * FROM pg_proc WHERE proname='__procedurename__';
List view (including the definition)
SELECT * FROM pg_views WHERE viewname='__viewname__';
Show DB table space in use
SELECT pg_size_pretty(pg_total_relation_size('__table_name__'));
Show DB space in use
SELECT pg_size_pretty(pg_database_size('__database_name__'));
SELECT pg_database_size(current_database());
SELECT pg_database_size('postgres');
select pg_size_pretty(pg_database_size(current_database()));
Show current user's statement timeout
show statement_timeout;
Show table indexes
SELECT * FROM pg_indexes WHERE tablename ='__table_name__' AND schemaname='__schema_name__';
Get all indexes from all tables of a schema:
SELECT t.relname AS table_name, i.relname AS index_name, a.attname AS column_name FROM pg_class t, pg_class i, pg_index ix, pg_attribute a, pg_namespace n WHERE
t.oid = ix.indrelid AND i.oid = ix.indexrelid AND a.attrelid = t.oid AND a.attnum = ANY(ix.indkey)AND t.relnamespace = n.oid AND n.nspname = 'kartones' ORDER BY t.relname,
i.relname;
15. •Execution data:
o Queries being executed at a certain DB:
SELECT datname, application_name, pid, backend_start, query_start, state_change, state, query FROM pg_stat_activity WHERE
datname='__database_name__';
•Get all queries from all dbs waiting for data (might be hung):
SELECT * FROM pg_stat_activity WHERE waiting='t'
•Currently running queries with process pid:
SELECT pg_stat_get_backend_pid(s.backendid) AS procpid, pg_stat_get_backend_activity(s.backendid) AS current_query
FROM (SELECT pg_stat_get_backend_idset() AS backendid) AS s;
Casting:
•CAST (column AS type) or column::type
•'__table_name__'::regclass::oid: Get oid having a table name
Query analysis:
•EXPLAIN __query__: see the query plan for the given query
•EXPLAIN ANALYZE __query__: see and execute the query plan for the given query
•ANALYZE [__table__]: collect statistics
Keyboard shortcuts
• CTRL + R: reverse-i-search
Tools
•ptop and pg_top: top for PG. Available on the APT repository from apt.postgresql.org.
•pg_activity: Command line tool for PostgreSQL server activity monitoring.
•Unix-like reverse search in psql:
$ echo "bind "^R" em-inc-search-prev" > $HOME/.editrc
$ source $HOME/.editrc
16. Find Out Essential Settings in a Postgres DB
show config_file;
show effective_cache_size;
show shared_buffers;
show all;
How to calculate postgreSQL database size in disk ?
SELECT pg_database_size('postgres');
SELECT pg_size_pretty(pg_database_size('postgres'));
How to calculate postgreSQL table size in disk ?
SELECT pg_size_pretty(pg_total_relation_size('k1'));
SELECT pg_size_pretty(pg_relation_size('k1'));
select pg_relation_size('audit_hist');
How to view the indexes of an existing postgreSQL table ?
Syntax: # d k1
d pg_attribute
17. Useful command :
# l : to list all databases
# ? : to list help for all commands
# h create table : show help with topics
# d : to list all tables
# df : to list all functions
# da : to list all aggregate functions
# dn : to list all schemas
# du : to list all users/privileges
# r : to reset the query buffer
# db : to list all table space
dt – table list
dv views
d+ students – table description
dy: List events
di: List indexes
dT+: List all data types
18. PostgreSQL Scripts
Table Count SELECT table_schema as Schema_Name ,count(*) as Table_Count FROM information_schema.tables WHERE table_schema IN ('testdb') and table_type='BASE TABLE' group by table_schema
Table List & Row
count
SELECT nspname AS schemaname,relname as Tablename ,reltuples as numRows
FROM pg_class C LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace) WHERE nspname IN (‘testdb') AND relkind='r’ ORDER BY relname;
Column list select table_schema, table_name, ordinal_position as position, column_name,
data_type, case when character_maximum_length is not null
then character_maximum_length
else numeric_precision end as max_length, is_nullable, column_default as default_value from information_schema.columns where table_schema in ('testdb')
order by table_schema, table_name, ordinal_position;
Constraint
list
select
tc.constraint_schema as Owner, tc.table_name as TableName, kcu.column_name as ColumnName, tc.constraint_name,
tc.constraint_type
from
information_schema.table_constraints as tc
join information_schema.key_column_usage as kcu on (tc.constraint_name = kcu.constraint_name and tc.table_name = kcu.table_name)
join information_schema.constraint_column_usage as ccu on ccu.constraint_name = tc.constraint_name where tc.constraint_schema = 'testdb'
--- No search CONDITION in Postgresql
Indexes list SELECT indexname,tablename FROM pg_indexes WHERE schemaname = 'testdb' ORDER BY tablename, indexname;
Partition list (Table
Partition)
select c.relnamespace::regnamespace::text as schema, c.relname as table_name,
pg_get_partkeydef(c.oid) as partition_key from pg_class c
where c.relkind = 'p' and c.relnamespace::regnamespace::text='testdb';
Function &
Procedure
List
SELECT routine_name as objects FROM information_schema.routines
WHERE routine_type='FUNCTION' AND specific_schema IN ('testdb') and routine_name not like '%$%'
Triggers list select trigger_schema, trigger_name from information_schema.triggers
where trigger_schema='testdb' order by trigger_name;
Views list select table_schema as schema_name, table_name as view_name from information_schema.views where table_schema in ('testdb') order by schema_name, view_name;
User Lists select s.nspname as table_schema, s.oid as schema_id, u.usename as owner
from pg_catalog.pg_namespace s join pg_catalog.pg_user u on u.usesysid = s.nspowner
where nspname IN (testdb') order by table_schema;
19. lists all
materialized
views
select schemaname as schema_name, matviewname as view_name, matviewowner as owner, ispopulated as is_populated, definition
from pg_matviews where schemaname='testdb_schema'order by schema_name, view_name;
lists tables with
primary key
columns
select kcu.table_schema, kcu.table_name, tco.constraint_name,
kcu.ordinal_position as position, kcu.column_name as key_column
from information_schema.table_constraints tco
join information_schema.key_column_usage kcu
on kcu.constraint_name = tco.constraint_name
and kcu.constraint_schema = tco.constraint_schema
and kcu.constraint_name = tco.constraint_name
where tco.constraint_type = 'PRIMARY KEY' and kcu.table_schema='testdb_schema'
order by kcu.table_schema, kcu.table_name, position;
Sequence SELECT sequence_schema,sequence_name FROM information_schema.sequences where sequence_schema=‘testdb';
Foreign
Constraints
SELECT
tc.table_schema, tc.constraint_name, tc.table_name, kcu.column_name,
ccu.table_schema AS foreign_table_schema, ccu.table_name AS foreign_table_name,
ccu.column_name AS foreign_column_name
FROM
information_schema.table_constraints AS tc JOIN information_schema.key_column_usage AS kcu ON tc.constraint_name = kcu.constraint_name
AND tc.table_schema = kcu.table_schema JOIN information_schema.constraint_column_usage AS ccu ON ccu.constraint_name =
tc.constraint_name AND ccu.table_schema = tc.table_schema
WHERE tc.constraint_type = 'FOREIGN KEY' and tc.table_schema=‘testdb'
DB LINK SELECT PRONAME,PRONAMESPACE,PROOWNER FROM pg_catalog.pg_proc
WHERE prosrc ILIKE '%dblink%';