Whats wrong with postgres | PGConf EU 2019 | Craig KerstiensCitus Data
Postgres is a powerful database, it continues to improve in terms of performance, extensibility, and more broadly in features. However it is not perfect.
Here I'll cover a highly opinionated view of all the areas Postgres falls flat, with some rough thought ideas on how we can make it better. Opinions are all informed by 10 years of interacting with customers running literally millions of databases for users.
New monetization models such as open core have presented questions for IT professionals. Do we stick with the freedom and agility of community releases, or do we pay for the enterprise counterparts? Explore patterns in Enterprise Edition add-ons, look at some concrete examples such as Confluent vs. Kafka, and be better prepared to decide where you spend your open source license dollars.
Open Source Library System Software: Libraries Are Doing it For Themselvesloriayre
One of the great advantages of an Open Source Library System (OSLS) such as Koha or Evergreen is the ability to empower staff and optimize the user's experience by getting involved in improving the software. This is in contrast to the traditional integrated library system (ILS) model where all the software development was done by "the vendor," creating a condition of "learned helplessness" on the part of library staff. By making the transition to OSLS, you can shift the culture of your organization from "learned helplessness" to one in which everyone can contribute to enhancing their work environment.
This webinar will describe all the ways to get involved with an OSLS project -- even if you aren't a programmer. By the end of the webinar, you will understand why involving your organization in an OSLS project creates opportunities for delivering new services to customers and optimizing the work of your staff.
Nancy CLI, a unified way to manage automated database experiments. Nancy CLI is an automated database management framework based on well-known open-source projects and incorporating major open-source tools.
Using these tools, casual DBAs can conduct automated experiments today, either on AWS EC2 Spot instances or on any other servers. All you need is to tell Nancy which database to use, how to determine workloads and what you want to verify – say, check how some index will help, or compare various values of "default_statistics_target" for your database and your workload.
Everything else Nancy will do for you, in fully automated fashion, in the end presenting you detailed results for comparison.
Another day, another buzzword in the world of software development! ‘Microservices’ is a new approach to structuring server-side software. But is it really new? In this talk I’ll walk you through the birth and ‘raison d’etre’ of microservices and tell about pro’s and con’s of the approach.
Having laid the foundation, we will take a look at best-practices and patterns for building micro service architectures and combine this with a tour of current technologies and development tools.
Finally, I will take a quick look at the future and discuss some of the remaining challenges. All parts of the presentation will be accompanied by structural examples based on a real ecommerse system.
Whats wrong with postgres | PGConf EU 2019 | Craig KerstiensCitus Data
Postgres is a powerful database, it continues to improve in terms of performance, extensibility, and more broadly in features. However it is not perfect.
Here I'll cover a highly opinionated view of all the areas Postgres falls flat, with some rough thought ideas on how we can make it better. Opinions are all informed by 10 years of interacting with customers running literally millions of databases for users.
New monetization models such as open core have presented questions for IT professionals. Do we stick with the freedom and agility of community releases, or do we pay for the enterprise counterparts? Explore patterns in Enterprise Edition add-ons, look at some concrete examples such as Confluent vs. Kafka, and be better prepared to decide where you spend your open source license dollars.
Open Source Library System Software: Libraries Are Doing it For Themselvesloriayre
One of the great advantages of an Open Source Library System (OSLS) such as Koha or Evergreen is the ability to empower staff and optimize the user's experience by getting involved in improving the software. This is in contrast to the traditional integrated library system (ILS) model where all the software development was done by "the vendor," creating a condition of "learned helplessness" on the part of library staff. By making the transition to OSLS, you can shift the culture of your organization from "learned helplessness" to one in which everyone can contribute to enhancing their work environment.
This webinar will describe all the ways to get involved with an OSLS project -- even if you aren't a programmer. By the end of the webinar, you will understand why involving your organization in an OSLS project creates opportunities for delivering new services to customers and optimizing the work of your staff.
Nancy CLI, a unified way to manage automated database experiments. Nancy CLI is an automated database management framework based on well-known open-source projects and incorporating major open-source tools.
Using these tools, casual DBAs can conduct automated experiments today, either on AWS EC2 Spot instances or on any other servers. All you need is to tell Nancy which database to use, how to determine workloads and what you want to verify – say, check how some index will help, or compare various values of "default_statistics_target" for your database and your workload.
Everything else Nancy will do for you, in fully automated fashion, in the end presenting you detailed results for comparison.
Another day, another buzzword in the world of software development! ‘Microservices’ is a new approach to structuring server-side software. But is it really new? In this talk I’ll walk you through the birth and ‘raison d’etre’ of microservices and tell about pro’s and con’s of the approach.
Having laid the foundation, we will take a look at best-practices and patterns for building micro service architectures and combine this with a tour of current technologies and development tools.
Finally, I will take a quick look at the future and discuss some of the remaining challenges. All parts of the presentation will be accompanied by structural examples based on a real ecommerse system.
AD113 Speed Up Your Applications w/ Nginx and PageSpeededm00se
My slide deck from my session, AD113: Speed Up Your Applications with Nginx + PageSpeed, at MWLUG 2015 in Atlanta, GA at the Ritz-Carlton.
For more, see:
- https://edm00se.io/self-promotion/mwlug-ad113-success
- https://github.com/edm00se/AD113-Speed-Up-Your-Apps-with-Nginx-and-PageSpeed
The Art of Database Experiments – PostgresConf Silicon Valley 2018 / San JoseNikolay Samokhvalov
Future database administration will be highly automated. Until then, we still live in a world where extensive manual interactions are required from a skilled DBA. This will change soon as more "autonomous databases" reach maturity and enter the production environment.
Postgres-specific monitoring tools and systems continue to improve, detecting and analyzing performance issues and bottlenecks in production databases. However, while these tools can detect current issues, they require highly-experienced DBAs to analyze and recommend mitigations.
In this session, the speaker will present the initial results of the POSTGRES.AI project – Nancy CLI, a unified way to manage automated database experiments. Nancy CLI is an automated database management framework based on well-known open-source projects and incorporating major open-source tools and Postgres modules: pgBadger, pg_stat_kcache, auto_explain, pgreplay, and others.
Originally developed with the goal to simulate various SQL query use cases in various environments and collect data to train ML models, Nancy CLI turned out to be very a universal framework that can play a crucial role in CI/CD pipelines in any company.
Using Nancy CLI, casual DBAs and any engineers can easily conduct automated experiments today, either on AWS EC2 Spot instances or on any other servers. All you need is to tell Nancy which database to use, specify workload (synthetic or "real", generated based on the Postgres logs), and what you want to test – say, check how a new index will affect all most expensive query groups from pg_stat_statements, or compare various values of "default_statistics_target". All the collected information with a very high level of confidence will give you understanding, how various queries and overall Postgres performance will be affected when you apply this change to production.
A Comparison of EDB Postgres to Self-Supported PostgreSQLEDB
Like all database management systems, PostgreSQL requires additional enterprise tools and capabilities to ensure high availability at scale. These include additional tools for backup, disaster recovery, replication, monitoring and data migration.
To meet these needs, EnterpriseDB created the EDB Postgres Platform.
This document explains the key differences between PostgreSQL using the EDB Postgres Platform compared to self-supported PostgreSQL alone.
During this webinar, we will review best practices and lessons learned from working with large and mid-size companies on their deployment of PostgreSQL. We will explore the practices that helped industry leaders move through these stages quickly, and get as much value out of PostgreSQL as possible without incurring undue risk.
We have identified a set of levers that companies can use to accelerate their success with PostgreSQL:
- Application Tiering
- Collaboration between DBAs and Development Teams
- Evangelizing
- Standardization and Automation
- Balance of Migration and New Development
This ppt is an quick introduction to sqlmap which is a tool used in ethical hacking for detecting and exploiting sql injection flaws and taking over of database servers. This slide covers the history of sqlmap, how it works and important sqlmap queries.
Michael Choi's process for designing web application(s), including which programming language to use, when to use Node.js, when to use a light-weight framework vs a heavy MVC framework, how to set up git for collaboration based on complexity of the project, how a tool like Jenkins can be used for continuous integration, continuous delivery, and continuous deployment, where to host the data, what services to use for orchestrating containers or servers.
Presentation that I gave at the August Ruby Meetup at ODEO in San Francisco. A bit light on content as most of the presentation was a live coding session.
This presentation is to introduce you to different ways of getting involved in open source community and how to get support. Most people are not aware of Open Source culture and they don't know how to be part of it. I hope this presentation will throw some light. This presentation was presented at Drupal Workshop held in May, 2010 in Hyderabad, India.
ChatGPT - a Conversational AI is a powerful tool that enables companies to provide exceptional customer experiences while safeguarding data privacy.
Building in-house Conversional AI capabilities, such as using the tools of ColossalAI, Haystack and Coati, allow organizations to maintain full control over customer data and ensure its confidentiality.
PostgreSQL 10 is the breakthrough revision of RDBMS with its real parallelism and Table Replication, Quorum Commit, and many more cool features. It is inevitable solution for Enterprises to replace any RDBMS to PostgreSQL.
AD113 Speed Up Your Applications w/ Nginx and PageSpeededm00se
My slide deck from my session, AD113: Speed Up Your Applications with Nginx + PageSpeed, at MWLUG 2015 in Atlanta, GA at the Ritz-Carlton.
For more, see:
- https://edm00se.io/self-promotion/mwlug-ad113-success
- https://github.com/edm00se/AD113-Speed-Up-Your-Apps-with-Nginx-and-PageSpeed
The Art of Database Experiments – PostgresConf Silicon Valley 2018 / San JoseNikolay Samokhvalov
Future database administration will be highly automated. Until then, we still live in a world where extensive manual interactions are required from a skilled DBA. This will change soon as more "autonomous databases" reach maturity and enter the production environment.
Postgres-specific monitoring tools and systems continue to improve, detecting and analyzing performance issues and bottlenecks in production databases. However, while these tools can detect current issues, they require highly-experienced DBAs to analyze and recommend mitigations.
In this session, the speaker will present the initial results of the POSTGRES.AI project – Nancy CLI, a unified way to manage automated database experiments. Nancy CLI is an automated database management framework based on well-known open-source projects and incorporating major open-source tools and Postgres modules: pgBadger, pg_stat_kcache, auto_explain, pgreplay, and others.
Originally developed with the goal to simulate various SQL query use cases in various environments and collect data to train ML models, Nancy CLI turned out to be very a universal framework that can play a crucial role in CI/CD pipelines in any company.
Using Nancy CLI, casual DBAs and any engineers can easily conduct automated experiments today, either on AWS EC2 Spot instances or on any other servers. All you need is to tell Nancy which database to use, specify workload (synthetic or "real", generated based on the Postgres logs), and what you want to test – say, check how a new index will affect all most expensive query groups from pg_stat_statements, or compare various values of "default_statistics_target". All the collected information with a very high level of confidence will give you understanding, how various queries and overall Postgres performance will be affected when you apply this change to production.
A Comparison of EDB Postgres to Self-Supported PostgreSQLEDB
Like all database management systems, PostgreSQL requires additional enterprise tools and capabilities to ensure high availability at scale. These include additional tools for backup, disaster recovery, replication, monitoring and data migration.
To meet these needs, EnterpriseDB created the EDB Postgres Platform.
This document explains the key differences between PostgreSQL using the EDB Postgres Platform compared to self-supported PostgreSQL alone.
During this webinar, we will review best practices and lessons learned from working with large and mid-size companies on their deployment of PostgreSQL. We will explore the practices that helped industry leaders move through these stages quickly, and get as much value out of PostgreSQL as possible without incurring undue risk.
We have identified a set of levers that companies can use to accelerate their success with PostgreSQL:
- Application Tiering
- Collaboration between DBAs and Development Teams
- Evangelizing
- Standardization and Automation
- Balance of Migration and New Development
This ppt is an quick introduction to sqlmap which is a tool used in ethical hacking for detecting and exploiting sql injection flaws and taking over of database servers. This slide covers the history of sqlmap, how it works and important sqlmap queries.
Michael Choi's process for designing web application(s), including which programming language to use, when to use Node.js, when to use a light-weight framework vs a heavy MVC framework, how to set up git for collaboration based on complexity of the project, how a tool like Jenkins can be used for continuous integration, continuous delivery, and continuous deployment, where to host the data, what services to use for orchestrating containers or servers.
Presentation that I gave at the August Ruby Meetup at ODEO in San Francisco. A bit light on content as most of the presentation was a live coding session.
This presentation is to introduce you to different ways of getting involved in open source community and how to get support. Most people are not aware of Open Source culture and they don't know how to be part of it. I hope this presentation will throw some light. This presentation was presented at Drupal Workshop held in May, 2010 in Hyderabad, India.
ChatGPT - a Conversational AI is a powerful tool that enables companies to provide exceptional customer experiences while safeguarding data privacy.
Building in-house Conversional AI capabilities, such as using the tools of ColossalAI, Haystack and Coati, allow organizations to maintain full control over customer data and ensure its confidentiality.
PostgreSQL 10 is the breakthrough revision of RDBMS with its real parallelism and Table Replication, Quorum Commit, and many more cool features. It is inevitable solution for Enterprises to replace any RDBMS to PostgreSQL.
Collapsing Narratives: Exploring Non-Linearity • a micro report by Rosie WellsRosie Wells
Insight: In a landscape where traditional narrative structures are giving way to fragmented and non-linear forms of storytelling, there lies immense potential for creativity and exploration.
'Collapsing Narratives: Exploring Non-Linearity' is a micro report from Rosie Wells.
Rosie Wells is an Arts & Cultural Strategist uniquely positioned at the intersection of grassroots and mainstream storytelling.
Their work is focused on developing meaningful and lasting connections that can drive social change.
Please download this presentation to enjoy the hyperlinks!
This presentation, created by Syed Faiz ul Hassan, explores the profound influence of media on public perception and behavior. It delves into the evolution of media from oral traditions to modern digital and social media platforms. Key topics include the role of media in information propagation, socialization, crisis awareness, globalization, and education. The presentation also examines media influence through agenda setting, propaganda, and manipulative techniques used by advertisers and marketers. Furthermore, it highlights the impact of surveillance enabled by media technologies on personal behavior and preferences. Through this comprehensive overview, the presentation aims to shed light on how media shapes collective consciousness and public opinion.
2. Your speaker
+61 2 9452 9087
haribabuk@fast.au.fujitsu.com
postgresql.fastware.com
3. Contents
Need of contribution to the PostgreSQL community
Advantages of contributing to community
Contribution methods
4. Need of contribution
PostgreSQL is a “world’s most advanced open source database” that powers many
mission critical systems around the world.
It needs support from individuals and companies to continue it’s development
further to compete against major commercial databases.
5. Advantages of contribution
Contributing to community will benefit both individuals and companies.
All the individuals that contributes to PostgreSQL echo system are listed as
contributors.
https://www.postgresql.org/community/contributors/
All the companies that contributes to PostgreSQL echo system are listed as
sponsors.
https://www.postgresql.org/about/sponsors/
6. Contribution methods
There are plenty of ways to contribute to PostgreSQL
(echo system) and many people are already doing
the same.
This topic is to list out the some of the possible ways
to contribute to PostgreSQL database by individuals
and companies.
7. Contribution methods
Donation
Participation in Survey
Hardware/Infrastructure support
Web updates/support
Events/Meetup groups
Documentation
Translation support
Answering questions
Build farm support
Bugs
Writing tools/extensions
Writing feature/bug fix patches
Reviewing of Patches
8. Donation
The Simple way to support PostgreSQL NFO is by
contributing some donation.
The funds that are collected by the PostgreSQL
groups are used for various activities such as
education, user groups and advocacy.
https://www.postgresql.org/about/donate/
9. Participation in Survey
PostgreSQL usually conducts surveys for every 4 to
6 months.
https://www.postgresql.org/community/
Why is it so important to participate in survey?
Where the survey results are used and how?
10. Participation in Survey
The results of the latest survey are available in the
following link:
https://www.postgresql.org/community/survey/93-what-
postgresql-10-feature-are-you-most-excited-about/
As you observe that, there are less number of
people responded to the survey.
Currently the open survey is not reaching many
users, needs to identify other approaches by the
infrastructure team.
11. Hardware/Infrastructure support
The servers that power the services of
postgresql.org are provided by different
companies and organisations around the world.
https://www.postgresql.org/about/servers/
Hardware support can be a test machine or a
performance machine
I listed a Power2 machine sponsored by IBM to
PostgreSQL for performance testing.
Power2 configuration
Architecture: ppc64le
Byte Order: Little Endian
CPU(s): 192
On-line CPU(s) list: 0-191
Thread(s) per core: 8
Core(s) per socket: 1
Socket(s): 24
NUMA node(s): 4
Model: IBM,8286-42A
L1d cache: 64K
L1i cache: 32K
L2 cache: 512K
L3 cache: 8192K
NUMA node0 CPU(s): 0-47
NUMA node1 CPU(s): 48-95
NUMA node2 CPU(s): 96-143
NUMA node3 CPU(s): 144-191
12. Web updates/support
The infrastructure team itself takes care of web
updates/support. Interested people can register with the
following.
https://lists.postgresql.org/manage/ (pgsql-www)
Currently PostgreSQL website doesn’t scale well on
mobile devices.
There is no proper details/issues that are available
anywhere in the website, so that people can participate in
the development. It needs an improvement from
community to increase the contribution.
13. Events/Meetup groups
Thanks for the Organizers and Sponsors of the PG Day
Australia event.
Conducting events/local meetups will help in growing
the PostgreSQL community.
Try to conduct monthly/quarterly meetups.
Following are some of the local meetup groups in
Australia.
https://www.meetup.com/Sydney-PostgreSQL-User-Group/
https://www.meetup.com/melpug/
https://www.meetup.com/Brisbane-PostgreSQL-User-Group/
14. Documentation
PostgreSQL community provides a very high quality
documentation, but there are cases that it may not handle
everything and some information may misleading and etc.
Register to the following mailing list and provide your
updates including the review of the changes suggested
by others.
https://lists.postgresql.org/manage/ (pgsql-docs)
If you’re a document developer or knows the tools that
are used for the documentation, you valuable information
is welcomed.
15. Translations support
PostgreSQL programs (server and client) can issue their messages in your
favourite language.
Creating and maintaining translated message sets needs the help of people who
speak their own language well and want to contribute to the PostgreSQL effort.
All the translation issues are tracked in redmine.
https://redmine.postgresql.org/projects/pgtranslation
There is a dedicated mailing list to discuss the translations updates/issues.
https://www.postgresql.org/list/pgsql-translators/
16. Answering questions
Try to answer the simple to complex questions that are
raised by the users not only related to the following,
Performance
General discussions
Administration
Etc.
PostgreSQL community mailing list is very supportive,
you will receive all the information that you needed.
17. Build farm support
Build farm is something like PostgreSQL continuous integration environment.
Currently there are extensive set of animals that are present in the build farm, but
still there is a need of some more machines with different set of configurations that
are used by users in the production environment.
If you have some specific hardware/software where you want to PostgreSQL to be
running without fail, it is better add that configuration as a build farm animal if that
doesn’t exist.
18. Bugs
PostgreSQL community review and it’s extensive build farm
support catches most of the problems, but still there may be
problems that can occur only with production data.
Need your support in validating new features or
performance enhancements with respect to quality and
functionality.
Many people will face problems in sharing the bug details to
the community. Steps to follow for quicker bug fix,
A reproducible test steps
If not, try to provide a call stack with debug symbols.
Or provide a core dump in a shared location.
19. Bugs
Once all details are captured, raise the bug with the
following bug reporting form.
https://www.postgresql.org/account/submitbug/
Try to register to the following mailing list to check the
progress of the bug and also the bug patterns, that can help
in further testing.
https://lists.postgresql.org/manage/ (pgsql-bugs)
20. Writing tools/extensions
PostgreSQL have many variety of tools and extensions that are useful for many
applications.
Before writing any tool/extension by your own for your needs, please make sure
that the existing are supported the same.
https://www.postgresql.org/download/product-categories/
Writing an extension is easier than writing a core feature.
21. Writing feature/bug fix patches
If you found a bug or interested in fixing bugs raised by
others, generate a patch and post it to the mailing list.
Community acts very quickly in fixing any bugs that arise.
Try to clarify the users when they raised a bug, but actually
that is an expected behavior with details.
22. Writing feature/bug fix patches
If you found some interesting feature and thought of to be
present in PostgreSQL.
Find out the use case scenario
Check for any older discussion on the mailing list
Post your idea to community.
Once everyone agrees and come to a common approach,
generate a POC patch.
Every feature that is submitted to PostgreSQL, has to follow the
review process.
https://commitfest.postgresql.org/
Definitely there may be complete rewrite of the patch based on
the feedback.
23. Reviewing Patches
PostgreSQL is a community database, it needs support from you not
just only features, but also from reviewing the submitted patches.
Please make sure that if you submit a patch, make sure that you
reviewed another patch.
Review can be anything from source code, documentation, test and
etc.
PostgreSQL community started recognizing the contribution from the
reviewers also by listing their names in the release notes from
version 10.
In the past, many patches are not received the much feedback from
reviewers especially from actual users of those features.