It has just been a few months since the PostgreSQL9.5 is released. We have got some of our customers excited about great new features and performance enhancements in v9.5. But here we are already taking a peak into the next version, and we find it awesome! One of the most awaited features – parallelism makes it to Postgres. The infrastructure for parallelism has been added over last few releases but the first parallel operation in query execution will be seen only in v9.6.
Asynchronous cascading master to multiple replicas
Asynchronous multi-master
Can be used for:
Improved performance for geographically dispersed users
High availability
Load distribution (OLTP vs. reporting)
Countdown to PostgreSQL v9.5 - Foriegn Tables can be part of Inheritance Tree Ashnikbiz
Distributed databases and horizontal scale up is one of the key demands in today's date. PostgreSQL already had some vertical scaling features and horizontal scale-up by adding disks and table partitioning/child tables. With release of v9.5, PostgreSQL will get basic foundation for native sharing capability. From v9.5 Foreign Tables will be able to participate in Inheritance Tree as a child or parent table i.e. one can have table partitions residing on different system.
In our countdown to v9.5 series of hangouts, we will be covering some of the great features of PostgreSQL v9.5 and what is their real life applicability. In the first hangout in this series we will be talking about-
- The feature of foreign partitions/child tables
- Syntax and usage
- EXPLAIN plan demo
- Use cases and benefits
Join us for more and send us your queries on success@ashnik.com
Transform your DBMS to drive engagement innovation with Big DataAshnikbiz
Erik Baardse and Ajit Gadge from EDB Postgres presented on how to transform your DBMS in order to drive digital business. How Postgres enables you to support a wider range of workloads with your relational database which opens the Big Data doors. They also cover EnterpriseDB’s Strategy around Big Data which focuses on 3 areas and finally last but not the last how to find money in IT with Big Data and digital transformation
Various HA and DR setups for Postgres Plus Advanced Server -
Active – Passive OS HA Clustering
Log Shipping Replication (Hot Standby Mode)
Hot Streaming Replication (Hot Standby Mode)
EDB Postgres Plus Failover Manager
HA with read scaling (with pg-pool)
xDB Single Master Replication (SMR)
xDB Multi Master Replication (MMR)
Use Cases
Tuning the Memory and Optimizer Parameters - This topic has come quite often in requests and suggestions from our regular viewers and followers of our Postgres Hangouts.
So we picked it up as the topic for this month's Postgres Hangout. This time we will try to explore and share on Memory and Memory .
Join us as we discuss what impacts your decision before you put a number against each of these important parameters!
This technical presentation by EDB Dave Thomas, Systems Engineer provides an overview of:
1) BGWriter/Writer Process
2) Wall Writer Process
3) Stats Collector Process
4) Autovacuum Launch Process
5) Syslogger Process/Logger process
6) Archiver Process
7) WAL Send/Receive Processes
PostgreSQL Hangout Replication Features v9.4Ashnikbiz
See the new enhancements in v9.4 which takes away the pain of guessing right wal_keep_segment
See the new time lagging replication capability in v9.4
Short intro to logical replication introduced in v9.4
Asynchronous cascading master to multiple replicas
Asynchronous multi-master
Can be used for:
Improved performance for geographically dispersed users
High availability
Load distribution (OLTP vs. reporting)
Countdown to PostgreSQL v9.5 - Foriegn Tables can be part of Inheritance Tree Ashnikbiz
Distributed databases and horizontal scale up is one of the key demands in today's date. PostgreSQL already had some vertical scaling features and horizontal scale-up by adding disks and table partitioning/child tables. With release of v9.5, PostgreSQL will get basic foundation for native sharing capability. From v9.5 Foreign Tables will be able to participate in Inheritance Tree as a child or parent table i.e. one can have table partitions residing on different system.
In our countdown to v9.5 series of hangouts, we will be covering some of the great features of PostgreSQL v9.5 and what is their real life applicability. In the first hangout in this series we will be talking about-
- The feature of foreign partitions/child tables
- Syntax and usage
- EXPLAIN plan demo
- Use cases and benefits
Join us for more and send us your queries on success@ashnik.com
Transform your DBMS to drive engagement innovation with Big DataAshnikbiz
Erik Baardse and Ajit Gadge from EDB Postgres presented on how to transform your DBMS in order to drive digital business. How Postgres enables you to support a wider range of workloads with your relational database which opens the Big Data doors. They also cover EnterpriseDB’s Strategy around Big Data which focuses on 3 areas and finally last but not the last how to find money in IT with Big Data and digital transformation
Various HA and DR setups for Postgres Plus Advanced Server -
Active – Passive OS HA Clustering
Log Shipping Replication (Hot Standby Mode)
Hot Streaming Replication (Hot Standby Mode)
EDB Postgres Plus Failover Manager
HA with read scaling (with pg-pool)
xDB Single Master Replication (SMR)
xDB Multi Master Replication (MMR)
Use Cases
Tuning the Memory and Optimizer Parameters - This topic has come quite often in requests and suggestions from our regular viewers and followers of our Postgres Hangouts.
So we picked it up as the topic for this month's Postgres Hangout. This time we will try to explore and share on Memory and Memory .
Join us as we discuss what impacts your decision before you put a number against each of these important parameters!
This technical presentation by EDB Dave Thomas, Systems Engineer provides an overview of:
1) BGWriter/Writer Process
2) Wall Writer Process
3) Stats Collector Process
4) Autovacuum Launch Process
5) Syslogger Process/Logger process
6) Archiver Process
7) WAL Send/Receive Processes
PostgreSQL Hangout Replication Features v9.4Ashnikbiz
See the new enhancements in v9.4 which takes away the pain of guessing right wal_keep_segment
See the new time lagging replication capability in v9.4
Short intro to logical replication introduced in v9.4
hbaseconasia2017: Large scale data near-line loading method and architectureHBaseCon
Shuaifeng Zhou
When we do real-time data loading to HBase, we use put/putlist interface. After receiving put request, regionserver will write WAL, write data into memory store, flush memory store to disk-store, then compact files again and again. That precedure occupies too much resource and causing read/write performance decrease. To solve the problem, we provide a kind of near-line loading method and architecture, greatly increase the loading bandwidth, and decrease the influence to read operations.
hbaseconasia2017 hbasecon hbase https://www.eventbrite.com/e/hbasecon-asia-2017-tickets-34935546159#
hbaseconasia2017: HBase Disaster Recovery Solution at HuaweiHBaseCon
Ashish Singhi
HBase Disaster recovery solution aims to maintain high availability of HBase service in case of disaster of one HBase cluster with very minimal user intervention. This session will introduce the HBase disaster recovery use cases and the various solutions adopted at Huawei like.
a) Cluster Read-Write mode
b) DDL operations synchronization with standby cluster
c) Mutation and bulk loaded data replication
d) Further challenges and pending work
hbaseconasia2017 hbasecon hbase https://www.eventbrite.com/e/hbasecon-asia-2017-tickets-34935546159#
Architecture for building scalable and highly available Postgres ClusterAshnikbiz
As PostgreSQL has made way into business critical applications, many customers who are using Oracle RAC for high availability and load balancing have asked for similar functionality for using PostgreSQL.
In this Hangout session we would discuss architecture and alternatives, based on real life experience, for achieving high availability and load balancing functionality when you deploy PostgreSQL. We will also present some of the key tools and how to deploy them for effectiveness of this architecture.
HBaseCon2017 Removable singularity: a story of HBase upgrade in PinterestHBaseCon
HBase is used to serve online facing traffic in Pinterest. It means no downtime is allowed. However, we were on HBase 94. To upgrade to latest version, we need to figure out a way to live upgrade while keeping Pinterest site live. Recently, we successfully upgrade 94 HBase cluster to 1.2 with no downtime. We made change to both Asynchbase and HBase server side. We will talk about what we did and how we did it. We will also talk about the finding in config and performance tuning we did to achieve low latency.
This technical presentation shows you the best practices with EDB Postgres tools, that are designed to make database administration easier and more efficient:
● Tune a new database using Postgres Expert
● Set up streaming replication in EDB Postgres Enterprise Manager (PEM)
● Create a backup schedule in EDB Postgres Backup and Recovery
● Automatically failover with EDB Postgres Failover Manager
● Use SQL Profiler and Index Advisor to add indexes
The presentation also included a demonstration. To access the recording visit www.enterprisedb.com and access the webcast recordings section or email info@enterprisedb.com.
Security Best Practices for your Postgres DeploymentPGConf APAC
These slides were used by Sameer Kumar of Ashnik for presenting his topic at pgDay Asia 2016. He took audience through some of the security best practices for deploying and hardening PostgreSQL
Kafka is becoming an ever more popular choice for users to help enable fast data and Streaming. Kafka provides a wide landscape of configuration to allow you to tweak its performance profile. Understanding the internals of Kafka is critical for picking your ideal configuration. Depending on your use case and data needs, different settings will perform very differently. Lets walk through performance essentials of Kafka. Let's talk about how your Consumer configuration, can speed up or slow down the flow of messages to Brokers. Lets talk about message keys, their implications and their impact on partition performance. Lets talk about how to figure out how many partitions and how many Brokers you should have. Let's discuss consumers and what effects their performance. How do you combine all of these choices and develop the best strategy moving forward? How do you test performance of Kafka? I will attempt a live demo with the help of Zeppelin to show in real time how to tune for performance.
Lessons PostgreSQL learned from commercial databases, and didn’tPGConf APAC
This is the ppt used by Illay for his presentation at pgDay Asia 2016 - "Lessons PostgreSQL learned from commercial
databases, and didn’t". The talk takes you through some of the really good things that PostgreSQL has done really well and somethings that PostgreSQL can learn from other databases
Xinxin Fan and Hongxiang Jiang
First, we will give a brief introduction about the HBase service at Netease,include the basic cluster info and the key HBase service. And then we will talk same tips about the tuning practices for HBase. Last, we will introduce some improvements at the internal HBase version.
hbaseconasia2017 hbasecon hbase https://www.eventbrite.com/e/hbasecon-asia-2017-tickets-34935546159#
HBase-2.0.0 has been a couple of years in the making. It is chock-a-block full of a long list of new features and fixes. In this session, the 2.0.0 release manager will perform the impossible, describing the release content inside the session time bounds.
hbaseconasia2017 hbasecon hbase https://www.eventbrite.com/e/hbasecon-asia-2017-tickets-34935546159#
Hardware planning & sizing for sql serverDavide Mauri
Purchasing a dedicated server to SQL Server is still a necessary operation. The cloud is a great choice but if you need to create a data warehouse of non-trivial size or if you have the need for optimal performance and control of your production database server, the choice of on-premise server is still an optimal choice. So, how not to throw away money on unnecessary hardware? In this session we will see how each component works together to form a balanced hardware (this is the key word!), without bottlenecks, maximizing the investment made. We'll talk about SAN, CPU, HBA, Fibre Channel, Memory and everything you thought you knew well...
How to tune slow running SQLs in PostgreSQL? See this to know (with screenshots) -
1. See the explain plan and analyze the slow running query
2. Some basic tips for tuning the query
FOSSASIA 2016 - 7 Tips to design web centric high-performance applicationsAshnikbiz
Ashnik Database Solution Architect, Sameer Kumar, an Open Source evangelist shared some tips at FOSSASIA 2016 about how to design web-centric high-performance applications.
hbaseconasia2017: Large scale data near-line loading method and architectureHBaseCon
Shuaifeng Zhou
When we do real-time data loading to HBase, we use put/putlist interface. After receiving put request, regionserver will write WAL, write data into memory store, flush memory store to disk-store, then compact files again and again. That precedure occupies too much resource and causing read/write performance decrease. To solve the problem, we provide a kind of near-line loading method and architecture, greatly increase the loading bandwidth, and decrease the influence to read operations.
hbaseconasia2017 hbasecon hbase https://www.eventbrite.com/e/hbasecon-asia-2017-tickets-34935546159#
hbaseconasia2017: HBase Disaster Recovery Solution at HuaweiHBaseCon
Ashish Singhi
HBase Disaster recovery solution aims to maintain high availability of HBase service in case of disaster of one HBase cluster with very minimal user intervention. This session will introduce the HBase disaster recovery use cases and the various solutions adopted at Huawei like.
a) Cluster Read-Write mode
b) DDL operations synchronization with standby cluster
c) Mutation and bulk loaded data replication
d) Further challenges and pending work
hbaseconasia2017 hbasecon hbase https://www.eventbrite.com/e/hbasecon-asia-2017-tickets-34935546159#
Architecture for building scalable and highly available Postgres ClusterAshnikbiz
As PostgreSQL has made way into business critical applications, many customers who are using Oracle RAC for high availability and load balancing have asked for similar functionality for using PostgreSQL.
In this Hangout session we would discuss architecture and alternatives, based on real life experience, for achieving high availability and load balancing functionality when you deploy PostgreSQL. We will also present some of the key tools and how to deploy them for effectiveness of this architecture.
HBaseCon2017 Removable singularity: a story of HBase upgrade in PinterestHBaseCon
HBase is used to serve online facing traffic in Pinterest. It means no downtime is allowed. However, we were on HBase 94. To upgrade to latest version, we need to figure out a way to live upgrade while keeping Pinterest site live. Recently, we successfully upgrade 94 HBase cluster to 1.2 with no downtime. We made change to both Asynchbase and HBase server side. We will talk about what we did and how we did it. We will also talk about the finding in config and performance tuning we did to achieve low latency.
This technical presentation shows you the best practices with EDB Postgres tools, that are designed to make database administration easier and more efficient:
● Tune a new database using Postgres Expert
● Set up streaming replication in EDB Postgres Enterprise Manager (PEM)
● Create a backup schedule in EDB Postgres Backup and Recovery
● Automatically failover with EDB Postgres Failover Manager
● Use SQL Profiler and Index Advisor to add indexes
The presentation also included a demonstration. To access the recording visit www.enterprisedb.com and access the webcast recordings section or email info@enterprisedb.com.
Security Best Practices for your Postgres DeploymentPGConf APAC
These slides were used by Sameer Kumar of Ashnik for presenting his topic at pgDay Asia 2016. He took audience through some of the security best practices for deploying and hardening PostgreSQL
Kafka is becoming an ever more popular choice for users to help enable fast data and Streaming. Kafka provides a wide landscape of configuration to allow you to tweak its performance profile. Understanding the internals of Kafka is critical for picking your ideal configuration. Depending on your use case and data needs, different settings will perform very differently. Lets walk through performance essentials of Kafka. Let's talk about how your Consumer configuration, can speed up or slow down the flow of messages to Brokers. Lets talk about message keys, their implications and their impact on partition performance. Lets talk about how to figure out how many partitions and how many Brokers you should have. Let's discuss consumers and what effects their performance. How do you combine all of these choices and develop the best strategy moving forward? How do you test performance of Kafka? I will attempt a live demo with the help of Zeppelin to show in real time how to tune for performance.
Lessons PostgreSQL learned from commercial databases, and didn’tPGConf APAC
This is the ppt used by Illay for his presentation at pgDay Asia 2016 - "Lessons PostgreSQL learned from commercial
databases, and didn’t". The talk takes you through some of the really good things that PostgreSQL has done really well and somethings that PostgreSQL can learn from other databases
Xinxin Fan and Hongxiang Jiang
First, we will give a brief introduction about the HBase service at Netease,include the basic cluster info and the key HBase service. And then we will talk same tips about the tuning practices for HBase. Last, we will introduce some improvements at the internal HBase version.
hbaseconasia2017 hbasecon hbase https://www.eventbrite.com/e/hbasecon-asia-2017-tickets-34935546159#
HBase-2.0.0 has been a couple of years in the making. It is chock-a-block full of a long list of new features and fixes. In this session, the 2.0.0 release manager will perform the impossible, describing the release content inside the session time bounds.
hbaseconasia2017 hbasecon hbase https://www.eventbrite.com/e/hbasecon-asia-2017-tickets-34935546159#
Hardware planning & sizing for sql serverDavide Mauri
Purchasing a dedicated server to SQL Server is still a necessary operation. The cloud is a great choice but if you need to create a data warehouse of non-trivial size or if you have the need for optimal performance and control of your production database server, the choice of on-premise server is still an optimal choice. So, how not to throw away money on unnecessary hardware? In this session we will see how each component works together to form a balanced hardware (this is the key word!), without bottlenecks, maximizing the investment made. We'll talk about SAN, CPU, HBA, Fibre Channel, Memory and everything you thought you knew well...
How to tune slow running SQLs in PostgreSQL? See this to know (with screenshots) -
1. See the explain plan and analyze the slow running query
2. Some basic tips for tuning the query
FOSSASIA 2016 - 7 Tips to design web centric high-performance applicationsAshnikbiz
Ashnik Database Solution Architect, Sameer Kumar, an Open Source evangelist shared some tips at FOSSASIA 2016 about how to design web-centric high-performance applications.
NGINX Plus PLATFORM For Flawless Application DeliveryAshnikbiz
Flawless Application Delivery using Nginx Plus
By leveraging these latest features:
• Support for HTTP/2 standard
• Thread pools and socket sharding and how it can help improve performance
• NTLM support and new TCP security enhancements
• Advanced NGINX Plus monitoring, management and visibility of health & load checks
Catch this exclusive Google Hangout live!
November 4th, 2015 | 2.00-2.30PM IST | 4.30-5.00PM SGT
About the speaker: Sandeep Khuperkar, Director and CTO at Ashnik will be heading this session. He is an author, enthusiast and community moderator at opensource.com. He is also member of Open Source Initiative, Linux Foundation and Open Source Consortium Of India.
FOSSASIA 2015 - 10 Features your developers are missing when stuck with Propr...Ashnikbiz
Ashnik Database Solution Architect, Sameer Kumar, an Open Source evangelist presented at FOSSASIA 2015 about the features of open source database like PostgreSQL which are missed by developers stuck on proprietary databases.
10 Features you would love as an Open Source developer!
- New JSON Datatype
- Vast set of datatypes supported
- Rich support for foreign Data Wrap
- User Defined Operators
- User Defined Extensions
- Filter Based Indexes or Partial Indexes
- Granular control of parameters at User, Database, Connection or Transaction Level
- Use of indexes to get statistics
- JDBC API for COPY -Command
- Full Text Search
Building Data Integration and Transformations using PentahoAshnikbiz
This presentation will showcase the Data Integration capabilities of Pentaho which helps in building data transformations, through two demonstrations:
- How to build your first transformation to extract, transform and blend the data from various data sources
- How to add additional steps and filters to your transformation
PgDay Asia 2016 - Security Best Practices for your Postgres DeploymentAshnikbiz
Ashnik Database Solution Architect, Sameer Kumar, an Open Source database evangelist talked about the "Security Best Practices for your Postgres Deployment" at the recent pgDAy Asia event held in Singapore in March 2016.
Key areas he presented were:
- Security Model
- Security Features in Postgres
- Securing the access
- Avoiding common attacks
- Access Control and Securing data
- Logging and Auditing
- Patching – OS and PostgreSQL
This presentation goes over the PGSQL 9.3 key features along with several other exciting additions released in early September 2013. These features will also be available in the 9.3 release of Postgres Plus Advanced Server.
Include_dir configuration directive
Copy freeze
Custom background workers
Additional JSON functionality
Lateral join
Parallel pg_dump pg_isready
Posix shared memory/mmap
Event triggers
Materialized views
Recursive views
Updateable views
Writeable foreign tables / postgres_fdw
Streaming only remastering
Fast failover
Architecture-independent streaming
pg_basebackup recovery.conf autosetup
This EnterpriseDB presentation reviews backup and recovery best practices and the following helpful features in EnterpriseDB’s Backup and Recovery Tool (BART) and more:
Retention Policies: Control which backups are retained and for how long
Disk space awareness: Ensures you don’t start a backup if there’s not enough disk space.
Friendly backup names: Support for naming rules to make it easy to identify backups.
If you are seeking ways to improve your cloud database environment with EDB Postgres, this presentation reviews how you can create a Database-as-a-Service (DBaaS) with EDB Postgres on AWS.
This presentation outlines how EDB Ark can play a key role in your digital transformation with more agility and speed.
It highlights:
● How EDB Ark can integrate with your existing AWS environment and other clouds
● How you can automate your database deployments to instantly spin up new databases
● How to manage your database environment easier using the same GUI for all clouds
● How to boost developer efficiency and satisfaction
Whether your database is currently in the cloud or you are considering the cloud as an option, this presentation will provide you with the information you need to evaluate EDB Postgres and EDB Ark.
The recording of this presentation includes a demonstration. Visit www.edbpostgres.com > resources > webcasts
Big Data Business Transformation - Big Picture and BlueprintsAshnikbiz
Kaustubh Patwardhan, Head of Strategy and Business Development at Ashnik presents the big picture and blueprints of a big data journey for enterprises. The Value of Big Data – Machine Learning and its big impact. He covers a spectrum of Big Data use cases where right data storage, integration & data consolidation plays a big role.
This presentation introduces the following functionalities of pgAdmin and PEM that make database management more efficient:
1. Examining the performance of a query using the explain plan visualizer in pgAdmin’s Query Tool
2. Examining the performance of a process or session consisting of multiple queries in PEM’s SQL Profiler
3. 24/7 monitoring of Postgres and the underlying host system
4. Capacity management and reporting
5. Alerting the DBA or System Administrator to potential problems
In September 2016, the PostgreSQL community is rolling out PostgreSQL 9.6 which includes improvements in parallelism for query performance, overall performance improvements and the integration of foreign data sources.
This presentation introduces the new features of 9.6 and how they will benefit you.
- Parallel sequential scans, joins and aggregates
- Elimination of repetitive scanning of old data by autovacuum
- Synchronous replication now allows multiple standby servers for increased reliability
- Full-text search for phrases
- Support for remote joins, sorts, and updates in postgres_fdw
- Substantial performance improvements, especially in the area of improving scalability on many-CPU servers
If you have any questions on how to get started with Postgres, please email sales@enterprisedb.com
These slides were used by Victor from Tantan, a company who provides dating app which is very popular in China. He spoke about a key feature of PostGIS (a Geo-spatial extenion of PostgreSQL), which they used for finding perfect match.
This is the presentation used by Umari Shahid or 2nd Quadrant for his Presentation at pgDay Asia 2016. It takes you through usage of TABLESAMPLE clause of SELECT queries introduced in PostgreSQL v9.5.
Speaker: Darlene Nerden, IBM
Overview: In this session will review the Maximo architecture and factors that influence performance. We will discuss some details for those factors regarding tuning for a performance impact. We will look at troubleshooting tools and Maximo settings to help identify and resolve a Maximo performance issue.
MongoDB: How We Did It – Reanimating Identity at AOLMongoDB
AOL experienced explosive growth and needed a new database that was both flexible and easy to deploy with little effort. They chose MongoDB. Due to the complexity of internal systems and the data, most of the migration process was spent building a new identity platform and adapters for legacy apps to talk to MongoDB. Systems were migrated in 4 phases to ensure that users were not impacted during the switch. Turning on dual reads/writes to both legacy databases and MongoDB also helped get production traffic into MongoDB during the process. Ultimately, the project was successful with the help of MongoDB support. Today, the team has 15 shards, with 60-70 GB per shard.
The venerable Servlet Container still has some performance tricks up its sleeve - this talk will demonstrate Apache Tomcat's stability under high load, describe some do's (and some don'ts!), explain how to performance test a Servlet-based application, troubleshoot and tune the container and your application and compare the performance characteristics of the different Tomcat connectors. The presenters will share their combined experience supporting real Tomcat applications for over 20 years and show how a few small changes can make a big, big difference.
There are many computational paradigms that could be used to harness the power of the herd of computers. In financial services, a share-nothing approach could be used to speed up CPU intensive calculations while the hierarchal nature of rollups requires tight synchronization. Some interesting use cases are:
In Wealth Management, the SQL approach is traditionally used, but it lacks efficient support of hierarchal structures, iterative calculation, and provides limited scalability. Unlike traditional, centralized scale-up enterprise systems, an in-memory-based architecture scales out and takes advantage of cost-effective high volume commodity hardware that maximizes compute power efficiently. It makes the user experience better by speeding up response time utilizing distributed implementation of calculation algorithms. OData enables DaaS to expose financial data and calculation capabilities.
In the insurance industry, in-memory computing was used for Monte-Carlo to estimate the value of life insurance policies. This is a very CPU-intensive task, which requires 2000 cores to build ~1 million simulated policies in 30 minutes (about 25 trillion numbers or 100TB of data), which then aggregates and compresses into 40GB of data for analysis.
To speed up CPU-intensive iterative financial calculations, we use a share-nothing approach while the hierarchal nature of rollups requires tight synchronization. Several algorithms that are typical for the financial industry, different approaches on distribution and synchronization, and the benefits of in-memory data grid technologies will be discussed.
How to get the maximum performance from your AEP server. This will discuss ways to improve execution time of short running jobs and how to properly configure the server depending on the expected number of users as well as the average size and duration of individual jobs. Included will be examples of making use of job pooling, Database connection sharing, and parallel subprotocol tuning. Determining when to make use of cluster, grid, or load balanced configurations along with memory and CPU sizing guidelines will also be discussed.
Grails has great performance characteristics but as with all full stack frameworks, attention must be paid to optimize performance. In this talk Lari will discuss common missteps that can easily be avoided and share tips and tricks which help profile and tune Grails applications.
Performance Benchmarking: Tips, Tricks, and Lessons LearnedTim Callaghan
Presentation covering 25 years worth of lessons learned while performance benchmarking applications and databases. Presented at Percona Live London in November 2014.
Grails has great performance characteristics but as with all full stack frameworks, attention must be paid to optimize performance. In this talk Lari will discuss common missteps that can easily be avoided and share tips and tricks which help profile and tune Grails applications.
Intro to Apache Apex - Next Gen Platform for Ingest and TransformApache Apex
Introduction to Apache Apex - The next generation native Hadoop platform. This talk will cover details about how Apache Apex can be used as a powerful and versatile platform for big data processing. Common usage of Apache Apex includes big data ingestion, streaming analytics, ETL, fast batch alerts, real-time actions, threat detection, etc.
Bio:
Pramod Immaneni is Apache Apex PMC member and senior architect at DataTorrent, where he works on Apache Apex and specializes in big data platform and applications. Prior to DataTorrent, he was a co-founder and CTO of Leaf Networks LLC, eventually acquired by Netgear Inc, where he built products in core networking space and was granted patents in peer-to-peer VPNs.
Hangfire
An easy way to perform background processing in .NET and .NET Core applications. No Windows Service or separate process required.
Why Background Processing?
Lengthy operations like updating lot of records in DB
Checking every 2 hours for new data or files
Invoice generation at the end of every billing period
Monthly Reporting
Rebuild data, indexes or search-optimized index after data change
Automatic subscription renewal
Regular Mailings
Send an email due to an action
Background service provisioning
Apache Big Data 2016: Next Gen Big Data Analytics with Apache ApexApache Apex
Apache Apex is a next gen big data analytics platform. Originally developed at DataTorrent it comes with a powerful stream processing engine, rich set of functional building blocks and an easy to use API for the developer to build real-time and batch applications. Apex runs natively on YARN and HDFS and is used in production in various industries. You will learn about the Apex architecture, including its unique features for scalability, fault tolerance and processing guarantees, programming model and use cases.
http://apachebigdata2016.sched.org/event/6M0L/next-gen-big-data-analytics-with-apache-apex-thomas-weise-datatorrent
Similar to 2016 may-countdown-to-postgres-v96-parallel-query (20)
CloudDBOps is Ashnik's automation focussed UI tool which can help you seamlessly install and configure multiple technologies like Postgres, MongoDB, Elastic(ELK), Monitoring
Deploy and automate ‘Secrets Management’ for a multi-cloud environmentAshnikbiz
Over the years, there has been a massive transition from on-premise environments to hybrid or multi-cloud, resulting in a significant increase in the adoption of cloud-native practices and technologies. However, while cloud-native methodologies offer growing benefits and are instrumental to digitalization, they can pose considerable challenges in managing secrets.
Secrets management aims to solve a lack of visibility and control on handling these highly-trusted credentials.
Deploy, move and manage Postgres across cloud platformsAshnikbiz
Running applications in a hybrid set-up creates complexities that can increase downtime and maintenance. PostgreSQL runs across virtual, cloud, and container environments; minimizing complexity without sacrificing the performance, so you can take control. Being today’s undisputed leader of relational databases for new and modern applications, Postgres’ tools and features will enable you to swiftly deploy, move and manage your database across platforms.
Deploy, move and manage Postgres across cloud platformsAshnikbiz
Running applications in a hybrid setup creates complexities that can increase downtime and maintenance. PostgreSQL runs across virtual, cloud, and container environments; minimizing complexity without sacrificing the performance, so you can take control. Being today’s undisputed leader of relational databases for new and modern applications, Postgres’ tools and features will enable you to swiftly deploy, move and manage your database across platforms.
Webinar Covers:
Multi-cloud strategy and trends
How EDB Postgres can pillar your cloud platform
Use cases: Postgres and its tools on-premises and multi-cloud platforms
Demo: Using Postgres tools on-premises and for diverse cloud platforms – handling back-up, monitoring, and ensuring Business Continuity Process (BCP)
Which PostgreSQL is right for your multi cloud strategy? P2Ashnikbiz
The adoption of PostgreSQL in enterprises is becoming a strategic choice, more so with the adoption of Multi-Cloud now becoming a need for enterprise deployment. This availability creates multiple combinations of deployment options for you. So, it is important to identify the right strategy fitting into your organization’s needs.
Which PostgreSQL is right for your multi cloud strategy? P1Ashnikbiz
The adoption of PostgreSQL in enterprises is becoming a strategic choice, more so with the adoption of Multi-Cloud now becoming a need for enterprise deployment. This availability creates multiple combinations of deployment options for you. So, it is important to identify the right strategy fitting into your organization’s needs.
Reduce the complexities of managing Kubernetes clusters anywhere 2Ashnikbiz
Learn how Kubernetes has become a critical component for deploying applications on multi-platform / multi-cloud environments and how to manage and monitor clusters running Mirantis Kubernetes Engine (formerly Docker Enterprise) using Mirantis Container Cloud, AWS, VMware and other providers.
Reduce the complexities of managing Kubernetes clusters anywhereAshnikbiz
Learn how Kubernetes has become a critical component for deploying applications on multi-platform / multi-cloud environments and how to manage and monitor clusters running Mirantis Kubernetes Engine (formerly Docker Enterprise) using Mirantis Container Cloud, AWS, VMware and other providers.
Enhance your multi-cloud application performance using Redis Enterprise P2Ashnikbiz
In this webinar, our team will share how no matter which flavour of multi-cloud you choose, as a cloud-agnostic database, Redis Enterprise offers the best experience for multi and hybrid cloud deployments.
Enhance your multi-cloud application performance using Redis Enterprise P1Ashnikbiz
In this webinar, our team will share how no matter which flavour of multi-cloud you choose, as a cloud-agnostic database, Redis Enterprise offers the best experience for multi and hybrid cloud deployments.
Gain multi-cloud versatility with software load balancing designed for cloud-...Ashnikbiz
Over 50% organizations today are changing how they develop applications to support their digital transformation goals, and a multi-cloud strategy often plays a big role in that. For many organizations, it’s just not practical to be tied to one cloud anymore, given the flexibility of choosing the right cloud for each application.
Gain multi-cloud versatility with software load balancing designed for cloud-...Ashnikbiz
Over 50% organizations today are changing how they develop applications to support their digital transformation goals, and a multi-cloud strategy often plays a big role in that.
Enterprise-class security with PostgreSQL - 1Ashnikbiz
For businesses that handle personal data everyday, the security aspect of their database is of utmost importance.
With an increasing number of hack attacks and frauds, organizations want their open source databases to be fully equipped with the top security features.
Enterprise-class security with PostgreSQL - 2Ashnikbiz
For businesses that handle personal data everyday, the security aspect of their database is of utmost importance.
With an increasing number of hack attacks and frauds, organizations want their open source databases to be fully equipped with the top security features.
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
Kubernetes & AI - Beauty and the Beast !?! @KCD Istanbul 2024Tobias Schneck
As AI technology is pushing into IT I was wondering myself, as an “infrastructure container kubernetes guy”, how get this fancy AI technology get managed from an infrastructure operational view? Is it possible to apply our lovely cloud native principals as well? What benefit’s both technologies could bring to each other?
Let me take this questions and provide you a short journey through existing deployment models and use cases for AI software. On practical examples, we discuss what cloud/on-premise strategy we may need for applying it to our own infrastructure to get it to work from an enterprise perspective. I want to give an overview about infrastructure requirements and technologies, what could be beneficial or limiting your AI use cases in an enterprise environment. An interactive Demo will give you some insides, what approaches I got already working for real.
Connector Corner: Automate dynamic content and events by pushing a buttonDianaGray10
Here is something new! In our next Connector Corner webinar, we will demonstrate how you can use a single workflow to:
Create a campaign using Mailchimp with merge tags/fields
Send an interactive Slack channel message (using buttons)
Have the message received by managers and peers along with a test email for review
But there’s more:
In a second workflow supporting the same use case, you’ll see:
Your campaign sent to target colleagues for approval
If the “Approve” button is clicked, a Jira/Zendesk ticket is created for the marketing design team
But—if the “Reject” button is pushed, colleagues will be alerted via Slack message
Join us to learn more about this new, human-in-the-loop capability, brought to you by Integration Service connectors.
And...
Speakers:
Akshay Agnihotri, Product Manager
Charlie Greenberg, Host
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
Smart TV Buyer Insights Survey 2024 by 91mobiles.pdf91mobiles
91mobiles recently conducted a Smart TV Buyer Insights Survey in which we asked over 3,000 respondents about the TV they own, aspects they look at on a new TV, and their TV buying preferences.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
2. • Current State of Parallelism in PostgreSQL
• What was needed to bring server side parallelism – Work done in
v9.4 and v9.5
• Parallel Query in v9.6
• Review some parallel plans
• Parallelism may not be used always
• Parallelism may not be useful always
• Parameters
• Benefits
• Questions
Agenda
2
3. • Client side parallelism – Application can open multiple sessions
• One can run a batch with multiple application threads
• Server side languages can potentially do parallel operations
• I/O activity is taken off from main query execution process by wal
writer and bgwriter
• effective_io_concurrency allows page prefetch requests to the
kernel, for bitmap joins
• But there is no server side parallelism for dividing the same task
among multiple-workers
Current State (v9.5) of Parallelism in PostgreSQL
3
4. v9.4
• Dynamic background workers
• Dynamic shared memory
• Implementation of shared
memory message queues
v9.5
• Message propagation i.e. error
messages from background
worker can be sent to master
and received by master
• Synchronization of state (GUC
values, XID, CID mapping,
current user and current db
etc)
• Parallel Contexts can be used
by backend code to launch
worker processes
A lot of work was needed and was done!
4
5. • Parallel Sequential Scan
• Parallel Joins
• Parallel Aggregates
• Though these are not in their best forms and have certain
exceptions/limitations but they still work and quite useful!
v9.6: We have something that users can use!
5
13. Wow! So using ‘Parallel
Workers’ should be
preferred!
No, not really!
14. Parallel Query May not be used all the time
• Cost of working and coordinating among multiple worker
processes defeats the advantage of parallelism
• Cost of setting up parallelism infrastructure is too high
• No worker process is available
14
17. Parallel Query may not be good all the time
• It depends a lot on your hardware resources and process scheduling by
your OS
• I tried various degree of parallelism on a test machine
• 3 CPU, 3GB RAM
• VM Running CentOS
• Single I/O disk
• A simple ‘count’ on a table with 100million rows and 8 byte width
• explain analyze select count(*) from pgbench_accounts ;
• It performs faster with parallel degree set to 0, as index scan is
performed
• Make sure you have tuned your parameters well to help optimizer
decide
17
19. Parameters which govern parallel query execution
• parallel_setup_cost
• parallel_tuple_cost
• max_worker_processes
• max_parallel_degree
• force_parallel_mode
• ALTER TABLE … SET (parallel_degree=n)
• ALTER FUNCTION … PARALLEL SAFE
• ALTER FUNCTION … COST
19
20. Benefits to the users
• Sequential scan on large tables would be faster
• Analytics workload involve aggregates would be faster
• Faster JOINs between large tables
• PostgreSQL v9.6 can be a good candidate for the backend
database of data warehouse
• More parallel operations to come in future releases
20
21. What can you do?
• PostgreSQL Beta 1 is out
• Try it out…
• Test it…
• Break it…
• Report it
• Help PostgreSQL community make it better
21
22. Further Reading
• PGCon 2014: Implementing Parallelism in PostgreSQL, Robert
Haas
• PGConf.US, 2016: PostgreSQL 9.6, Magnus Hagander
• PGCon, Ottawa 2015: Parallel Sequential Scan, Robert Haas and
Amit Kapila at
• EnterpriseDB Blog: Parallelism Progress, Robert Haas
• Parallel Sequential Scan is Committed, Robert Haas
• EnterpriseDB Blog: Parallelism Becomes a Reality in Postgres, Amit
Kapila
22
23. Send us your suggestions and questions
success@ashnik.com
Stay Tuned!
Website: www.ashnik.com