Oracle Partitioning for DBAs and Developers

•

4 likes•614 views

This document discusses Oracle partitioning for DBAs and developers. It begins with an introduction of the presenter and an agenda. It then covers partitioning concepts, background/history, benefits including increased availability and performance, implementation scenarios, types of partitioning like range and hash, how to implement range, list, interval and hash partitioning, partitioning maintenance operations, and concludes with a discussion of new features in Oracle releases.

Technology

1© The Pythian Group Inc., 2018
MAY 2018
WHY, WHEN AND HOW TO DO IT
ORACLE
PARTITIONING FOR
DBAS AND
DEVELOPERS

2© The Pythian Group Inc., 2018 2
FRANKY WEBER FAUST
● Senior Oracle Database Consultant at
Pythian
● 27 years old
● Writer at OTNLA and Lore Data Blog
● Speaker at some conferences
● High Availability specialist
● Performance researcher
● Exadata, RAC, DataGuard, Grid
Infrastructure, GoldenGate

© 2017 Pythian. Confidential 3
Years in Business
20 Pythian Experts
in 35 Countries
400+ Current Clients
Globally
350+

4© The Pythian Group Inc., 2018
AGENDA
4
● Partitioning concept
● Background
● Partitioning benefits (Why)
● Implementation scenarios (When)
● Types of partitioning (When)
● Range, List, Interval e Hash partitioning (How)
● The execution plan
● Partitioning maintenance operations*
● Historical data archiving*
● Converting partitions*

© 2017 Pythian. Confidential 5
PARTITIONING CONCEPTS
● Break down a table/index into
smaller pieces
● Partition Key
● Single or multicolumn
● Up to 16 columns
● Unique values are not required
● Every partition is a segment
● Tables
● Heap tables
● Index-organized tables
● Indexes
● Materialized Views
● Clusters
● Performance
● Partition Pruning

© 2017 Pythian. Confidential 6
BACKGROUND HISTORY
● Since Oracle 8
● Requires extra licensing and EE
● Manual partitioning using views
● New features every new release
Oracle 12cR2 Auto-list and Multicolumn list
partitioning, Partitioned
external tables
Table creation for partition exchange,
read-only partitions, Filtered partition
maintenance, Online table
conversion to partitioned table
Oracle 18c Online Merging of Partitions
and Subpartitions
Parallel Partition-Wise
SQL Operations
Modifying the Partitioning
Strategy
Partition Maintenance on multiple
partitions, Asynchronous global
index maintenance, Zone maps

© 2017 Pythian. Confidential 7
PARTITIONING BENEFITS (WHY)
● Increases availability of data
● Eases administration of large
segments
● Might improve the performance of
some queries
● Might reduce contention on hot
OLTP segments

© 2017 Pythian. Confidential 8
IMPLEMENTATION SCENARIOS (WHEN)
● Large volume of data when
administration becomes difficult
● Query performance on large tables
suffers
● Buffer busy waits on high-concurrent
tables
● Horizontal scalability in RAC
● Decreased storage cost
● Compression and row archival

© 2017 Pythian. Confidential 9
TYPES OF PARTITIONING (WHEN)
● Range Partitioning
● Dates and numbers are good
candidates
● Upper boundary
● Hash Partitioning
● High-volume OLTP segments
● Data is evenly distributed across
partitions
● List Partitioning
● Categories, colors, sports, brands
● Default values accepted
● Interval Partitioning
● Automated range partitions
● Reference Partitioning
● Relational model PK-FK
● Interval Reference Partitioning
● All child tables will be automatically
maintained
● Virtual Column Based Partitioning

© 2017 Pythian. Confidential 10
RANGE PARTITIONING (HOW)

© 2017 Pythian. Confidential 11
RANGE PARTITIONING (HOW)

© 2017 Pythian. Confidential 12
LIST PARTITIONING (HOW)

© 2017 Pythian. Confidential 13
LIST PARTITIONING (HOW)

© 2017 Pythian. Confidential 14
HASH PARTITIONING (HOW)

© 2017 Pythian. Confidential 15
HASH PARTITIONING (HOW)

© 2017 Pythian. Confidential 16
HASH PARTITIONING (HOW)

© 2017 Pythian. Confidential 17
HASH PARTITIONING (HOW)

© 2017 Pythian. Confidential 18
HASH PARTITIONING (HOW)

© 2017 Pythian. Confidential 19
HASH PARTITIONING (HOW)

© 2017 Pythian. Confidential 20
HASH PARTITIONING (HOW)

© 2017 Pythian. Confidential 21
HASH PARTITIONING (HOW)

© 2017 Pythian. Confidential 22
HASH PARTITIONING (HOW)

© 2017 Pythian. Confidential 23
HASH PARTITIONING (HOW)

© 2017 Pythian. Confidential 24
HASH PARTITIONING (HOW)

© 2017 Pythian. Confidential 25
INTERVAL PARTITIONING (HOW)

© 2017 Pythian. Confidential 26
INTERVAL PARTITIONING (HOW)

© 2017 Pythian. Confidential 27
INTERVAL PARTITIONING (HOW)

© 2017 Pythian. Confidential 28
INTERVAL PARTITIONING (HOW)

© 2017 Pythian. Confidential 29
INTERVAL PARTITIONING (HOW)

© 2017 Pythian. Confidential 30
THE EXECUTION PLAN

© 2017 Pythian. Confidential 31
THE EXECUTION PLAN

© 2017 Pythian. Confidential 32
THE EXECUTION PLAN

© 2017 Pythian. Confidential 33
THE EXECUTION PLAN

© 2017 Pythian. Confidential 34
THE EXECUTION PLAN

© 2017 Pythian. Confidential 35
PARTITIONING MAINTENANCE OPERATIONS

© 2017 Pythian. Confidential 36
PARTITIONING MAINTENANCE OPERATIONS

© 2017 Pythian. Confidential 37
PARTITIONING MAINTENANCE OPERATIONS

© 2017 Pythian. Confidential 38
CONCLUSION
● Range Partitioning, Hash
Partitioning, List Partitioning, Interval
Partitioning
● 12cR2 onwards can do most
partitioning operations online
● Performance problems in read or
write operations can get better or
worse with partitioning, so plan well
● New releases -> New features, it's
been like that for years.
● Questions?

Designing Apache Hudi for Incremental Processing With Vinoth Chandar and Ethan Guo | Current 2022 Back in 2016, Apache Hudi brought transactions, change capture on top of data lakes, what is today referred to as the Lakehouse architecture. In this session, we first introduce Apache Hudi and the key technology gaps it fills in the modern data architecture. Bridging traditional data lakes and warehouses, Hudi helps realize the Lakehouse vision, by bringing transactions, optimized table metadata to data lakes and powerful storage layout optimizations, moving them closer to cloud warehouses of today. Viewed from a data engineering lens, Hudi also plays a key unifying role between the batch and stream processing worlds, by acting as a columnar, server-less ""state store"" for batch jobs, ushering in what we call the incremental processing model, where batch jobs can consume new data, update/delete intermediate results in a Hudi table, instead of re-computing/re-write entire output like old-school big batch jobs. Rest of talk focusses on a deep dive into the some of the time-tested design choices and tradeoffs in Hudi, that helps power some of the largest transactional data lakes on the planet today. We will start by describing a tour of the storage format design, including data, metadata layouts and of course Hudi's timeline, an event log that is central to implementing ACID transactions and concurrency control. We will delve deeper into the practical concurrency control pitfalls in data lakes, and show how Hudi's hybrid approach combining MVCC with optimistic concurrency control, lowers contention and unlocks minute-level near real-time commits to Hudi tables. We will conclude with code examples that showcase Hudi's rich set of table services that perform vital table management such as cleaning older file versions, compaction of delta logs into base files, dynamic re-clustering for faster query performance, or the more recently introduced indexing service that maintains Hudi's multi-modal indexing capabilities.

Internals of Speeding up PySpark with Arrow

Databricks

Back in the old days of Apache Spark, using Python with Spark was an exercise in patience. Data was moving up and down from Python to Scala, being serialised constantly. Leveraging SparkSQL and avoiding UDFs made things better, likewise did the constant improvement of the optimisers (Catalyst and Tungsten). But, after Spark 2.3, PySpark has sped up tremendously thanks to the addition of the Arrow serialisers. In this talk you will learn how the Spark Scala core communicates with the Python processes, how data is exchanged across both sub-systems and the development efforts present and underway to make it as fast as possible.

Data Security at Scale through Spark and Parquet Encryption

Databricks

Big data presents new challenges for protection of privacy and integrity of sensitive information. Straightforward application of traditional file encryption and MAC techniques can’t cope with staggering volumes of data, flowing in modern analytic pipelines. Apple addresses these challenges by leveraging the new capabilities in the Apache Parquet format. We work with the Apache Parquet community on a modular data security mechanism, that provides privacy and integrity guarantees for sensitive information at scale; the encryption specification has been approved and released by the Apache Parquet Format project. Today, there are two open source implementations of this specification – in Apache Arrow (C++) and in Apache Parquet-MR (Java) repositories. The latter has just been released in the parquet-mr-1.12 version – which means the Apache Spark and other Java/Scala based analytic frameworks can start working with Apache Parquet encryption. In this talk, Gidon Gershinsky and Tim Perelmutov will outline the challenges of protecting the privacy of data at scale and describe the Apache Parquet encryption technology security approach. We will give a quick intro to usage of Apache Parquet encryption API in pure Java and in Apache Spark applications. We will also discuss the roadmap of the community work on new encryption features and on deeper integration with Apache Spark and other analytic frameworks. Finally, we will show a demo of the Apache Parquet modular encryption in action, sharing our learnings using it at scale.

Apache Kafka’s Transactions in the Wild! Developing an exactly-once KafkaSink...

HostedbyConfluent

Apache Kafka is one of the most commonly used connectors with Apache Flink for exactly-once streaming use cases. The combination of both systems allows you to build mission-critical systems that require low end-to-end latency and exactly-once processing eg. banks processing transactions. In Apache Flink 1.14, we released a new KafkaSink based on Apache Flink’s unified Sink interface that natively supports streaming and batch executions. However, we needed to stretch Kafka’s transactions API to fully support exactly-once processing in Flink. In this talk, we will start with a quick recap of Apache Kafka’s transactions and Flink’s checkpointing mechanism. Then, we describe the two-phase commit protocol implemented in KafkaSink in-depth and emphasize the difficulties we have overcome when applying Kafka’s transaction API to longer-lasting transactions. We explain how we ensure performant writing to Apache Kafka and how the KafkaSink recovery works. In summary, this talk should give users a deep dive into how Apache Flink leverages Apache Kafka’s transactions and developers an overview of what they have to consider when using Apache Kafka’s transactions.

Oracle GoldenGate and Apache Kafka: A Deep Dive Into Real-Time Data Streaming

Michael Rainey

We produce quite a lot of data! Much of the data are business transactions stored in a relational database. More frequently, the data are non-structured, high volume and rapidly changing datasets known in the industry as Big Data. The challenge for data integration professionals is to combine and transform the data into useful information. Not just that, but it must also be done in near real-time and using a target system such as Hadoop. The topic of this session, real-time data streaming, provides a great solution for this challenging task. By integrating GoldenGate, Oracle’s premier data replication technology, and Apache Kafka, the latest open-source streaming and messaging system, we can implement a fast, durable, and scalable solution. Presented at Oracle OpenWorld 2016

Parquet Hadoop Summit 2013

Julien Le Dem

Data Engineer's Lunch #83: Strategies for Migration to Apache Iceberg

Anant Corporation

Spark SQL is a highly scalable and efficient relational processing engine with ease-to-use APIs and mid-query fault tolerance. It is a core module of Apache Spark. Spark SQL can process, integrate and analyze the data from diverse data sources (e.g., Hive, Cassandra, Kafka and Oracle) and file formats (e.g., Parquet, ORC, CSV, and JSON). This talk will dive into the technical details of SparkSQL spanning the entire lifecycle of a query execution. The audience will get a deeper understanding of Spark SQL and understand how to tune Spark SQL performance.

Apache Spark Data Source V2 with Wenchen Fan and Gengliang Wang

Databricks

As a general computing engine, Spark can process data from various data management/storage systems, including HDFS, Hive, Cassandra and Kafka. For flexibility and high throughput, Spark defines the Data Source API, which is an abstraction of the storage layer. The Data Source API has two requirements. 1) Generality: support reading/writing most data management/storage systems. 2) Flexibility: customize and optimize the read and write paths for different systems based on their capabilities. Data Source API V2 is one of the most important features coming with Spark 2.3. This talk will dive into the design and implementation of Data Source API V2, with comparison to the Data Source API V1. We also demonstrate how to implement a file-based data source using the Data Source API V2 for showing its generality and flexibility.

Snowflake essentials

qureshihamid

Simplify CDC Pipeline with Spark Streaming SQL and Delta Lake

Databricks

Change Data Capture CDC is a typical use case in Real-Time Data Warehousing. It tracks the data change log -binlog- of a relational database [OLTP], and replay these change log timely to an external storage to do Real-Time OLAP, such as delta/kudu. To implement a robust CDC streaming pipeline, lots of factors should be concerned, such as how to ensure data accuracy , how to process OLTP source schema changed, whether it is easy to build for variety databases with less code.

Redpanda and ClickHouse

Altinity Ltd

ORC Deep Dive 2020

Owen O'Malley

Zero to Snowflake Presentation

Brett VanderPlaats

Using Apache Arrow, Calcite, and Parquet to Build a Relational Cache

Dremio Corporation

From DataEngConf 2017 - Everybody wants to get to data faster. As we move from more general solution to specific optimization techniques, the level of performance impact grows. This talk will discuss how layering in-memory caching, columnar storage and relational caching can combine to provide a substantial improvement in overall data science and analytical workloads. It will include a detailed overview of how you can use Apache Arrow, Calcite and Parquet to achieve multiple magnitudes improvement in performance over what is currently possible.

Examining Oracle GoldenGate Trail Files

Bobby Curtis

Extreme Replication - Performance Tuning Oracle GoldenGate

Bobby Curtis

YugaByte DB Internals - Storage Engine and Transactions

Yugabyte

Snowflake for Data Engineering

Harald Erb

Iceberg + Alluxio for Fast Data Analytics

Alluxio, Inc.

Squirreling Away $640 Billion: How Stripe Leverages Flink for Change Data Cap...

Flink Forward

Flink Forward San Francisco 2022. Being in the payments space, Stripe requires strict correctness and freshness guarantees. We rely on Flink as the natural solution for delivering on this in support of our Change Data Capture (CDC) infrastructure. We heavily rely on CDC as a tool for capturing data change streams from our databases without critically impacting database reliability, scalability, and maintainability. Data derived from these streams is used broadly across the business and powers many of our critical financial reporting systems totalling over $640 Billion in payment volume annually. We use many components of Flink’s flexible DataStream API to perform aggregations and abstract away the complexities of stream processing from our downstreams. In this talk, we’ll walk through our experience from the very beginning to what we have in production today. We’ll share stories around the technical details and trade-offs we encountered along the way. by Jeff Chao

Building an open data platform with apache iceberg

Alluxio, Inc.

A Thorough Comparison of Delta Lake, Iceberg and Hudi

Databricks

Maximum Availability Architecture - Best Practices for Oracle Database 19c

Glen Hawkins

Provides the latest updates on high availability (HA) best practices in this well-established technical deep-dive session. Learn how to optimize all aspects of Oracle Active Data Guard 19c. See how to use session draining, transparent application continuity, Oracle RAC, and Oracle GoldenGate to mask outages and planned maintenance from users and to accelerate time to repair for single database or your fleet of databases. Hear about the latest HA best practices with Oracle Multitenant and understand how the new sharded architecture can achieve even higher levels of HA and fault isolation for OLTP applications. Find out how everything you know about Oracle Maximum Availability Architecture (MAA) on-premises can be deployed in the cloud.

The Apache Spark File Format Ecosystem

Databricks

Oracle 12c PDB insights

Kirill Loifman

File Format Benchmark - Avro, JSON, ORC & Parquet

DataWorks Summit/Hadoop Summit

PL/SQL Camp MeetUp 2019 - Oracle Partitioning for Developers

Franky Weber Faust

Leveraging partition enhancements

Aishwarya Kala

What's hot

Deep Dive into Spark SQL with Advanced Performance Tuning with Xiao Li & Wenc...

Databricks

Apache Spark Data Source V2 with Wenchen Fan and Gengliang Wang

Databricks

Snowflake essentials

qureshihamid

Simplify CDC Pipeline with Spark Streaming SQL and Delta Lake

Databricks

Redpanda and ClickHouse

Altinity Ltd

ORC Deep Dive 2020

Owen O'Malley

Zero to Snowflake Presentation

Brett VanderPlaats

Using Apache Arrow, Calcite, and Parquet to Build a Relational Cache

Dremio Corporation

Examining Oracle GoldenGate Trail Files

Bobby Curtis

Extreme Replication - Performance Tuning Oracle GoldenGate

Bobby Curtis

YugaByte DB Internals - Storage Engine and Transactions

Yugabyte

Snowflake for Data Engineering

Harald Erb

Iceberg + Alluxio for Fast Data Analytics

Alluxio, Inc.

Squirreling Away $640 Billion: How Stripe Leverages Flink for Change Data Cap...

Flink Forward

Building an open data platform with apache iceberg

Alluxio, Inc.

A Thorough Comparison of Delta Lake, Iceberg and Hudi

Databricks

Maximum Availability Architecture - Best Practices for Oracle Database 19c

Glen Hawkins

The Apache Spark File Format Ecosystem

Databricks

Oracle 12c PDB insights

Kirill Loifman

File Format Benchmark - Avro, JSON, ORC & Parquet

DataWorks Summit/Hadoop Summit

What's hot (20)

Deep Dive into Spark SQL with Advanced Performance Tuning with Xiao Li & Wenc...

Apache Spark Data Source V2 with Wenchen Fan and Gengliang Wang

Snowflake essentials

Simplify CDC Pipeline with Spark Streaming SQL and Delta Lake

Redpanda and ClickHouse

ORC Deep Dive 2020

Zero to Snowflake Presentation

Using Apache Arrow, Calcite, and Parquet to Build a Relational Cache

Examining Oracle GoldenGate Trail Files

Extreme Replication - Performance Tuning Oracle GoldenGate

YugaByte DB Internals - Storage Engine and Transactions

Snowflake for Data Engineering

Iceberg + Alluxio for Fast Data Analytics

Squirreling Away $640 Billion: How Stripe Leverages Flink for Change Data Cap...

Building an open data platform with apache iceberg

A Thorough Comparison of Delta Lake, Iceberg and Hudi

Maximum Availability Architecture - Best Practices for Oracle Database 19c

The Apache Spark File Format Ecosystem

Oracle 12c PDB insights

File Format Benchmark - Avro, JSON, ORC & Parquet

Similar to Oracle Partitioning for DBAs and Developers

PL/SQL Camp MeetUp 2019 - Oracle Partitioning for Developers

Franky Weber Faust

Leveraging partition enhancements

Aishwarya Kala

graymatter-pentaho-consulting-services-.pdf

GrayMatter Software Services

GrayMatter's partnership with Pentaho, spanning over a decade, reflects their deep expertise in business analytics. They have successfully served 100+ global companies, establishing best practices and a profound understanding of data-driven solutions. Their performance dashboard and select clientele demonstrate their global reach and customer engagement metrics. GrayMatter's Top Projects Showcase provides concise insights into their Pentaho consulting, Big Data solutions, and stress performance testing capabilities. At the heart of their services are Business Analytics, covering dashboards, reporting, ad-hoc analysis, visualization, and data mining. Data Integration Services ensure data quality, governance, ETL, master data management, and data warehousing, facilitating data accessibility and reliability. In the realm of Big Data, GrayMatter's Pentaho proficiency empowers businesses to harness the potential of vast data sources. They also offer seamless migration services and support for embedding Pentaho within organizations, ensuring its seamless integration. Their extensive project portfolio spans various industries, with notable case studies such as Stream Global, highlighting their remarkable scalability in implementing Enterprise Data Warehouses with Pentaho Data Integration. In summary, GrayMatter's Pentaho Competency Centre is a testament to their dedication to data-driven excellence, making them a leading player in the business analytics arena. Their partnership with Pentaho isn't just about technology; it's about transforming businesses through data-driven insights that drive success.

[db tech showcase Tokyo 2017] C13:There and back again or how to connect Orac...

Insight Technology, Inc.

The presentation will show how we can combine Oracle relation database and ''Big Data'' platforms using Oracle and 3-d party tools and utilities. We will discuss ''Big Data'' and what we mean by it, talk about different topologies and how they can be applied to the business needs. Primarily the presentation will focus on Oracle integration tools such as Oracle GoldenGate, ODI, BD SQL and how they can help us with data ingestion and further usage of the data in our workflow. In addition, we are going to touch alternative tools and utilities and how they can be used.

There and back_again_oracle_and_big_data_16x9

Gleb Otochkin

Delicious : EDQ, OGG and ODI over Exadata for Perfection

Gurcan Orhan

Oracle Golden Gate (OGG) handles extraction phase for operational reporting, Enterprise Data Quality (EDQ) leverage source system based data quality problems (misspells/duplications in defined or undefined conditions) and Oracle Data Integrator (ODI) handles and controls both of those tools and moreover loading your data into data warehouse. These 3 tools are integrated to work together while you have control of OGG and EDQ via ODI over Exadata machine in a faster manner with minimum effort of development and tools already exist in ODI. At the same time use Exadata's extensive features to decrease ETL jobs' duration and obtain high availibility. In this presentation, see how this 3 tools are merged to be used together.

Why are we excited about MySQL 8? / Петр Зайцев (Percona)

Ontico

HighLoad++ 2017 Зал «Мумбай», 7 ноября, 17:00 Тезисы: http://www.highload.ru/2017/abstracts/2953.html MySQL 8 is coming! As large jump in the version implies this is the largest update in the MySQL space since MySQL 5.0 was released over 12 years ago. Are you excited about MySQL 8? We are! In this presentation we will talk about features new to MySQL 8, taking a practical look - how do those features are useful for me as developer using MySQL or as a person responsible for MySQL Operations.

How to scale MongoDB

Igor Donchovski

Redundancy and high availability are the basis for all production deployments. Database systems with large data sets or high throughput applications can challenge the capacity of a single server like CPU for high query rates or RAM for large working sets. Adding more CPU and RAM for vertical scaling is limited. Systems need horizontal scaling by distributing data across multiple servers. MongoDB supports horizontal scaling through sharding.

Mysql query optimization best practices and indexing

Alkin Tezuysal

Why Data Vault?

TESCHGlobal

One bridge to connect them all. Oracle GoldenGate for Big Data.UKOUG Tech 2018

Gleb Otochkin

YugaByte DB - "Designing a Distributed Database Architecture for GDPR Complia...

Jimmy Guerrero

Join Karthik Ranganathan (YugaByte CTO) for an in-depth technical webinar to understand how developers and administrators alike, can design systems that enable users to control the sharing and protection of their personal data so that it complies with GDPR. Topics covered include schema design, data partitioning, encryption and replication. Karthik will draw on his experience helping scale Facebook's Messenger and Inbox Search along with real-world implementations which make use of YugaByte DB.

Oracle Database Cloud Service - Provisioning Your First DBaaS Instance

Revelation Technologies

Moving to the cloud is inevitable, yet many haven’t started getting their hands dirty yet. This presentation will walk you through key Oracle Cloud concepts for those who have never seen or used it before, and walk through provisioning a cloud database, which includes storage and infrastructure. We explain how to access it, and discuss various administration concepts. This presentation is perfect for the Oracle DBA who wants to understand how to get started administering an Oracle DBaaS environment, from a technical perspective.

Alkin Tezuysal "MySQL Query Optimization Best Practices and Indexing"

Fwdays

Make your data fly - Building data platform in AWS

Kimmo Kantojärvi

Implementing MySQL Database-as-a-Service using open source tools

All Things Open

A Planet-Scale Database for Low Latency Transactional Apps by Yugabyte

Carlos Andrés García

A Planet-Scale Database for Low Latency Transactional Apps by Yugabyte

VMware Tanzu

How YugaByte DB Implements Distributed PostgreSQL

Yugabyte

Building applications on PostgreSQL that require automatic data sharding and replication, fault tolerance, distributed transactions and geographic data distribution has been hard. In this 3 hour workshop, we will look at how to do this using a real-world example running on top of YugaByte DB, a distributed database that is fully wire-compatible with PostgreSQL and NoSQL APIs (Apache Cassandra and Redis). We will look at the design and architecture of YugaByte DB and how it reuses the PostgreSQL codebase to achieve full API compatibility. YugaByte DB support for PostgreSQL includes most data types, queries, stored procedures, etc. We will also take a look at how to build applications that are planet scale (requiring geographic distribution of data) and how to run them in cloud-native environments (for example, Kubernetes, hybrid or multi-cloud deployments).

Mastering Data Modeling for NoSQL Platforms

DATAVERSITY

Data is proliferating at an accelerated rate, with all the mobile and desktop apps, social media, online purchasing, and consumer loyalty programs available today. All of these data sources have not just changed the way we operate on a day-to-day basis, but it has immensely increased the volume, velocity, and variety of data being created. Faced with this growing trend, data professionals now often have to look beyond the relational database to NoSQL database technologies to fully address their data management needs for data lakes, data warehouses, and other data stores. IDERA’s Ron Huizenga will discuss the NoSQL data modeling support included in ER/Studio, including round-trip engineering for Hadoop Hive and MongoDB.

Similar to Oracle Partitioning for DBAs and Developers (20)

PL/SQL Camp MeetUp 2019 - Oracle Partitioning for Developers

Leveraging partition enhancements

graymatter-pentaho-consulting-services-.pdf

[db tech showcase Tokyo 2017] C13:There and back again or how to connect Orac...

There and back_again_oracle_and_big_data_16x9

Delicious : EDQ, OGG and ODI over Exadata for Perfection

Why are we excited about MySQL 8? / Петр Зайцев (Percona)

How to scale MongoDB

Mysql query optimization best practices and indexing

Why Data Vault?

One bridge to connect them all. Oracle GoldenGate for Big Data.UKOUG Tech 2018

YugaByte DB - "Designing a Distributed Database Architecture for GDPR Complia...

Oracle Database Cloud Service - Provisioning Your First DBaaS Instance

Alkin Tezuysal "MySQL Query Optimization Best Practices and Indexing"

Make your data fly - Building data platform in AWS

Implementing MySQL Database-as-a-Service using open source tools

A Planet-Scale Database for Low Latency Transactional Apps by Yugabyte

How YugaByte DB Implements Distributed PostgreSQL

Mastering Data Modeling for NoSQL Platforms

Recently uploaded

LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...

DanBrown980551

Do you want to learn how to model and simulate an electrical network from scratch in under an hour? Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)! During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook. PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides: - A fully editable and extendable library for grid component modelling; - Visualization tools to display your network; - Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses; The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well. What you will learn during the webinar: - For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills; - For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.

FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf

FIDO Alliance

Pushing the limits of ePRTC: 100ns holdover for 100 days

Adtran

FIDO Alliance Osaka Seminar: Overview.pdf

FIDO Alliance

Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...

SOFTTECHHUB

The choice of an operating system plays a pivotal role in shaping our computing experience. For decades, Microsoft's Windows has dominated the market, offering a familiar and widely adopted platform for personal and professional use. However, as technological advancements continue to push the boundaries of innovation, alternative operating systems have emerged, challenging the status quo and offering users a fresh perspective on computing. One such alternative that has garnered significant attention and acclaim is Nitrux Linux 3.5.0, a sleek, powerful, and user-friendly Linux distribution that promises to redefine the way we interact with our devices. With its focus on performance, security, and customization, Nitrux Linux presents a compelling case for those seeking to break free from the constraints of proprietary software and embrace the freedom and flexibility of open-source computing.

DevOps and Testing slides at DASA Connect

Kari Kakkonen

GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...

James Anderson

Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management. The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM). Speakers: Bob Boule Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle. Gopinath Rebala Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.

Le nuove frontiere dell'AI nell'RPA con UiPath Autopilot™

UiPathCommunity

In questo evento online gratuito, organizzato dalla Community Italiana di UiPath, potrai esplorare le nuove funzionalità di Autopilot, il tool che integra l'Intelligenza Artificiale nei processi di sviluppo e utilizzo delle Automazioni. 📕 Vedremo insieme alcuni esempi dell'utilizzo di Autopilot in diversi tool della Suite UiPath: Autopilot per Studio Web Autopilot per Studio Autopilot per Apps Clipboard AI GenAI applicata alla Document Understanding 👨‍🏫👨‍💻 Speakers: Stefano Negro, UiPath MVPx3, RPA Tech Lead @ BSP Consultant Flavio Martinelli, UiPath MVP 2023, Technical Account Manager @UiPath Andrei Tasca, RPA Solutions Team Lead @NTT Data

Encryption in Microsoft 365 - ExpertsLive Netherlands 2024

Albert Hoitingh

GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...

Sri Ambati

Free Complete Python - A step towards Data Science

RinaMondal9

Enhancing Performance with Globus and the Science DMZ

Globus

State of ICS and IoT Cyber Threat Landscape Report 2024 preview

Prayukth K V

The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development. The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers: State of global ICS asset and network exposure Sectoral targets and attacks as well as the cost of ransom Global APT activity, AI usage, actor and tactic profiles, and implications Rise in volumes of AI-powered cyberattacks Major cyber events in 2024 Malware and malicious payload trends Cyberattack types and targets Vulnerability exploit attempts on CVEs Attacks on counties – USA Expansion of bot farms – how, where, and why In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East Why are attacks on smart factories rising? Cyber risk predictions Axis of attacks – Europe Systemic attacks in the Middle East Download the full report from here: https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/

Secstrike : Reverse Engineering & Pwnable tools for CTF.pptx

nkrafacyberclub

Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf

Paige Cruz

Monitoring and observability aren’t traditionally found in software curriculums and many of us cobble this knowledge together from whatever vendor or ecosystem we were first introduced to and whatever is a part of your current company’s observability stack. While the dev and ops silo continues to crumble….many organizations still relegate monitoring & observability as the purview of ops, infra and SRE teams. This is a mistake - achieving a highly observable system requires collaboration up and down the stack. I, a former op, would like to extend an invitation to all application developers to join the observability party will share these foundational concepts to build on:

Monitoring Java Application Security with JDK Tools and JFR Events

Ana-Maria Mihalceanu

FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf

FIDO Alliance

By Design, not by Accident - Agile Venture Bolzano 2024

Pierluigi Pugliese

The Metaverse and AI: how can decision-makers harness the Metaverse for their...

Jen Stirrup

The Metaverse is popularized in science fiction, and now it is becoming closer to being a part of our daily lives through the use of social media and shopping companies. How can businesses survive in a world where Artificial Intelligence is becoming the present as well as the future of technology, and how does the Metaverse fit into business strategy when futurist ideas are developing into reality at accelerated rates? How do we do this when our data isn't up to scratch? How can we move towards success with our data so we are set up for the Metaverse when it arrives? How can you help your company evolve, adapt, and succeed using Artificial Intelligence and the Metaverse to stay ahead of the competition? What are the potential issues, complications, and benefits that these technologies could bring to us and our organizations? In this session, Jen Stirrup will explain how to start thinking about these technologies as an organisation.

The Future of Platform Engineering

Jemma Hussein Allen

Recently uploaded (20)

LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...

FIDO Alliance Osaka Seminar: The WebAuthn API and Discoverable Credentials.pdf

Pushing the limits of ePRTC: 100ns holdover for 100 days

FIDO Alliance Osaka Seminar: Overview.pdf

Why You Should Replace Windows 11 with Nitrux Linux 3.5.0 for enhanced perfor...

DevOps and Testing slides at DASA Connect

GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...

Le nuove frontiere dell'AI nell'RPA con UiPath Autopilot™

Encryption in Microsoft 365 - ExpertsLive Netherlands 2024

GenAISummit 2024 May 28 Sri Ambati Keynote: AGI Belongs to The Community in O...

Free Complete Python - A step towards Data Science

Enhancing Performance with Globus and the Science DMZ

State of ICS and IoT Cyber Threat Landscape Report 2024 preview

Secstrike : Reverse Engineering & Pwnable tools for CTF.pptx

Observability Concepts EVERY Developer Should Know -- DeveloperWeek Europe.pdf

Monitoring Java Application Security with JDK Tools and JFR Events

FIDO Alliance Osaka Seminar: FIDO Security Aspects.pdf

By Design, not by Accident - Agile Venture Bolzano 2024

The Metaverse and AI: how can decision-makers harness the Metaverse for their...

The Future of Platform Engineering

Oracle Partitioning for DBAs and Developers

2. 2© The Pythian Group Inc., 2018 2 FRANKY WEBER FAUST ● Senior Oracle Database Consultant at Pythian ● 27 years old ● Writer at OTNLA and Lore Data Blog ● Speaker at some conferences ● High Availability specialist ● Performance researcher ● Exadata, RAC, DataGuard, Grid Infrastructure, GoldenGate

4. 4© The Pythian Group Inc., 2018 AGENDA 4 ● Partitioning concept ● Background ● Partitioning benefits (Why) ● Implementation scenarios (When) ● Types of partitioning (When) ● Range, List, Interval e Hash partitioning (How) ● The execution plan ● Partitioning maintenance operations* ● Historical data archiving* ● Converting partitions*

5. © 2017 Pythian. Confidential 5 PARTITIONING CONCEPTS ● Break down a table/index into smaller pieces ● Partition Key ● Single or multicolumn ● Up to 16 columns ● Unique values are not required ● Every partition is a segment ● Tables ● Heap tables ● Index-organized tables ● Indexes ● Materialized Views ● Clusters ● Performance ● Partition Pruning

6. © 2017 Pythian. Confidential 6 BACKGROUND HISTORY ● Since Oracle 8 ● Requires extra licensing and EE ● Manual partitioning using views ● New features every new release Oracle 12cR2 Auto-list and Multicolumn list partitioning, Partitioned external tables Table creation for partition exchange, read-only partitions, Filtered partition maintenance, Online table conversion to partitioned table Oracle 18c Online Merging of Partitions and Subpartitions Parallel Partition-Wise SQL Operations Modifying the Partitioning Strategy Partition Maintenance on multiple partitions, Asynchronous global index maintenance, Zone maps

7. © 2017 Pythian. Confidential 7 PARTITIONING BENEFITS (WHY) ● Increases availability of data ● Eases administration of large segments ● Might improve the performance of some queries ● Might reduce contention on hot OLTP segments

8. © 2017 Pythian. Confidential 8 IMPLEMENTATION SCENARIOS (WHEN) ● Large volume of data when administration becomes difficult ● Query performance on large tables suffers ● Buffer busy waits on high-concurrent tables ● Horizontal scalability in RAC ● Decreased storage cost ● Compression and row archival

9. © 2017 Pythian. Confidential 9 TYPES OF PARTITIONING (WHEN) ● Range Partitioning ● Dates and numbers are good candidates ● Upper boundary ● Hash Partitioning ● High-volume OLTP segments ● Data is evenly distributed across partitions ● List Partitioning ● Categories, colors, sports, brands ● Default values accepted ● Interval Partitioning ● Automated range partitions ● Reference Partitioning ● Relational model PK-FK ● Interval Reference Partitioning ● All child tables will be automatically maintained ● Virtual Column Based Partitioning

38. © 2017 Pythian. Confidential 38 CONCLUSION ● Range Partitioning, Hash Partitioning, List Partitioning, Interval Partitioning ● 12cR2 onwards can do most partitioning operations online ● Performance problems in read or write operations can get better or worse with partitioning, so plan well ● New releases -> New features, it's been like that for years. ● Questions?

39. © 2017 Pythian. Confidential 39 STAY IN TOUCH! ● E-mail: faust@pythian.com ● Blog: http://loredata.com.br/blog ● Facebook: https://facebook.com/08Franky.Weber ● Twitter: https://twitter.com/frankyweber ● LinkedIn: https://linkedin.com/in/frankyweber/en ● Oracle ACE profile: https://apex.oracle.com/pls/apex/f?p=19297:4:::NO:4:P4_ID:17002

Oracle Partitioning for DBAs and Developers

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to Oracle Partitioning for DBAs and Developers

Similar to Oracle Partitioning for DBAs and Developers (20)

Recently uploaded

Recently uploaded (20)

Oracle Partitioning for DBAs and Developers