This document summarizes Stew Ashton's presentation on using Oracle Database 12c's MATCH_RECOGNIZE clause to solve various "row pattern matching" problems in a more efficient way than pre-12c solutions. The document provides examples of using MATCH_RECOGNIZE for problems involving identifying consecutive values, grouping data into fixed bins, and distributing values evenly across bins. It shows that MATCH_RECOGNIZE offers performance improvements and simpler solutions compared to earlier approaches using window functions, self-joins and the MODEL clause.
Recognizing patterns in a sequence of rows has been a capability that was widely desired, but not possible with SQL until now. There were many workarounds, but these were difficult to write, hard to understand, and inefficient to execute. Beginning in Oracle Database 12c, you can use the MATCH_RECOGNIZE clause to achieve this capability in native SQL that executes efficiently. This presentation discusses how to do this.
ISO SQL:2016 introduced Row Pattern Matching: a feature to apply (limited) regular expressions on table rows and perform analysis on each match. As of writing, this feature is only supported by the Oracle Database 12c.
Recognizing patterns in a sequence of rows has been a capability that was widely desired, but not possible with SQL until now. There were many workarounds, but these were difficult to write, hard to understand, and inefficient to execute. Beginning in Oracle Database 12c, you can use the MATCH_RECOGNIZE clause to achieve this capability in native SQL that executes efficiently. This presentation discusses how to do this.
ISO SQL:2016 introduced Row Pattern Matching: a feature to apply (limited) regular expressions on table rows and perform analysis on each match. As of writing, this feature is only supported by the Oracle Database 12c.
Functions can be used anywhere in SQL, like AVG, COUNT, SUM, MIN, DATE and so on with select statements. Functions compile every time. Functions must return a value or result. Functions only work with input parameters.
Introduction
Plotting basic 2-D plots.
The plot command
The fplot command
Plotting multiple graphs in the same plot
Formatting plots
USING THE plot() COMMAND TO PLOT
MULTIPLE GRAPHS IN THE SAME PLOT
MATLAB PROGRAM TO PLOT VI CHARACTERISTICS OF A DIODE
SUMMARY
1. Introduction to MATLAB and programming
2. Workspace, variables and arrays
3. Using operators, expressions and statements
4. Repeating and decision-making
5. Different methods for input and output
6. Common functions
7. Logical vectors
8. Matrices and string arrays
9. Introduction to graphics
10. Loops
11. Custom functions and M-files
As part of the GSP’s capacity development and improvement programme, FAO/GSP have organised a one week training in Izmir, Turkey. The main goal of the training was to increase the capacity of Turkey on digital soil mapping, new approaches on data collection, data processing and modelling of soil organic carbon. This 5 day training is titled ‘’Training on Digital Soil Organic Carbon Mapping’’ was held in IARTC - International Agricultural Research and Education Center in Menemen, Izmir on 20-25 August, 2017.
Use Cases of Row Pattern Matching in Oracle 12cGerger
In this presentation, Oracle ACE Kim Berg Hansen explains the use cases of the MATCH_RECOGNIZE feature in Oracle 12c. You can watch the recording of this presentation at http://www.prohuddle.com/webinars/KimBergHansen/Use_Cases_of_Row_Pattern_Matching_in_Oracle_12c.php
This presentation is part of the Oracle OpenWorld 2016 session: EOUC Database ACES Share Their Favorite Database Things: Part II. In this session (UGF-2632) ACE Directors share their favorite database features in our now traditional quick-fire sessions (of 5 minutes per speaker).
Functions can be used anywhere in SQL, like AVG, COUNT, SUM, MIN, DATE and so on with select statements. Functions compile every time. Functions must return a value or result. Functions only work with input parameters.
Introduction
Plotting basic 2-D plots.
The plot command
The fplot command
Plotting multiple graphs in the same plot
Formatting plots
USING THE plot() COMMAND TO PLOT
MULTIPLE GRAPHS IN THE SAME PLOT
MATLAB PROGRAM TO PLOT VI CHARACTERISTICS OF A DIODE
SUMMARY
1. Introduction to MATLAB and programming
2. Workspace, variables and arrays
3. Using operators, expressions and statements
4. Repeating and decision-making
5. Different methods for input and output
6. Common functions
7. Logical vectors
8. Matrices and string arrays
9. Introduction to graphics
10. Loops
11. Custom functions and M-files
As part of the GSP’s capacity development and improvement programme, FAO/GSP have organised a one week training in Izmir, Turkey. The main goal of the training was to increase the capacity of Turkey on digital soil mapping, new approaches on data collection, data processing and modelling of soil organic carbon. This 5 day training is titled ‘’Training on Digital Soil Organic Carbon Mapping’’ was held in IARTC - International Agricultural Research and Education Center in Menemen, Izmir on 20-25 August, 2017.
Use Cases of Row Pattern Matching in Oracle 12cGerger
In this presentation, Oracle ACE Kim Berg Hansen explains the use cases of the MATCH_RECOGNIZE feature in Oracle 12c. You can watch the recording of this presentation at http://www.prohuddle.com/webinars/KimBergHansen/Use_Cases_of_Row_Pattern_Matching_in_Oracle_12c.php
This presentation is part of the Oracle OpenWorld 2016 session: EOUC Database ACES Share Their Favorite Database Things: Part II. In this session (UGF-2632) ACE Directors share their favorite database features in our now traditional quick-fire sessions (of 5 minutes per speaker).
One key area of Oracle OpenWorld 2016 was data in various shapes. Big Data, streaming data and traditional transactional data. The power of SQL to access and unleash all data - even data in NoSQL databases. The advent of the citizen data scientist. Streaming data analysis in real time on vast and fast and vast data, data discovery. And the new Oracle Database 12cR2 release. Forms, APEX, SQL and PL/SQL.
Introducing Node.js in an Oracle technology environment (including hands-on)Lucas Jellema
This presentation introduces Node.js in a few simple, straightforward steps. First, Node.js is presented as just JavaScript on the browser, then HTTP handling is discussed with core module http and subsequently using Express. Running Oracle JET from Node.js is explained. The implementation of APIs - REST services supporting various [operations on] resources is discussed. The single-thread nature of Node.js is presented, along with the essentials of asynchrous programming, working with callbacks and using the async module. The Node Oracle DB Database driver is introduced and demonstrated. Finally, further steps are suggested. This presentation is supported by a set of resources that constitute a three hour hands on session - sources are in GitHub https://github.com/lucasjellema/sig-nodejs-amis-2016.
What is the Oracle PaaS Cloud for Developers (Oracle Cloud Day, The Netherlan...Lucas Jellema
The promise of the cloud is substantial. Oracle's public cloud promise goes beyond the generic promise. This presentation describes the promise of the Oracle Public Cloud specifically for developers. It describes the current state of the PaaS Platform, the actual and coming services and what they could mean to a developer. From same platform, different location (DBaaS, JCS) to cloud native stack (ICS, MCS) and services for Citizen Developers, the presentation touches upon virtually all services relevant to developers. The presentation concludes with first the steps enterprises can start taking to move to the cloud and second the steps individual developers could and perhaps should take in order to conquer the clouds.
In the last few years, Apache Kafka has been used extensively in enterprises for real-time data collecting, delivering, and processing. In this presentation, Jun Rao, Co-founder, Confluent, gives a deep dive on some of the key internals that help make Kafka popular.
- Companies like LinkedIn are now sending more than 1 trillion messages per day to Kafka. Learn about the underlying design in Kafka that leads to such high throughput.
- Many companies (e.g., financial institutions) are now storing mission critical data in Kafka. Learn how Kafka supports high availability and durability through its built-in replication mechanism.
- One common use case of Kafka is for propagating updatable database records. Learn how a unique feature called compaction in Apache Kafka is designed to solve this kind of problem more naturally.
A stream processing platform is not an island unto itself; it must be connected to all of your existing data systems, applications, and sources. In this talk we will provide different options for integrating systems and applications with Apache Kafka, with a focus on the Kafka Connect framework and the ecosystem of Kafka connectors. We will discuss the intended use cases for Kafka Connect and share our experience and best practices for building large-scale data pipelines using Apache Kafka.
Introduction To Streaming Data and Stream Processing with Apache Kafkaconfluent
Modern businesses have data at their core, and this data is changing continuously. How can we harness this torrent of continuously changing data in real time? The answer is stream processing, and one system that has become a core hub for streaming data is Apache Kafka.
This presentation will give a brief introduction to Apache Kafka and describe its usage as a platform for streaming data. It will explain how Kafka serves as a foundation for both streaming data pipelines and applications that consume and process real-time data streams. It will introduce some of the newer components of Kafka that help make this possible, including Kafka Connect, a framework for capturing continuous data streams, and Kafka Streams, a lightweight stream processing library.
This is talk 1 out of 6 from the Kafka Talk Series.
http://www.confluent.io/apache-kafka-talk-series/introduction-to-stream-processing-with-apache-kafka
Oracle OpenWorld 2016 Review - High Level Overview of major themes and grand ...Lucas Jellema
Overview of the highlights, main themes and grand announcements during Oracle OpenWorld 2016. Cloud, Big Data, Machine Learning, Infrastructure, raging against AWS and the Oracle future strategy are the chief topics.
Handson Oracle Management Cloud with Application Performance Monitoring and L...Lucas Jellema
This presentation gives an overview of the steps in the workshop labs for Oracle Management Cloud APM and Log Analytics. The labs themselves and all sources are found at GitHub: https://github.com/lucasjellema/APM-Demo-App-WorldView .
Streaming in Practice - Putting Apache Kafka in Productionconfluent
This presentation focuses on how to integrate all these components into an enterprise environment and what things you need to consider as you move into production.
We will touch on the following topics:
- Patterns for integrating with existing data systems and applications
- Metadata management at enterprise scale
- Tradeoffs in performance, cost, availability and fault tolerance
- Choosing which cross-datacenter replication patterns fit with your application
- Considerations for operating Kafka-based data pipelines in production
Paolo Castagna is a Senior Sales Engineer at Confluent. His background is on 'big data' and he has, first hand, saw the shift happening in the industry from batch to stream processing and from big data to fast data. His talk will introduce Kafka Streams and explain why Apache Kafka is a great option and simplification for stream processing.
Data Streaming with Apache Kafka & MongoDBconfluent
Explore the use-cases and architecture for Apache Kafka, and how it integrates with MongoDB to build sophisticated data-driven applications that exploit new sources of data.
The Data Dichotomy- Rethinking the Way We Treat Data and Servicesconfluent
Presenter: Ben Stopford, Engineer, Confluent
Services come with a problem: they’re not well suited to sharing data. This talk will examine the underlying dichotomy we all face as we piece such systems together. One that is not well served today. The solution lies in blending the old with the new and Apache Kafka plays a central role.
AMIS SIG - Introducing Apache Kafka - Scalable, reliable Event Bus & Message ...Lucas Jellema
Introduction of Apache Kafka - the open source platform for real time message queuing and reliable, scalable, distributed event handling and high volume pub/sub implementation.
see GitHub https://github.com/MaartenSmeets/kafka-workshop for the workshop resources.
Modern businesses have data at their core, and this data is changing continuously. How can we harness this torrent of information in real-time? The answer is stream processing, and the technology that has since become the core platform for streaming data is Apache Kafka. Among the thousands of companies that use Kafka to transform and reshape their industries are the likes of Netflix, Uber, PayPal, and AirBnB, but also established players such as Goldman Sachs, Cisco, and Oracle.
Unfortunately, today’s common architectures for real-time data processing at scale suffer from complexity: there are many technologies that need to be stitched and operated together, and each individual technology is often complex by itself. This has led to a strong discrepancy between how we, as engineers, would like to work vs. how we actually end up working in practice.
In this session we talk about how Apache Kafka helps you to radically simplify your data processing architectures. We cover how you can now build normal applications to serve your real-time processing needs — rather than building clusters or similar special-purpose infrastructure — and still benefit from properties such as high scalability, distributed computing, and fault-tolerance, which are typically associated exclusively with cluster technologies. Notably, we introduce Kafka’s Streams API, its abstractions for streams and tables, and its recently introduced Interactive Queries functionality. As we will see, Kafka makes such architectures equally viable for small, medium, and large scale use cases.
Performance improvements in PostgreSQL 9.5 and beyondTomas Vondra
Let's see what major performance improvements PostgreSQL 9.5 brings, measure the impact on simple examples and also briefly look at improvements likely to appear in PostgreSQL 9.6 or some of the following releases.
What's the great thing about a database? Why, it stores data of course! However, one feature that makes a database useful is the different data types that can be stored in it, and the breadth and sophistication of the data types in PostgreSQL is second-to-none, including some novel data types that do not exist in any other database software!
This talk will take an in-depth look at the special data types built right into PostgreSQL version 9.4, including:
* INET types
* UUIDs
* Geometries
* Arrays
* Ranges
* Document-based Data Types:
* Key-value store (hstore)
* JSON (text [JSON] & binary [JSONB])
We will also have some cleverly concocted examples to show how all of these data types can work together harmoniously.
If you are worried about completing your R homework, you can connect with us at Statisticshomeworkhelper.com. We have a team of experts who are professionals in R programming homework help and have years of experience in working on any problem related to R. Visit statisticshomeworkhelper.com or email info@statisticshomeworkhelper.com. You can also call +1 (315) 557-6473 for assistance with Statistics Homework.
How the query planner in PostgreSQL works? Index access methods, join execution types, aggregation & pipelining. Optimizing queries with WHERE conditions, ORDER BY and GROUP BY. Composite indexes, partial and expression indexes. Exploiting assumptions about data and denormalization.
R is a fun and versatile language for statistical analysis, visualization, and data exploration. Target audience are software engineers/programmers who can code comfortably in another language. Emphasis in this lesson is on data structures, and light on analysis examples (to be covered at later date) but you are exposed to the basic concepts and commands. Email me for the pptx file which has notes.
To Compute GCD and LCM String Manipulation , Operations on string length, concatenation, reverse, counting, and copy of a string to another, Matrices, Matrix Addition, Subtraction, Multiplication, Transpose of n x m matrices, Inverse of a square matrix, Searching, Binary Search,GCD , LCM String Manipulation , Operations STRING,string length, concatenation, reverse, counting, copy of a string, Matrices, Matrix Addition, Subtraction, Multiplication, Transpose,n x m matrices, Inverse MATRIX,square matrix, Searching, Binary Search,C PROGRAM,C,PROGRAMMING,SIMPLE C, EXAMPLE C,LAB PROGRAMS,LAB, C EXERCISES,thiruvalluvar university,lab manual,c lab manual
Dev Dives: Train smarter, not harder – active learning and UiPath LLMs for do...UiPathCommunity
💥 Speed, accuracy, and scaling – discover the superpowers of GenAI in action with UiPath Document Understanding and Communications Mining™:
See how to accelerate model training and optimize model performance with active learning
Learn about the latest enhancements to out-of-the-box document processing – with little to no training required
Get an exclusive demo of the new family of UiPath LLMs – GenAI models specialized for processing different types of documents and messages
This is a hands-on session specifically designed for automation developers and AI enthusiasts seeking to enhance their knowledge in leveraging the latest intelligent document processing capabilities offered by UiPath.
Speakers:
👨🏫 Andras Palfi, Senior Product Manager, UiPath
👩🏫 Lenka Dulovicova, Product Program Manager, UiPath
GraphRAG is All You need? LLM & Knowledge GraphGuy Korland
Guy Korland, CEO and Co-founder of FalkorDB, will review two articles on the integration of language models with knowledge graphs.
1. Unifying Large Language Models and Knowledge Graphs: A Roadmap.
https://arxiv.org/abs/2306.08302
2. Microsoft Research's GraphRAG paper and a review paper on various uses of knowledge graphs:
https://www.microsoft.com/en-us/research/blog/graphrag-unlocking-llm-discovery-on-narrative-private-data/
A tale of scale & speed: How the US Navy is enabling software delivery from l...sonjaschweigert1
Rapid and secure feature delivery is a goal across every application team and every branch of the DoD. The Navy’s DevSecOps platform, Party Barge, has achieved:
- Reduction in onboarding time from 5 weeks to 1 day
- Improved developer experience and productivity through actionable findings and reduction of false positives
- Maintenance of superior security standards and inherent policy enforcement with Authorization to Operate (ATO)
Development teams can ship efficiently and ensure applications are cyber ready for Navy Authorizing Officials (AOs). In this webinar, Sigma Defense and Anchore will give attendees a look behind the scenes and demo secure pipeline automation and security artifacts that speed up application ATO and time to production.
We will cover:
- How to remove silos in DevSecOps
- How to build efficient development pipeline roles and component templates
- How to deliver security artifacts that matter for ATO’s (SBOMs, vulnerability reports, and policy evidence)
- How to streamline operations with automated policy checks on container images
GDG Cloud Southlake #33: Boule & Rebala: Effective AppSec in SDLC using Deplo...James Anderson
Effective Application Security in Software Delivery lifecycle using Deployment Firewall and DBOM
The modern software delivery process (or the CI/CD process) includes many tools, distributed teams, open-source code, and cloud platforms. Constant focus on speed to release software to market, along with the traditional slow and manual security checks has caused gaps in continuous security as an important piece in the software supply chain. Today organizations feel more susceptible to external and internal cyber threats due to the vast attack surface in their applications supply chain and the lack of end-to-end governance and risk management.
The software team must secure its software delivery process to avoid vulnerability and security breaches. This needs to be achieved with existing tool chains and without extensive rework of the delivery processes. This talk will present strategies and techniques for providing visibility into the true risk of the existing vulnerabilities, preventing the introduction of security issues in the software, resolving vulnerabilities in production environments quickly, and capturing the deployment bill of materials (DBOM).
Speakers:
Bob Boule
Robert Boule is a technology enthusiast with PASSION for technology and making things work along with a knack for helping others understand how things work. He comes with around 20 years of solution engineering experience in application security, software continuous delivery, and SaaS platforms. He is known for his dynamic presentations in CI/CD and application security integrated in software delivery lifecycle.
Gopinath Rebala
Gopinath Rebala is the CTO of OpsMx, where he has overall responsibility for the machine learning and data processing architectures for Secure Software Delivery. Gopi also has a strong connection with our customers, leading design and architecture for strategic implementations. Gopi is a frequent speaker and well-known leader in continuous delivery and integrating security into software delivery.
LF Energy Webinar: Electrical Grid Modelling and Simulation Through PowSyBl -...DanBrown980551
Do you want to learn how to model and simulate an electrical network from scratch in under an hour?
Then welcome to this PowSyBl workshop, hosted by Rte, the French Transmission System Operator (TSO)!
During the webinar, you will discover the PowSyBl ecosystem as well as handle and study an electrical network through an interactive Python notebook.
PowSyBl is an open source project hosted by LF Energy, which offers a comprehensive set of features for electrical grid modelling and simulation. Among other advanced features, PowSyBl provides:
- A fully editable and extendable library for grid component modelling;
- Visualization tools to display your network;
- Grid simulation tools, such as power flows, security analyses (with or without remedial actions) and sensitivity analyses;
The framework is mostly written in Java, with a Python binding so that Python developers can access PowSyBl functionalities as well.
What you will learn during the webinar:
- For beginners: discover PowSyBl's functionalities through a quick general presentation and the notebook, without needing any expert coding skills;
- For advanced developers: master the skills to efficiently apply PowSyBl functionalities to your real-world scenarios.
Epistemic Interaction - tuning interfaces to provide information for AI supportAlan Dix
Paper presented at SYNERGY workshop at AVI 2024, Genoa, Italy. 3rd June 2024
https://alandix.com/academic/papers/synergy2024-epistemic/
As machine learning integrates deeper into human-computer interactions, the concept of epistemic interaction emerges, aiming to refine these interactions to enhance system adaptability. This approach encourages minor, intentional adjustments in user behaviour to enrich the data available for system learning. This paper introduces epistemic interaction within the context of human-system communication, illustrating how deliberate interaction design can improve system understanding and adaptation. Through concrete examples, we demonstrate the potential of epistemic interaction to significantly advance human-computer interaction by leveraging intuitive human communication strategies to inform system design and functionality, offering a novel pathway for enriching user-system engagements.
SAP Sapphire 2024 - ASUG301 building better apps with SAP Fiori.pdfPeter Spielvogel
Building better applications for business users with SAP Fiori.
• What is SAP Fiori and why it matters to you
• How a better user experience drives measurable business benefits
• How to get started with SAP Fiori today
• How SAP Fiori elements accelerates application development
• How SAP Build Code includes SAP Fiori tools and other generative artificial intelligence capabilities
• How SAP Fiori paves the way for using AI in SAP apps
Accelerate your Kubernetes clusters with Varnish CachingThijs Feryn
A presentation about the usage and availability of Varnish on Kubernetes. This talk explores the capabilities of Varnish caching and shows how to use the Varnish Helm chart to deploy it to Kubernetes.
This presentation was delivered at K8SUG Singapore. See https://feryn.eu/presentations/accelerate-your-kubernetes-clusters-with-varnish-caching-k8sug-singapore-28-2024 for more details.
State of ICS and IoT Cyber Threat Landscape Report 2024 previewPrayukth K V
The IoT and OT threat landscape report has been prepared by the Threat Research Team at Sectrio using data from Sectrio, cyber threat intelligence farming facilities spread across over 85 cities around the world. In addition, Sectrio also runs AI-based advanced threat and payload engagement facilities that serve as sinks to attract and engage sophisticated threat actors, and newer malware including new variants and latent threats that are at an earlier stage of development.
The latest edition of the OT/ICS and IoT security Threat Landscape Report 2024 also covers:
State of global ICS asset and network exposure
Sectoral targets and attacks as well as the cost of ransom
Global APT activity, AI usage, actor and tactic profiles, and implications
Rise in volumes of AI-powered cyberattacks
Major cyber events in 2024
Malware and malicious payload trends
Cyberattack types and targets
Vulnerability exploit attempts on CVEs
Attacks on counties – USA
Expansion of bot farms – how, where, and why
In-depth analysis of the cyber threat landscape across North America, South America, Europe, APAC, and the Middle East
Why are attacks on smart factories rising?
Cyber risk predictions
Axis of attacks – Europe
Systemic attacks in the Middle East
Download the full report from here:
https://sectrio.com/resources/ot-threat-landscape-reports/sectrio-releases-ot-ics-and-iot-security-threat-landscape-report-2024/
Builder.ai Founder Sachin Dev Duggal's Strategic Approach to Create an Innova...Ramesh Iyer
In today's fast-changing business world, Companies that adapt and embrace new ideas often need help to keep up with the competition. However, fostering a culture of innovation takes much work. It takes vision, leadership and willingness to take risks in the right proportion. Sachin Dev Duggal, co-founder of Builder.ai, has perfected the art of this balance, creating a company culture where creativity and growth are nurtured at each stage.
The Art of the Pitch: WordPress Relationships and SalesLaura Byrne
Clients don’t know what they don’t know. What web solutions are right for them? How does WordPress come into the picture? How do you make sure you understand scope and timeline? What do you do if sometime changes?
All these questions and more will be explored as we talk about matching clients’ needs with what your agency offers without pulling teeth or pulling your hair out. Practical tips, and strategies for successful relationship building that leads to closing the deal.
Encryption in Microsoft 365 - ExpertsLive Netherlands 2024Albert Hoitingh
In this session I delve into the encryption technology used in Microsoft 365 and Microsoft Purview. Including the concepts of Customer Key and Double Key Encryption.
PHP Frameworks: I want to break free (IPC Berlin 2024)Ralf Eggert
In this presentation, we examine the challenges and limitations of relying too heavily on PHP frameworks in web development. We discuss the history of PHP and its frameworks to understand how this dependence has evolved. The focus will be on providing concrete tips and strategies to reduce reliance on these frameworks, based on real-world examples and practical considerations. The goal is to equip developers with the skills and knowledge to create more flexible and future-proof web applications. We'll explore the importance of maintaining autonomy in a rapidly changing tech landscape and how to make informed decisions in PHP development.
This talk is aimed at encouraging a more independent approach to using PHP frameworks, moving towards a more flexible and future-proof approach to PHP development.
PHP Frameworks: I want to break free (IPC Berlin 2024)
Row patternmatching12ctech14
1. “Row Pattern Matching” with
Database 12c MATCH_RECOGNIZE
Beating the Best Pre-12c Solutions
Stew Ashton UKOUG Tech 14 Stew ASHTON
UKOUG Tech 14
2. Agenda
• Who am I?
• Pre-12c solutions compared to row pattern
matching with MATCH_RECOGNIZE
– For all sizes of data
– Thinking in patterns
• Watch out for “catastrophic backtracking”
• Other things to keep in mind (time permitting)
2
3. Who am I?
• 33 years in IT
– Developer, Technical Sales Engineer, Technical Architect
– Aeronautics, IBM, Finance
– Mainframe, client-server, Web apps
• 25 years as an American in Paris
• 9 years using Oracle database
– Performance analysis
– Replace Java with SQL
• 2 years as internal “Oracle Development Expert”
3
4. 1) “Fixed Difference”
• Identify and group rows with consecutive values
• My presentation: print slides to keep
• Math: subtract known consecutives
– If A-1 = B-2 then A = B-1
– Else A <> B-1
– Consecutive becomes equality,
non-consecutive becomes inequality
• “Consecutive” = fixed difference of 1
PAGE
1
2
3
5
6
7
10
11
12
36
4
5. 1) Pre-12c
select min(page) firstpage,
max(page) lastpage,
count(*) cnt
FROM (
SELECT page,
page –
Row_Number() over(order by page)
as grp_id
FROM t
)
GROUP BY grp_id;
FIRSTPAGE PAGE [RN] GRP_LASTPAGE ID
CNT
1 1 0
2 2 0
3 3 0
5 4 1
6 5 1
7 6 1
10 7 3
11 8 3
12 9 3
42 10 32
1 3 3
5 7 3
10 12 3
36 36 1
5
6. Think “match a row pattern”
• PATTERN
– Uninterrupted series of input rows
– Described as a list of conditions (“regular expressions”)
PATTERN (A B*)
"A" : 1 row, "B" : 0 or more rows, as many as possible
• DEFINE each row condition
[A undefined = TRUE]
B AS page = PREV(page)+1
• Each series that matches the pattern is a “match”
– "A" and "B" identify the rows that meet their conditions
6
7. Input, Processing, Output
1. Define input
2. Order input
3. Process pattern
4. using defined conditions
5. Output: rows per match
6. Output: columns per row
7. Go where after match?
SELECT *
FROM t
MATCH_RECOGNIZE (
ORDER BY page
MEASURES
PATTERN (A B*)
DEFINE B AS page = PREV(page)+1
ONE ROW PER MATCH
MEASURES
A.page firstpage,
LAST(page) lastpage,
COUNT(*) cnt
AFTER MATCH SKIP PAST LAST ROW
);
A.page firstpage,
LAST(page) lastpage,
COUNT(*) cnt
ONE ROW PER MATCH
AFTER MATCH SKIP PAST LAST ROW
PATTERN (A B*)
DEFINE B AS page = PREV(page)+1
7
8. 1) Run_Stats comparison
For one million rows:
Stat Pre 12c Match_R Pct
Latches 4090 4079 100%
Elapsed Time 5.51 5.56 101%
CPU used by this session 5.5 5.55 101%
“Latches” are serialization devices: fewer means more scalable
8
9. 1) Execution Plans
Operation Used-Mem
SELECT STATEMENT
HASH GROUP BY 40M (0)
Id Operation Name Starts E-Rows A-Rows A-Time Buffers OMem 1Mem Used-Mem
0 SELECT STATEMENT 1 400K 00:00:01.83 1594
1 VIEW
HASH GROUP BY 1 1000K 400K 00:00:01.83 1594 41M 5035K 40M (0)
2 VIEW WINDOW SORT 1 1000K 1000K 00:00:12.69 1594
3 WINDOW SORT 1 1000K 1000K 00:00:03.46 1594 22M 20M 1749K (0)
20M (0)
4 TABLE ACCESS FULL T 1 1000K 1000K 00:00:02.53 1594
TABLE ACCESS FULL
Id Operation Name Starts E-Rows A-Rows A-Time Buffers OMem 1Mem Used-Mem
0 SELECT STATEMENT 1 400K 00:00:03.45 1594
1 VIEW 1 1000K 400K 00:00:03.45 1594
2
Operation Used-Mem
MATCH RECOGNIZE SORT DETERMINISTIC FINITE
SELECT AUTO
STATEMENT
VIEW
1 1000K 400K 00:00:01.87 1594 22M 1749K 20M (0)
3 TABLE ACCESS FULL T 1 1000K 1000K 00:00:02.09 1594
MATCH RECOGNIZE SORT DETERMINISTIC FINITE AUTO 20M (0)
TABLE ACCESS FULL
9
10. 2) “Start of Group”
• Identify group boundaries, often using LAG()
• 3 steps instead of 2:
1. For each row: if start of group, assign 1
Else assign 0
2. Running total of 1s and 0s produces a group
identifier
3. Group by the group identifier
10
11. 2) Requirement
GROUP_NAME START_TS END_TS
X 2014-01-01 00:00 2014-02-01 00:00
X 2014-03-01 00:00 2014-04-01 00:00
X 2014-04-01 00:00 2014-05-01 00:00
X 2014-06-01 00:00 2014-06-01 01:00
X 2014-06-01 01:00 2014-06-01 02:00
X 2014-06-01 02:00 2014-06-01 03:00
Y 2014-06-01 03:00 2014-06-01 04:00
Y 2014-06-01 04:00 2014-06-01 05:00
Y 2014-07-03 08:00 2014-09-29 17:00
Merge contiguous date ranges in same group
11
12. 1
2
2
3
3
3
1
1
2
X X 05-X 06-06-03:Y 03:05:Y 07-03 08:09-29 17:X 01-01 00:00 02-01 00:00
1
X 03-01 00:00 04-01 00:00
1
X 04-01 00:00 05-01 00:00
0
X 06-01 00:00 06-01 01:00
1
X 06-01 01:00 06-01 02:00
0
X 06-01 02:00 06-01 03:00 0
Y 06-01 03:00 06-01 04:00 1
Y 06-01 04:00 06-01 05:00 0
Y 07-03 08:00 09-29 17:00 1
with grp_starts as (
select a.*,
case when start_ts =
lag(end_ts) over(
partition by group_name
order by start_ts
)
then 0 else 1 end grp_start
from t a
), grps as (
select b.*,
sum(grp_start) over(
partition by group_name
order by start_ts
) grp_id
from grp_starts b)
select group_name,
min(start_ts) start_ts,
max(end_ts) end_ts
from grps
group by group_name, grp_id;
12
13. 2) Match_Recognize
SELECT * FROM t
MATCH_RECOGNIZE(
PARTITION BY group_name
ORDER BY start_ts
MEASURES
A.start_ts start_ts,
end_ts end_ts,
next(start_ts) - end_ts gap
PATTERN(A B*)
DEFINE B AS start_ts = prev(end_ts)
);
New this time:
• Added PARTITION BY
• MEASURES
added gap using row
outside the match!
• ONE ROW PER MATCH
and
SKIP PAST LAST ROW
are the defaults
One solution replaces two methods: simple!
13
14. Which row do we mean?
14
Column name by itself = « current » row
• Define: row being evaluated
• All rows: each row being output
• One row: last row being output
START_TS END_TS DEFINE
MEASURES
ALL ROWS ONE ROW
00:00 01:00 FIRST() FIRST() FIRST()
01:00 02:00 Current Current Current
02:00 03:00 LAST() LAST() LAST()
04:00 05:00 FINAL LAST FINAL LAST
15. Which row do we mean?
Expression DEFINE
MEASURES
ALL ROWS… ONE ROW…
FIRST(start_ts) First row of match
start_ts current row last row of match
LAST(end_ts) current row last row of match
FINAL
ORA-62509 last row of match
LAST(end_ts)
B.start_ts most recent B row last B row
PREV(), NEXT() Physical offset from referenced row
COUNT(*) from first to current row all rows in match
COUNT(B.*) B rows including current row all B rows
15
16. 2) Run_Stats comparison
For 500,000 rows:
Stat Pre 12c Match_R Pct
Latches 10165 8066 79%
Elapsed Time 32,16 20,58 64%
CPU used by this session 31,94 19,67 62%
16
17. 2) Execution Plans
Operation Used-Mem
SELECT STATEMENT
HASH GROUP BY 20M (0)
VIEW
WINDOW BUFFER 32M (0)
VIEW
WINDOW SORT 27M (0)
TABLE ACCESS FULL
Operation Used-Mem
SELECT STATEMENT
VIEW
MATCH RECOGNIZE SORT DETERMINISTIC FINITE AUTO 27M (0)
TABLE ACCESS FULL
17
18. 2) Matching within a group
18
SELECT * FROM (
SELECT * from t
WHERE group_name = 'X'
)
MATCH_RECOGNIZE
…
);
Filter before MATCH_RECOGNIZE to avoid extra work
19. 2) Predicate pushing
Select * from <view> where group_name = 'X'
Operation Name A-Rows Buffers
SELECT STATEMENT 3 4
VIEW 3 4
MATCH RECOGNIZE SORT DETERMINISTIC
FINITE AUTO
3 4
TABLE ACCESS BY INDEX ROWID
BATCHED
T 6 4
INDEX RANGE SCAN TI 6 3
19
21. SELECT s first_site, MAX(e) last_site, MAX(sm) sum_cnt FROM (
SELECT s, e, cnt, sm FROM t
MODEL
DIMENSION BY (row_number() over(order by study_site) rn)
MEASURES (study_site s, study_site e, cnt, cnt sm)
RULES (
sm[ > 1] =
CASE WHEN sm[cv() - 1] + cnt[cv()] > 65000 OR cnt[cv()]
> 65000
THEN cnt[cv()]
ELSE sm[cv() - 1] + cnt[cv()]
END,
s[ > 1] =
CASE WHEN sm[cv() - 1] + cnt[cv()] > 65000 OR cnt[cv()]
> 65000
THEN s[cv()]
ELSE s[cv() - 1]
END
)
)
GROUP BY s;
• DIMENSION with row_number
orders data and processing
• rn can be used like a subscript
• cv() means current row
• cv()-1 means previous row
rn
[– [[[[– [rn
[[[[[– 21
22. SELECT * FROM t
MATCH_RECOGNIZE (
ORDER BY study_site
MEASURES
FIRST(study_site) first_site,
LAST(study_site) last_site,
SUM(cnt) sum_cnt
PATTERN (A+)
DEFINE A AS SUM(cnt) <= 65000
);
New this time:
• PATTERN
(A+) replaces (A B*)
means 1 or more rows
• Why? In previous
examples I used PREV(),
which returns NULL on
the first row.
One solution replaces 3 methods: simpler!
22
23. 3) Run_Stats comparison
For one million rows:
Stat Pre 12c Match_R Pct
Latches 357448 4622 1%
Elapsed Time 32.85 2.9 9%
CPU used by this session 31.31 2.88 9%
23
24. 3) Execution Plans
Id Operation Used-Mem
0 SELECT STATEMENT
1 HASH GROUP BY 7534K (0)
2 VIEW
3 SQL MODEL ORDERED 105M (0)
4 WINDOW SORT 27M (0)
5 TABLE ACCESS FULL
Id Operation Used-Mem
0 SELECT STATEMENT
1 VIEW
2 MATCH RECOGNIZE SORT DETERMINISTIC FINITE AUTO 27M (0)
3 TABLE ACCESS FULL
24
25. 4) “Bin fitting”: fixed number
Name Val Val BIN1 BIN2 BIN3
1 1 10 10
2 2 9 10 9
3 3 8 10 9 8
4 4 7 10 9 15
5 5 6 10 15 15
6 6 5 15 15 15
7 7 4 19 15 15
8 8 3 19 18 15
9 9 2 19 18 17
10 10 1 19 18 18
• Requirement
– Distribute values in 3
parts as equally as
possible
• “Best fit decreasing”
– Sort values in
decreasing order
– Put each value in least
full “bin”
25
26. 4) Brilliant pre 12c solution
SELECT bin, Max (bin_value) bin_value
FROM (
SELECT * FROM items
MODEL
DIMENSION BY
(Row_Number() OVER
(ORDER BY item_value DESC) rn)
MEASURES (
item_name,
item_value,
Row_Number() OVER
(ORDER BY item_value DESC) bin,
item_value bin_value,
Row_Number() OVER
(ORDER BY item_value DESC) rn_m,
0 min_bin,
Count(*) OVER () - 3 - 1 n_iters
)
RULES ITERATE(100000)
UNTIL (ITERATION_NUMBER >= n_iters[1]) (
min_bin[1] = Min(rn_m) KEEP (DENSE_RANK
FIRST ORDER BY bin_value)[rn<= 3],
bin[ITERATION_NUMBER + 3 + 1] =
min_bin[1],
bin_value[min_bin[1]] =
bin_value[CV()] +
Nvl(item_value[ITERATION_NUMBER+4], 0))
)
WHERE item_name IS NOT NULL
group by bin;
26
27. SELECT * from items
MATCH_RECOGNIZE (
ORDER BY item_value desc
MEASURES
sum(bin1.item_value) bin1,
sum(bin2.item_value) bin2,
sum(bin3.item_value) bin3
PATTERN ((bin1|bin2|bin3)+)
DEFINE
bin1 AS count(bin1.*) = 1
OR sum(bin1.item_value)-bin1.item_value
<= least(
sum(bin2.item_value),
sum(bin3.item_value)
),
bin2 AS count(bin2.*) = 1
OR sum(bin2.item_value)-bin2.item_value
<= sum(bin3.item_value)
);
• ()+ = 1 or more of whatever
is inside
• '|' = alternatives,
“preferred in the order
specified”
• Bin1 condition:
• No rows here yet,
• Or this bin least full
• Bin2 condition
• No rows here yet, or
• This bin less full than 3
27
28. 4) Run_Stats comparison
For 10,000 rows:
Stat Pre 12c Match_R Pct
Latches 3124 47 2%
Elapsed Time 28 0.02 0%
CPU used by this session 26.39 0.03 0%
28
29. 4) Execution Plans
Id Operation Used-Mem
0 SELECT STATEMENT
1 HASH GROUP BY 817K (0)
2 VIEW
3 SQL MODEL ORDERED 1846K (0)
4 WINDOW SORT 424K (0)
5 TABLE ACCESS FULL
Id Operation Used-Mem
0 SELECT STATEMENT
1 VIEW
2 MATCH RECOGNIZE SORT 330K (0)
3 TABLE ACCESS FULL
29
30. Backtracking
• What happens when there is no match???
• “Greedy” quantifiers - * + {2,}
– are not that greedy
– Take all the rows they can, BUT
give rows back if necessary – one at a time
• Regular expression engines will test all possible
combinations to find a match
30
31. Repeating conditions
select 'match' from (
select level n from dual
connect by level <= 100
)
match_recognize(
pattern(a b* c)
define b as n > prev(n)
, c as n = 0
);
Runs in 0.005 secs
select 'match' from (
select level n from dual
connect by level <= 100
)
match_recognize(
pattern(a b* b* b* c)
define b as n > prev(n)
, c as n = 0
);
Runs in 5.4 secs
31
32. 32
123456789
A
AB
ABBB
ABBBB
ABBBBB
ABBBBBB
ABBBBBBB
ABBBBBBBC
ABBBBBBC
ABBBBBC
ABBBBC
ABBBC
ABBC
ABC
AC
Backtracking in action:
1. Find A
2. Find all the Bs you can
3. At the end, look for a C
4. No C? Backtrack through the Bs
5. Still no C? No Match!
33. Imprecise Conditions
SELECT * FROM Ticker
MATCH_RECOGNIZE (
PARTITION BY symbol
ORDER BY tstamp
MEASURES FIRST(tstamp) AS start_tstamp,
LAST(tstamp) AS end_tstamp
AFTER MATCH SKIP TO LAST UP
PATTERN (STRT DOWN+ UP+ DOWN+ UP+)
DEFINE DOWN AS price < PREV(price),
UP AS price > PREV(price),
STRT AS price >= nvl(PREV(PRICE),0)
);
Runs in 0.02 seconds
CREATE TABLE Ticker (
SYMBOL VARCHAR2(10),
tstamp DATE,
price NUMBER
);
insert into ticker
select 'ACME',
sysdate + level/24/60/60,
10000-level
from dual
connect by level <= 5000;
price)
);
Runs in 24 seconds
INMEMORY: 13 seconds 33
34. Keep in Mind
• Backtracking
– Precise conditions
– Test data with no matches
• To debug:
Measures classifier() cl,
match_number() mn
All rows per match with
unmatched rows
• No DISTINCT, no LISTAGG
• MEASURES columns must
have aliases
• “Reluctant quantifier” = ?
= JDBC bind variable
• “Pattern variables” are
range variables, not bind
variables
34
35. Output Row “shape”
Per Match PARTITION BY ORDER BY MEASURES Other input
ONE ROW X Omitted X omitted
ALL ROWS X X X X
ORA-00918, anyone?
35