Analytical Queries with Hive: SQL Windowing and Table FunctionsDataWorks Summit
Hive Query Language (HQL) is excellent for productivity and enables reuse of SQL skills, but falls short in advanced analytic queries. Hive`s Map & Reduce scripts mechanism lacks the simplicity of SQL and specifying new analysis is cumbersome. We developed SQLWindowing for Hive(SQW) to overcome these issues. SQW introduces both Windowing and Table Functions to the Hive user. SQW appears as a HQL extension with table functions and windowing clauses interspersed with HQL. This means the user stays within a SQL-like interface, while simultaneously having these capabilities available. SQW has been published as an open source project. It is available as both a CLI and an embeddable jar with a simple query API. There are pre-built functions for windowing to do Ranking, Aggregation, Navigation and Linear Regression. There are Table functions to do Time Series Analysis, Allocations, and Data Densification. Functions can be chained for more complex analysis. Under the covers MR mechanics are used to partition and order data. The fundamental interface is the tableFunction, whose core job is to operate on data partitions. Function implemenations are isolated from MR mechanics, focus purely on computation logic. Groovy scripting can be used for core implementation and parameterizing behavior. Writing functions typically involves extending one of the existing Abstract functions.
In diesem Vortrag wird ein Vergleich analytischer Funktionen vorgenommen und auf die Grundlage für selbstgebaute Funktionen, das Oracle Data Cartridge Interface (ODCI) näher eingegegangen. Auch eigene nützliche Data Cartridges werden demonstriert und des Weiteren die Data Cartridge Nutzung im Oracle Warehouse Builder (OWB) vorgestellt. OPITZ CONSULTING Senior Architect Gero Kanpstein hielt diesen Vortrag im Rahmen der DOAG Konferenz 2010.
Various use cases for Oracle database version 12c MATCH_RECOGNIZE data pattern matching functionality, not only for classic pattern matching like finding W patterns in stock ticker data, but also used for more general purpose SQL as "declarative analytics." Presentation given at OUGN Spring Conference 2016.
Real cases of indispensability of Oracle SQL analytic functionsKim Berg Hansen
Kim Berg Hansen will present on real cases where analytic functions in SQL were indispensably useful. Some of the cases covered include analyzing top selling items, efficient order picking using a FIFO principle, optimizing picking routes, measuring picking efficiency, sales forecasting, and forecasting inventory needs. The presentation argues that analytic functions allow SQL queries to efficiently operate across result sets rather than single rows, enabling new types of analyses.
This document describes using analytic functions in SQL to optimize warehouse picking based on the FIFO principle. It shows how to:
1. Join order lines to inventory data and order by purchase date to pick oldest items first.
2. Use an analytic sum to calculate a running total of quantities picked to determine when enough has been picked to fulfill an order.
3. Generate a picklist that orders picks by location to minimize travel within the warehouse.
4. Add aisle and warehouse information to further optimize the pick route by picking contiguous aisles and positions.
Hive is a data warehouse infrastructure built on top of Hadoop for querying and managing large datasets stored in Hadoop Distributed File System (HDFS). It provides SQL-like interface to query data, and uses MapReduce to parallelize the execution of queries across clusters. The document discusses Hive architecture, how it works, HiveQL syntax, data types, storage formats, DDL commands, data loading, functions, optimizations and getting started with Hive.
Analytical Queries with Hive: SQL Windowing and Table FunctionsDataWorks Summit
Hive Query Language (HQL) is excellent for productivity and enables reuse of SQL skills, but falls short in advanced analytic queries. Hive`s Map & Reduce scripts mechanism lacks the simplicity of SQL and specifying new analysis is cumbersome. We developed SQLWindowing for Hive(SQW) to overcome these issues. SQW introduces both Windowing and Table Functions to the Hive user. SQW appears as a HQL extension with table functions and windowing clauses interspersed with HQL. This means the user stays within a SQL-like interface, while simultaneously having these capabilities available. SQW has been published as an open source project. It is available as both a CLI and an embeddable jar with a simple query API. There are pre-built functions for windowing to do Ranking, Aggregation, Navigation and Linear Regression. There are Table functions to do Time Series Analysis, Allocations, and Data Densification. Functions can be chained for more complex analysis. Under the covers MR mechanics are used to partition and order data. The fundamental interface is the tableFunction, whose core job is to operate on data partitions. Function implemenations are isolated from MR mechanics, focus purely on computation logic. Groovy scripting can be used for core implementation and parameterizing behavior. Writing functions typically involves extending one of the existing Abstract functions.
In diesem Vortrag wird ein Vergleich analytischer Funktionen vorgenommen und auf die Grundlage für selbstgebaute Funktionen, das Oracle Data Cartridge Interface (ODCI) näher eingegegangen. Auch eigene nützliche Data Cartridges werden demonstriert und des Weiteren die Data Cartridge Nutzung im Oracle Warehouse Builder (OWB) vorgestellt. OPITZ CONSULTING Senior Architect Gero Kanpstein hielt diesen Vortrag im Rahmen der DOAG Konferenz 2010.
Various use cases for Oracle database version 12c MATCH_RECOGNIZE data pattern matching functionality, not only for classic pattern matching like finding W patterns in stock ticker data, but also used for more general purpose SQL as "declarative analytics." Presentation given at OUGN Spring Conference 2016.
Real cases of indispensability of Oracle SQL analytic functionsKim Berg Hansen
Kim Berg Hansen will present on real cases where analytic functions in SQL were indispensably useful. Some of the cases covered include analyzing top selling items, efficient order picking using a FIFO principle, optimizing picking routes, measuring picking efficiency, sales forecasting, and forecasting inventory needs. The presentation argues that analytic functions allow SQL queries to efficiently operate across result sets rather than single rows, enabling new types of analyses.
This document describes using analytic functions in SQL to optimize warehouse picking based on the FIFO principle. It shows how to:
1. Join order lines to inventory data and order by purchase date to pick oldest items first.
2. Use an analytic sum to calculate a running total of quantities picked to determine when enough has been picked to fulfill an order.
3. Generate a picklist that orders picks by location to minimize travel within the warehouse.
4. Add aisle and warehouse information to further optimize the pick route by picking contiguous aisles and positions.
Hive is a data warehouse infrastructure built on top of Hadoop for querying and managing large datasets stored in Hadoop Distributed File System (HDFS). It provides SQL-like interface to query data, and uses MapReduce to parallelize the execution of queries across clusters. The document discusses Hive architecture, how it works, HiveQL syntax, data types, storage formats, DDL commands, data loading, functions, optimizations and getting started with Hive.
The document provides examples of SQL queries and solutions to interview questions related to SQL queries on Oracle databases. It includes queries to find products with continuously increasing sales, products with no sales, products whose sales decreased in 2012 vs 2011, the top product sold each year, and total sales of each product. Tables called PRODUCTS and SALES are created with sample data on products and sales to demonstrate the example queries.
The document discusses information management and business intelligence. It defines information management as collecting, storing, manipulating, processing, and managing business data and glossaries. Business intelligence involves using the products of information management like data warehouses and data marts in strategic business ways. Good business intelligence requires understanding business models and processes, clear objectives, and effective data modeling to transform business needs into usable data models. Popular business intelligence tools are also listed.
This document provides an overview and analysis of sales forecasting data for Walmart. It includes:
- An introduction to the business problem of forecasting sales for Walmart stores and departments on a weekly basis.
- A description and exploration of the train and features datasets, including checking statistics, plotting sales over time, and outlier treatment.
- Preliminary analysis of the train dataset to understand sales patterns and relationships with features like temperature and holidays.
- Checks of the features dataset and treatment of missing/outlier markdown data.
The document analyzes the datasets to understand the data and perform initial exploratory analysis in preparation for building a sales forecasting model.
Reporting at Motorola - Predictive analytics & business insights 2014Patrick Deglon
In this presentation, Patrick Deglon will share his learnings and provide best practices when using open Google tools & API. He will present his daily email report that hundreds of key Motorola stakeholders are receiving to drive the business, as well as a mobile solution based on the latest web technologies, including Google Visualization, Bootstrap CSS and many of the Google APIs (Gmail, BigQuery, Analytics, Drive, App Engine, Users authentication, etc.).
The document presents on multidimensional data models. It discusses the key components of multidimensional data models including dimensions and facts. It describes different types of multidimensional data models such as data cube model, star schema model, snowflake schema model, and fact constellations. The star schema model and snowflake schema model are explained in more detail through examples and their benefits are highlighted.
Alteryx and Tableau: Iron Mountain's Sherpa to business insightMattSemple1
Presentation made to Alteryx Inspire Conference (London 2019) by Matt Semple (Iron Mountain) and David Turley (InterWorks) explaining the data journey undertaken by Iron Mountain to transform its analytics function and deliver advanced commercial and operational analytics to its global management teams.
A transformation journey leveraging the advanced technology of Alteryx and Tableau aligned to a clear vision of the data roadmap. All supported by a great consulting partnership with the InterWorks team in London.
This document provides an overview and introduction to Oracle SQL basics. It covers topics such as installing Oracle software like the database, Java SDK, and SQL Developer tool. It then discusses database concepts like what a database and table are. It also covers database fundamentals including SQL queries, functions, joins, constraints, views and other database objects. The document provides examples and explanations of SQL statements and database components.
This document provides an overview of various data visualization tools including Teradata Aster Lens, Tibco Spotfire, RStudio, Tableau, D3, and Qlik. It demonstrates how to create different types of charts and graphs using each tool, including chord charts, sankey charts, scatterplots, and maps. The goal is to help users understand how to leverage these visualization tools to gain insights from data.
Building a semantic/metrics layer using CalciteJulian Hyde
A semantic layer, also known as a metrics layer, lies between business users and the database, and lets those users compose queries in the concepts that they understand. It also governs access to the data, manages data transformations, and can tune the database by defining materializations.
Like many new ideas, the semantic layer is a distillation and evolution of many old ideas, such as query languages, multidimensional OLAP, and query federation.
In this talk, we describe the features we are adding to Calcite to define business views, query measures, and optimize performance.
A talk given at Community over Code, the annual conference of the Apache Software Foundation, in Halifax, NS, on 9th October, 2023.
This document summarizes a presentation about operationalizing linked data to transform industries using a multi-model approach. It discusses the importance of data and challenges with traditional data approaches. It promotes using linked data and semantic techniques to create flexible, contextual data layers that can be used across business units. Examples are provided of companies using these approaches for regulatory compliance, integrated digital delivery for auto repair, and open data sharing without data silos.
Validating, Generating, Deploying, and Executing Objects: Validating,
Validating in the Design Center, Validating from the editors, Validating in
the Data Object Editor, Validating in the Mapping, Editor, Generating,
Generating in the Design Center, Generating from the editors, Generating in
the Data Object Editor, Generating in the Mapping Editor, Deploying, The
Control Center Service, Deploying in the Design Center and Data Object
Editor, The Control Center Manager, The Control Center Manager window
overview, Deploying in the Control Center ,Manager, Executing, Deploying
and executing remaining objects, Deployment Order, Execution order.
Starting from the database used in Project 1 (see the slightly cha.docxdessiechisomjj4
Starting from the database used in Project 1 (see the slightly changed schema from the original version used in P1, defined in the attached DDL file), a data warehouse star schema with the following characteristics will be defined:
· Dimension tables:
1. Date
2. Product
3. Customer
· Fact table:
1. Sales
For this final project, perform the following steps:
· Create the tables defined above in a star schema. Add the necessary columns in each table to facilitate the implementation of the queries defined below. Only the four tables listed above are allowed in that star schema
· Write PL/SQL code (anonymous blocks and/or subprograms) to populate the warehouse schema with data from the normalized database provided in the attached DDL script
· Write SQL code to perform the following queries:
1. What customer age group spent the most money in the last year? An age group is defined as a ten years interval, such as: 11 – 20, 21 – 30, etc
2. In what zip codes did the highest number of sales (number of items) occur during April 2015?
3. What day of the week did they do most business (by value of sales) in the last year?
4. What quarter is the worst (by value of sales) for each product category, using the whole set of historical data available in the warehouse?
5. What was the best sales month for each product in the last year?
· Write a couple of paragraphs describing:
. How this small data warehouse can help decision making
. How is it different from the original database used as data source
Submit the PL/SQL blocks and SQL statements as a text file (Notepad) following the document naming convention FirstLastFP.txt.
Grading: this project is awarded 100 points
15 points
DDL Script to create the tables in the star schema
25 points
PL/SQL code to populate the star schema from the original database
5 x 10 points
SQL statements for each of the requested queries
10 points
Description of how this data warehouse helps decision making and how it differs from regular databases
DROP TABLE ORDER_ITEMS;
DROP TABLE ORDERS;
DROP TABLE CUSTOMERS;
DROP TABLE PRODUCTS;
DROP TABLE CATEGORIES;
CREATE TABLE CATEGORIES (
ID NUMBER PRIMARY KEY,
Name VARCHAR2(20) NOT NULL);
CREATE TABLE PRODUCTS (
ID NUMBER PRIMARY KEY,
CatID NUMBER,
Name VARCHAR2(20) NOT NULL,
Price NUMBER NOT NULL,
FOREIGN KEY (CatID) REFERENCES CATEGORIES(ID));
CREATE TABLE CUSTOMERS (
ID NUMBER PRIMARY KEY,
Name VARCHAR2(50) NOT NULL,
DOB DATE NOT NULL,
Email VARCHAR2(50),
ZipCode CHAR(5) NOT NULL);
CREATE TABLE ORDERS (
ID NUMBER PRIMARY KEY,
CustID NUMBER NOT NULL,
DatePlaced DATE DEFAULT SYSDATE,
FOREIGN KEY (CustID) REFERENCES CUSTOMERS(ID));
CREATE TABLE ORDER_ITEMS (
OrderID NUMBER NOT NULL,
ProdID NUMBER NOT NULL,
Quantity NUMBER DEFAULT 1,
DateShipped DATE,
FOREIGN KEY (OrderID) REFERENCES ORDERS(ID),
FOREIGN KEY (ProdID) REFERENCES PRODUCTS(ID));
INSERT INTO CATEGORIES VALUES (1, 'Books');
INSERT INTO CATEGORIES VALUES (2, '.
Extract, Transform, and Load Basics: ETL, Manual ETL processes,
Staging, To stage or not to stage, Configuration of a staging area, Mappings
and operators in OWB, The canvas layout, OWB operators, Source and
target operators, Data flow operators, Pre/post-processing operators.
Imhotep is Indeed's open-source analytics platform that allows for easy upload and compression of data as well as fast, interactive querying. It uses an SQL-like query language called IQL to perform aggregate analytics on datasets within specified date ranges and with optional filters, groupings, and metrics. The platform aims to provide an interactive experience where questions can be quickly refined, show ground truth data through a web interface, and instantly share insights through caching.
La programmation concurrente par flux de donnéesMicrosoft
Session Acensi : Avec le développement des ordinateurs à plusieurs cœurs, la programmation concurrente est devenue incontournable pour optimiser l’utilisation de toutes les ressources disponibles. Sa complexité limite pourtant son adoption par un grand nombre de personnes. La librairie Task Parrallel Library (TPL) cherche à simplifier ce paradigme de programmation. Dans cette session, nous allons vous faire découvrir ou redécouvrir l’une des nouveautés du Framework .NET 4.5 à savoir l’introduction de la librairie DataFlow dans la Task Parallel Library (TPL). Cette librairie fournit un ensemble de primitives permettant d’implémenter le pattern « Acteur / Agent ». Elle permet notamment de créer très facilement des pipelines complexes de tâches grâce à un enchaînement de « Blocks ». Dans cette session nous allons donc vous démontrer avec un exemple d’application concret comment TPL DataFlow peut nous aider à simplifier un problème de programmation concurrente.
This project focused on creating data frames, filtering data, grouping data, merging, and displaying data. Furthermore, it also includes creating new columns in which specific conditions can be applied. The data is used to solve business problems within a superstore.
The first problem statement is determining the prizes taken from the Top 5 products from the Mobiles & Tablet Category. Second, the data is processed to fulfill the requirement to check whether there is a decrease in the sales of the Others Category in 2022. The task also requires the display of the top 20 products that have the highest decrease. Third, I utilize the data to process the Customer ID and Registered Data of the consumers who have checked out but have not yet made payment. Fourth, the data is sorted and analyzed to compare the average daily sales on the weekends and those on the weekdays in the time range of 3 months.
- The document provides an investor presentation for Teradata's second quarter 2016 results.
- It highlights Teradata's portfolio of workload-specific platforms and hybrid cloud offerings that provide scalability, performance, and flexibility.
- Charts show Teradata's revenue mix by industry and type over recent years as well as its balance sheet, cash flow, and share repurchase activity.
The purpose of Simple Input Window is to input data easy and quickly.
Simple Input Window can edit only one tab of standard window. In stead of that, Simple Input Window can input data easy and quickly, and can operate like a spreadsheet.
External Tables - not just loading a csv fileKim Berg Hansen
Giving an overview of many of the details of the external table syntax in Oracle that enables you from SQL to access files that reside outside the database
When 7 bit-ascii ain't enough - about NLS, collation, charsets, unicode and s...Kim Berg Hansen
Revised version of the presentation - as presented at #POUG2018.
About problems that can occur if NLS_LANG is not set correctly, which may lead to corrupt data.
About different NLS settings and related topics, collation, and the use of DMU to migrate to UTF or check for corrupt text.
More Related Content
Similar to Really using Oracle analytic SQL functions
The document provides examples of SQL queries and solutions to interview questions related to SQL queries on Oracle databases. It includes queries to find products with continuously increasing sales, products with no sales, products whose sales decreased in 2012 vs 2011, the top product sold each year, and total sales of each product. Tables called PRODUCTS and SALES are created with sample data on products and sales to demonstrate the example queries.
The document discusses information management and business intelligence. It defines information management as collecting, storing, manipulating, processing, and managing business data and glossaries. Business intelligence involves using the products of information management like data warehouses and data marts in strategic business ways. Good business intelligence requires understanding business models and processes, clear objectives, and effective data modeling to transform business needs into usable data models. Popular business intelligence tools are also listed.
This document provides an overview and analysis of sales forecasting data for Walmart. It includes:
- An introduction to the business problem of forecasting sales for Walmart stores and departments on a weekly basis.
- A description and exploration of the train and features datasets, including checking statistics, plotting sales over time, and outlier treatment.
- Preliminary analysis of the train dataset to understand sales patterns and relationships with features like temperature and holidays.
- Checks of the features dataset and treatment of missing/outlier markdown data.
The document analyzes the datasets to understand the data and perform initial exploratory analysis in preparation for building a sales forecasting model.
Reporting at Motorola - Predictive analytics & business insights 2014Patrick Deglon
In this presentation, Patrick Deglon will share his learnings and provide best practices when using open Google tools & API. He will present his daily email report that hundreds of key Motorola stakeholders are receiving to drive the business, as well as a mobile solution based on the latest web technologies, including Google Visualization, Bootstrap CSS and many of the Google APIs (Gmail, BigQuery, Analytics, Drive, App Engine, Users authentication, etc.).
The document presents on multidimensional data models. It discusses the key components of multidimensional data models including dimensions and facts. It describes different types of multidimensional data models such as data cube model, star schema model, snowflake schema model, and fact constellations. The star schema model and snowflake schema model are explained in more detail through examples and their benefits are highlighted.
Alteryx and Tableau: Iron Mountain's Sherpa to business insightMattSemple1
Presentation made to Alteryx Inspire Conference (London 2019) by Matt Semple (Iron Mountain) and David Turley (InterWorks) explaining the data journey undertaken by Iron Mountain to transform its analytics function and deliver advanced commercial and operational analytics to its global management teams.
A transformation journey leveraging the advanced technology of Alteryx and Tableau aligned to a clear vision of the data roadmap. All supported by a great consulting partnership with the InterWorks team in London.
This document provides an overview and introduction to Oracle SQL basics. It covers topics such as installing Oracle software like the database, Java SDK, and SQL Developer tool. It then discusses database concepts like what a database and table are. It also covers database fundamentals including SQL queries, functions, joins, constraints, views and other database objects. The document provides examples and explanations of SQL statements and database components.
This document provides an overview of various data visualization tools including Teradata Aster Lens, Tibco Spotfire, RStudio, Tableau, D3, and Qlik. It demonstrates how to create different types of charts and graphs using each tool, including chord charts, sankey charts, scatterplots, and maps. The goal is to help users understand how to leverage these visualization tools to gain insights from data.
Building a semantic/metrics layer using CalciteJulian Hyde
A semantic layer, also known as a metrics layer, lies between business users and the database, and lets those users compose queries in the concepts that they understand. It also governs access to the data, manages data transformations, and can tune the database by defining materializations.
Like many new ideas, the semantic layer is a distillation and evolution of many old ideas, such as query languages, multidimensional OLAP, and query federation.
In this talk, we describe the features we are adding to Calcite to define business views, query measures, and optimize performance.
A talk given at Community over Code, the annual conference of the Apache Software Foundation, in Halifax, NS, on 9th October, 2023.
This document summarizes a presentation about operationalizing linked data to transform industries using a multi-model approach. It discusses the importance of data and challenges with traditional data approaches. It promotes using linked data and semantic techniques to create flexible, contextual data layers that can be used across business units. Examples are provided of companies using these approaches for regulatory compliance, integrated digital delivery for auto repair, and open data sharing without data silos.
Validating, Generating, Deploying, and Executing Objects: Validating,
Validating in the Design Center, Validating from the editors, Validating in
the Data Object Editor, Validating in the Mapping, Editor, Generating,
Generating in the Design Center, Generating from the editors, Generating in
the Data Object Editor, Generating in the Mapping Editor, Deploying, The
Control Center Service, Deploying in the Design Center and Data Object
Editor, The Control Center Manager, The Control Center Manager window
overview, Deploying in the Control Center ,Manager, Executing, Deploying
and executing remaining objects, Deployment Order, Execution order.
Starting from the database used in Project 1 (see the slightly cha.docxdessiechisomjj4
Starting from the database used in Project 1 (see the slightly changed schema from the original version used in P1, defined in the attached DDL file), a data warehouse star schema with the following characteristics will be defined:
· Dimension tables:
1. Date
2. Product
3. Customer
· Fact table:
1. Sales
For this final project, perform the following steps:
· Create the tables defined above in a star schema. Add the necessary columns in each table to facilitate the implementation of the queries defined below. Only the four tables listed above are allowed in that star schema
· Write PL/SQL code (anonymous blocks and/or subprograms) to populate the warehouse schema with data from the normalized database provided in the attached DDL script
· Write SQL code to perform the following queries:
1. What customer age group spent the most money in the last year? An age group is defined as a ten years interval, such as: 11 – 20, 21 – 30, etc
2. In what zip codes did the highest number of sales (number of items) occur during April 2015?
3. What day of the week did they do most business (by value of sales) in the last year?
4. What quarter is the worst (by value of sales) for each product category, using the whole set of historical data available in the warehouse?
5. What was the best sales month for each product in the last year?
· Write a couple of paragraphs describing:
. How this small data warehouse can help decision making
. How is it different from the original database used as data source
Submit the PL/SQL blocks and SQL statements as a text file (Notepad) following the document naming convention FirstLastFP.txt.
Grading: this project is awarded 100 points
15 points
DDL Script to create the tables in the star schema
25 points
PL/SQL code to populate the star schema from the original database
5 x 10 points
SQL statements for each of the requested queries
10 points
Description of how this data warehouse helps decision making and how it differs from regular databases
DROP TABLE ORDER_ITEMS;
DROP TABLE ORDERS;
DROP TABLE CUSTOMERS;
DROP TABLE PRODUCTS;
DROP TABLE CATEGORIES;
CREATE TABLE CATEGORIES (
ID NUMBER PRIMARY KEY,
Name VARCHAR2(20) NOT NULL);
CREATE TABLE PRODUCTS (
ID NUMBER PRIMARY KEY,
CatID NUMBER,
Name VARCHAR2(20) NOT NULL,
Price NUMBER NOT NULL,
FOREIGN KEY (CatID) REFERENCES CATEGORIES(ID));
CREATE TABLE CUSTOMERS (
ID NUMBER PRIMARY KEY,
Name VARCHAR2(50) NOT NULL,
DOB DATE NOT NULL,
Email VARCHAR2(50),
ZipCode CHAR(5) NOT NULL);
CREATE TABLE ORDERS (
ID NUMBER PRIMARY KEY,
CustID NUMBER NOT NULL,
DatePlaced DATE DEFAULT SYSDATE,
FOREIGN KEY (CustID) REFERENCES CUSTOMERS(ID));
CREATE TABLE ORDER_ITEMS (
OrderID NUMBER NOT NULL,
ProdID NUMBER NOT NULL,
Quantity NUMBER DEFAULT 1,
DateShipped DATE,
FOREIGN KEY (OrderID) REFERENCES ORDERS(ID),
FOREIGN KEY (ProdID) REFERENCES PRODUCTS(ID));
INSERT INTO CATEGORIES VALUES (1, 'Books');
INSERT INTO CATEGORIES VALUES (2, '.
Extract, Transform, and Load Basics: ETL, Manual ETL processes,
Staging, To stage or not to stage, Configuration of a staging area, Mappings
and operators in OWB, The canvas layout, OWB operators, Source and
target operators, Data flow operators, Pre/post-processing operators.
Imhotep is Indeed's open-source analytics platform that allows for easy upload and compression of data as well as fast, interactive querying. It uses an SQL-like query language called IQL to perform aggregate analytics on datasets within specified date ranges and with optional filters, groupings, and metrics. The platform aims to provide an interactive experience where questions can be quickly refined, show ground truth data through a web interface, and instantly share insights through caching.
La programmation concurrente par flux de donnéesMicrosoft
Session Acensi : Avec le développement des ordinateurs à plusieurs cœurs, la programmation concurrente est devenue incontournable pour optimiser l’utilisation de toutes les ressources disponibles. Sa complexité limite pourtant son adoption par un grand nombre de personnes. La librairie Task Parrallel Library (TPL) cherche à simplifier ce paradigme de programmation. Dans cette session, nous allons vous faire découvrir ou redécouvrir l’une des nouveautés du Framework .NET 4.5 à savoir l’introduction de la librairie DataFlow dans la Task Parallel Library (TPL). Cette librairie fournit un ensemble de primitives permettant d’implémenter le pattern « Acteur / Agent ». Elle permet notamment de créer très facilement des pipelines complexes de tâches grâce à un enchaînement de « Blocks ». Dans cette session nous allons donc vous démontrer avec un exemple d’application concret comment TPL DataFlow peut nous aider à simplifier un problème de programmation concurrente.
This project focused on creating data frames, filtering data, grouping data, merging, and displaying data. Furthermore, it also includes creating new columns in which specific conditions can be applied. The data is used to solve business problems within a superstore.
The first problem statement is determining the prizes taken from the Top 5 products from the Mobiles & Tablet Category. Second, the data is processed to fulfill the requirement to check whether there is a decrease in the sales of the Others Category in 2022. The task also requires the display of the top 20 products that have the highest decrease. Third, I utilize the data to process the Customer ID and Registered Data of the consumers who have checked out but have not yet made payment. Fourth, the data is sorted and analyzed to compare the average daily sales on the weekends and those on the weekdays in the time range of 3 months.
- The document provides an investor presentation for Teradata's second quarter 2016 results.
- It highlights Teradata's portfolio of workload-specific platforms and hybrid cloud offerings that provide scalability, performance, and flexibility.
- Charts show Teradata's revenue mix by industry and type over recent years as well as its balance sheet, cash flow, and share repurchase activity.
The purpose of Simple Input Window is to input data easy and quickly.
Simple Input Window can edit only one tab of standard window. In stead of that, Simple Input Window can input data easy and quickly, and can operate like a spreadsheet.
External Tables - not just loading a csv fileKim Berg Hansen
Giving an overview of many of the details of the external table syntax in Oracle that enables you from SQL to access files that reside outside the database
When 7 bit-ascii ain't enough - about NLS, collation, charsets, unicode and s...Kim Berg Hansen
Revised version of the presentation - as presented at #POUG2018.
About problems that can occur if NLS_LANG is not set correctly, which may lead to corrupt data.
About different NLS settings and related topics, collation, and the use of DMU to migrate to UTF or check for corrupt text.
When 7-bit ASCII ain't enough - about NLS, Collation, Charsets, Unicode and suchKim Berg Hansen
This document discusses character sets, encodings, and national language support (NLS) settings in Oracle databases. It begins with an introduction and agenda, then covers character sets and encodings like ASCII, ISO-8859, Windows codepages, and Unicode. It discusses database and national character sets. It also covers byte versus character length semantics, viewing NLS parameter values, and setting the session NLS environment.
Presentation on Analytic View syntax - a new feature in Oracle 12.2 that works like a "virtual" OLAP cube, so you can dimensionally query your relational data in real-time without copying to a datawarehouse.
Using Oracle database and Oracle SQL to retrieve XML and JSON from for example webservices, store XML and JSON in the database, query XML and JSON with SELECT as relational data, create XML and JSON in various manners from relational data. Presentation given at OUGN Spring Conference 2016.
Use of Oracle SQL to twist data around - unpivot columns to rows, pivot rows to columns, parse delimited strings to columns and rows, turn rows into delimited strings. Presentation given at OUGN Spring Conference 2016.
HCL Notes and Domino License Cost Reduction in the World of DLAUpanagenda
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-and-domino-license-cost-reduction-in-the-world-of-dlau/
The introduction of DLAU and the CCB & CCX licensing model caused quite a stir in the HCL community. As a Notes and Domino customer, you may have faced challenges with unexpected user counts and license costs. You probably have questions on how this new licensing approach works and how to benefit from it. Most importantly, you likely have budget constraints and want to save money where possible. Don’t worry, we can help with all of this!
We’ll show you how to fix common misconfigurations that cause higher-than-expected user counts, and how to identify accounts which you can deactivate to save money. There are also frequent patterns that can cause unnecessary cost, like using a person document instead of a mail-in for shared mailboxes. We’ll provide examples and solutions for those as well. And naturally we’ll explain the new licensing model.
Join HCL Ambassador Marc Thomas in this webinar with a special guest appearance from Franz Walder. It will give you the tools and know-how to stay on top of what is going on with Domino licensing. You will be able lower your cost through an optimized configuration and keep it low going forward.
These topics will be covered
- Reducing license cost by finding and fixing misconfigurations and superfluous accounts
- How do CCB and CCX licenses really work?
- Understanding the DLAU tool and how to best utilize it
- Tips for common problem areas, like team mailboxes, functional/test users, etc
- Practical examples and best practices to implement right away
Communications Mining Series - Zero to Hero - Session 1DianaGray10
This session provides introduction to UiPath Communication Mining, importance and platform overview. You will acquire a good understand of the phases in Communication Mining as we go over the platform with you. Topics covered:
• Communication Mining Overview
• Why is it important?
• How can it help today’s business and the benefits
• Phases in Communication Mining
• Demo on Platform overview
• Q/A
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/building-and-scaling-ai-applications-with-the-nx-ai-manager-a-presentation-from-network-optix/
Robin van Emden, Senior Director of Data Science at Network Optix, presents the “Building and Scaling AI Applications with the Nx AI Manager,” tutorial at the May 2024 Embedded Vision Summit.
In this presentation, van Emden covers the basics of scaling edge AI solutions using the Nx tool kit. He emphasizes the process of developing AI models and deploying them globally. He also showcases the conversion of AI models and the creation of effective edge AI pipelines, with a focus on pre-processing, model conversion, selecting the appropriate inference engine for the target hardware and post-processing.
van Emden shows how Nx can simplify the developer’s life and facilitate a rapid transition from concept to production-ready applications.He provides valuable insights into developing scalable and efficient edge AI solutions, with a strong focus on practical implementation.
Full-RAG: A modern architecture for hyper-personalizationZilliz
Mike Del Balso, CEO & Co-Founder at Tecton, presents "Full RAG," a novel approach to AI recommendation systems, aiming to push beyond the limitations of traditional models through a deep integration of contextual insights and real-time data, leveraging the Retrieval-Augmented Generation architecture. This talk will outline Full RAG's potential to significantly enhance personalization, address engineering challenges such as data management and model training, and introduce data enrichment with reranking as a key solution. Attendees will gain crucial insights into the importance of hyperpersonalization in AI, the capabilities of Full RAG for advanced personalization, and strategies for managing complex data integrations for deploying cutting-edge AI solutions.
Pushing the limits of ePRTC: 100ns holdover for 100 daysAdtran
At WSTS 2024, Alon Stern explored the topic of parametric holdover and explained how recent research findings can be implemented in real-world PNT networks to achieve 100 nanoseconds of accuracy for up to 100 days.
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
Best 20 SEO Techniques To Improve Website Visibility In SERPPixlogix Infotech
Boost your website's visibility with proven SEO techniques! Our latest blog dives into essential strategies to enhance your online presence, increase traffic, and rank higher on search engines. From keyword optimization to quality content creation, learn how to make your site stand out in the crowded digital landscape. Discover actionable tips and expert insights to elevate your SEO game.
Removing Uninteresting Bytes in Software FuzzingAftab Hussain
Imagine a world where software fuzzing, the process of mutating bytes in test seeds to uncover hidden and erroneous program behaviors, becomes faster and more effective. A lot depends on the initial seeds, which can significantly dictate the trajectory of a fuzzing campaign, particularly in terms of how long it takes to uncover interesting behaviour in your code. We introduce DIAR, a technique designed to speedup fuzzing campaigns by pinpointing and eliminating those uninteresting bytes in the seeds. Picture this: instead of wasting valuable resources on meaningless mutations in large, bloated seeds, DIAR removes the unnecessary bytes, streamlining the entire process.
In this work, we equipped AFL, a popular fuzzer, with DIAR and examined two critical Linux libraries -- Libxml's xmllint, a tool for parsing xml documents, and Binutil's readelf, an essential debugging and security analysis command-line tool used to display detailed information about ELF (Executable and Linkable Format). Our preliminary results show that AFL+DIAR does not only discover new paths more quickly but also achieves higher coverage overall. This work thus showcases how starting with lean and optimized seeds can lead to faster, more comprehensive fuzzing campaigns -- and DIAR helps you find such seeds.
- These are slides of the talk given at IEEE International Conference on Software Testing Verification and Validation Workshop, ICSTW 2022.
Unlocking Productivity: Leveraging the Potential of Copilot in Microsoft 365, a presentation by Christoforos Vlachos, Senior Solutions Manager – Modern Workplace, Uni Systems
How to Get CNIC Information System with Paksim Ga.pptxdanishmna97
Pakdata Cf is a groundbreaking system designed to streamline and facilitate access to CNIC information. This innovative platform leverages advanced technology to provide users with efficient and secure access to their CNIC details.
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc
How does your privacy program stack up against your peers? What challenges are privacy teams tackling and prioritizing in 2024?
In the fifth annual Global Privacy Benchmarks Survey, we asked over 1,800 global privacy professionals and business executives to share their perspectives on the current state of privacy inside and outside of their organizations. This year’s report focused on emerging areas of importance for privacy and compliance professionals, including considerations and implications of Artificial Intelligence (AI) technologies, building brand trust, and different approaches for achieving higher privacy competence scores.
See how organizational priorities and strategic approaches to data security and privacy are evolving around the globe.
This webinar will review:
- The top 10 privacy insights from the fifth annual Global Privacy Benchmarks Survey
- The top challenges for privacy leaders, practitioners, and organizations in 2024
- Key themes to consider in developing and maintaining your privacy program
Maruthi Prithivirajan, Head of ASEAN & IN Solution Architecture, Neo4j
Get an inside look at the latest Neo4j innovations that enable relationship-driven intelligence at scale. Learn more about the newest cloud integrations and product enhancements that make Neo4j an essential choice for developers building apps with interconnected data and generative AI.
Building Production Ready Search Pipelines with Spark and MilvusZilliz
Spark is the widely used ETL tool for processing, indexing and ingesting data to serving stack for search. Milvus is the production-ready open-source vector database. In this talk we will show how to use Spark to process unstructured data to extract vector representations, and push the vectors to Milvus vector database for search serving.
Sudheer Mechineni, Head of Application Frameworks, Standard Chartered Bank
Discover how Standard Chartered Bank harnessed the power of Neo4j to transform complex data access challenges into a dynamic, scalable graph database solution. This keynote will cover their journey from initial adoption to deploying a fully automated, enterprise-grade causal cluster, highlighting key strategies for modelling organisational changes and ensuring robust disaster recovery. Learn how these innovations have not only enhanced Standard Chartered Bank’s data infrastructure but also positioned them as pioneers in the banking sector’s adoption of graph technology.
Programming Foundation Models with DSPy - Meetup SlidesZilliz
Prompting language models is hard, while programming language models is easy. In this talk, I will discuss the state-of-the-art framework DSPy for programming foundation models with its powerful optimizers and runtime constraint system.
Goodbye Windows 11: Make Way for Nitrux Linux 3.5.0!SOFTTECHHUB
As the digital landscape continually evolves, operating systems play a critical role in shaping user experiences and productivity. The launch of Nitrux Linux 3.5.0 marks a significant milestone, offering a robust alternative to traditional systems such as Windows 11. This article delves into the essence of Nitrux Linux 3.5.0, exploring its unique features, advantages, and how it stands as a compelling choice for both casual users and tech enthusiasts.
3. What’s up?
• Why analytics?
• Case 1: Top selling items
• Case 2: Picking by FIFO
• Case 3: Efficient picking route
• Case 4: Picking efficiency
• Case 5: Forecasting sales
• Case 6: Forecast zero firework stock
• Case 7: Multi-order FIFO picking (time permitting)
• Any questions?
3 2012-12-05 #ukoug2012 Really Using Analytic Functions
4. Why analytics?
• Normal SQL functions operate on one row
• Aggregates can do more rows but loose detail
• When you need details together with subtotals, ranks,
ratios, comparisons, you could do:
Client operations (tool or code with variables/arrays)
Scalar subqueries (multiple access of same data)
Analytic functions (often much more efficient )
• Analytics allow you to operate across the entire resultset,
not just a single row
4 2012-12-05 #ukoug2012 Really Using Analytic Functions
5. Top Selling Items
Case 1
5 2012-12-05 #ukoug2012 Really Using Analytic Functions
6. Top selling items
• Classic task for a programmer:
• Show top three by product group
• Also show how big percentage they
sold of the total
– Both of the total by product group
– And of the grand total
6 2012-12-05 #ukoug2012 Really Using Analytic Functions
7. Tables
create table items(
item varchar2(10) primary key,
Items with groups
grp varchar2(10),
name varchar2(20)
)
/
create table sales ( Sales per month
item varchar2(10) references items (item),
mth date,
qty number
)
/
7 2012-12-05 #ukoug2012 Really Using Analytic Functions
8. Data - items
insert into items values ('101010','AUTO','Brake disc');
insert into items values ('102020','AUTO','Snow chain');
5 autoparts
insert into items values ('103030','AUTO','Sparc plug');
insert into items values ('104040','AUTO','Oil filter');
insert into items values ('105050','AUTO','Light bulb');
insert into items values ('201010','MOBILE','Handsfree');
insert into items values ('202020','MOBILE','Charger'); 5 mobile
insert into items values ('203030','MOBILE','iGloves');
insert into items values ('204040','MOBILE','Headset');
accessories
insert into items values ('205050','MOBILE','Cover');
8 2012-12-05 #ukoug2012 Really Using Analytic Functions
9. Data – sales AUTO
insert into sales values ('101010',date '2011-04-01',10);
insert into sales values ('101010',date '2011-05-01',11);
Sales for various
insert into sales values ('101010',date '2011-06-01',12); months of 2011 for
insert into sales values ('102020',date '2011-03-01', 7); the autoparts
insert into sales values ('102020',date '2011-07-01', 8);
insert into sales values ('103030',date '2011-01-01', 6);
insert into sales values ('103030',date '2011-02-01', 9);
insert into sales values ('103030',date '2011-11-01', 4);
insert into sales values ('103030',date '2011-12-01',14);
insert into sales values ('104040',date '2011-08-01',22);
insert into sales values ('105050',date '2011-09-01',13);
insert into sales values ('105050',date '2011-10-01',15);
9 2012-12-05 #ukoug2012 Really Using Analytic Functions
10. Data – sales MOBILE
insert into sales values ('201010',date '2011-04-01', 5);
insert into sales values ('201010',date '2011-05-01', 6);
Sales for various
insert into sales values ('201010',date '2011-06-01', 7); months of 2011 for
insert into sales values ('202020',date '2011-03-01',21); the mobile
insert into sales values ('202020',date '2011-07-01',23);
insert into sales values ('203030',date '2011-01-01', 7);
accessories
insert into sales values ('203030',date '2011-02-01', 7);
insert into sales values ('203030',date '2011-11-01', 6);
insert into sales values ('203030',date '2011-12-01', 8);
insert into sales values ('204040',date '2011-08-01',35);
insert into sales values ('205050',date '2011-09-01',13);
insert into sales values ('205050',date '2011-10-01',15);
10 2012-12-05 #ukoug2012 Really Using Analytic Functions
11. Base select
select i.grp
, i.item
Join items and sales
, max(i.name) name
, sum(s.qty) qty
from items i Sales for 2011
join sales s
on s.item = i.item
where s.mth between date '2011-01-01' and date '2011-12-01' Group by to get
group by i.grp, i.item
order by i.grp, sum(s.qty) desc, i.item
total sales for 2011
per item
11 2012-12-05 #ukoug2012 Really Using Analytic Functions
12. Base select
GRP ITEM NAME QTY
---------- ---------- -------------------- -----
Couple of items in
AUTO 101010 Brake disc 33 each group have
AUTO 103030 Sparc plug 33 identical sales
AUTO 105050 Light bulb 28
AUTO 104040 Oil filter 22
AUTO 102020 Snow chain 15
MOBILE 202020 Charger 44
MOBILE 204040 Headset 35
MOBILE 203030 iGloves 28
MOBILE 205050 Cover 28
MOBILE 201010 Handsfree 18
12 2012-12-05 #ukoug2012 Really Using Analytic Functions
13. Which TOP?
select g.grp, g.item, g.name, g.qty
, dense_rank() over (partition by g.grp order by g.qty desc) drnk
, rank() over (partition by g.grp order by g.qty desc) rnk
, row_number() over (partition by g.grp order by g.qty desc, g.item) rnum
from (
select i.grp
, i.item
, max(i.name) name
,
from
sum(s.qty) qty
items i
Base select as
join
on
sales s
s.item = i.item inline view
where s.mth between date '2011-01-01' and date '2011-12-01'
group by i.grp, i.item
) g
order by g.grp, g.qty desc, g.item
13 2012-12-05 #ukoug2012 Really Using Analytic Functions
14. Which TOP?
GRP ITEM NAME QTY DRNK RNK RNUM
---------- ---------- -------------------- ----- ----- ----- -----
The three different
AUTO 101010 Brake disc 33 1 1 1 functions handle
AUTO 103030 Sparc plug 33 1 1 2 ties differently
AUTO 105050 Light bulb 28 2 3 3
AUTO 104040 Oil filter 22 3 4 4
AUTO 102020 Snow chain 15 4 5 5
MOBILE 202020 Charger 44 1 1 1
MOBILE 204040 Headset 35 2 2 2
MOBILE 203030 iGloves 28 3 3 3
MOBILE 205050 Cover 28 3 3 4
MOBILE 201010 Handsfree 18 4 5 5
14 2012-12-05 #ukoug2012 Really Using Analytic Functions
15. Without inline view
select i.grp
, i.item
Analytics calculated last, so can include
, max(i.name) name group by expressions as well as aggregates
, sum(s.qty) qty
, dense_rank() over (partition by i.grp order by sum(s.qty) desc) drnk
, rank() over (partition by i.grp order by sum(s.qty) desc) rnk
, row_number() over (partition by i.grp order by sum(s.qty) desc, i.item) rnum
from items i
join sales s
on s.item = i.item
where s.mth between date '2011-01-01' and date '2011-12-01'
group by i.grp, i.item
order by i.grp, sum(s.qty) desc, i.item
15 2012-12-05 #ukoug2012 Really Using Analytic Functions
16. Without inline view
GRP ITEM NAME QTY DRNK RNK RNUM
---------- ---------- -------------------- ----- ----- ----- -----
Identical results
AUTO 101010 Brake disc 33 1 1 1
AUTO 103030 Sparc plug 33 1 1 2
AUTO 105050 Light bulb 28 2 3 3
AUTO 104040 Oil filter 22 3 4 4
AUTO 102020 Snow chain 15 4 5 5
MOBILE 202020 Charger 44 1 1 1
MOBILE 204040 Headset 35 2 2 2
MOBILE 203030 iGloves 28 3 3 3
MOBILE 205050 Cover 28 3 3 4
MOBILE 201010 Handsfree 18 4 5 5
16 2012-12-05 #ukoug2012 Really Using Analytic Functions
17. TOP 3 – rank()
select g.grp, g.item, g.name, g.qty, g.rnk
from (
Analytic function
select i.grp cannot be in
, i.item
, max(i.name) name
where clause
, sum(s.qty) qty
, rank() over (partition by i.grp order by sum(s.qty) desc) rnk
from items i So inline view
join sales s
on s.item = i.item and filter on the
where s.mth between date '2011-01-01' and date '2011-12-01' alias
group by i.grp, i.item
) g
where g.rnk <= 3
order by g.grp, g.rnk, g.item
17 2012-12-05 #ukoug2012 Really Using Analytic Functions
18. TOP 3 – rank()
GRP ITEM NAME QTY RNK
---------- ---------- -------------------- ----- -----
rank() works like the
AUTO 101010 Brake disc 33 1 olympics – two gold
AUTO 103030 Sparc plug 33 1 medals mean no
AUTO 105050 Light bulb 28 3
MOBILE 202020 Charger 44 1
silver medal
MOBILE 204040 Headset 35 2
MOBILE 203030 iGloves 28 3
MOBILE 205050 Cover 28 3
18 2012-12-05 #ukoug2012 Really Using Analytic Functions
19. TOP 3 – dense_rank()
select g.grp, g.item, g.name, g.qty, g.rnk
from (
select i.grp
, i.item
, max(i.name) name
, sum(s.qty) qty
, dense_rank() over (partition by i.grp order by sum(s.qty) desc) rnk
from items i
join sales s
on s.item = i.item
where s.mth between date '2011-01-01' and date '2011-12-01'
group by i.grp, i.item
) g
where g.rnk <= 3
order by g.grp, g.rnk, g.item
19 2012-12-05 #ukoug2012 Really Using Analytic Functions
20. TOP 3 – dense_rank()
GRP ITEM NAME QTY RNK
---------- ---------- -------------------- ----- -----
dense_rank() also
AUTO 101010 Brake disc 33 1 gives equal rank at
AUTO 103030 Sparc plug 33 1 ties, but does not
AUTO 105050 Light bulb 28 2
AUTO 104040 Oil filter 22 3
skip ranks
MOBILE 202020 Charger 44 1
MOBILE 204040 Headset 35 2
MOBILE 203030 iGloves 28 3
MOBILE 205050 Cover 28 3
20 2012-12-05 #ukoug2012 Really Using Analytic Functions
21. TOP 3 – row_number()
select g.grp, g.item, g.name, g.qty, g.rnk
from (
select i.grp
, i.item
, max(i.name) name
, sum(s.qty) qty
, row_number() over (partition by i.grp order by sum(s.qty) desc, i.item) rnk
from items i
join sales s
on s.item = i.item
where s.mth between date '2011-01-01' and date '2011-12-01'
group by i.grp, i.item
) g
where g.rnk <= 3
order by g.grp, g.rnk, g.item
21 2012-12-05 #ukoug2012 Really Using Analytic Functions
22. TOP 3 – row_number()
GRP ITEM NAME QTY RNK
---------- ---------- -------------------- ----- -----
row_number() just
AUTO 101010 Brake disc 33 1 numbers
AUTO 103030 Sparc plug 33 2 consecutively
AUTO 105050 Light bulb 28 3
MOBILE 202020 Charger 44 1
MOBILE 204040 Headset 35 2
MOBILE 203030 iGloves 28 3
If ties, then result is
”random”, so a good
idea always to use
”unique” order by
22 2012-12-05 #ukoug2012 Really Using Analytic Functions
23. Percent of total
select g.grp, g.item, g.name, g.qty, g.rnk
, round(g.g_pct,1) g_pct ratio_to_report()
, round(g.t_pct,1) t_pct
from ( returns number
select i.grp
, i.item between 0 and 1
, max(i.name) name
, sum(s.qty) qty
, rank() over (partition by i.grp order by sum(s.qty) desc) rnk
, 100 * ratio_to_report(sum(s.qty)) over (partition by i.grp) g_pct
, 100 * ratio_to_report(sum(s.qty)) over () t_pct Multiply with 100 to
from items i
join sales s get percent
on s.item = i.item
where s.mth between date '2011-01-01' and date '2011-12-01'
group by i.grp, i.item
) g
where g.rnk <= 3
order by g.grp, g.rnk, g.item
23 2012-12-05 #ukoug2012 Really Using Analytic Functions
24. Percent of total
GRP ITEM NAME QTY RNK G_PCT T_PCT
---------- ---------- -------------------- ----- ----- ------ ------
AUTO 101010 Brake disc 33 1 25.2 11.6
AUTO 103030 Sparc plug 33 1 25.2 11.6
AUTO 105050 Light bulb 28 3 21.4 9.9
MOBILE 202020 Charger 44 1 28.8 15.5
MOBILE 204040 Headset 35 2 22.9 12.3
MOBILE 203030 iGloves 28 3 18.3 9.9
MOBILE 205050 Cover 28 3 18.3 9.9
24 2012-12-05 #ukoug2012 Really Using Analytic Functions
25. Top selling items
• What kind of top three do you wish?
– DENSE_RANK()
– RANK()
– ROW_NUMBER()
• PARTITION BY groups of items
• RATIO_TO_REPORT for percentages
25 2012-12-05 #ukoug2012 Really Using Analytic Functions
26. Picking by FIFO
Case 2
26 2012-12-05 #ukoug2012 Really Using Analytic Functions
27. Picking by FIFO
• Items stored in different locations in warehouse
• Pick an order by First-In First-Out principle
27 2012-12-05 #ukoug2012 Really Using Analytic Functions
28. Tables
create table inventory (
item varchar2(10),
Item, location, quan
loc varchar2(10), tity and date of
qty number, purchase
purch date
)
/
create table orderline ( Order number,
ordno number,
item varchar2(10),
item and quantity
qty number ordered
)
/
28 2012-12-05 #ukoug2012 Really Using Analytic Functions
29. Data
insert into inventory values('A1', '1-A-20', 18, DATE '2004-11-01');
insert into inventory values('A1', '1-A-31', 12, DATE '2004-11-05');
2 items each 5
insert into inventory values('A1', '1-C-05', 18, DATE '2004-11-03'); locations various
insert into inventory values('A1', '2-A-02', 24, DATE '2004-11-02');
insert into inventory values('A1', '2-D-07', 9, DATE '2004-11-04');
purchase dates
insert into inventory values('B1', '1-A-02', 18, DATE '2004-11-06');
insert into inventory values('B1', '1-B-11', 4, DATE '2004-11-05');
insert into inventory values('B1', '1-C-04', 12, DATE '2004-11-03');
insert into inventory values('B1', '1-B-15', 2, DATE '2004-11-02');
insert into inventory values('B1', '2-D-23', 1, DATE '2004-11-04');
insert into orderline values (1,'A1',24); One order with a
insert into orderline values (1,'B1',18);
quantity of both
items
29 2012-12-05 #ukoug2012 Really Using Analytic Functions
30. What to pick
Picking application sets bind variable for which order to pick
(Could be sales order, batch order, shop refill order)
variable pick_order number;
begin
:pick_order := 1;
end;
/
30 2012-12-05 #ukoug2012 Really Using Analytic Functions
31. What can we pick
select o.item
, o.qty ord_qty
Join orderline to
, i.loc inventory to see all
, i.purch that potentially can
, i.qty loc_qty
from orderline o
be picked
join inventory i
on i.item = o.item
where o.ordno = :pick_order
order by o.item, i.purch, i.loc
31 2012-12-05 #ukoug2012 Really Using Analytic Functions
32. What can we pick
ITEM ORD_QTY LOC PURCH LOC_QTY
---------- ------- ---------- ---------- -------
Visually we can see
A1 24 1-A-20 2004-11-01 18 we need 18 A1 from
A1 24 2-A-02 2004-11-02 24 first location and 6
A1 24 1-C-05 2004-11-03 18
A1 24 2-D-07 2004-11-04 9
from second loc.
A1 24 1-A-31 2004-11-05 12
B1 18 1-B-15 2004-11-02 2
B1 18 1-C-04 2004-11-03 12 Likewise we will
B1 18 2-D-23 2004-11-04 1 empty first 3 locs of
B1 18 1-B-11 2004-11-05 4
B1 18 1-A-02 2004-11-06 18 B1 and pick 3 from
fourth location
32 2012-12-05 #ukoug2012 Really Using Analytic Functions
33. Accumulate
select o.item
, o.qty ord_qty
Let’s try to create a
, i.loc rolling sum of the
, i.purch
, i.qty loc_qty
qty for each item
, sum(i.qty) over (
partition by i.item
order by i.purch, i.loc
rows between unbounded preceding and current row
) sum_qty
from orderline o
join inventory i
on i.item = o.item
where o.ordno = :pick_order
order by o.item, i.purch, i.loc
33 2012-12-05 #ukoug2012 Really Using Analytic Functions
34. Accumulate
ITEM ORD_QTY LOC PURCH LOC_QTY SUM_QTY
---------- ------- ---------- ---------- ------- -------
Yup, when our sum
A1 24 1-A-20 2004-11-01 18 18 is greater than the
A1 24 2-A-02 2004-11-02 24 42 ordered qty, it looks
A1 24 1-C-05 2004-11-03 18 60
A1 24 2-D-07 2004-11-04 9 69
like we have enough
A1 24 1-A-31 2004-11-05 12 81
B1 18 1-B-15 2004-11-02 2 2
B1 18 1-C-04 2004-11-03 12 14
B1 18 2-D-23 2004-11-04 1 15
B1 18 1-B-11 2004-11-05 4 19
B1 18 1-A-02 2004-11-06 18 37
34 2012-12-05 #ukoug2012 Really Using Analytic Functions
35. Filter accumulated
select s.*
from ( So let’s try to filter
select o.item, o.qty ord_qty
, i.loc, i.purch, i.qty loc_qty
on that
, sum(i.qty) over (
partition by i.item
order by i.purch, i.loc
rows between unbounded preceding and current row
) sum_qty
from orderline o
join inventory i
on i.item = o.item
where o.ordno = :pick_order
) s
where s.sum_qty < s.ord_qty
order by s.item, s.purch, s.loc
35 2012-12-05 #ukoug2012 Really Using Analytic Functions
36. Filter accumulated
ITEM ORD_QTY LOC PURCH LOC_QTY SUM_QTY
---------- ------- ---------- ---------- ------- -------
FAIL!
A1 24 1-A-20 2004-11-01 18 18
B1 18 1-B-15 2004-11-02 2 2
B1 18 1-C-04 2004-11-03 12 14 Missing the last
B1 18 2-D-23 2004-11-04 1 15 location for each
item
36 2012-12-05 #ukoug2012 Really Using Analytic Functions
37. Accumulate previous
select o.item
, o.qty ord_qty
One small change to
, i.loc our rolling sum:
, i.purch
, i.qty loc_qty
, sum(i.qty) over (
partition by i.item Sum of rows up to
order by i.purch, i.loc but not including
rows between unbounded preceding and 1 preceding
) sum_prv_qty the current row
from orderline o
join inventory i
on i.item = o.item
where o.ordno = :pick_order
order by o.item, i.purch, i.loc
37 2012-12-05 #ukoug2012 Really Using Analytic Functions
38. Accumulate previous
ITEM ORD_QTY LOC PURCH LOC_QTY SUM_PRV_QTY
---------- ------- ---------- ---------- ------- -----------
As long as the sum
A1 24 1-A-20 2004-11-01 18 of the previous rows
A1 24 2-A-02 2004-11-02 24 18 are not enough, we
A1 24 1-C-05 2004-11-03 18 42
A1 24 2-D-07 2004-11-04 9 60
continue
A1 24 1-A-31 2004-11-05 12 69
B1 18 1-B-15 2004-11-02 2
B1 18 1-C-04 2004-11-03 12 2 When the previous
B1 18 2-D-23 2004-11-04 1 14 rows are sufficient,
B1 18 1-B-11 2004-11-05 4 15
B1 18 1-A-02 2004-11-06 18 19 we stop
38 2012-12-05 #ukoug2012 Really Using Analytic Functions
39. Filter previous
select s.*
, least(s.loc_qty, s.ord_qty - s.sum_prv_qty) pick_qty Now we can filter
from (
select o.item, o.qty ord_qty correctly
, i.loc, i.purch, i.qty loc_qty
, nvl(sum(i.qty) over (
partition by i.item
order by i.purch, i.loc nvl() for first row
rows between unbounded preceding and 1 preceding
),0) sum_prv_qty
from orderline o
join inventory i
on i.item = o.item
least() to get qty to
where o.ordno = :pick_order be picked at that
) s
where s.sum_prv_qty < s.ord_qty location
order by s.item, s.purch, s.loc
39 2012-12-05 #ukoug2012 Really Using Analytic Functions
41. Picking list FIFO
select s.loc
, s.item Now order by
, least(s.loc_qty, s.ord_qty - s.sum_prv_qty) pick_qty
from ( location to make a
select o.item, o.qty ord_qty
, i.loc, i.purch, i.qty loc_qty
pick list
, nvl(sum(i.qty) over (
partition by i.item
order by i.purch, i.loc
rows between unbounded preceding and 1 preceding
),0) sum_prv_qty
from orderline o
join inventory i
on i.item = o.item
where o.ordno = :pick_order
) s
where s.sum_prv_qty < s.ord_qty
order by s.loc
41 2012-12-05 #ukoug2012 Really Using Analytic Functions
42. Picking list FIFO
LOC ITEM PICK_QTY
---------- ---------- --------
Ready for operator
1-A-20 A1 18 to go picking
1-B-11 B1 3
1-B-15 B1 2
1-C-04 B1 12
2-A-02 A1 6
2-D-23 B1 1
42 2012-12-05 #ukoug2012 Really Using Analytic Functions
43. Picking small qty
select s.loc
, s.item Change pick policy
, least(s.loc_qty, s.ord_qty - s.sum_prv_qty) pick_qty
from ( by changing order:
select o.item, o.qty ord_qty
, i.loc, i.purch, i.qty loc_qty
, nvl(sum(i.qty) over (
partition by i.item Here empty small
order by i.qty, i.loc -- << only line changed
rows between unbounded preceding and 1 preceding quantities first to
),0) sum_prv_qty
from orderline o clean out locations
join inventory i
on i.item = o.item
where o.ordno = :pick_order
) s
where s.sum_prv_qty < s.ord_qty
order by s.loc
43 2012-12-05 #ukoug2012 Really Using Analytic Functions
44. Picking small qty
LOC ITEM PICK_QTY
---------- ---------- --------
Lots of picks
1-A-20 A1 3
1-A-31 A1 12
1-B-11 B1 4 Will clean locations
1-B-15 B1 2 quickly for new
1-C-04 B1 11
2-D-07 A1 9
incoming goods
2-D-23 B1 1
44 2012-12-05 #ukoug2012 Really Using Analytic Functions
45. Picking few picks
select s.loc
, s.item Or policy of picking
, least(s.loc_qty, s.ord_qty - s.sum_prv_qty) pick_qty
from ( as few times as
select o.item, o.qty ord_qty
, i.loc, i.purch, i.qty loc_qty
possible
, nvl(sum(i.qty) over (
partition by i.item
order by i.qty desc, i.loc -- << only line changed
rows between unbounded preceding and 1 preceding
),0) sum_prv_qty
from orderline o
join inventory i
on i.item = o.item
where o.ordno = :pick_order
) s
where s.sum_prv_qty < s.ord_qty
order by s.loc
45 2012-12-05 #ukoug2012 Really Using Analytic Functions
46. Picking few picks
LOC ITEM PICK_QTY
---------- ---------- --------
Only two picks
1-A-02 B1 18
2-A-02 A1 24
But will be at
expense of leaving
small quantities all
over warehouse
46 2012-12-05 #ukoug2012 Really Using Analytic Functions
47. Picking short route
select s.loc
, s.item Policy of not driving
, least(s.loc_qty, s.ord_qty - s.sum_prv_qty) pick_qty
from ( to the far
select o.item, o.qty ord_qty
, i.loc, i.purch, i.qty loc_qty
warehouse if
, nvl(sum(i.qty) over (
partition by i.item
possible
order by i.loc -- << only line changed
rows between unbounded preceding and 1 preceding
),0) sum_prv_qty
from orderline o
join inventory i
on i.item = o.item
where o.ordno = :pick_order
) s
where s.sum_prv_qty < s.ord_qty
order by s.loc
47 2012-12-05 #ukoug2012 Really Using Analytic Functions
48. Picking short route
LOC ITEM PICK_QTY
---------- ---------- --------
All picked in the
1-A-02 B1 18 very fist aisle in the
1-A-20 A1 18 warehouse
1-A-31 A1 6
48 2012-12-05 #ukoug2012 Really Using Analytic Functions
49. Picking by FIFO
• SUM() by item
• Ordered by purchase date
• Rolling sum to find how much was picked by ”previous
rows”
• Filter away rows where sufficient has already been
picked
49 2012-12-05 #ukoug2012 Really Using Analytic Functions
51. Picking small qty
select s.loc
, s.item Same data as FIFO
, least(s.loc_qty, s.ord_qty - s.sum_prv_qty) pick_qty
from ( picking case
select o.item, o.qty ord_qty
, i.loc, i.purch, i.qty loc_qty
, nvl(sum(i.qty) over (
partition by i.item But for this case we
order by i.qty, i.loc -- << only line changed
rows between unbounded preceding and 1 preceding will use the policy of
),0) sum_prv_qty
from orderline o picking small
join inventory i
on i.item = o.item
quantities
where o.ordno = :pick_order
) s
where s.sum_prv_qty < s.ord_qty
order by s.loc
51 2012-12-05 #ukoug2012 Really Using Analytic Functions
52. Picking small qty
LOC ITEM PICK_QTY
---------- ---------- --------
Because that gives
1-A-20 A1 3 many picks and
1-A-31 A1 12 shows this case best
1-B-11 B1 4
1-B-15 B1 2
1-C-04 B1 11
2-D-07 A1 9
Notice anything
2-D-23 B1 1 about these data?
52 2012-12-05 #ukoug2012 Really Using Analytic Functions
53. Picking route
• Is this a smart route to drive?
53 2012-12-05 #ukoug2012 Really Using Analytic Functions
54. Better picking route
• We need to change direction every other aisle
54 2012-12-05 #ukoug2012 Really Using Analytic Functions
55. Decipher loc
select to_number(substr(s.loc,1,1)) warehouse
, substr(s.loc,3,1) aisle In this case location
,
,
to_number(substr(s.loc,5,2)) position
s.loc
can be split into
, s.item warehouse, aisle
, least(s.loc_qty, s.ord_qty - s.sum_prv_qty) pick_qty
from ( and position simply
select o.item, o.qty ord_qty
, i.loc, i.purch, i.qty loc_qty
, nvl(sum(i.qty) over (
by substr()
partition by i.item
order by i.qty, i.loc
rows between unbounded preceding and 1 preceding
),0) sum_prv_qty
from orderline o
join inventory i
on i.item = o.item
where o.ordno = :pick_order
) s
where s.sum_prv_qty < s.ord_qty
order by s.loc
55 2012-12-05 #ukoug2012 Really Using Analytic Functions
56. Decipher loc
WAREHOUSE AISLE POSITION LOC ITEM PICK_QTY
--------- ----- -------- ---------- ---------- --------
Now we can use
1 A 20 1-A-20 A1 3 analytics on the
1 A 31 1-A-31 A1 12 individual parts of
1 B 11 1-B-11 B1 4
1 B 15 1-B-15 B1 2
the location
1 C 4 1-C-04 B1 11
2 D 7 2-D-07 A1 9
2 D 23 2-D-23 B1 1
56 2012-12-05 #ukoug2012 Really Using Analytic Functions
57. Rank aisles
select to_number(substr(s.loc,1,1)) warehouse
, substr(s.loc,3,1) aisle Ordering by
, dense_rank() over (
order by to_number(substr(s.loc,1,1)) -- warehouse warehouse and aisle
, substr(s.loc,3,1) -- aisle
) aisle_no will give same rank
, to_number(substr(s.loc,5,2)) position
, s.loc to positions in same
, s.item
, least(s.loc_qty, s.ord_qty - s.sum_prv_qty) pick_qty
from (
aisle
select o.item, o.qty ord_qty, i.loc, i.purch, i.qty loc_qty
, nvl(sum(i.qty) over (
partition by i.item
order by i.qty, i.loc
rows between unbounded preceding and 1 preceding
),0) sum_prv_qty dense_rank()
from orderline o
join inventory i
on i.item = o.item
ensures consecutive
where o.ordno = :pick_order
) s
where s.sum_prv_qty < s.ord_qty
ranks
order by s.loc
57 2012-12-05 #ukoug2012 Really Using Analytic Functions
58. Rank aisles
WAREHOUSE AISLE AISLE_NO POSITION LOC ITEM PICK_QTY
--------- ----- -------- -------- ---------- ---------- --------
Now we have
1 A 1 20 1-A-20 A1 3 numbered the aisles
1 A 1 31 1-A-31 A1 12 in the order they
1 B 2 11 1-B-11 B1 4
1 B 2 15 1-B-15 B1 2
are to be visited
1 C 3 4 1-C-04 B1 11
2 D 4 7 2-D-07 A1 9
2 D 4 23 2-D-23 B1 1
58 2012-12-05 #ukoug2012 Really Using Analytic Functions
59. Odd up, even down
select
s2.warehouse, s2.aisle, s2.aisle_no, s2.position, s2.loc, s2.item, s2.pick_qty mod() and case
from (
select to_number(substr(s.loc,1,1)) warehouse
, substr(s.loc,3,1) aisle
, dense_rank() over (
order by to_number(substr(s.loc,1,1)) -- warehouse
allows us to order
, substr(s.loc,3,1) -- aisle
) aisle_no
, to_number(substr(s.loc,5,2)) position
, s.loc, s.item
, least(s.loc_qty, s.ord_qty - s.sum_prv_qty) pick_qty
positive on odd
from (
select o.item, o.qty ord_qty, i.loc, i.purch, i.qty loc_qty
, nvl(sum(i.qty) over (
partition by i.item
order by i.qty, i.loc
aisles and negative
rows between unbounded preceding and 1 preceding
),0) sum_prv_qty
from orderline o
join inventory i
on i.item = o.item
on even aisles
where o.ordno = :pick_order
) s
where s.sum_prv_qty < s.ord_qty
) s2
order by s2.warehouse
, s2.aisle_no
, case
when mod(s2.aisle_no,2) = 1 then s2.position
else -s2.position
end
59 2012-12-05 #ukoug2012 Really Using Analytic Functions
60. Odd up, even down
WAREHOUSE AISLE AISLE_NO POSITION LOC ITEM PICK_QTY
--------- ----- -------- -------- ---------- ---------- --------
And so aisle
1 A 1 20 1-A-20 A1 3 1-A is ascending,
1 A 1 31 1-A-31 A1 12 1-B is descending,
1 B 2 15 1-B-15 B1 2
1 B 2 11 1-B-11 B1 4
1-C is ascending,
1 C 3 4 1-C-04 B1 11 2-D is descending
2 D 4 23 2-D-23 B1 1
2 D 4 7 2-D-07 A1 9
60 2012-12-05 #ukoug2012 Really Using Analytic Functions
61. Single door
• Direction has to ”restart” per warehouse
61 2012-12-05 #ukoug2012 Really Using Analytic Functions
62. Partition warehouse
select s2.warehouse, s2.aisle, s2.aisle_no, s2.position, s2.loc, s2.item, s2.pick_qty
from (
select to_number(substr(s.loc,1,1)) warehouse Move the
, substr(s.loc,3,1) aisle
, dense_rank() over (
partition by to_number(substr(s.loc,1,1)) -- warehouse
warehouse part
order by
) aisle_no
substr(s.loc,3,1) -- aisle from the order by to
, to_number(substr(s.loc,5,2)) position
, s.loc, s.item the partition by
, least(s.loc_qty, s.ord_qty - s.sum_prv_qty) pick_qty
from (
select o.item, o.qty ord_qty, i.loc, i.purch, i.qty loc_qty
, nvl(sum(i.qty) over (
partition by i.item
order by i.qty, i.loc
rows between unbounded preceding and 1 preceding
),0) sum_prv_qty
from orderline o
join inventory i
on i.item = o.item
where o.ordno = :pick_order
) s
where s.sum_prv_qty < s.ord_qty
) s2
order by s2.warehouse
, s2.aisle_no
, case
when mod(s2.aisle_no,2) = 1 then s2.position
else -s2.position
end
62 2012-12-05 #ukoug2012 Really Using Analytic Functions
63. Partition warehouse
WAREHOUSE AISLE AISLE_NO POSITION LOC ITEM PICK_QTY
--------- ----- -------- -------- ---------- ---------- --------
Now the aisle_no is
1 A 1 20 1-A-20 A1 3 restarted for each
1 A 1 31 1-A-31 A1 12 warehouse, so the
1 B 2 15 1-B-15 B1 2
1 B 2 11 1-B-11 B1 4
first visited aisle of a
1 C 3 4 1-C-04 B1 11 warehouse is always
2 D 1 7 2-D-07 A1 9
odd and therefore
2 D 1 23 2-D-23 B1 1
sorted ascending
63 2012-12-05 #ukoug2012 Really Using Analytic Functions
64. Efficient picking route
• DENSE_RANK() to number the aisles in order visited
• Order the output
– ”Up” on odd aisles
– ”Down” on even aisles
• Partition by warehouse if door is missing
64 2012-12-05 #ukoug2012 Really Using Analytic Functions
65. Picking efficiency
Case 4
65 2012-12-05 #ukoug2012 Really Using Analytic Functions
66. Picking efficiency
• How fast can operators
pick items?
• How much do they wait
idle for totes to arrive?
66 2012-12-05 #ukoug2012 Really Using Analytic Functions
67. Table
create table missions (
missionid number primary key,
Missions are
loadunit number, everytime a tote
departpos varchar2(10), (loadunit) goes from
departtime date,
arrivepos varchar2(10),
one position to
arrivetime date another position on
)
the conveyor system
/
67 2012-12-05 #ukoug2012 Really Using Analytic Functions
69. Arrivals
select a.arrivepos pos
, a.arrivetime time
All missions arriving
, a.loadunit at picking stations
, a.missionid PLF4 and PLF5 on
from missions a
where a.arrivepos in ('PLF4','PLF5')
April 12th after
and a.arrivetime >= to_date('2012-04-12 08:00:00', 08:00
'YYYY-MM-DD HH24:MI:SS')
and a.arrivetime <= to_date('2012-04-12 23:59:59',
'YYYY-MM-DD HH24:MI:SS')
order by a.arrivepos, a.arrivetime
69 2012-12-05 #ukoug2012 Really Using Analytic Functions
71. Departures
select d.departpos pos
, d.departtime time
All missions
, d.loadunit departing from
, d.missionid picking stations
from missions d
where d.departpos in ('PLF4','PLF5')
PLF4 and PLF5 on
and d.departtime >= to_date('2012-04-12 08:00:00', April 12th after
'YYYY-MM-DD HH24:MI:SS')
and d.departtime <= to_date('2012-04-12 23:59:59',
08:00
'YYYY-MM-DD HH24:MI:SS')
order by d.departpos, d.departtime
71 2012-12-05 #ukoug2012 Really Using Analytic Functions
73. Combined events
select pos, time, ad, loadunit, missionid
from (
select a.arrivepos pos, a.arrivetime time, 'A' ad, a.loadunit, a.missionid
from missions a
where a.arrivepos in ('PLF4','PLF5')
and a.arrivetime >= to_date('2012-04-12 08:00:00','YYYY-MM-DD HH24:MI:SS')
and a.arrivetime <= to_date('2012-04-12 23:59:59','YYYY-MM-DD HH24:MI:SS')
union all
select d.departpos pos, d.departtime time, 'D' ad, d.loadunit, d.missionid
from missions d
where d.departpos in ('PLF4','PLF5')
and d.departtime >= to_date('2012-04-12 08:00:00','YYYY-MM-DD HH24:MI:SS')
and d.departtime <= to_date('2012-04-12 23:59:59','YYYY-MM-DD HH24:MI:SS')
) s1
order by pos, time
73 2012-12-05 #ukoug2012 Really Using Analytic Functions
74. Combined events
POS TIME AD LOADUNIT MISSIONID
---- -------- -- --------- --------- Arrivals and
PLF4
PLF4
08:00:00
08:00:03
D
A
10067235 35988299
10063485 35986751
departures joined
PLF4 08:00:08 D 10063485 35988300 allows us to see the
PLF4 08:00:11 A 10069588 35986762
PLF4 08:01:07 D 10069588 35988307 loadunit arriving
PLF4 08:01:09 A 10059580 35986754 and a little bit later
PLF4 08:01:14 D 10059580 35988308
PLF4 08:01:16 A 10056277 35986755 departing
PLF4 08:01:24 D 10056277 35988309
PLF4 08:01:26 A 10081310 35986764
PLF4 08:01:39 D 10081310 35988310
PLF4 08:01:41 A 10016906 35986752
...
2700 rows selected.
74 2012-12-05 #ukoug2012 Really Using Analytic Functions
75. Lead the next event
with s1 as (
select a.arrivepos pos, a.arrivetime time,
from missions a
'A' ad, a.loadunit, a.missionid The analytic
where a.arrivepos in ('PLF4','PLF5')
and a.arrivetime >= to_date('2012-04-12
and a.arrivetime <= to_date('2012-04-12
08:00:00','YYYY-MM-DD HH24:MI:SS')
23:59:59','YYYY-MM-DD HH24:MI:SS')
function lead() gives
union all
select d.departpos pos, d.departtime time,
from missions d
'D' ad, d.loadunit, d.missionid for each row the
where d.departpos in ('PLF4','PLF5')
and d.departtime >= to_date('2012-04-12
and d.departtime <= to_date('2012-04-12
08:00:00','YYYY-MM-DD HH24:MI:SS')
23:59:59','YYYY-MM-DD HH24:MI:SS')
time of the next row
)
select pos
, time
, lead(time) over (
partition by pos
order by time, missionid
) nexttime
, ad
, loadunit
from s1
order by pos, time
75 2012-12-05 #ukoug2012 Really Using Analytic Functions
76. Lead the next event
POS TIME NEXTTIME AD LOADUNIT
---- -------- -------- -- --------- So on each ’D’ row
PLF4
PLF4
08:00:00
08:00:03
08:00:03
08:00:08
D
A
10067235
10063485
NEXTTIME is the
PLF4 08:00:08 08:00:11 D 10063485 time of the
PLF4 08:00:11 08:01:07 A 10069588
PLF4 08:01:07 08:01:09 D 10069588 following ’A’ row
PLF4 08:01:09 08:01:14 A 10059580
PLF4 08:01:14 08:01:16 D 10059580
PLF4 08:01:16 08:01:24 A 10056277
PLF4 08:01:24 08:01:26 D 10056277
And on each ’A’ row
PLF4 08:01:26 08:01:39 A 10081310 NEXTTIME is the
PLF4 08:01:39 08:01:41 D 10081310
PLF4 08:01:41 08:01:57 A 10016906 time of the
...
2700 rows selected.
following ’D’ row
76 2012-12-05 #ukoug2012 Really Using Analytic Functions
77. Lead on
with s1 as (
select a.arrivepos pos, a.arrivetime time,
from missions a
where a.arrivepos in ('PLF4','PLF5')
'A' ad, a.loadunit, a.missionid
lead() accepts a
second parameter
and a.arrivetime >= to_date('2012-04-12 08:00:00','YYYY-MM-DD HH24:MI:SS')
and a.arrivetime <= to_date('2012-04-12 23:59:59','YYYY-MM-DD HH24:MI:SS')
union all
select d.departpos pos, d.departtime time, 'D' ad, d.loadunit, d.missionid
from missions d
where d.departpos in ('PLF4','PLF5')
and d.departtime >= to_date('2012-04-12
and d.departtime <= to_date('2012-04-12
08:00:00','YYYY-MM-DD HH24:MI:SS')
23:59:59','YYYY-MM-DD HH24:MI:SS') telling how many
)
select pos, time rows forward the
, lead(time) over (
partition by pos function should
order by time, missionid
) nexttime
”look”
, lead(time,2) over (
partition by pos
order by time, missionid
) next2time
, ad, loadunit
from s1
order by pos, time
77 2012-12-05 #ukoug2012 Really Using Analytic Functions
78. Lead on
POS TIME NEXTTIME NEXT2TIM AD LOADUNIT
---- -------- -------- -------- -- --------- The NEXT2TIME
PLF4
PLF4
08:00:00
08:00:03
08:00:03
08:00:08
08:00:08
08:00:11
D
A
10067235
10063485
column ”looks” 2
PLF4 08:00:08 08:00:11 08:01:07 D 10063485 rows forward
PLF4 08:00:11 08:01:07 08:01:09 A 10069588
PLF4 08:01:07 08:01:09 08:01:14 D 10069588
PLF4 08:01:09 08:01:14 08:01:16 A 10059580
PLF4 08:01:14 08:01:16 08:01:24 D 10059580
PLF4 08:01:16 08:01:24 08:01:26 A 10056277
PLF4 08:01:24 08:01:26 08:01:39 D 10056277
PLF4 08:01:26 08:01:39 08:01:41 A 10081310
PLF4 08:01:39 08:01:41 08:01:57 D 10081310
PLF4 08:01:41 08:01:57 08:01:59 A 10016906
...
2700 rows selected.
78 2012-12-05 #ukoug2012 Really Using Analytic Functions
79. Filter double lead
with s1 as (
select a.arrivepos pos, a.arrivetime time, 'A' ad, a.loadunit, a.missionid
from missions a
where a.arrivepos in ('PLF4','PLF5')
and a.arrivetime >= to_date('2012-04-12
and a.arrivetime <= to_date('2012-04-12
union all
select d.departpos pos, d.departtime time,
08:00:00','YYYY-MM-DD HH24:MI:SS')
23:59:59','YYYY-MM-DD HH24:MI:SS')
'D' ad, d.loadunit, d.missionid
Since we use the
double lead we now
from missions d
where d.departpos in ('PLF4','PLF5')
and d.departtime >= to_date('2012-04-12 08:00:00','YYYY-MM-DD HH24:MI:SS')
and d.departtime <= to_date('2012-04-12 23:59:59','YYYY-MM-DD HH24:MI:SS')
)
select pos, time arrive, nexttime depart, next2time nextarrive, loadunit
from ( have all the data
select pos, time
, lead(time) over ( necessary on the ’A’
partition by pos
order by time, missionid rows and do not
) nexttime
, lead(time,2) over ( need the ’D’ rows
partition by pos
order by time, missionid anymore
) next2time
, ad, loadunit
from s1
) s2
where ad = 'A'
order by pos, arrive
79 2012-12-05 #ukoug2012 Really Using Analytic Functions
80. Filter double lead
POS ARRIVE DEPART NEXTARRI LOADUNIT
---- -------- -------- -------- ---------
We can now see a tote
PLF4 08:00:03 08:00:08 08:00:11 10063485 arrives 08:00:03, leaves
PLF4 08:00:11 08:01:07 08:01:09 10069588
PLF4 08:01:09 08:01:14 08:01:16 10059580
again at 08:00:08, and a
PLF4 08:01:16 08:01:24 08:01:26 10056277 new tote arrives at
PLF4 08:01:26 08:01:39 08:01:41 10081310
PLF4 08:01:41 08:01:57 08:01:59 10016906 08:00:11
...
PLF4 10:59:47 10:59:54 10:59:56 10076144
PLF4 10:59:56 11:00:11 11:00:12 10012882 Note the tote that
PLF4 11:00:12 11:00:28 11:00:29 10035898
PLF4 11:00:29 11:00:42 11:00:44 10076793 arrived 10:59:56 leaves
... after 11:00:00
1453 rows selected.
80 2012-12-05 #ukoug2012 Really Using Analytic Functions
81. Pick and wait
with s1 as ( ... )
select pos, arrive, depart, nextarrive Calculate pick
, (depart - arrive) * 24 * 60 * 60 pickseconds
, (nextarrive - depart) * 24 * 60 * 60 waitseconds seconds and wait
from (
select pos, time arrive, nexttime depart, next2time nextarrive, loadunit
from (
seconds
select pos, time
, lead(time) over (
partition by pos order by time, missionid
) nexttime
, lead(time,2) over (
partition by pos order by time, missionid
) next2time
, ad, loadunit
from s1
) s2
where ad = 'A'
) s3 Filter on desired 3
where arrive >= to_date('2012-04-12 08:00:00','YYYY-MM-DD HH24:MI:SS')
and arrive <= to_date('2012-04-12 10:59:59','YYYY-MM-DD HH24:MI:SS') hour interval
order by pos, arrive
81 2012-12-05 #ukoug2012 Really Using Analytic Functions
82. Pick and wait
POS ARRIVE DEPART NEXTARRI PICKSECONDS WAITSECONDS
---- -------- -------- -------- ----------- ----------- How fast did the
PLF4 08:00:03 08:00:08 08:00:11 5 3
PLF4 08:00:11 08:01:07 08:01:09 56 2 operator pick and
PLF4 08:01:09 08:01:14 08:01:16 5 2
PLF4 08:01:16 08:01:24 08:01:26 8 2 how long time did
...
PLF4 08:58:08 08:58:08 09:11:36 0 808 he wait for a new
PLF4 09:11:36 09:12:55 09:12:56 79 1
... tote to arrive
PLF4 10:59:47 10:59:54 10:59:56 7 2
PLF4 10:59:56 11:00:11 11:00:12 15 1
PLF5 08:00:00 08:00:06 08:00:07 6 1
PLF5 08:00:07 08:00:13 08:00:14 6 1
...
PLF5 10:57:54 10:59:58 10:59:59 124 1
PLF5 10:59:59 11:00:09 11:00:10 10 1
1155 rows selected.
82 2012-12-05 #ukoug2012 Really Using Analytic Functions
83. Hourly stats
with s1 as ( ... )
select
,
pos
trunc(arrive,'HH24') hour
Now we can use the
,
,
count(*) picks
avg(pickseconds) secondsprpick
previous select as
,
,
sum(pickseconds)/60 minutespicked
100*sum(pickseconds)/sum(pickseconds+waitseconds) pickpct basis for some plain
, avg(waitseconds) secondsprwait
, sum(waitseconds)/60 minuteswaited statistics by the
, 100*sum(waitseconds)/sum(pickseconds+waitseconds) waitpct
,
,
avg(pickseconds+waitseconds) secondsprcycle
sum(pickseconds+waitseconds)/60 minutestotal
hour
, 60 * count(*) / sum(pickseconds+waitseconds) cyclesprmin
from (
select pos, arrive, depart, nextarrive
, (depart - arrive) * 24 * 60 * 60 pickseconds
, (nextarrive - depart) * 24 * 60 * 60 waitseconds
from (
select pos, time arrive, nexttime depart, next2time nextarrive, loadunit
from (
select pos, time
, lead(time) over (
partition by pos order by time, missionid
) nexttime
, lead(time,2) over (
partition by pos order by time, missionid
) next2time
, ad, loadunit
from s1
) s2
where ad = 'A'
) s3
where arrive >= to_date('2012-04-12 08:00:00','YYYY-MM-DD HH24:MI:SS')
and arrive <= to_date('2012-04-12 10:59:59','YYYY-MM-DD HH24:MI:SS')
) s4
group by pos, trunc(arrive,'HH24')
order by pos, trunc(arrive,'HH24')
83 2012-12-05 #ukoug2012 Really Using Analytic Functions
85. Picking efficiency
• Log over tote missions arriving and departing the
picking stations
• LEAD() on mission log to find the departure following
an arrival => picking time
• LEAD(,2) on mission log to find the arrival following a
departure => waiting time
85 2012-12-05 #ukoug2012 Really Using Analytic Functions
86. Forecasting sales
Case 5
86 2012-12-05 #ukoug2012 Really Using Analytic Functions
87. Forecasting sales
• Forecast the sales
of next year
• But follow the
trend of the item
87 2012-12-05 #ukoug2012 Really Using Analytic Functions
88. Table
create table sales (
item varchar2(10),
Simple table of
mth date, monthly sales by
qty number item
)
/
88 2012-12-05 #ukoug2012 Really Using Analytic Functions
89. Data
insert into sales values ('Snowchain', date '2008-01-01', 79);
insert into sales values ('Snowchain', date '2008-02-01', 133);
Item Snowchain
insert into sales values ('Snowchain', date '2008-03-01', 24); sells good in winter
... and trends up
insert into sales values ('Snowchain', date '2010-10-01', 1);
insert into sales values ('Snowchain', date '2010-11-01', 73);
insert into sales values ('Snowchain', date '2010-12-01', 160);
insert into sales values ('Sunshade' , date '2008-01-01', 4);
insert into sales values ('Sunshade' , date '2008-02-01', 6); Item Sunshade sells
insert into sales values ('Sunshade' , date '2008-03-01', 32);
...
good in summer
insert into sales values ('Sunshade' , date '2010-10-01', 11); and trends down
insert into sales values ('Sunshade' , date '2010-11-01', 3);
insert into sales values ('Sunshade' , date '2010-12-01', 5);
89 2012-12-05 #ukoug2012 Really Using Analytic Functions
90. Slope
select sales.item Graph slope: y-axis is qty
, sales.mth
, sales.qty
x-axis is a number with the scale of 1=a month
, regr_slope( Range between gives sliding 2-year window
sales.qty
, extract(year from sales.mth) * 12 + extract(month from sales.mth)
) over (
partition by sales.item
order by sales.mth
range between interval '23' month preceding and current row
) slope
from sales
order by sales.item, sales.mth
90 2012-12-05 #ukoug2012 Really Using Analytic Functions
91. Slope
ITEM MTH QTY SLOPE
---------- ---------- ----- -------- Slope value most
Snowchain 2008-01-01 79
Snowchain 2008-02-01 133 54.000 accurate for 2010
Snowchain
...
2008-03-01 24 -27.500
data where 2 year
Snowchain
Snowchain
2010-10-01
2010-11-01
1
73
-2.274
-2.363
sliding window
Snowchain 2010-12-01 160 -.991 contains full set of
Sunshade 2008-01-01 4
Sunshade 2008-02-01 6 2.000 data
Sunshade 2008-03-01 32 14.000
...
Sunshade 2010-10-01 11 .217
Sunshade 2010-11-01 3 -.200
Sunshade 2010-12-01 5 -.574
72 rows selected.
91 2012-12-05 #ukoug2012 Really Using Analytic Functions
92. Transpose using slope
select item, mth, qty
, qty + 12 * slope qty_next_year
As x-axis had scale
from ( of 1=a month and y-
select sales.item, sales.mth, sales.qty
, regr_slope( axis was qty,
sales.qty
, extract(year from sales.mth) * 12 + extract(month from sales.mth) multiplying slope
) over (
partition by sales.item
with 12 gives how
order by sales.mth
range between interval '23' month preceding and current row
much qty goes up or
) slope down in a year
from sales
)
where mth >= date '2010-01-01'
order by item, mth
92 2012-12-05 #ukoug2012 Really Using Analytic Functions
94. Forecast
select item
, add_months(mth, 12) mth
Rather than column
, greatest(round(qty + 12 * slope), 0) forecast QTY_NEXT_YEAR we
from (
select sales.item, sales.mth, sales.qty add a year to the
, regr_slope(
sales.qty
month and call it a
, extract(year from sales.mth) * 12 + extract(month from sales.mth)
) over (
forecast
partition by sales.item
order by sales.mth
range between interval '23' month preceding and current row
) slope We round the
from sales
)
numbers and skip
where mth >= date '2010-01-01' any negatives
order by item, mth
94 2012-12-05 #ukoug2012 Really Using Analytic Functions
96. Actual + forecast
select item, mth, qty, type
from (
select sales.item, sales.mth, sales.qty, 'Actual' type
UNION ALL of the
from sales
union all
actual data and the
select item
, add_months(mth, 12) mth forecast data for a
, greatest(round(qty + 12 * slope), 0) qty
, 'Forecast' type complete set of
from (
select sales.item, sales.mth, sales.qty
, regr_slope(
sales data that can
sales.qty
, extract(year from sales.mth) * 12 + extract(month from sales.mth)
) over (
be shown in a graph
partition by sales.item
order by sales.mth
range between interval '23' month preceding and current row
) slope
from sales
)
where mth >= date '2010-01-01'
)
order by item, mth
96 2012-12-05 #ukoug2012 Really Using Analytic Functions
97. Actual + forecast
ITEM MTH QTY TYPE
---------- ---------- ----- ----------
Snowchain 2008-01-01 79 Actual Sunshade 2008-01-01 4 Actual
Snowchain 2008-02-01 133 Actual Sunshade 2008-02-01 6 Actual
Snowchain 2008-03-01 24 Actual Sunshade 2008-03-01 32 Actual
... ...
Snowchain 2010-10-01 1 Actual Sunshade 2010-10-01 11 Actual
Snowchain 2010-11-01 73 Actual Sunshade 2010-11-01 3 Actual
Snowchain 2010-12-01 160 Actual Sunshade 2010-12-01 5 Actual
Snowchain 2011-01-01 188 Forecast Sunshade 2011-01-01 0 Forecast
Snowchain 2011-02-01 305 Forecast Sunshade 2011-02-01 0 Forecast
Snowchain 2011-03-01 96 Forecast Sunshade 2011-03-01 9 Forecast
... ...
Snowchain 2011-10-01 0 Forecast Sunshade 2011-10-01 14 Forecast
Snowchain 2011-11-01 45 Forecast Sunshade 2011-11-01 1 Forecast
Snowchain 2011-12-01 148 Forecast Sunshade 2011-12-01 0 Forecast
97 2012-12-05 #ukoug2012 Really Using Analytic Functions
98. Actual + forecast
Data from previous
slide is the graph
• ”Actual” is
normal lines
• ”Forecast” is
stapled lines
98 2012-12-05 #ukoug2012 Really Using Analytic Functions
99. Forecasting sales
• REGR_SLOPE() to calculate trend
• RANGE window for sliding trend calculation over
three years
• ”Transpose” last years sales by the slope to get
next years forecast
99 2012-12-05 #ukoug2012 Really Using Analytic Functions
100. Forecast zero stock
Case 6
100 2012-12-05 #ukoug2012 Really Using Analytic Functions
101. Forecast zero stock
• Fireworks sell like crazy
last week of December
• What hour will a store
run out of stock?
101 2012-12-05 #ukoug2012 Really Using Analytic Functions
102. Tables
create table fw_store (
shopid varchar2(10) primary key,
Stores are defined
containers integer by how many
) storage containers
/
create table fw_sales (
shopid varchar2(10) references fw_store (shopid),
Sales are hourly
saleshour date, data per shop in
salesnem number Net Explosive Mass
)
/
102 2012-12-05 #ukoug2012 Really Using Analytic Functions
103. Tables
create table fw_daybudget (
shopid varchar2(10) references fw_store (shopid),
Daily budget of
budgetdate date, Net Explosive Mass
budgetnem number per shop
)
/
create table fw_hourbudget (
Percentage of a
hour integer, days budget
percent number expected to be in
)
/ each hour
103 2012-12-05 #ukoug2012 Really Using Analytic Functions
104. Data - store
insert into fw_store values ('AALBORG' , 4);
insert into fw_store values ('GLOSTRUP' , 4);
insert into fw_store values ('HADERSLEV', 3);
104 2012-12-05 #ukoug2012 Really Using Analytic Functions
105. Data - sales
insert into fw_sales
select shopid, day + numtodsinterval(hour,'hour') saleshour, salesnem
from (
select 'AALBORG' shopid, date '2011-12-27' day,
4 h9, 6 h10, 5 h11, 20 h12, 19 h13, 22 h14, 27 h15, 11 h16, 16 h17, 4 h18 from dual union all
select 'AALBORG' , date '2011-12-28', 7, 17, 18, 13, 27, 28, 20, 14, 10, 19 from dual union all
select 'AALBORG' , date '2011-12-29', 10, 14, 20, null, null, null, null, null, null, null from dual union all
select 'GLOSTRUP' , date '2011-12-27', 1, 6, 6, 14, 17, 17, 13, 15, 7, 7 from dual union all
select 'GLOSTRUP' , date '2011-12-28', 4, 14, 30, 35, 22, 21, 35, 34, 15, 25 from dual union all
select 'GLOSTRUP' , date '2011-12-29', 6, 13, 50, null, null, null, null, null, null, null from dual union all
select 'HADERSLEV', date '2011-12-27', 4, 7, 13, 15, 17, 13, 18, 19, 10, 3 from dual union all
select 'HADERSLEV', date '2011-12-28', 8, 5, 14, 18, 20, 18, 15, 24, 12, 1 from dual union all
select 'HADERSLEV', date '2011-12-29', 1, 19, 33, null, null, null, null, null, null, null from dual
) s1
unpivot exclude nulls (
salesnem for hour in (
h9 as 9, h10 as 10, h11 as 11, h12 as 12, h13 as 13, h14 as 14, h15 as 15, h16 as 16, h17 as 17, h18 as 18
)
)
105 2012-12-05 #ukoug2012 Really Using Analytic Functions