The document discusses point processing and histograms. It covers various point processing techniques including linear, logarithmic, exponential, and power-law transformations that process each pixel based on its value. It also defines histograms as the distribution of pixel intensities in an image and how they can be used to analyze properties such as contrast. Methods for computing histograms in MATLAB and C/C++ are also presented.
The document discusses various image enhancement techniques in digital image processing. It describes point operations like image negative, contrast stretching, thresholding, brightness enhancement, log transformation, and power law transformation. Contrast stretching expands the range of intensity levels and can be done by multiplying pixels with a constant, using a transfer function, or histogram equalization. Thresholding converts an image to binary by assigning pixel values above a threshold to one level and below to another. Log and power law transformations compress high intensity values and expand low values to enhance an image. Matlab code examples are provided for each technique.
The document discusses various image enhancement techniques in digital image processing. It describes point operations like image negative, contrast stretching, thresholding, brightness enhancement, log transformation, and power law transformation. Contrast stretching expands the range of intensity levels and can be done by multiplying pixels with a constant, using a transfer function, or histogram equalization. Thresholding converts an image to binary by assigning pixel values above a threshold to one level and below to another. Log and power law transformations compress high intensity values and expand low values to enhance an image. Matlab code examples are provided for each technique.
Image enhancement techniques can be divided into two categories: spatial domain and frequency domain. Spatial domain techniques include point and mask operations. Point operations modify pixel intensities individually without considering neighboring pixels, such as image negative, contrast stretching, thresholding, and brightness enhancement. Mask operations involve multiplying the input image with a small matrix mask to blur, sharpen, or detect edges by considering neighboring pixel values. Common image enhancement techniques in the spatial domain include logarithmic and power law transformations to expand the dynamic range of darker pixels.
This document discusses image enhancement techniques in the spatial domain. It describes two categories of spatial domain operations: point processing and neighborhood processing. Point processing involves direct manipulation of pixel values through techniques like contrast stretching and thresholding. Neighborhood processing considers pixels in a local region and applies techniques like averaging filters. The document outlines several gray level transformations for enhancement, including logarithmic, power-law, piecewise linear, and bit-plane slicing transformations. It also discusses arithmetic and logic operations on images.
This document describes the design of an IEEE 754 single precision floating point multiplier. It discusses how floating point numbers are represented in binary format according to the IEEE 754 standard. The designed multiplier handles overflow and underflow cases and implements floating point multiplication by multiplying significands, adding exponents, determining the sign, and normalizing the result. Simulation results show the multiplier works as intended according to the IEEE 754 standard.
The document discusses various image enhancement techniques in digital image processing. It describes how image enhancement can be done using point operations, mask operations, and in the spatial and frequency domains. Some common point operations discussed include image negative, contrast stretching, thresholding, brightness enhancement, log transformation, and power law transformation. Contrast stretching can expand the range of intensity levels in an image. Thresholding converts an image to binary black and white. Log and power law transformations are used to expand values of dark pixels and compress values of bright pixels. Mask operations involve multiplying an image with a small matrix mask to perform operations like blurring and sharpening.
The document provides the MATLAB code to solve systems of linear equations using Gauss elimination with partial pivoting. It takes the matrix A and column matrix B as input from the user. It finds the maximum element in each column of A using partial pivoting to maintain stability. It then performs forward elimination on the augmented matrix [A B]. It displays the updated matrices after each step. Finally, it performs back substitution to calculate the values of the unknown matrix X and displays the result.
The document discusses various image enhancement techniques in digital image processing. It describes point operations like image negative, contrast stretching, thresholding, brightness enhancement, log transformation, and power law transformation. Contrast stretching expands the range of intensity levels and can be done by multiplying pixels with a constant, using a transfer function, or histogram equalization. Thresholding converts an image to binary by assigning pixel values above a threshold to one level and below to another. Log and power law transformations compress high intensity values and expand low values to enhance an image. Matlab code examples are provided for each technique.
The document discusses various image enhancement techniques in digital image processing. It describes point operations like image negative, contrast stretching, thresholding, brightness enhancement, log transformation, and power law transformation. Contrast stretching expands the range of intensity levels and can be done by multiplying pixels with a constant, using a transfer function, or histogram equalization. Thresholding converts an image to binary by assigning pixel values above a threshold to one level and below to another. Log and power law transformations compress high intensity values and expand low values to enhance an image. Matlab code examples are provided for each technique.
Image enhancement techniques can be divided into two categories: spatial domain and frequency domain. Spatial domain techniques include point and mask operations. Point operations modify pixel intensities individually without considering neighboring pixels, such as image negative, contrast stretching, thresholding, and brightness enhancement. Mask operations involve multiplying the input image with a small matrix mask to blur, sharpen, or detect edges by considering neighboring pixel values. Common image enhancement techniques in the spatial domain include logarithmic and power law transformations to expand the dynamic range of darker pixels.
This document discusses image enhancement techniques in the spatial domain. It describes two categories of spatial domain operations: point processing and neighborhood processing. Point processing involves direct manipulation of pixel values through techniques like contrast stretching and thresholding. Neighborhood processing considers pixels in a local region and applies techniques like averaging filters. The document outlines several gray level transformations for enhancement, including logarithmic, power-law, piecewise linear, and bit-plane slicing transformations. It also discusses arithmetic and logic operations on images.
This document describes the design of an IEEE 754 single precision floating point multiplier. It discusses how floating point numbers are represented in binary format according to the IEEE 754 standard. The designed multiplier handles overflow and underflow cases and implements floating point multiplication by multiplying significands, adding exponents, determining the sign, and normalizing the result. Simulation results show the multiplier works as intended according to the IEEE 754 standard.
The document discusses various image enhancement techniques in digital image processing. It describes how image enhancement can be done using point operations, mask operations, and in the spatial and frequency domains. Some common point operations discussed include image negative, contrast stretching, thresholding, brightness enhancement, log transformation, and power law transformation. Contrast stretching can expand the range of intensity levels in an image. Thresholding converts an image to binary black and white. Log and power law transformations are used to expand values of dark pixels and compress values of bright pixels. Mask operations involve multiplying an image with a small matrix mask to perform operations like blurring and sharpening.
The document provides the MATLAB code to solve systems of linear equations using Gauss elimination with partial pivoting. It takes the matrix A and column matrix B as input from the user. It finds the maximum element in each column of A using partial pivoting to maintain stability. It then performs forward elimination on the augmented matrix [A B]. It displays the updated matrices after each step. Finally, it performs back substitution to calculate the values of the unknown matrix X and displays the result.
The document discusses various types of transformations in computer graphics, including translation, scaling, and rotation in both 2D and 3D. Translation moves an object by adding offsets to its coordinates. Scaling changes the size of an object by multiplying its coordinates by scaling factors. Rotation changes the orientation of an object by applying trigonometric functions to its coordinates based on a rotation angle and axis. Transformation matrices are used to represent and apply these operations to objects uniformly.
Chapter 8 discusses converter transfer functions and Bode plots. It reviews common transfer function elements like poles, zeros and their impact on Bode plots. Specific topics covered include the single pole response, single zero response, right half-plane zeros, and combinations of elements. It also discusses how to analyze converter transfer functions, construct them graphically, and measure real converter transfer functions and impedances. The chapter aims to provide engineers with the tools needed to model, analyze and design power converters.
Image Processing using Matlab . Useful for beginners to learn Image ProcessingAshok Kumar
Matlab can be used for image processing tasks such as loading, displaying, and manipulating images. Images are represented as matrices where each element corresponds to the pixel intensity values. Common operations include convolutions using various kernel filters to perform tasks like smoothing, sharpening, and edge detection. Functions such as imread, image, and imshow can load and display images. Built-in functions such as fspecial generate common kernel filters. Convolution functions convolve images with kernels to apply filtering effects.
A Dimension Abstraction Approach to Vectorization in MatlabaiQUANT
The document presents an approach to vectorizing Matlab source code while preserving correctness and improving efficiency. It describes representing variable dimensionalities, rules for determining when vectorization is valid, and techniques like transpose transformations and additive reduction to enable more vectorizations. Evaluation on image processing and linear algebra examples showed speedups of 1.56x to 4.6x from vectorizing loop-based code.
The document presents an approach to vectorizing Matlab source code while preserving correctness and improving efficiency. It describes representing variable dimensionalities, rules for determining when vectorization is valid, and techniques like transpose transformations and additive reduction to enable more vectorizations. Evaluation on image processing and linear algebra examples showed speedups of 1.56x to 4.6x from vectorizing loop-based code.
IRJET-Design and Implementation of LNS based Approximate Multiplier using Mit...IRJET Journal
This document describes a design for an approximate logarithmic multiplier that uses Mitchell's algorithm and operand decomposition to improve accuracy while maintaining low power consumption. It is intended for applications like convolutional neural networks and digital signal/image processing that require multiplication but not highly accurate results. The design takes two binary inputs, divides them into four parts using logic operations, and applies Mitchell's algorithm to each part separately before adding the results. This operand decomposition approach increases the accuracy over using Mitchell's alone. The design is implemented using Xilinx tools and simulation waveforms and block diagrams are provided to illustrate its operation. Experimental results show this approach reduces error compared to using just Mitchell's algorithm for multiplication.
Novel Parallel - Perfix Structure Binary to Residue Number System Conversion ...CSCJournals
In the present world there is always a demand for faster algorithms and techniques that could boost up the speed of the computations. With the help of VLSI fabrication techniques and using residue number system (RNS) arithmetic we can achieve the faster speeds. In this paper we propose a novel parallel prefix binary to residue number system conversion method. The method that we present in this paper utilizes parallel-prefix technique with multiplexers and modulo adders as the main building blocks which makes it practical and suitable for VLSI implementation.
The document proposes an algorithm called Robust Tensor Decomposition (RTD) to recover a low-rank tensor from observations that are corrupted by block sparse errors. RTD iteratively estimates the low-rank and sparse components using hard thresholding and tensor projections. It compares RTD to existing approaches on video sequences, finding RTD yields more accurate recovery with fewer assumptions.
Small Signal Modelling and Controller Design of Boost Converter using MATLABPremier Publishers
This document presents a systematic approach for designing a PID controller for a boost converter using MATLAB. It begins with deriving the small-signal model of the boost converter theoretically and comparing it to the model generated in MATLAB. It then uses the linearized model from MATLAB to design the PID controller. Finally, it verifies the controller's performance for variations in input voltage and load current.
This presentation describes briefly about the image enhancement in spatial domain, basic gray level transformation, histogram processing, enhancement using arithmetic/ logical operation, basics of spatial filtering and local enhancements.
This document provides an overview of image processing using MATLAB. It discusses how images are represented as matrices in MATLAB and demonstrates various image processing functions and techniques.
Key points covered include:
- Loading and displaying an image using imread and image commands
- Converting between intensity, indexed, and RGB image representations
- Exploring image histograms and equalization
- Performing operations like resizing, rotation and filtering using functions like imresize, imrotate, and filters from fspecial
- Implementing convolution using custom kernels and built-in filters
- Understanding effects of different kernels on images
Digital image enhancement involves modifying images to improve visual quality. It can be done in the spatial or frequency domain. Spatial domain enhancement works directly with pixel values. Key point processing techniques in the spatial domain include: digital negative, contrast stretching, thresholding, gray level slicing, bit plane slicing, and dynamic range compression. These techniques apply mathematical transforms to pixel values to darken or lighten regions of the image for better visualization.
This document provides an overview of constructing Bode plots to analyze the frequency response of control systems. It discusses:
1) How Bode plots show the magnitude and phase shift of a transfer function as functions of frequency on logarithmic axes.
2) The steps to compute magnitude and phase values at different frequencies from a transfer function and plot the results.
3) How to use MATLAB functions like tf, zpk and bode to define transfer functions as linear time-invariant systems and generate Bode plots, including examples comparing different transfer functions on the same plot.
The document discusses a workshop on image processing using MATLAB. It provides an overview of MATLAB and its image processing toolbox. It describes how to read, display, and convert between different image formats in MATLAB. It also demonstrates various image processing operations that can be performed, such as arithmetic operations, conversion between color and grayscale, image rotation, blurring and deblurring, and filling regions of interest. The document aims to introduce the basics of working with images in the MATLAB environment.
The document discusses a workshop on image processing using MATLAB. It provides an overview of MATLAB and its image processing toolbox. It describes how to read, display, and convert between different image formats in MATLAB. It also demonstrates various image processing operations like arithmetic operations, conversion between color and grayscale, image rotation, blurring and deblurring, and filling regions of interest. The document aims to introduce the basics of image processing using MATLAB.
This document provides an introduction to Matlab and Simulink for control applications. It discusses what Matlab and Simulink are, their main components and functions. It then gives examples of using Matlab to model control systems using transfer functions, Laplace transforms and PID controllers. Specific examples include modeling and controlling a mass-spring-damper system and designing a cruise control system for a car.
This document summarizes key concepts about combinational logic circuits. It defines combinational logic as circuits whose outputs depend only on the current inputs, in contrast to sequential logic which also depends on prior inputs. Common combinational circuits are described like half and full adders used for arithmetic, as well as decoders. The design process for combinational circuits is outlined involving specification, formulation, optimization and technology mapping. Implementation of functions using NAND and NOR gates is also discussed.
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...sameer shah
"Join us for STATATHON, a dynamic 2-day event dedicated to exploring statistical knowledge and its real-world applications. From theory to practice, participants engage in intensive learning sessions, workshops, and challenges, fostering a deeper understanding of statistical methodologies and their significance in various fields."
The document discusses various types of transformations in computer graphics, including translation, scaling, and rotation in both 2D and 3D. Translation moves an object by adding offsets to its coordinates. Scaling changes the size of an object by multiplying its coordinates by scaling factors. Rotation changes the orientation of an object by applying trigonometric functions to its coordinates based on a rotation angle and axis. Transformation matrices are used to represent and apply these operations to objects uniformly.
Chapter 8 discusses converter transfer functions and Bode plots. It reviews common transfer function elements like poles, zeros and their impact on Bode plots. Specific topics covered include the single pole response, single zero response, right half-plane zeros, and combinations of elements. It also discusses how to analyze converter transfer functions, construct them graphically, and measure real converter transfer functions and impedances. The chapter aims to provide engineers with the tools needed to model, analyze and design power converters.
Image Processing using Matlab . Useful for beginners to learn Image ProcessingAshok Kumar
Matlab can be used for image processing tasks such as loading, displaying, and manipulating images. Images are represented as matrices where each element corresponds to the pixel intensity values. Common operations include convolutions using various kernel filters to perform tasks like smoothing, sharpening, and edge detection. Functions such as imread, image, and imshow can load and display images. Built-in functions such as fspecial generate common kernel filters. Convolution functions convolve images with kernels to apply filtering effects.
A Dimension Abstraction Approach to Vectorization in MatlabaiQUANT
The document presents an approach to vectorizing Matlab source code while preserving correctness and improving efficiency. It describes representing variable dimensionalities, rules for determining when vectorization is valid, and techniques like transpose transformations and additive reduction to enable more vectorizations. Evaluation on image processing and linear algebra examples showed speedups of 1.56x to 4.6x from vectorizing loop-based code.
The document presents an approach to vectorizing Matlab source code while preserving correctness and improving efficiency. It describes representing variable dimensionalities, rules for determining when vectorization is valid, and techniques like transpose transformations and additive reduction to enable more vectorizations. Evaluation on image processing and linear algebra examples showed speedups of 1.56x to 4.6x from vectorizing loop-based code.
IRJET-Design and Implementation of LNS based Approximate Multiplier using Mit...IRJET Journal
This document describes a design for an approximate logarithmic multiplier that uses Mitchell's algorithm and operand decomposition to improve accuracy while maintaining low power consumption. It is intended for applications like convolutional neural networks and digital signal/image processing that require multiplication but not highly accurate results. The design takes two binary inputs, divides them into four parts using logic operations, and applies Mitchell's algorithm to each part separately before adding the results. This operand decomposition approach increases the accuracy over using Mitchell's alone. The design is implemented using Xilinx tools and simulation waveforms and block diagrams are provided to illustrate its operation. Experimental results show this approach reduces error compared to using just Mitchell's algorithm for multiplication.
Novel Parallel - Perfix Structure Binary to Residue Number System Conversion ...CSCJournals
In the present world there is always a demand for faster algorithms and techniques that could boost up the speed of the computations. With the help of VLSI fabrication techniques and using residue number system (RNS) arithmetic we can achieve the faster speeds. In this paper we propose a novel parallel prefix binary to residue number system conversion method. The method that we present in this paper utilizes parallel-prefix technique with multiplexers and modulo adders as the main building blocks which makes it practical and suitable for VLSI implementation.
The document proposes an algorithm called Robust Tensor Decomposition (RTD) to recover a low-rank tensor from observations that are corrupted by block sparse errors. RTD iteratively estimates the low-rank and sparse components using hard thresholding and tensor projections. It compares RTD to existing approaches on video sequences, finding RTD yields more accurate recovery with fewer assumptions.
Small Signal Modelling and Controller Design of Boost Converter using MATLABPremier Publishers
This document presents a systematic approach for designing a PID controller for a boost converter using MATLAB. It begins with deriving the small-signal model of the boost converter theoretically and comparing it to the model generated in MATLAB. It then uses the linearized model from MATLAB to design the PID controller. Finally, it verifies the controller's performance for variations in input voltage and load current.
This presentation describes briefly about the image enhancement in spatial domain, basic gray level transformation, histogram processing, enhancement using arithmetic/ logical operation, basics of spatial filtering and local enhancements.
This document provides an overview of image processing using MATLAB. It discusses how images are represented as matrices in MATLAB and demonstrates various image processing functions and techniques.
Key points covered include:
- Loading and displaying an image using imread and image commands
- Converting between intensity, indexed, and RGB image representations
- Exploring image histograms and equalization
- Performing operations like resizing, rotation and filtering using functions like imresize, imrotate, and filters from fspecial
- Implementing convolution using custom kernels and built-in filters
- Understanding effects of different kernels on images
Digital image enhancement involves modifying images to improve visual quality. It can be done in the spatial or frequency domain. Spatial domain enhancement works directly with pixel values. Key point processing techniques in the spatial domain include: digital negative, contrast stretching, thresholding, gray level slicing, bit plane slicing, and dynamic range compression. These techniques apply mathematical transforms to pixel values to darken or lighten regions of the image for better visualization.
This document provides an overview of constructing Bode plots to analyze the frequency response of control systems. It discusses:
1) How Bode plots show the magnitude and phase shift of a transfer function as functions of frequency on logarithmic axes.
2) The steps to compute magnitude and phase values at different frequencies from a transfer function and plot the results.
3) How to use MATLAB functions like tf, zpk and bode to define transfer functions as linear time-invariant systems and generate Bode plots, including examples comparing different transfer functions on the same plot.
The document discusses a workshop on image processing using MATLAB. It provides an overview of MATLAB and its image processing toolbox. It describes how to read, display, and convert between different image formats in MATLAB. It also demonstrates various image processing operations that can be performed, such as arithmetic operations, conversion between color and grayscale, image rotation, blurring and deblurring, and filling regions of interest. The document aims to introduce the basics of working with images in the MATLAB environment.
The document discusses a workshop on image processing using MATLAB. It provides an overview of MATLAB and its image processing toolbox. It describes how to read, display, and convert between different image formats in MATLAB. It also demonstrates various image processing operations like arithmetic operations, conversion between color and grayscale, image rotation, blurring and deblurring, and filling regions of interest. The document aims to introduce the basics of image processing using MATLAB.
This document provides an introduction to Matlab and Simulink for control applications. It discusses what Matlab and Simulink are, their main components and functions. It then gives examples of using Matlab to model control systems using transfer functions, Laplace transforms and PID controllers. Specific examples include modeling and controlling a mass-spring-damper system and designing a cruise control system for a car.
This document summarizes key concepts about combinational logic circuits. It defines combinational logic as circuits whose outputs depend only on the current inputs, in contrast to sequential logic which also depends on prior inputs. Common combinational circuits are described like half and full adders used for arithmetic, as well as decoders. The design process for combinational circuits is outlined involving specification, formulation, optimization and technology mapping. Implementation of functions using NAND and NOR gates is also discussed.
STATATHON: Unleashing the Power of Statistics in a 48-Hour Knowledge Extravag...sameer shah
"Join us for STATATHON, a dynamic 2-day event dedicated to exploring statistical knowledge and its real-world applications. From theory to practice, participants engage in intensive learning sessions, workshops, and challenges, fostering a deeper understanding of statistical methodologies and their significance in various fields."
Beyond the Basics of A/B Tests: Highly Innovative Experimentation Tactics You...Aggregage
This webinar will explore cutting-edge, less familiar but powerful experimentation methodologies which address well-known limitations of standard A/B Testing. Designed for data and product leaders, this session aims to inspire the embrace of innovative approaches and provide insights into the frontiers of experimentation!
Predictably Improve Your B2B Tech Company's Performance by Leveraging DataKiwi Creative
Harness the power of AI-backed reports, benchmarking and data analysis to predict trends and detect anomalies in your marketing efforts.
Peter Caputa, CEO at Databox, reveals how you can discover the strategies and tools to increase your growth rate (and margins!).
From metrics to track to data habits to pick up, enhance your reporting for powerful insights to improve your B2B tech company's marketing.
- - -
This is the webinar recording from the June 2024 HubSpot User Group (HUG) for B2B Technology USA.
Watch the video recording at https://youtu.be/5vjwGfPN9lw
Sign up for future HUG events at https://events.hubspot.com/b2b-technology-usa/
The Ipsos - AI - Monitor 2024 Report.pdfSocial Samosa
According to Ipsos AI Monitor's 2024 report, 65% Indians said that products and services using AI have profoundly changed their daily life in the past 3-5 years.
ViewShift: Hassle-free Dynamic Policy Enforcement for Every Data LakeWalaa Eldin Moustafa
Dynamic policy enforcement is becoming an increasingly important topic in today’s world where data privacy and compliance is a top priority for companies, individuals, and regulators alike. In these slides, we discuss how LinkedIn implements a powerful dynamic policy enforcement engine, called ViewShift, and integrates it within its data lake. We show the query engine architecture and how catalog implementations can automatically route table resolutions to compliance-enforcing SQL views. Such views have a set of very interesting properties: (1) They are auto-generated from declarative data annotations. (2) They respect user-level consent and preferences (3) They are context-aware, encoding a different set of transformations for different use cases (4) They are portable; while the SQL logic is only implemented in one SQL dialect, it is accessible in all engines.
#SQL #Views #Privacy #Compliance #DataLake
Orchestrating the Future: Navigating Today's Data Workflow Challenges with Ai...Kaxil Naik
Navigating today's data landscape isn't just about managing workflows; it's about strategically propelling your business forward. Apache Airflow has stood out as the benchmark in this arena, driving data orchestration forward since its early days. As we dive into the complexities of our current data-rich environment, where the sheer volume of information and its timely, accurate processing are crucial for AI and ML applications, the role of Airflow has never been more critical.
In my journey as the Senior Engineering Director and a pivotal member of Apache Airflow's Project Management Committee (PMC), I've witnessed Airflow transform data handling, making agility and insight the norm in an ever-evolving digital space. At Astronomer, our collaboration with leading AI & ML teams worldwide has not only tested but also proven Airflow's mettle in delivering data reliably and efficiently—data that now powers not just insights but core business functions.
This session is a deep dive into the essence of Airflow's success. We'll trace its evolution from a budding project to the backbone of data orchestration it is today, constantly adapting to meet the next wave of data challenges, including those brought on by Generative AI. It's this forward-thinking adaptability that keeps Airflow at the forefront of innovation, ready for whatever comes next.
The ever-growing demands of AI and ML applications have ushered in an era where sophisticated data management isn't a luxury—it's a necessity. Airflow's innate flexibility and scalability are what makes it indispensable in managing the intricate workflows of today, especially those involving Large Language Models (LLMs).
This talk isn't just a rundown of Airflow's features; it's about harnessing these capabilities to turn your data workflows into a strategic asset. Together, we'll explore how Airflow remains at the cutting edge of data orchestration, ensuring your organization is not just keeping pace but setting the pace in a data-driven future.
Session in https://budapestdata.hu/2024/04/kaxil-naik-astronomer-io/ | https://dataml24.sessionize.com/session/667627
End-to-end pipeline agility - Berlin Buzzwords 2024Lars Albertsson
We describe how we achieve high change agility in data engineering by eliminating the fear of breaking downstream data pipelines through end-to-end pipeline testing, and by using schema metaprogramming to safely eliminate boilerplate involved in changes that affect whole pipelines.
A quick poll on agility in changing pipelines from end to end indicated a huge span in capabilities. For the question "How long time does it take for all downstream pipelines to be adapted to an upstream change," the median response was 6 months, but some respondents could do it in less than a day. When quantitative data engineering differences between the best and worst are measured, the span is often 100x-1000x, sometimes even more.
A long time ago, we suffered at Spotify from fear of changing pipelines due to not knowing what the impact might be downstream. We made plans for a technical solution to test pipelines end-to-end to mitigate that fear, but the effort failed for cultural reasons. We eventually solved this challenge, but in a different context. In this presentation we will describe how we test full pipelines effectively by manipulating workflow orchestration, which enables us to make changes in pipelines without fear of breaking downstream.
Making schema changes that affect many jobs also involves a lot of toil and boilerplate. Using schema-on-read mitigates some of it, but has drawbacks since it makes it more difficult to detect errors early. We will describe how we have rejected this tradeoff by applying schema metaprogramming, eliminating boilerplate but keeping the protection of static typing, thereby further improving agility to quickly modify data pipelines without fear.
End-to-end pipeline agility - Berlin Buzzwords 2024
2 - PointProcessing.pdf
1. Point Processing
and Histogram
LE Thanh Sach
Point processing
Linear transformation
Logarithmic transformation
Exponential transformation
Power-law transformation
Histogram
Definition
Computation
Equalization
Specification
2.1
Chapter 2
Point Processing and Histogram
Image Processing and Computer Vision
LE Thanh Sach
Faculty of Computer Science and Engineering
Ho Chi Minh University of Technology, VNU-HCM
2. Point Processing
and Histogram
LE Thanh Sach
Point processing
Linear transformation
Logarithmic transformation
Exponential transformation
Power-law transformation
Histogram
Definition
Computation
Equalization
Specification
2.2
Overview
1 Point processing
Linear transformation
Logarithmic transformation
Exponential transformation
Power-law transformation
2 Histogram
Definition
Computation
Equalization
Specification
3. Point Processing
and Histogram
LE Thanh Sach
Point processing
Linear transformation
Logarithmic transformation
Exponential transformation
Power-law transformation
Histogram
Definition
Computation
Equalization
Specification
2.3
Point processing
Concepts
• Image: f(x, y)
1 x: discrete variable, in [0, 1, .., N]
2 y: discrete variable, in [0, 1, .., M]
3 f(x, y): discrete value, in [0, 1, .., 255]
• point ≡ pixel
• f(x, y): has M × N points (or pixels)
4. Point Processing
and Histogram
LE Thanh Sach
Point processing
Linear transformation
Logarithmic transformation
Exponential transformation
Power-law transformation
Histogram
Definition
Computation
Equalization
Specification
2.4
Point processing
Definition and Notation
• Point processing: Process each point by a function
that depends ONLY the pixel’s value and that does not
depend on the point’s neighbors.
• Pixel processing function:
• referred to as transfer function
• denoted as T[.]
• f(x, y): input image
• g(x, y) = T[f(x, y)]: output image
5. Point Processing
and Histogram
LE Thanh Sach
Point processing
Linear transformation
Logarithmic transformation
Exponential transformation
Power-law transformation
Histogram
Definition
Computation
Equalization
Specification
2.5
Point processing
Linear transformation: Math
• g(x, y) = a × f(x, y) + b
• Where, a and b: pre-defined parameters
Linear transformation: Applications
• General: Change the image’s intensity, cause the input
bighter or darker
• Users have to choose appropriate a and b (manually)
• Specific:
• Create negative images
• Convert to back-white image
6. Point Processing
and Histogram
LE Thanh Sach
Point processing
Linear transformation
Logarithmic transformation
Exponential transformation
Power-law transformation
Histogram
Definition
Computation
Equalization
Specification
2.6
Point processing
Linear transformation: with Matlab
%read input image
im = imread ( ’ cameraman . t i f ’ ) ;
%s e t parameters
a1 = 1 . 5 ; b1 = 20;
a2 = 1 . 5 ; b2 = 50;
a3 = 0 . 5 ; b3 = 0;
%transform the input
im1 = a1 ∗ im + b1 ; %c l i p p e d to [0 ,255] auto
im2 = a2 ∗ im + b2 ; %c l i p p e d to [0 ,255] auto
im3 = a3 ∗ im + b3 ; %c l i p p e d to [0 ,255] auto
7. Point Processing
and Histogram
LE Thanh Sach
Point processing
Linear transformation
Logarithmic transformation
Exponential transformation
Power-law transformation
Histogram
Definition
Computation
Equalization
Specification
2.7
Point processing
Linear transformation: with Matlab(continued)
%show output image
f i g u r e ;
subplot ( 2 , 2 , 1 ) ; imshow ( im ) ; t i t l e ( ’ i n p ut image ’ )
subplot ( 2 , 2 , 2 ) ; imshow ( im1 ) ;
t1 = s p r i n t f ( ’ output image [ a =%5.2f , b = %5.2 f ] ’ ,
a1 , b1 ) ;
t i t l e ( t1 )
subplot ( 2 , 2 , 3 ) ; imshow ( im2 ) ;
t2 = s p r i n t f ( ’ output image [ a =%5.2f , b = %5.2 f ] ’ ,
a2 , b2 ) ;
t i t l e ( t2 )
subplot ( 2 , 2 , 4 ) ; imshow ( im3 ) ;
t3 = s p r i n t f ( ’ output image [ a =%5.2f , b = %5.2 f ] ’ ,
a3 , b3 ) ;
t i t l e ( t3 )
s u p t i t l e ( ’ L i n e a r Transformation : output = a∗ i n pu t + b ’ ) ;
8. Point Processing
and Histogram
LE Thanh Sach
Point processing
Linear transformation
Logarithmic transformation
Exponential transformation
Power-law transformation
Histogram
Definition
Computation
Equalization
Specification
2.8
Point processing
Linear transformation: Illustration
Input image Output Image
[a = 1.5, b = 20]
Output Image Output Image
[a = 1.5, b = 50] [a = 0.5, b = 0]
Table: Linear Transformation: output = a * input + b
9. Point Processing
and Histogram
LE Thanh Sach
Point processing
Linear transformation
Logarithmic transformation
Exponential transformation
Power-law transformation
Histogram
Definition
Computation
Equalization
Specification
2.9
Point processing
Linear transformation: Illustration
Table: Creating negative image: output = a * input + b; where,
a = −1, b = 255
10. Point Processing
and Histogram
LE Thanh Sach
Point processing
Linear transformation
Logarithmic transformation
Exponential transformation
Power-law transformation
Histogram
Definition
Computation
Equalization
Specification
2.10
Point processing
Linear transformation: Illustration
Table: Creating negative image: output = a * input + b; where,
a = −1, b = 255
11. Point Processing
and Histogram
LE Thanh Sach
Point processing
Linear transformation
Logarithmic transformation
Exponential transformation
Power-law transformation
Histogram
Definition
Computation
Equalization
Specification
2.11
Point processing
Logarithmic transformation: Math
• g(x, y) = a × log[1 + f(x, y)] + b
• Where, a: a pre-defined parameter
Logarithm function
log2(x)
x
f(x) = log2(x)
• a large value (x) → mapped to a smaller value (log2 x)
• a large distance between two intensities in the input
image → mapped to a smaller distance in the output
12. Point Processing
and Histogram
LE Thanh Sach
Point processing
Linear transformation
Logarithmic transformation
Exponential transformation
Power-law transformation
Histogram
Definition
Computation
Equalization
Specification
2.12
Point processing
Logarithmic transformation: Applications
• Input image:
• have some regions with too dark intensities (near 0)
• have some regions with too bright intensities (near 255)
• Requirement:
• Make dark regions brighter while keeping the intensity
in bright regions lower than 255.
• Or, make bright regions darker while keeping the
intensity in dark regions lower than 0.
• Solution: use logarithmic transformation
• Often case:
• Should use logarithmic transformation to enhance the
result of FFT before displaying
13. Point Processing
and Histogram
LE Thanh Sach
Point processing
Linear transformation
Logarithmic transformation
Exponential transformation
Power-law transformation
Histogram
Definition
Computation
Equalization
Specification
2.13
Point processing
Logarithmic transformation: Illustration
Table: Samples
14. Point Processing
and Histogram
LE Thanh Sach
Point processing
Linear transformation
Logarithmic transformation
Exponential transformation
Power-law transformation
Histogram
Definition
Computation
Equalization
Specification
2.14
Point processing
(Inverse-log) Exponential transformation: Math
• g(x, y) = a × ef(x,y) + b
• Where, a and b: pre-defined parameters
Exponential function
−4 −2 2 4
2
4
6
8
10
y = ex
• a small value (x) → mapped to a larger value ex
• a small distance between two intensities in the input →
mapped to a larger distance in the output image
15. Point Processing
and Histogram
LE Thanh Sach
Point processing
Linear transformation
Logarithmic transformation
Exponential transformation
Power-law transformation
Histogram
Definition
Computation
Equalization
Specification
2.15
Point processing
Exponential transformation: Applications
• map small differences between two intensities in the
input to a larger differences in the output image
(contrast input images )
Exponential transformation: with Matlab
%read input image
path = ’ coffebean . png ’
im = imread ( path ) ; %range [0−>255]
%convert double image : [0−>1]
im = im2double ( im ) ;
%s e t parameters
a = 1; b = −1;
%transform input image
im1 = a ∗ exp ( im ) + b ;
%normalize the output , to [0−>1]
im1 = im1 ./max( im1 ( : ) ) ;
16. Point Processing
and Histogram
LE Thanh Sach
Point processing
Linear transformation
Logarithmic transformation
Exponential transformation
Power-law transformation
Histogram
Definition
Computation
Equalization
Specification
2.16
Point processing
Exponential transformation: with Matlab (continued)
%show image
figure ; imshow ( im ) ; t i t l e ( ’ input image ’ ) ;
figure ; imshow ( im1 ) ; t i t l e ( ’ output image ’ ) ;
Input image output image
Table: g(x, y) = a × ef(x,y)
+ b; where, a = 1, b = −1
17. Point Processing
and Histogram
LE Thanh Sach
Point processing
Linear transformation
Logarithmic transformation
Exponential transformation
Power-law transformation
Histogram
Definition
Computation
Equalization
Specification
2.17
Point processing
Power-law transformation: Math
• g(x, y) = a × f(x, y)γ + b
• Where, a and b: pre-defined parameters
Power function
18. Point Processing
and Histogram
LE Thanh Sach
Point processing
Linear transformation
Logarithmic transformation
Exponential transformation
Power-law transformation
Histogram
Definition
Computation
Equalization
Specification
2.18
Point processing
Power-law function: characteristics
• Depend on γ, power-law transformation can be either
1 linear transformation: for γ = 1
2 log transformation: for γ < 1
3 inverse-log transformation: for γ > 1
19. Point Processing
and Histogram
LE Thanh Sach
Point processing
Linear transformation
Logarithmic transformation
Exponential transformation
Power-law transformation
Histogram
Definition
Computation
Equalization
Specification
2.19
Point processing
Power-law transformation: Illustration
20. Point Processing
and Histogram
LE Thanh Sach
Point processing
Linear transformation
Logarithmic transformation
Exponential transformation
Power-law transformation
Histogram
Definition
Computation
Equalization
Specification
2.20
Histogram
Definition
Histogram computed for image I is a statistical quantity,
contains the following information
• number of pixels in I for each intensity (for each value
in [0,255]). This kind of histogram is referred to as
unnormalized histogram.
• Sum of all values in this kind of histogram = total
number of pixels in I
• probability distribution of intensities in image I. This
kind of histogram is referred to as normalized
histogram.
• Sum of all values in this kind of histogram = 1
21. Point Processing
and Histogram
LE Thanh Sach
Point processing
Linear transformation
Logarithmic transformation
Exponential transformation
Power-law transformation
Histogram
Definition
Computation
Equalization
Specification
2.21
Histogram
(a) An image (b) Its Histogram
• From histogram (b): almost of pixels in the image have
the intensity near 0. Therefore, the image is too dark,
confirmed by image (a).
22. Point Processing
and Histogram
LE Thanh Sach
Point processing
Linear transformation
Logarithmic transformation
Exponential transformation
Power-law transformation
Histogram
Definition
Computation
Equalization
Specification
2.22
Histogram
(a) An image (b) Its Histogram
• From histogram (b): almost of pixels in the image have
the intensity near 255. Therefore, the image is too
bright, confirmed by image (a).
23. Point Processing
and Histogram
LE Thanh Sach
Point processing
Linear transformation
Logarithmic transformation
Exponential transformation
Power-law transformation
Histogram
Definition
Computation
Equalization
Specification
2.23
Histogram
(a) An image (b) Its Histogram
• From histogram (b): almost of pixels in the image have
the intensity compacted in short range [100, 130].
Therefore, the image has very low contrast, confirmed
by image (a).
24. Point Processing
and Histogram
LE Thanh Sach
Point processing
Linear transformation
Logarithmic transformation
Exponential transformation
Power-law transformation
Histogram
Definition
Computation
Equalization
Specification
2.24
Histogram
(a) An image (b) Its Histogram
• From histogram (b): the pixels in the image are
distributed in a full range [0, 255]. Therefore, the image
is balanced (not to dark, not to bright) and has a high
contrast (preferred).
25. Point Processing
and Histogram
LE Thanh Sach
Point processing
Linear transformation
Logarithmic transformation
Exponential transformation
Power-law transformation
Histogram
Definition
Computation
Equalization
Specification
2.25
Histogram
Histogram: with C/C++/Java
Excercise:
• Write a function for computing the histogram
(unnormalized and normalized) of the input image
passed to the function.
26. Point Processing
and Histogram
LE Thanh Sach
Point processing
Linear transformation
Logarithmic transformation
Exponential transformation
Power-law transformation
Histogram
Definition
Computation
Equalization
Specification
2.26
Histogram
Histogram: with Matlab
• use functions: imhist
%read input images
path1 = ’ Fig3 .15( a ) 1 . jpg ’ ;
path2 = ’ Fig3 .15( a ) 2 . jpg ’ ;
path3 = ’ Fig3 .15( a ) 3 . jpg ’ ;
path4 = ’ Fig3 .15( a ) 4 . jpg ’ ;
im1 = imread ( path1 ) ;
im2 = imread ( path2 ) ;
im3 = imread ( path3 ) ;
im4 = imread ( path4 ) ;
%Compute histogram
[ count1 , x1 ] = i m h i s t ( im1 ) ;
[ count2 , x2 ] = i m h i s t ( im2 ) ;
[ count3 , x3 ] = i m h i s t ( im3 ) ;
[ count4 , x4 ] = i m h i s t ( im4 ) ;
27. Point Processing
and Histogram
LE Thanh Sach
Point processing
Linear transformation
Logarithmic transformation
Exponential transformation
Power-law transformation
Histogram
Definition
Computation
Equalization
Specification
2.27
Histogram
Histogram: with Matlab (continued)
%show input images
figure ; imshow ( im1 ) ; t i t l e ( ’ input image 1 ’ ) ;
figure ; imshow ( im2 ) ; t i t l e ( ’ input image 2 ’ ) ;
figure ; imshow ( im3 ) ; t i t l e ( ’ input image 3 ’ ) ;
figure ; imshow ( im4 ) ; t i t l e ( ’ input image 4 ’ ) ;
%draw histograms
figure ; stem( x1 , count1 , ’ . ’ ) ;
xlim ( [ 0 , 2 5 5 ] ) ; t i t l e ( ’ histogram 1 ’ ) ;
figure ; stem( x2 , count2 , ’ . ’ ) ;
xlim ( [ 0 , 2 5 5 ] ) ; t i t l e ( ’ histogram 2 ’ ) ;
figure ; stem( x3 , count3 , ’ . ’ ) ;
xlim ( [ 0 , 2 5 5 ] ) ; t i t l e ( ’ histogram 3 ’ ) ;
figure ; stem( x4 , count4 , ’ . ’ ) ;
xlim ( [ 0 , 2 5 5 ] ) ; t i t l e ( ’ histogram 4 ’ ) ;
28. Point Processing
and Histogram
LE Thanh Sach
Point processing
Linear transformation
Logarithmic transformation
Exponential transformation
Power-law transformation
Histogram
Definition
Computation
Equalization
Specification
2.28
Histogram: Equalization
Histogram Equalization: What?
Histogram Equalization is an operation to create an image
(from a given image) whose pixels’s value distributed
uniformly in [0, 255]
• Purpose of histogram equalization: to create image not
too dark, not too bright, and high contrast
(a) An image (b) Its normalized histogram
(not preferred) (not preferred)
29. Point Processing
and Histogram
LE Thanh Sach
Point processing
Linear transformation
Logarithmic transformation
Exponential transformation
Power-law transformation
Histogram
Definition
Computation
Equalization
Specification
2.29
Histogram: Equalization
Histogram Equalization: What?
Histogram Equalization is an operation to create an image
(from a given image) whose pixels’s value distributed
uniformly in [0, 255]
• Purpose of histogram equalization: to create image not
too dark, not too bright, and high contrast
(a) Image, preferred (b) Histogram, preferred
(after equalization) (after equalization)
30. Point Processing
and Histogram
LE Thanh Sach
Point processing
Linear transformation
Logarithmic transformation
Exponential transformation
Power-law transformation
Histogram
Definition
Computation
Equalization
Specification
2.30
Histogram: Equalization
Histogram Equalization: How?
Input:
1 Input image I
• So, compute the normalized-histogram of image I, to
obtain: px(x):
2 Expected normalized-histogram of the output image:
pz(z)
• with histogram equalization: pz(z) is a uniform
distribution over [0, 255]
• with histogram specification: pz(z) is specified by users
(interactively)
31. Point Processing
and Histogram
LE Thanh Sach
Point processing
Linear transformation
Logarithmic transformation
Exponential transformation
Power-law transformation
Histogram
Definition
Computation
Equalization
Specification
2.31
Histogram: Equalization
Histogram Equalization: How?
We can determine z (e.g., pixel value of the output image),
if we know:
1 x: pixel value of the input image
2 px(x): pdf of x
3 pz(z): pdf of z
Assume that z = f(x) has been determined, the output
image g(u, v) can be produced from the input f(u, v) as
follows:
• foreach point (u, v) in f(u, v):
• x = f(u, v) (get the value of the input pixel)
• z = f(x) (map to the output value)
• g(u, v) = z (assign to the output pixel)
32. Point Processing
and Histogram
LE Thanh Sach
Point processing
Linear transformation
Logarithmic transformation
Exponential transformation
Power-law transformation
Histogram
Definition
Computation
Equalization
Specification
2.32
Histogram: Equalization
Determination of z = f(x)
Let cx(x) and cz(z) be cumulative density functions of x
and z.
cx(x) =
x
X
i=0
px(i)
cz(z) =
z
X
i=0
pz(i)
33. Point Processing
and Histogram
LE Thanh Sach
Point processing
Linear transformation
Logarithmic transformation
Exponential transformation
Power-law transformation
Histogram
Definition
Computation
Equalization
Specification
2.33
Histogram: Equalization
z = f(x) be a non-decreasing function that map one-to-one
between x and z.
Determination of z = f(x)
• Assume that we have z1 = f(x1)
• Then, cz(z1) = cx(x1). This is because of one-one
mapping
• A value x < x1 will be mapped into z < z1
• Define w ≡ cx(x1), i.e., w ≡ cz(z1) ≡ cx(x1)
Therefore,
• z1 = c−1
z (w) ≡ c−1
z (cx(x1))
Mapping function f(x)
z = f(x) = c−1
z (cx(x))
34. Point Processing
and Histogram
LE Thanh Sach
Point processing
Linear transformation
Logarithmic transformation
Exponential transformation
Power-law transformation
Histogram
Definition
Computation
Equalization
Specification
2.34
Histogram: Equalization
Implemtation of z = f(x) = c−1
z (cx(x))
1 If we have closed-form of z = c−1
z (w) then can use this
function to determine z.
2 If we do not have closed-form of c−1
z (z)
• Create a lookup table at the beginning for mapping
z → c−1
z (p), for discrete values p. We can do this
because we know cz(z) in advance. This task is equal
to rasterize p = cz(z) and store pairs into lookup table
• Determine z according to the lookup table.