QQ Plot introduction

1. QQ Plot

2. Introduction
Engineers and scientists work with data. Without data, they are not able to draw
any conclusion. Now is the era of creation of data everyday from every aspects of
our lives. Some data are random and some are biased. Some may suffer from bias
because of the data collection process.
One very important aspect of data is the distribution profile. The collected data
may have normal distribution or may be far from normal. It may also be skewed one
either side or may follow multimodal pattern. It may be discrete or may be
continuous. For continuous data, normal distribution brings a whole lot of
advantages compared to its counterparts.
Various inferential statistical process assume that the distribution is normal. A
bell-shaped curve is easy to describe with mean and standard deviation.

3. Introduction

4. Why Q-Q plot?
Since normal distribution is of so much importance, we need to check if the
collected data is normal or not.
Here, we will demonstrate the Q-Q plot to check the normality of skewness of
data. Q stands for quantile and therefore, Q-Q plot represents quantile-quantile
plot.
To determine the normality, there are also several statistical tests out there such
as the Kolmogorov–Smirnov test and the Shapiro– Wilk test.
However, it is difficult to do so by looking at the table.

5. Brief explanation?
We now know Q-Q plot is quantile-quantile plot but what is quantile at the
first place?
When the whole data is sorted, 50th quantile means 50% of the data falls
below that point and 50% of the data falls above that point. That is the
median point.
When we say 1st quantile, only 1% of the data falls below that point and
99% is above that. 25th and 75th quantile points are also known as quartiles.
There are three quartiles is the dataset.
Q1 = first quartile = 25th quantile
Q2 = second quartile = 50th quantile = median
Q3 = third quartile = 75th quantile

6. Brief explanation?
Quantile are sometimes called percentile.
A typical Q-Q plot is sown below. Let’s explain this plot which seems pretty much a
straight line.
Axes
The x-axis of a Q-Q plot represents the quantiles of
standard normal distribution.
Let’s say we have a normal data and we want to
standardize it. Standardizing means subtracting mean
from each data point and dividing it by standard
deviation.
The resultant is also known as z-score. Let’s sort those
z-scores and plot again. The plot below shows that the
x-axis is now centered at 0 and extended up to 3

7. Network Graph: Motivation
Wouldn’t it be nice if you could visualize their connections using an
interactive network graph like below?

8. Network Graph: Pyvis
 What is Pyvis?
Pyvis is a Python library that allows you to create interactive network graphs
in a few lines of code.
To install pyvis, type:
pip install pyvis

9. What is a Tree map?
 What is a Tree map?
A tree map is a special type of chart for visualization using a set of nested
rectangles of categorical data that is preferably hierarchical.
In Hierarchical data, the categories or items share parent-child type
relationships in an overall tree structure.
The simplest example of this type of data structure can be seen in a
company where all individuals and their designations within teams could
be grouped under one entity i.e., the company itself

10. What is a Tree map?
 When to use a Tree map?
These are some key points to consider before using tree maps for
visualization.
Tree maps work well when there is a clear ‘Part-to-whole’ relationship
amongst multiple categories present in the data.
Hierarchical Data is needed. This indicates that the data could be arranged
in branches and sub-branches.
The focus is not on precise comparisons between categories but rather on
spotting the key factors/trends or patterns.

11. Benefits of using a Tree map?
Benefits of using a Tree map:
Space constraint: There is a large amount of hierarchical data that needs
to be visualized in a smaller space.
Easier to read: When compared to a circular multi-level pie chart, the tree
map is easier to read due to its linear visual appearance.
Quickly spot patterns: Since each group is represented by a rectangle
and the area of this rectangle is always proportional to its value, trends
and patterns (similarities and anomalies) are quickly visible in tree maps.

12. Real-world use cases for Tree map Charts?
1. Displaying region-wise customer complaints about a product
Suppose there are 10 different types of complaints (assume these are
denoted as C1 to C10) about a product and the company wants to
visualize which complaints are relevant to a region then in such a case a
tree map could be used. Here, it can be clearly seen how different regions
have specific types of user complaints.

13. Real-world use cases for Tree map Charts?
2. Showcasing category-wise product availability like mobile phones
Let us assume that there are four categories of mobile phones with their
market share percentages i.e., Low-end (up to 10,000 INR – 15%), Mid-
Range (10,000-25000 INR- 55%), Premium (above 25,000 to 50,000 INR-
25%), and Top-end (above 50,000 INR-10%).
From this tree map, we can gauge that there is a bigger demand and
market for Mid-Range phones while there are limited phones available in the
Top-End category.

14. Real-world use cases for Tree map Charts?
3. Explore customer segmentation for a product
Usually, companies for apparel or personal products divide their customers
based on their age. This way they can categorize their products and the
product variants separately for each age group.
In the case of this tree map, the company could decide whether to launch
more products for particular customer segments based on the distribution.

15. Challenges associated with a Tree map
Tree maps also come with a set of limitations as outlined below-
– Tree maps built with large data points on a single level could be hard to
read as well as print for reporting purposes.
– Sometimes, additional sorting might be required to understand the data
better. However, all the rectangles are automatically ordered within the parent
node by area.
– With too many categories and colors to represent these, the tree map
becomes overwhelming for the reader.
– Tree maps become ineffective for datasets with balanced trees i.e., when
items are of a similar value. In these cases, the main purpose of a tree map
of highlighting the largest item in a given category becomes impossible.

16. Dendrograms in Python
A dendrogram is a diagram that depicts a tree.
The create_dendrogram figure factory conducts hierarchical clustering on data
and depicts the resultant tree.
Distances between clusters are represented by the values on the tree depth axis.
Dendrogram plots are often used in computational biology to depict gene
or sample grouping, occasionally in the margins of heatmaps.
Hierarchical clustering produces dendrograms as an output. Many people
claim that dendrograms of this type may be used to determine the number of
clusters.

17. Dendrograms in Python
Wholesale Customer Segmentation Problem using Hierarchical Clustering
We will be working on a wholesale customer segmentation problem. You
can download the dataset using this link.
The data is hosted on the UCI Machine Learning repository. The aim of this
problem is to segment the clients of a wholesale distributor based on their
annual spending on diverse product categories, like milk, grocery, region, etc.

18. Scree Plot in Python
How to Create a Scree Plot in Python
Principal components analysis (PCA) is an unsupervised machine learning
technique that finds principal components (linear combinations of the
predictor variables) that explain a large portion of the variation in a dataset.
When we perform PCA, we’re interested in understanding what percentage
of the total variation in the dataset can be explained by each principal
component.
One of the easiest ways to visualize the percentage of variation explained
by each principal component is to create a scree plot.