The document outlines a comprehensive workshop on data science, covering key topics such as data cleaning, exploratory data analysis, feature engineering, model selection, evaluation, and deployment. It provides practical prerequisites, tasks, and insights on analyzing car price datasets, including visualization techniques and feature transformations. Model evaluation metrics for regression, clustering, classification, and anomaly detection are also discussed to guide participants in assessing their data science models.

1. Data Science in a Day -
Workshop
Karthikeyan VK
Cloud Native Architect & Microsoft MVP

2. Basics
Data Cleaning
Exploratory Data Analysis(EDA)
Feature Selection and Engineering
Data Pre-Processing
Model Selection/Training/Evaluation
Deployment

3. Basics of Data

4. Types of Data
• Quantitative
• Qualitative

5. Attributes
Attribute is a property or characteristic of an
object that may vary, either from one object
to another or form one time another
Eg:- Temperature of an object varies over
time

6. Properties of Attributes
Distinctness
= and ≠
Order <,
>,≤,≥
Addition +
and –
Multiplication
* and /

7. Types of Attributes
Nominal Ordinal Interval Ratio

8. Types of
Attributes

9. WORKSHOP -
PREREQUISITES

10. Prerequisites – Readme.md
https://bit.ly/predtemplate

11. Prerequisites
Open Visual Studio Code and make sure the code is cloned.

12. Load Car
price Dataset

13. Check
Car price
Dataset

14. Check Car price Dataset

15. Check Car price Dataset

16. Check Car
price Dataset

17. Data Cleaning

18. Data Quality
Accuracy
Completeness
Consistencies
Interpretability

19. Clean up Car Name

20. Spelling
Mistake -
Car
Name

21. Exploratory Data
Analysis(EDA)

22. Exploratory Data Analysis
Visualization Insights for data Identify any patterns,
trends, or relationships.

23. Analysis

24. Visualize Cars vs Price

25. Visualize Cars vs Price
Insights
•
Jaguar & Buick seems to have the highest price range cars.
• Car companies like Nissan, Renault & Mercury are having only one
to two datapoints.
• So we can't make any inference related to lowest price range car
companies.
Note:
Since there are too many categories in car compnay feature. So we
can derive a new feature Company Price Range which will show the price
range as Low Range, Medium Range, High Range.

26. Feature
Engineering

27. Feature Engineering
Dimensionality reduction
Discretization and Binarization
Normalization or Standardization

28. Dimensionality Reduction
Feature Selection
Filter - Variance
Wrapper - Greedy approach
Embedded - L1 regularization(add bias)
Feature Extraction

29. Discretization and
binarization

30. Discretization
Unsupervised
Equal-Width
Equal-Frequency
K-Means
Supervised
Decision Trees

31. Binarization
Global
Thresholding
Local
Thresholding
Optimization
based
binarization

32. Check for mean to bin the data

33. Binning the Data

34. Data Pre-
Processing

35. Data Pre-processing
Scaling
Encoding
Normalization
Standardization

36. Normalization or
Standardization

37. Encoding

38. Standardization & Normalization
• Decimal Scaling
• Min-Max Normalization
• Z-Score
Normalization(Standariz
zation)
Standardization or Normalization when features of the
input dataset have anomalies or simply when they are
measured in different units (e.g., pounds, meters, miles,
etc.).

39. Standardization
• Before Principal Component Analysis
(PCA)
• Before Clustering
• Before K-Nearest Neighbors (KNN)
• Before Support Vector Machine (SVM)
• Before Measuring Variable Importance
in Regression Models

40. Select only useful features

41. Categorical to Numerical Transformation(Encoding)

42. Scaling Numerical data

43. Fit & Transform to remove the
anomalies

44. Model
Selection/Training/Evaluation

45. Model Selection
• Select a ML algorithm
• Selecting the one that performs
best on the problem and data.
• Linear regression, logistic regression,
decision trees, random forests, or
neural networks

46. Features for Training & Testing

47. Split the test and train data

48. Model Training
• Once the algorithm or model has been
selected, the next step is to train the
model on the training data.
• This involves fitting the model to the
data by adjusting the parameters of the
model to minimize the error or
maximize the accuracy.

49. Predict your car price

50. Model Evaluation – Regression
• Mean Squared Error (MSE)
• Root Mean Squared Error (RMSE)
• Mean Absolute Error (MAE)
• R-squared (R2)
• Adjusted R-squared
• Mean Absolute Percentage Error (MAPE)

51. Model Evaluation - Clustering
• Silhouette Coefficient
• Calinski-Harabasz Index
• Davies-Bouldin Index
• Dunn Index
• Rand Index
• Adjusted Rand Index
• Normalized Mutual Information

52. Model Evaluation - Classification
• Accuracy
• Precision
• Recall
• F1-score
• ROC curve
• AUC
• Confusion matrix

53. Model Evaluation – Anomaly Detection
• True Positive Rate (TPR)
• False Positive Rate (FPR)
• Precision
• F1-score
• Receiver Operating Characteristic (ROC) curve
• Area Under the ROC Curve (AUC)
• Confusion Matrix
• Precision-Recall (PR) Curve
• Average Precision (AP)

54. Model Evaluation – Reinforcement
• Rewards
• Average Reward per Episode
• Value Function
• Policy Gradient
• Exploration-Exploitation Trade-off
• Convergence Time
• Success Rate

55. Compare
your
models

56. Deployment

57. My Book

58. Thank You