Gain hidden insights from your data using IBM Watson Studio

Developer Series
Join our webinar series to expand your developer skills!
—
Security Tuesdays
Data Science and AI Wednesdays
Cloud Native and Red Hat OpenShift Thursdays
meetup.com/IBM-Cloud-MEA

Gain hidden insights from your data
using IBM Watson Studio
—
Anam Mahmood
Developer Advocate, UAE
Hashim Noor
Client Technical Specialist, UAE
Gain hidden insights from your data using IBM Watson Studio, Sept 20 2021/ © 2021 IBM Corporation
Get started at: https://ibm.biz/BdfpKw

Let’s get started
• Sign up/Log in to your IBM Cloud
Account: https://ibm.biz/BdfpKw
• Follow along for the hands-on:
https://github.com/IBMDeveloperMEA/
Gain-hidden-insights-from-your-data-
using-IBM-Watson-Studio
3

4
chat with everyone!
Q&A here!
Follow us to get notified
about upcoming events
View more info about this event
Workshop Resources

Agenda
5
Data Science and Subsets 8
Data Science Methodology 9
Data Types 16
Problems with Data 18
Data Preprocessing 22
Hands-on 27

Introduction to Artificial
Intelligence
“The science and engineering of
making intelligent machines”
- John McCarthy

7
The subsets of AI

What is Data Science?
Data science is an interdisciplinary
field leveraging insights from
many fields to extract knowledge
from data.
8
https://blog.finxter.com/artificial-intelligence-machine-learning-deep-learning-and-data-science-whats-the-difference/

Data Science Methodology
9
Requirements to
Collection
Understanding
to Preparation
Modelling to
Evaluation
Deployment to
Feedback
Problem to
Approach

From Problem to Approach
10
What is the problem we are trying to solve?
How can we use data to answer the question?
Analytic Approach
2. Descriptive Model
3. Statistical Analysis
4. Classification Model
1. Analytic Model
http://www.clipartpanda.com/clipart_images/the-nominal-group-technique-63420908

From Requirements to Collection
11
What data do we need to answer the question?
Where is the data coming from and how to get it?
Data
Collection
Data
Requirements

From Understanding to Preparation
12
Does the data represent the problem we are trying to
solve?
What additional work is required?
Run descriptive statistics
Dataframe.describe()
Data Prepation
1. Missing Data
2. Invalid Values
3. Remove Duplicates
4. Formatting
5. Feature Engineering

From Modeling to Evaluation
13
How can the data be visualized to get to the answer that is
required?
Does the model used really answer the initial question or
does it need to be adjusted?
DESCRIPTIVE
ANALYSIS
PREDICTIVE
ANALYSIS
The Diagnostic
Measures phase
The Statistical
Significance phase

From Deployment to Feedback
14
Can you put the model into practice?
Can you get constructive feedback into answering the
question?
https://www.newbreedmarketing.com/blog/how-to-translate-customer-feedback-into-action
https://giphy.com/gifs/producthunt-push-to-deploy-3og0IAQG2BtR13joe4

Data Science Methodology
https://www.geeksforgeeks.org/data-science-methodology-and-approach/

Types of Data
https://www.pinterest.com/pin/404127766560273784/

.
Problems with Data
18

Getting Data
19
1. Getting the right data.
2. Many often a problem needs data that
does not exist, or exists with another
entity.
3. Another challenge is that data may
come from a variety of different
sources.
4. Data also comes in different formats:
• Database
• CSV
• Unstructured
19

Completeness and Quality
20
• Once the data is obtained, it is important
to ensure that the data is usable.
1. Are there missing values?
2. Is the data reflective of the reality?
3. Are there outliers in the data?

Feature Engineering
21
• The process of using domain knowledge to
extract features from raw data via data
mining techniques.
• We use feature engineering when the data
or features we want don’t exist but are
related to other features.
• Example: Loan application example
Total Income
EMI
New feature added: Balance Income

Data Preprocessing
Converting or mapping data
from raw form into another form
to prepare data for further
analysis
22

Data Preprocessing
Deal with missing data
Check the Data Collection Source
Drop the missing
Value
Drop the variable
Drop the data entry
Replace the missing
values
Use average
Use frequency
Leave it as missing value
23

Data Preprocessing
Data Formatting Data Normalization
• Ensure data is consistent and
easily understandable to make
meaningful comparison.
• Bring data into similar range for
more useful comparison.
24

Data Preprocessing
Data Binning
• Data Binning gives a better
understanding of the data
distribution.
Turning categorical values into
numerical values
• For example: turn the values
of female/male into 0/1
25

Data
Machine Learning
Visualization
aiFairness360
IBM OpenScale
LIME
SHAP
Code
Ethics
Data ScienceTools
26

Let’s get started
• Sign up/Log in to your IBM Cloud
Account: https://ibm.biz/BdfpKw
• Follow along for the hands-on:
https://github.com/IBMDeveloperMEA
/Gain-hidden-insights-from-your-data-
using-IBM-Watson-Studio
27

Survey
https://ibm.biz/BdfpKk
28

.
.
Summary
29
• Introduction to AI
• Introduction to Data Science
• Data Science Methodology
o From problem to approach
o From requirements to collection
o From understanding to preparation
o From modelling to evaluation
o From deployment to feedback
• Problems with Data
o Getting data
o Completeness and quality
o Feature engineering
o Data Preprocessing
• Hands On: Use Watson Studio gather insights from Data

Resources
30
IBM Developer: https://developer.ibm.com/
Meetup: https://www.meetup.com/IBM-Cloud-MEA/
Learning:
– https://cognitiveclass.ai/
– https://learn.ibm.com/
Spark Fundamentals: https://cognitiveclass.ai/learn/spark
Data Science Methodology: https://cognitiveclass.ai/courses/data-science-methodology-2
Python for Data Science: https://cognitiveclass.ai/courses/python-for-data-science
Data visualization, preparation, and transformation using IBM Watson Studio: https://developer.ibm.com/tutorials/watson-studio-
data-visualization-preparation-transformation/?mhsrc=ibmsearch_a&mhq=data%20visualization
Take control of your data with Watson Studio: https://developer.ibm.com/learningpaths/get-started-watson-studio/

.
.
Thank you
31
.
Anam Mahmood
Developer Advocate, UAE
anam.mahmood@ibm.com
Hashim Noor
Client Technical Specialist, UAE
hashim.noor1@ibm.com

Gain hidden insights from your data using IBM Watson Studio

More Related Content

Recently uploaded

Featured

Gain hidden insights from your data using IBM Watson Studio

Editor's Notes