What are the challenges the IT industry is facing now? And how is data science the savior? How do you convert a real-world problem into a data science problem? An introduction to the modeling of a known problem to formulate and simulate the real world scenario

1. Intuitions and Formulations

2. THE MODERN IT SCENARIO
Challenges:
– Why IT?
What’s the problem?
What’s wrong?
Resource Utility
Economic Factors
Reputation and Competition

3. STEM TALENT DEMAND
Ever growing
Around 300,000 in demand
What comprises STEM?
The US crisis
Novel Definitions

4. THE ART OF DATA SCIENCE
Problem Definition
– Hypothesis Driven Approach
What’s the problem?
What’s probably a solution?
Analyze facts and data
Present the solution

5. THE ART OF DATA SCIENCE
Logic Trees
– How to solve the problem?
– How to cause the problem?
– How related is it to the
definitions we have in the world ?
– Formulation
– Validation
– Enhancement

6. A TEST ON YOUR BASICS
How would you explain a population at its best?
How would you define the income metrics of a nation at its
best?
How would you draw a border between two countries?
What is a straight line?
What is a basis function?
How would you store a 100000x100000 matrix

7. AN EXERCISE
Design a fair value predictor for used car price
How would you recommend a movie to an acquintance?
Real estate
– Design a fair value estimator for properties in Calicut
– Design a fair value predictor for properties in Calicut
– Design a representation of realtor activities in Calicut

8. SIGNAL PROCESSING?
Speech Analytics
Image Analytics
Weather Analytics
Finance and Economics
Engine Dynamics

9. SIGNAL PROCESSING: A USE CASE
Economic system as a signal processing problem?
Open Loop Systems
Assumptions and Hypothesis are non verifiable
Thus the Efficient Market Hypothesis
– All information is incorporated into price
– Market reaches equilibrium through rationale

10. UTILITY AS A FUNCTION
Utility a concave function of overall wealth
– Utility increases with wealth
– Wealth has diminishing marginal utility
RP Risk Premium, CE Certainty Equivalent

11. WE JUST MODELED
A rationale person is risk averse
– Would prefer riskless asset
– Certainty is the guaranteed wealth 𝐶𝐸 = 𝑈−1
𝐸 𝑈 𝑊
– 𝑅𝑃 = 𝐸 𝑊 − 𝐶𝐸
– Thus this is source of excess return or pay back of a stock, reason
to pay insurance
– This is how your brain takes decisions

12. UTILITY AS A FUNCTION
Utility a concave function of overall wealth
– Utility increases with wealth
– Wealth has diminishing marginal utility
RP Risk Premium, CE Certainty Equivalent

13. OPTIMIZING AND PORTFOLIO ANALYSIS
Mean Variance Portfolio – Solution of mean variance optimization
𝑥𝑖𝑝 = arg min{𝑥 𝑝
𝑇 𝑥 𝑝}
Let the net return for asset i at time t
𝑅𝑖.𝑡 =
𝑝𝑖.𝑡 − 𝑝𝑖.𝑡−1 + 𝑑𝑖.𝑡
𝑝𝑖.𝑡−1
𝑝 𝑖𝑠 𝑡ℎ𝑒 𝑝𝑟𝑖𝑐𝑒 𝑎𝑛𝑑 𝑑 𝑡ℎ𝑒 𝑑𝑖𝑣𝑖𝑑𝑒𝑛𝑡 from log of total return
Thus maximize the expected return of a given variance or minimize the
variance of expected return
Or people like signals but not the noise – Otherwise we defined
volatility

14. OTHER APPLICATIONS OF SP
Time series model
Gaussian process regression for stochastic volatility
Markov Chain Monte Carlo Learning
Kalman Filtering to estimate weights of mean reverting
problems for risk assessments