27 17 Yes
24 29 No
21 13 Yes
30 123 No
If x1 = Size of tumour ,
Then scaled value of x, x1norm = x1 /max(x1)
If x2 = Age of tumour,
Then scaled value of x, x2norm = x2 /max(x2)
Binary output can be converted into 0 and 1.
• Cost function - a way to estimate how far
the estimated value is from real-value
• 1/2 × (𝑦 − 𝑦)2
• The idea is: Inputs is the knowledge and
hence cannot be changed, what can be
changed to reduce error is : weights!!!
• The number of combination of possible
values of z is enormous. : (1000 for each
weight)^(number of weights)
• This is called ‘Curse of Dimensionality’!
• Gradient descent is the way to take lesser number of
steps of adjusting weights to reduce cost function.
• Use Partial differentiation:
> 0, 𝑡ℎ𝑒𝑛 𝑐𝑜𝑠𝑡 𝑓𝑢𝑛𝑐𝑡𝑖𝑜𝑛 𝑖𝑠 𝑖𝑛𝑐𝑟𝑒𝑎𝑠𝑖𝑛𝑔
𝑎𝑛𝑑 𝑤𝑒 𝑠ℎ𝑜𝑢𝑙𝑑 𝑚𝑜𝑣𝑒 𝑖𝑛 𝑜𝑝𝑝𝑜𝑠𝑖𝑡𝑒 𝑑𝑖𝑟𝑒𝑐𝑡𝑖𝑜𝑛!
< 0, 𝑡ℎ𝑒𝑛 cost function is decreasing
and we should move in this direction!
Choosing Learning Rate
lr_t = learning_rate * sqrt(1 - beta2^t) / (1 - beta1^t)
learning_rate =0.001 at t =0