explain time complexity, dynamic programming, backtracking

1. programming workshop

2. What will be covered
•Time complexity
•Dynamic Programming
• Backtracking

3. Time complexity of For loop
For (idx = 0; idx < N ; idx = idx + 1 )
{
print(idx)
}
Time complexity
is O(N)

4. Time complexity of For loop
For (idx = 1; idx < N ; idx = idx * 2)
{
print(idx)
}
Time complexity
is O(log2
(N))
idx = 1, 2, 4, 8, 16, 32…
N = 8, number of steps = 3
N = 32, number of steps = 5

5. Time complexity of For loops
For (idx = 1; idx < N ; idx = idx + 1)
{
aList.Append(idx)
}
Time complexity
is O(N2
)
1 2 N-5 N
Elements before you = 1 + 2 + 3 + 4 + … + N = N(N-1)/2

6. Time complexity of For loops
For (idx = 1; idx < N ; idx = idx + 1)
{
aList.Append(idx)
}
Time complexity
is O(N)
1 2 N-5 N
If you had a tail pointer ? Then 1 + 1 + 1 = N
Tail Pointer

7. Time complexity of For loops
For (idx = 1; idx < N ; idx = idx + 1)
{
binaryTree.Insert(idx)
}
Time complexity
is O(N . log2
(N))
A
K
Z S
T
E
Cost of one insert into Binary tree
= go down the depth of binary Tree
= log2
(N)

8. Time complexity of two For loops
For (i = 0; i < N ; i = i + 1)
for (j = 0; j < N ; j = j + 1)
{
print(i + j)
}
Time complexity
is O(N2
)

9. Time complexity of two For loops
for (i = 0; i < N ; i = i + 1)
for (j = 0; j < i ; j = j + 1)
{
print(i + j)
}
for (i = 0; i < N ; i = i + 1)
for (j = i; j < N ; j = j + 1)
{
print(i + j)
}
Left loop = 1 + 2 + 3 + 4 + … + N
= N(N-1)/2
Time complexity
is O(N2
)
Right loop = N + (N-1) + … + 2 + 1

10. Mark each algorithm on this chart
N2
N N3
exp(N)1
1
N
N.log2
(N)
N.log2
(N)
Exp(N)
TIME
SPAC
E
Double
For Loop
Single For
Loop
Tree
insert
List insert
List insert
with Tail
pointer

11. Many ways of solving a given problem
N2
N N3
exp(N)1
1
N
N.log2
(N)
N.log2
(N)
Exp(N)
TIME
SPAC
E
Time
efficient
Space
efficient
Space and
time
optimal
Cannot do better than Lower bound.
This depends on constraints of problem

12. Four ways to get home
Backtracking : Explore every point from current position. If you
don’t reach home, return to previous position and retry.
Divide and Conquer : Find way to get to Metro, then Majestic,
then home.
Dynamic Programming : Go home at lowest cost using Bus,
Metro, Cab and Rickshaw
Greedy : Find best next point in home-ward direction

13. Dynamic Programming

14. Get home via Dynamic Programming
Office Home
Jayanagar
Majestic
Cubbon
Park
TravelCost (Home) =
Min ( TravelCost(Majestic), TravelCost(Cubbon Park) )
TravelCost (Cubbon Park) =
Min ( TravelCost (Direct), TravelCost(Jayanagar) )
Silk Board Jn

15. Dynamic Programming - subset sum
4 7
7
5
3
3
5
3
3
Either add this element OR not
4 7 5 3
Find subset whose sum is 14
5
3

16. DP - Subset Sum – recursive
Boolean hasSubsetSum(int curIdx, int curSum)
{
// either you add this element OR not add it
return hasSubsetSum(curIdx - 1, curSum)
|| hasSubsetSum(curIdx - 1, curSum - array[curIdx - 1])
}

17. DP - Subset Sum – recursive
Boolean hasSubsetSum(int curIdx, int curSum)
{
if (curSum == 0) return True
// either you add this element OR not add it
return hasSubsetSum(curIdx - 1, curSum)
|| hasSubsetSum(curIdx - 1, curSum - array[curIdx - 1])
}

18. DP - Subset Sum – recursive
Boolean hasSubsetSum(int curIdx, int curSum)
{
if (curSum == 0) return True
if (curIdx == 0) || (curSum != 0) return False
// either you add this element OR not add it
return hasSubsetSum(curIdx - 1, curSum)
|| hasSubsetSum(curIdx - 1, curSum - array[curIdx - 1])
}

19. DP - Subset Sum – recursive to iterative
• REMEMBER : When the Recursion is a function of 2 variables, your iterative solution
requires a 2-dim matrix
….hasSubsetSum(curIdx - 1, curSum)
|| hasSubsetSum(curIdx - 1, curSum - array[curIdx - 1])
0
1
…
last array element n-1
0 1 expectedSum

20. DP - Subset sum – iterative
Boolean hasSubsetSum(int expectedSum)
{
for (i = 1; i < n; i ++)
for (j = 1; j < expectedSum; j ++ )
{
subset[ i ][ j ] = subset[i - 1][ j ]
|| subset[i -1][ j – array[curIdx]]
}
}

21. Backtracking

22. How to get home via backtracking
Boolean PathToHome(current position)
{
if (reachedHome(current position))
return True
for (all available Lanes starting from current position)
{
gotHome = PathToHome(current position + lane)
if (gotHome = false)
Go back to previous position
}
return False
}

23. Backtracking program template

24. Backtracking - subset sum
4 7
7
5
3
3
5
3
3
Either add this element OR not
4 7 5 3
Find subset whose sum is 14
5
3

25. Backtracking – subset sum

26. Backtracking – subset sum - Complexity
•Each element is either added or Not
•Number of such subsets = 2 * 2 * 2... (N times) = 2N
•Each subset has to be summed = sum of (roughly) N elements
Hence, time complexity
= number of elements in each subset x number of subsets
= O(N . 2N
)

27. Knights tour
Position = 2, 2
0,1 1,0 0,3 1,4 3,0 4,1 3,4 4,3
Each cell has 8 options
Depth of tree = 64
Check for all 64 cells
Time = O(864
)

28. Backtracking – Knights tour

29. Backtracking program template

30. Recap : Four ways to get home
Backtracking : Explore every point from current position. If you
don’t reach home, return to previous position and retry.
Divide and Conquer : Find way to get to Baiyappanahalli Metro,
then Majestic, then home.
Dynamic Programming : Go home at lowest cost using Bus,
Metro, Cab and Rickshaw
Greedy : Find best next point in home-ward direction

31. Difference between algorithm types
•“Greedy Algorithm” first makes choice and then solves sub-problem
•“Divide and conquer” generates non-overlapping sub-problems
•“Dynamic programming” finds optimal solution to sub-problem; but it
can generate overlapping sub-problems – hence “memoization”
•“Backtracking” can reach an invalid solution – that’s why its called
backtracking

32. Conclusion
To write good code…
•You have to read good code
•Work through many problems
•Memorize elegant patterns & tricks

33. Dynamic Programming - Rabbits
If rabbits die after 4 days, only those younger than 4 days are alive
Int getNumRabbits(int numDays)
{
if (n == 0) return 1
alive = [3 x getNumRabbits(numOfDays – 1) ]
- getNumOfRabbits[numOfDays – 4]
}

34. Invariants in an algorithm
•Invariants ensure that each step in your algorithm is correct
•Convert invariants into Assertions
•https://yourbasic.org/algorithms/loop-invariants-explained/
•Example : Take 2-3 algorithms and define one invariant in them

35. Think of the simplest cases
•Given a complicated problem
•E.g. In array of 100 elements, rearrange in positive & negative
Too hard to think of all combinations !
1. Lets reduce it to simplest possible examples
2. Take an array with 3 entries
3. Find a technique to solve this
4. Then expand it to solve larger problem

36. Think of the simplest cases
-3 -3 -3 4 4 4
-3 5 6
5 -3 6
5 -36
-3 -5 6
-3 -56
-3 -56
USE
SYMMETRY