2. DynamicProgramming
•DynamicProgramming is ageneralalgorithm designtechnique
•InventedbyAmerican mathematicianRichard Bellmaninthe 1950stosolve
optimizationproblems
•“Programming” heremeans“planning”
•Mainidea:
•solveseveralsmaller(overlapping)subproblems
•recordsolutionsinatableso thateachsubproblem isonlysolvedonce
•finalstateofthetablewillbe(orcontain)solution
29/8/2022
DesignandAnalysisofAlgorithms
GreedyTechnique
2
8. DesignandAnalysis ofAlgorithms
GreedyTechnique
8
Warshall’s Algorithm
•Main idea:apath existsbetweentwo verticesi,j,iff
•thereisanedgefromito j;or
•thereisapath fromitojgoingthroughvertex1;or
•thereisapath fromitojgoingthroughvertex1and/or 2;or
•thereisapath fromitojgoingthroughvertex1,2, and/or3;or
•...
•thereisapath fromitojgoingthroughanyoftheothervertices
3
4
2
1
3
4
2
1
3
4
2
1
3
4
2
1
R0
0 0 1 0
1 0 0 1
0 0 0 0
0 1 0 0
R1
0 0 1 0
1 0 1 1
0 0 0 0
0 1 0 0
R2
0 0 1 0
1 0 1 1
0 0 0 0
1 1 1 1
R3
0 0 1 0
1 0 1 1
0 0 0 0
1 1 1 1
R4
0 0 1 0
1 1 1 1
0 0 0 0
1 1 1 1
3
4
2
1
29/8/2022
9. DesignandAnalysis ofAlgorithms
GreedyTechnique
9
Warshall’s Algorithm
•Inthekth stage determine ifapathexistsbetween twovertices i,jusingjustverticesamong 1,…,k
R(k-1)[i,j] (path usingjust 1,…,k-1)
R(k)[i,j] = or
(R(k-1)[i,k] andR(k-1)[k,j]) (pathfrom itok
andfromkto i
usingjust 1,…,k-1)
i
j
k
kth stage
{
29/8/2022
12. DesignandAnalysis ofAlgorithms
GreedyTechnique
12
FinalComments
•Dynamicprogramming isatechnique for solvingproblems withoverlapping sub-
problems.
•Applicabilityofdynamicprogramming toanoptimizationproblem requiresthe
problemtosatisfytheprinciple ofoptimality:anoptimal solutionto anyofits
instancesmustbemadeupofoptimalsolutionstoitssub-instances.
•Warshall’salgorithmfor findingthetransitiveclosureandFloyd’s algorithm for
theall-pairsshortest-paths problemarebasedontheideathat anbeinterpretedas
anapplicationof thedynamicprogramming technique.
29/8/2022
13. DesignandAnalysis ofAlgorithms GreedyTechnique 13
GreedyAlgorithms
Optimizationproblemssolvedthrough a sequenceofchoicesthatare:
feasible
locallyoptimal
irrevocable
Notalloptimizationproblemscan beapproached inthismanner!
29/8/2022
14. DesignandAnalysis ofAlgorithms GreedyTechnique 14
ApplicationsoftheGreedyStrategy
Optimalsolutions:
• changemaking
• MinimumSpanningTree(MST)
• Single-sourceshortest paths
• simpleschedulingproblems
• Huffmancodes
Approximations:
• TravelingSalesmanProblem(TSP)
• Knapsackproblem
• other combinatorialoptimization problems
29/8/2022
15. DesignandAnalysis ofAlgorithms GreedyTechnique 15
MinimumSpanningTree(MST)
Spanningtreeof aconnectedgraph G:a connectedacyclic subgraph of Gthat
includesallof G’svertices.
MinimumSpanningTreeofaweighted,connectedgraph G:aspanningtreeof
Gof minimumtotalweight.
Example:
3
4
2
1
4
2
6
1
3
29/8/2022
16. DesignandAnalysis ofAlgorithms GreedyTechnique 16
Prim’sMSTalgorithm
Startwithtreeconsistingof onevertex
“grow” treeonevertex/edgeatatimetoproduce MST
Construct aseries ofexpandingsubtreesT1,T2,…
ateachstage construct Ti+1 from Ti:addminimumweightedgeconnecting a
vertexintree(Ti)toonenotyetintree
choose from “fringe”edges
(thisisthe“greedy”step!)
algorithmstops whenallverticesareincluded
29/8/2022
19. DesignandAnalysis ofAlgorithms GreedyTechnique 19
NotesaboutPrim’salgorithm
• Need toprovethat thisconstruction actuallyyieldsMST
• Need priorityqueue forlocatinglowest costfringeedge:use min-heap
• Efficiency: Forgraphwithnverticesandmedges:
(n–1+m)logn
Θ(mlog n)
numberofstages
(min-heap deletions) numberofedgesconsidered
(min-heapinsertions)
insertion/deletion frommin-heap
29/8/2022
20. DesignandAnalysis ofAlgorithms GreedyTechnique 20
Another Greedyalgorithm forMST:Kruskal
Startwithemptyforest of trees
“grow” MST oneedgeatatime
• intermediatestagesusuallyhaveforestoftrees(notconnected)
ateachstageaddminimumweightedgeamong thosenotyetusedthatdoesnot
createacycle
• edgesareinitiallysorted byincreasingweight
• ateachstagetheedgemay:
– expand anexisting tree
– combinetwo existing treesinto asingletree
– createanewtree
• needefficientwayofdetecting/avoidingcycles
algorithmstops whenallverticesareincluded
29/8/2022
22. DesignandAnalysis ofAlgorithms GreedyTechnique 22
NotesaboutKruskal’salgorithm
AlgorithmlookseasierthanPrim’sbutis
• hardertoimplement(checking forcycles!)
• lessefficient Θ(mlogm)
Cyclechecking:acycleexistsifedgeconnectsverticesinthesame
component.
29/8/2022
23. DesignandAnalysis ofAlgorithms GreedyTechnique 23
Shortestpaths-Dijkstra’salgorithm
SingleSourceShortestPathsProblem:Givenaweightedgraph G,findthe
shortest pathsfrom a source vertexs toeachof theother vertices.
Dijkstra’salgorithm:SimilartoPrim’sMSTalgorithm,withthefollowing
difference:
• Startwithtreeconsisting ofonevertex
• “grow”tree onevertex/edgeatatimetoproduce MST
– ConstructaseriesofexpandingsubtreesT1,T2,…
• Keep trackofshortest pathfromsourcetoeachoftheverticesin Ti
29/8/2022
26. DesignandAnalysis ofAlgorithms
GreedyTechnique
26
Huffman’salgorithm
Step 1 Initialize n one-node trees and label them with the symbols of the
alphabet given. Record the frequency of each symbol in its tree’s root to indicate
the tree’s weight. (More generally, the weight of a tree will be equal to the sum of
the frequencies inthe tree’sleaves.)
Step 2 Repeat the following operation until a single tree is obtained. Find two
trees with the smallest weight. Make them the left and right subtree of a new tree
andrecord the sumof theirweights inthe rootof thenewtreeasitsweight.
29/8/2022
