Divide and Conquer

1. Kamalesh Karmakar,
Assistant Professor,
Dept. of C.S.E.
Meghnad Saha Institute of Technology

2. Divide and Conquer:
Binary Search, Merge Sort, Quick Sort and their complexity.
Heap Sort and its complexity
Dynamic Programming:
Matrix Chain Manipulation, All pair shortest paths, single source shortest path.
Backtracking:
8 queens problem, Graph coloring problem.
Greedy Method:
Knapsack problem, Job sequencing with deadlines, Minimum cost spanning tree by Prim’s and Kruskal’salgorithm.

3. Insertionsortusesanincrementalapproach.
Anotherapproachofdesigningalgorithmsisdivide&conquerapproach.
Oneadvantageisthattheirrunningtimeareofteneasilydeterminedusingtechniquescalledrecurrencerelationsorrecurrenceequation.
Thedivideandconquerapproachfollowsthefollowingthreesteps:
Dividetheprobleminno.ofsubproblems.
Conquerthesubproblemsbysolvingrecursively.
Ifthesubproblemssizearesmallenoughjustsolvethesubproblemsinastraightforwardmanner.
Combinesolutionstothesubproblemsintothesolutionoftheoriginalproblem.

9. Divide:
Thedividestepjustcomputesthemiddleofthesub-array,whichtakesconstanttime.Thus,D(n)=Θ(1).
Conquer:
Werecursivelysolvetwosubproblems,eachofsizen/2,whichcontributes2T(n/2)totherunningtime.
Combine:
WehavealreadynotedthattheMERGEprocedureonann-elementsub-arraytakestimeΘ(n),soC(n)=Θ(n).

10. Tocomputethetotalcostrepresentedbytherecurrence,wesimplyaddupthecostsofallthelevels.Therearelgn+1levels,eachcostingcn,foratotalcostofcn(lgn+1)=cnlgn+cn.Ignoringthelow-ordertermandtheconstantcgivesthedesiredresultofΘ(nlgn).

13. Quicksortisasortingalgorithmwhoseworst-caserunningtimeisΘ(n2) onaninputarrayofnnumbers.Quicksortisoftenthebestpracticalchoiceforsortingbecauseitisremarkablyefficientontheaverage: itsexpectedrunningtimeisΘ(nlgn),andtheconstantfactorshiddenintheΘ(nlgn)notationarequitesmall.Italsohastheadvantageofsortinginplace,anditworkswelleveninvirtualmemoryenvironments.

14. Hereisthethree-stepdivide-and-conquerprocessforsortingatypicalsub-arrayA[p..r].
Divide:
Partition(rearrange)thearrayA[p..r]intotwo(possiblyempty) sub-arraysA[p..q−1]andA[q+1..r]suchthateachelementofA[p..q−1]islessthanorequaltoA[q],whichis,inturn,lessthanorequaltoeachelementofA[q+1..r].Computetheindexqaspartofthispartitioningprocedure.
Conquer:
Sortthetwosub-arraysA[p..q−1]andA[q+1..r]byrecursivecallstoquicksort.
Combine:
Sincethesub-arraysaresortedinplace,noworkisneededtocombinethem:theentirearrayA[p..r]isnowsorted.

16. A recursion tree for QUICKSORT in which PARTITION
always produces a 9-to-1 split

17. WorstCasePartitioning:
Theworst-casebehaviorforquicksortoccurswhenthepartitioningroutineproducesonesub-problemwithn−1elementsandonewith0elements.
Inunbalancedpartitioning,partitioningcostsΘ(n)time.Sincetherecursivecallonanarrayofsize0justreturns,T(0)=Θ(1),andtherecurrencefortherunningtimeis
ThisalgorithmevaluatestoΘ(n2).Indeed,itisstraightforwardtousethesubstitutionmethodtoprovethattherecurrenceT(n)=T(n−1)+ Θ(n)hasthesolutionT(n)=Θ(n2).Thus,ifthepartitioningismaximallyunbalancedateveryrecursivelevelofthealgorithm,therunningtimeisΘ(n2).
Moreover,the(n2)runningtimeoccurswhentheinputarrayisalreadycompletelysorted—acommonsituationinwhichinsertionsortrunsinO(n)time.

18. BestCasePartitioning:
Inthemostevenpossiblesplit,PARTITIONproducestwosubproblems,eachofsizenomorethann/2,sinceoneisofsizen/2andoneofsizen/2−1.Inthiscase,quicksortrunsmuchfaster.Therecurrencefortherunningtimeisthen
T(n)≤2T(n/2)+(n),
Bycase2ofthemastertheorem(Theorem4.1)hasthesolution T(n)=O(nlgn).Thus,theequalbalancingofthetwosidesofthepartitionateveryleveloftherecursionproducesanasymptoticallyfasteralgorithm.

19. BalancedPartitioning:
Suppose,forexample,thatthepartitioningalgorithmalwaysproducesa9-to-1proportionalsplit,whichatfirstblushseemsquiteunbalanced.Wethenobtaintherecurrence
T(n)≤T(9n/10)+T(n/10)+cn
ontherunningtimeofquicksort,wherewehaveexplicitlyincludedtheconstantchiddenintheΘ(n)term.
Noticethateverylevelofthetreehascostcn,untilaboundaryconditionisreachedatdepthlog10n=(lgn),andthenthelevelshavecostatmostcn.
Therecursionterminatesatdepthlog10/9n=(lgn).ThetotalcostofquicksortisthereforeO(nlgn).
Anysplitofconstantproportionalityyieldsarecursiontreeofdepth(lgn),wherethecostateachlevelisO(n).TherunningtimeisthereforeO(nlgn)wheneverthesplithasconstantproportionality.

21. The(binary)heapdatastructureisanarrayobjectthatcanbeviewedasanearlycompletebinarytree.Eachnodeofthetreecorrespondstoanelementofthearraythatstoresthevalueinthenode.Thetreeiscompletelyfilledonalllevelsexceptpossiblythelowest,whichisfilledfromtheleftuptoapoint.
AnarrayAthatrepresentsaheapisanobjectwithtwoattributes: length[A],whichisthenumberofelementsinthearray,andheap- size[A],thenumberofelementsintheheapstoredwithinarrayA.
So,heap-size[A]≤length[A]

22. A max-heap viewed as (a) a binary tree and (b) an array
Typesofbinaryheaps:max-heapsandmin-heaps.
Inbothkinds,thevaluesinthenodessatisfyaheapproperty.
Fortheheap-sortalgorithm,weusemax-heaps.
Min-heaps are commonly used in priority queues.

23. TheMAX-HEAPIFYprocedure,whichrunsinO(lgn)time,isthekeytomaintainingthemax-heapproperty.
TheBUILD-MAX-HEAPprocedure,whichrunsinlineartime, producesamaxheapfromanunorderedinputarray.
TheHEAPSORTprocedure,whichrunsinO(nlgn)time,sortsanarrayinplace.
TheMAX-HEAP-INSERT,HEAP-EXTRACT-MAX,HEAP- INCREASE-KEY,andHEAP-MAXIMUMprocedures,whichruninO(lgn)time,allowtheheapdatastructuretobeusedasapriorityqueue.

24. MAX-HEAPIFYisanimportantsubroutineformanipulatingmax- heaps.ItsinputsareanarrayAandanindexiintothearray.WhenMAX-HEAPIFYiscalled,itisassumedthatthebinarytreesrootedatLEFT(i)andRIGHT(i)aremax-heaps,butthatA[i]maybesmallerthanitschildren,thusviolatingthemax-heapproperty.
ThefunctionofMAX-HEAPIFYistoletthevalueatA[i]“floatdown” inthemaxheapsothatthesubtreerootedatindexibecomesamax-heap.

26. TheoperationofBUILD-MAX-HEAP,showingthedatastructurebeforethecalltoMAX-HEAPIFYinline3ofBUILD-MAX-HEAP.(a)A10- elementinputarrayAandthebinarytreeitrepresents.Thefigureshowsthattheloopindexireferstonode5beforethecallMAX-HEAPIFY(A, i).(b)Thedatastructurethatresults.Theloopindexiforthenextiterationreferstonode4.

27. (c)–(e)SubsequentiterationsoftheforloopinBUILD-MAX-HEAP. ObservethatwheneverMAX-HEAPIFYiscalledonanode,thetwosubtreesofthatnodearebothmax-heaps.
(f)Themax-heapafterBUILD-MAX-HEAPfinishes.

28. ThetimerequiredbyMAX-HEAPIFYwhencalledonanodeofheighthisO(h),sowecanexpressthetotalcostofBUILD-MAX-HEAPas
Thelastsummationcanbeevaluatedbysubstitutingx=1/2intheformula,whichyields
Thus,therunningtimeofBUILD-MAX-HEAPcanbeboundedas

29. TheHEAPSORTproceduretakestimeO(nlgn),sincethecalltoBUILD-MAXHEAPtakestimeO(n)andeachofthen−1callstoMAX-HEAPIFYtakestimeO(lgn).