Data Structure Puzzles

8,863 views

Published on

All code written in C
Data structure and algorithmic puzzles to improve thinking
Enumeration of possible solutions for each problem
Covers all topics for competitive exams
Covers interview questions on data structures and algorithms
Reference Manual for working people
Campus Preparation
Degree/Masters Course Preparation
Big Job Hunters: Microsoft, Google, Amazon, Yahoo, Oracle, Facebook and many more
Content has been reviewed by professors, architects and proof readers

Please have a look at sample chapters here:

https://www.createspace.com/Preview/1079368

Published in: Education, Technology
4 Comments
41 Likes
Statistics
Notes
No Downloads
Views
Total views
8,863
On SlideShare
0
From Embeds
0
Number of Embeds
27
Actions
Shares
0
Downloads
0
Comments
4
Likes
41
Embeds 0
No embeds

No notes for slide

Data Structure Puzzles

  1. 1. DATA STRUCTURE PUZZLESSuccess Key for:  Campus Preparation  Degree/Masters Course Preparation  Instructor’s  GATE Preparation  Big Job hunters: Microsoft, Google, Amazon, Yahoo, Facebook, Adobe and many more  Reference Manual for Working People CareerMonk Publications
  2. 2. To My Parents -Laxmi and ModaiahTo My Family MembersTo My FriendsTo IIT BombayTo All Hard Workers
  3. 3. Copyright ©2010 by CareerMonk.com All rights reserved.Designed by Narasimha Karumanchi Printed in India
  4. 4. AcknowledgementsI would like to express my gratitude to the many people who saw me through this book, to all thosewho provided support, talked things over, read, wrote, offered comments, allowed me to quote theirremarks and assisted in the editing, proofreading and design. In particular, I would like to thank thefollowing individuals.I would like to thank for encouraging me when I was at IIT Bombay. He isthe first person who taught me the importance of and its . From that day I keepon updating myself.I would like to thank [ ] and [ ] forspending time in reviewing this book and providing me the valuable suggestions almost every day.I would like to thank [ ] for spending his valuable time inreviewing the book and suggestions. His review gave me the confidence in the quality of the book.I would like to thank and [Founder’s of ] for approaching me forteaching Data Structures and Algorithms at their training centers. They are the primary reason forinitiation of this book.My and have contributed greatly to the quality of this book. I thank all of youfor your help and suggestions.Special thanks should go to my wife, for her encouragement and help during writing of thisbook.Last but not least, I would like to thank Director’s of , [Director of ], [Former Director of ] and [ , ] for helping me andmy family during our studies. -Narasimha Karumanchi M-Tech, IIT Bombay Founder of CareerMonk.com
  5. 5. PrefaceDear Reader,Please Hold on! I know many people do not read preface. But I would like to strongly recommendreading preface of this book at least. This preface has from regular prefaces.As a if you read complete book with good understanding, I am sure you will challenge theinterviewer’s and that is the objective of this book.If you read as an , you will give better lectures with easy go approach and as a result yourstudents will feel proud for selecting Computer Science / Information Technology as their degree.This book is very much useful for the of and during their academicpreparations. All the chapters of this book contain theory and their related problems as many aspossible. There a total of approximately algorithmic puzzles and all of them are with solutions.If you read as a preparing for competition exams for Computer Science/InformationTechnology], the content of this book covers the topics in full details. While writing thebook, an intense care has been taken to help students who are preparing for these kinds of exams.In all the chapters you will see more importance given to problems and analyzing them instead ofconcentrating more on theory. For each chapter, first you will see the basic required theory and thenfollowed by problems.For many of the problems, solutions are provided with different complexities. We start with solution and slowly move towards the possible for that problem. For eachproblem we will try to understand how much time the algorithm is taking and how much memorythe algorithm is taking.It is that, at least one complete reading of this book is required to get fullunderstanding of all the topics. In the subsequent readings, readers can directly go to any chapter andrefer. Even though, enough readings were given for correcting the errors, due to human tendencythere could be some minor typos in the book. If any such typos found, they will be updatedat . I request readers to constantly monitor this site for any corrections, newproblems and solutions. Also, please provide your valuable suggestions at: . Wish you all the best. Have a nice reading. -Narasimha Karumanchi M-Tech, IIT Bombay Founder of CareerMonk.com
  6. 6. Table of ContentsChapter 1 Introduction ............................................................................................................. 27 Variables ................................................................................................................................................. 27 Data types ............................................................................................................................................... 27 System defined data types (Primitive data types) ............................................................................ 28 User defined data types...................................................................................................................... 28 Data Structure ........................................................................................................................................ 28 Abstract Data Types (ADT’s) ................................................................................................................. 29 Memory and Variables .......................................................................................................................... 29 Size of a Variable................................................................................................................................ 30 Address of a Variable ......................................................................................................................... 30 Pointers................................................................................................................................................... 31 Declaration of Pointers ...................................................................................................................... 31 Pointers Usage .................................................................................................................................... 31 Pointer Manipulation ........................................................................................................................ 32 Arrays and Pointers ........................................................................................................................... 33 Dynamic Memory Allocation............................................................................................................ 33 Function Pointers............................................................................................................................... 34 Parameter Passing Techniques .............................................................................................................. 34 Actual and Formal Parameters .......................................................................................................... 34 Semantics of Parameter Passing ........................................................................................................ 35 Language Support for Parameter Passing Techniques ..................................................................... 35 Pass by Value...................................................................................................................................... 35 Pass by Result ..................................................................................................................................... 36 Pass by Value-Result.......................................................................................................................... 37 Pass by Reference (aliasing) .............................................................................................................. 38 Pass by Name...................................................................................................................................... 39 Binding ................................................................................................................................................... 40 Binding Times .................................................................................................................................... 40 Static Binding (Early binding) ........................................................................................................... 40 Dynamic Binding (Late binding)....................................................................................................... 41 Scope ....................................................................................................................................................... 41
  7. 7. Static Scope......................................................................................................................................... 41 Dynamic Scope ................................................................................................................................... 42 Storage Classes........................................................................................................................................ 43 Auto Storage Class.............................................................................................................................. 43 Extern storage class ............................................................................................................................ 44 Register Storage Class ........................................................................................................................ 49 Static Storage Class............................................................................................................................. 49 Storage Organization ............................................................................................................................. 50 Static Segment .................................................................................................................................... 50 Stack Segment .................................................................................................................................... 51 Heap Segment .................................................................................................................................... 53 Shallow Copy versus Deep Copy........................................................................................................... 54Chapter 2 Analysis of Algorithms ............................................................................................ 55 Introduction ........................................................................................................................................... 55 What is an Algorithm? .......................................................................................................................... 55 Why Analysis of Algorithms? ............................................................................................................... 55 Goal of Analysis of Algorithms?............................................................................................................ 56 What is Running Time Analysis?.......................................................................................................... 56 How to Compare Algorithms? .............................................................................................................. 56 What is Rate of Growth? ....................................................................................................................... 56 Commonly used Rate of Growths ......................................................................................................... 57 Types of Analysis ................................................................................................................................... 58 Asymptotic Notation? ............................................................................................................................ 59 Big-O Notation ....................................................................................................................................... 59 Big-O Visualization............................................................................................................................ 59 Big-O Examples .................................................................................................................................. 60 No Uniqueness? .................................................................................................................................. 61 Omega-Ω Notation................................................................................................................................. 61 Ω Examples ......................................................................................................................................... 61 Theta-θ Notation.................................................................................................................................... 62 Θ Examples......................................................................................................................................... 62 Important Notes ................................................................................................................................. 63 Why is it called Asymptotic Analysis? ................................................................................................. 63
  8. 8. Guidelines for Asymptotic Analysis? .................................................................................................... 63 Properties of Notations .......................................................................................................................... 66 Commonly used Logarithms and Summations ..................................................................................... 66 Master Theorem for Divide and Conquer ............................................................................................ 66 Problems Divide and Conquer Master Theorem.................................................................................. 67 Master Theorem for Subtract and Conquer Recurrences .................................................................... 69 Variant of subtraction and conquer master theorem ....................................................................... 69 Problems on Algorithms Analysis......................................................................................................... 69Chapter 3 Recursion and Backtracking .................................................................................... 84 Introduction ........................................................................................................................................... 84 What is Recursion? ................................................................................................................................ 84 Why Recursion?..................................................................................................................................... 84 Format of a Recursive Function ............................................................................................................ 84 Recursion and Memory (Visualization) ................................................................................................ 85 Recursion versus Iteration ..................................................................................................................... 86 Recursion ............................................................................................................................................ 86 Iteration .............................................................................................................................................. 87 Notes on Recursion ................................................................................................................................ 87 Example Algorithms of Recursion ........................................................................................................ 87 Problems on Recursion .......................................................................................................................... 87 What is Backtracking? ........................................................................................................................... 88 Example Algorithms of Backtracking ................................................................................................... 89 Problems on Backtracking..................................................................................................................... 89Chapter 4 Linked Lists .............................................................................................................. 91 What is a Linked List? ........................................................................................................................... 91 Linked Lists ADT ................................................................................................................................... 91 Why Linked Lists? ................................................................................................................................. 91 Arrays Overview ................................................................................................................................ 92 Why Constant Time for Accessing Array Elements?....................................................................... 92 Advantages of Arrays ......................................................................................................................... 92 Disadvantages of Arrays .................................................................................................................... 92 Dynamic Arrays ................................................................................................................................. 92 Advantages of Linked Lists ................................................................................................................ 93
  9. 9. Issues with Linked Lists (Disadvantages).......................................................................................... 93Comparison of Linked Lists with Arrays and Dynamic Arrays........................................................... 93Singly Linked Lists ................................................................................................................................. 93 Basic Operations on a List ................................................................................................................. 94 Traversing the Linked List................................................................................................................. 94Singly Linked List Insertion .................................................................................................................. 95 Inserting a Node in Singly Linked List at the Beginning ................................................................. 95 Inserting a Node in Singly Linked List at the Ending ...................................................................... 95 Inserting a Node in Singly Linked List in the Middle...................................................................... 96Singly Linked List Deletion ................................................................................................................... 98 Deleting the First Node in Singly Linked List .................................................................................. 98 Deleting the last node in Singly Linked List .................................................................................... 98 Deleting an Intermediate Node in Singly Linked List ..................................................................... 99Deleting Singly Linked List ................................................................................................................. 100Doubly Linked Lists ............................................................................................................................. 101Doubly Linked List Insertion .............................................................................................................. 101 Inserting a Node in Doubly Linked List at the Beginning ............................................................. 101 Inserting a Node in Doubly Linked List at the Ending .................................................................. 102 Inserting a Node in Doubly Linked List in the Middle .................................................................. 102Doubly Linked List Deletion ............................................................................................................... 104 Deleting the First Node in Doubly Linked List .............................................................................. 104 Deleting the Last Node in Doubly Linked List ............................................................................... 105 Deleting an Intermediate Node in Doubly Linked List ................................................................. 105Circular Linked Lists ............................................................................................................................ 107 Counting Nodes in a Circular List ................................................................................................... 107 Printing the contents of a circular list ............................................................................................ 108 Inserting a Node at the End of a Circular Linked List ................................................................... 108 Inserting a Node at Front of a Circular Linked List ....................................................................... 110 Deleting the Last Node in a Circular List ....................................................................................... 112 Deleting the First Node in a Circular List....................................................................................... 113 Applications of Circular List............................................................................................................ 114A Memory-Efficient Doubly Linked List ........................................................................................... 114Problems on Linked Lists .................................................................................................................... 116
  10. 10. Chapter 5 Stacks ...................................................................................................................... 137 What is a Stack? ................................................................................................................................... 137 How are Stacks Used? .......................................................................................................................... 137 Stack ADT ............................................................................................................................................ 138 Main stack operations ...................................................................................................................... 138 Auxiliary stack operations ............................................................................................................... 138 Exceptions ........................................................................................................................................ 138 Applications ......................................................................................................................................... 138 Implementation.................................................................................................................................... 139 Simple Array Implementation ........................................................................................................ 139 Dynamic Array Implementation ..................................................................................................... 141 Performance ..................................................................................................................................... 144 Linked List Implementation ............................................................................................................ 144 Performance ..................................................................................................................................... 146 Comparison of Implementations ......................................................................................................... 146 Comparing Incremental Strategy and Doubling Strategy.............................................................. 146 Comparing Array Implementation and Linked List Implementation........................................... 147 Problems on Stacks .............................................................................................................................. 147Chapter 6 Queues .................................................................................................................... 170 What is a Queue? ................................................................................................................................. 170 How are Queues Used? ........................................................................................................................ 170 Queue ADT .......................................................................................................................................... 170 Main queue operations .................................................................................................................... 171 Auxiliary queue operations ............................................................................................................. 171 Exceptions ............................................................................................................................................ 171 Applications ......................................................................................................................................... 171 Direct applications ........................................................................................................................... 171 Indirect applications ........................................................................................................................ 171 Implementation.................................................................................................................................... 171 Why Circular Arrays? ...................................................................................................................... 172 Simple Circular Array Implementation .......................................................................................... 172 Performance & Limitations ............................................................................................................. 175 Dynamic Circular Array Implementation ...................................................................................... 175
  11. 11. Performance ..................................................................................................................................... 177 Linked List Implementation ............................................................................................................ 178 Performance ..................................................................................................................................... 179 Comparison of Implementations ..................................................................................................... 180 Problems on Queues ............................................................................................................................ 180Chapter 7 Trees........................................................................................................................ 184 What is a Tree? .................................................................................................................................... 184 Glossary ................................................................................................................................................ 184 Binary Trees ......................................................................................................................................... 185 Types of Binary Trees ...................................................................................................................... 186 Properties of Binary Trees ............................................................................................................... 187 Structure of Binary Trees................................................................................................................. 187 Operations on Binary Trees ............................................................................................................. 188 Applications of Binary Trees ........................................................................................................... 188 Binary Tree Traversals ......................................................................................................................... 188 Traversal Possibilities ...................................................................................................................... 189 Classifying the Traversals ................................................................................................................ 189 PreOrder Traversal .......................................................................................................................... 190 InOrder Traversal ............................................................................................................................ 191 PostOrder Traversal ......................................................................................................................... 192 Level Order Traversal ...................................................................................................................... 194 Problems on Binary Trees ............................................................................................................... 195 Generic Trees (N-ary Trees) ................................................................................................................ 219 Representation of Generic Trees ..................................................................................................... 220 Problems on Generic Trees ............................................................................................................. 221 Threaded Binary Tree Traversals [Stack or Queue less Traversals] .................................................. 227 Issues with Regular Binary Trees .................................................................................................... 227 Motivation for Threaded Binary Trees ........................................................................................... 228 Classifying Threaded Binary Trees ................................................................................................. 228 Types of Threaded Binary Trees ..................................................................................................... 228 Threaded Binary Tree structure ...................................................................................................... 229 Difference between Binary Tree and Threaded Binary Tree Structures ...................................... 229 Finding Inorder Successor in Inorder Threaded Binary Tree ....................................................... 230
  12. 12. Inorder Traversal in Inorder Threaded Binary Tree ...................................................................... 231 Finding PreOrder Successor in InOrder Threaded Binary Tree ................................................... 231 PreOrder Traversal of InOrder Threaded Binary Tree .................................................................. 232 Insertion of Nodes in InOrder Threaded Binary Trees .................................................................. 233 Problems on Threaded binary Trees ............................................................................................... 234Expression Trees .................................................................................................................................. 235 Algorithm for Building Expression Tree from Postfix Expression ................................................ 236 Example ............................................................................................................................................ 237XOR Trees ............................................................................................................................................ 238Binary Search Trees (BSTs) ................................................................................................................. 239 Why Binary Search Trees? .............................................................................................................. 239 Binary Search Tree Property ........................................................................................................... 240 Binary Search Tree Declaration ...................................................................................................... 240 Operations on Binary Search Trees................................................................................................. 240 Important Notes on Binary Search Trees ....................................................................................... 241 Finding an Element in Binary Search Trees ................................................................................... 241 Finding an Minimum Element in Binary Search Trees ................................................................. 242 Finding an Maximum Element in Binary Search Trees ................................................................. 243 Where is Inorder Predecessor and Successor? ............................................................................... 244 Inserting an Element from Binary Search Tree.............................................................................. 244 Deleting an Element from Binary Search Tree .............................................................................. 246 Problems on Binary Search Trees ................................................................................................... 248Balanced Binary Search Trees ............................................................................................................. 257 Complete Balanced Binary Search Trees ........................................................................................ 257AVL (Adelson-Velskii and Landis) trees ............................................................................................ 258 Properties of AVL Trees .................................................................................................................. 258 Minimum/Maximum Number of Nodes in AVL Tree ................................................................... 258 AVL Tree Declaration...................................................................................................................... 259 Finding Height of an AVL tree ....................................................................................................... 260 Rotations........................................................................................................................................... 260 Observation ...................................................................................................................................... 260 Types of Violations .......................................................................................................................... 260 Single Rotations ............................................................................................................................... 261
  13. 13. Double Rotations ............................................................................................................................. 263 Insertion into an AVL tree .............................................................................................................. 264 Problems on AVL Trees ................................................................................................................... 266 Other Variations in Trees .................................................................................................................... 270 Red-Black Trees ............................................................................................................................... 270 Splay Trees ....................................................................................................................................... 271 Augmented Trees ............................................................................................................................. 271 Interval Trees ................................................................................................................................... 272Chapter 8 Priority Queue and Heaps ..................................................................................... 274 What is a Priority Queue? ................................................................................................................... 274 Priority Queue ADT ............................................................................................................................ 274 Main Priority Queues Operations ................................................................................................... 274 Auxiliary Priority Queues Operations ............................................................................................ 274 Priority Queue Applications ............................................................................................................... 275 Priority Queue Implementations ........................................................................................................ 275 Unordered Array Implementation .................................................................................................. 275 Unordered List Implementation ..................................................................................................... 275 Ordered Array Implementation ...................................................................................................... 275 Ordered List Implementation.......................................................................................................... 275 Binary Search Trees Implementation ............................................................................................. 276 Balanced Binary Search Trees Implementation ............................................................................. 276 Binary Heap Implementation .......................................................................................................... 276 Comparing Implementations........................................................................................................... 276 Heaps and Binary Heap ....................................................................................................................... 276 What is a Heap? ............................................................................................................................... 276 Types of Heaps? ................................................................................................................................ 277 Binary Heaps ........................................................................................................................................ 278 Representing Heaps ......................................................................................................................... 278 Declaration of Heap ......................................................................................................................... 278 Creating Heap .................................................................................................................................. 278 Parent of a Node............................................................................................................................... 279 Children of a Node ........................................................................................................................... 279 Getting the Maximum Element ...................................................................................................... 279
  14. 14. Heapifying an Element .................................................................................................................... 280 Deleting an Element ........................................................................................................................ 281 Inserting an Element ....................................................................................................................... 282 Destroying Heap .............................................................................................................................. 284 Heapifying the Array ....................................................................................................................... 284 Heapsort ........................................................................................................................................... 285 Problems on Priority Queues [Heaps] ................................................................................................ 286Chapter 9 Disjoint Sets ADT................................................................................................... 298 Introduction ......................................................................................................................................... 298 Equivalence Relations and Equivalence Classes................................................................................. 298 Disjoint Sets ADT................................................................................................................................. 299 Applications ......................................................................................................................................... 299 Tradeoffs in Implementing Disjoint Sets ADT ................................................................................... 299 Fast FIND Implementation (Quick FIND) ..................................................................................... 300 Fast UNION Implementation (Quick UNION) .............................................................................. 300 Fast UNION implementation (Slow FIND) ........................................................................................ 300 Fast UNION implementations (Quick FIND)..................................................................................... 304 UNION by Size................................................................................................................................. 304 UNION by Height (UNION by Rank) ............................................................................................ 305 Comparing UNION by Size and UNION by Height ...................................................................... 306 Path Compression ............................................................................................................................ 306 Summary .............................................................................................................................................. 307 Problems on Disjoint Sets .................................................................................................................... 308Chapter 10 Graph Algorithms .................................................................................................. 310 Introduction ......................................................................................................................................... 310 Glossary ................................................................................................................................................ 310 Applications of Graphs ........................................................................................................................ 313 Graph Representation .......................................................................................................................... 313 Adjacency Matrix ............................................................................................................................. 313 Adjacency List .................................................................................................................................. 315 Adjacency Set ................................................................................................................................... 317 Comparison of Graph Representations ........................................................................................... 317 Graph Traversals .................................................................................................................................. 318
  15. 15. Depth First Search [DFS] ................................................................................................................. 318 Breadth First Search [BFS]............................................................................................................... 323 Comparing DFS and BFS ................................................................................................................. 325 Topological Sort ................................................................................................................................... 326 Applications of Topological Sorting ................................................................................................ 327 Shortest Path Algorithms .................................................................................................................... 327 Shortest Path in Unweighted Graph ............................................................................................... 328 Shortest path in Weighted Graph [Dijkstra’s] ................................................................................ 329 Bellman-Ford Algorithm ................................................................................................................. 333 Overview of Shortest Path Algorithms........................................................................................... 334 Minimal Spanning Tree ....................................................................................................................... 334 Prims Algorithm ............................................................................................................................. 335 Kruskal’s Algorithm ......................................................................................................................... 336 Problems on Graph Algorithms .......................................................................................................... 339Chapter 11 Sorting .................................................................................................................... 368 What is Sorting? ................................................................................................................................... 368 Why Sorting? ....................................................................................................................................... 368 Classification ........................................................................................................................................ 368 By Number of Comparisons ............................................................................................................ 368 By Number of Swaps ........................................................................................................................ 368 By Memory Usage ............................................................................................................................ 368 By Recursion .................................................................................................................................... 369 By Stability ....................................................................................................................................... 369 By Adaptability ................................................................................................................................ 369 Other Classifications ............................................................................................................................ 369 Internal Sort ..................................................................................................................................... 369 External Sort..................................................................................................................................... 369 Bubble sort ........................................................................................................................................... 369 Implementation................................................................................................................................ 370 Performance ..................................................................................................................................... 371 Selection Sort ....................................................................................................................................... 371 Algorithm ......................................................................................................................................... 371 Implementation................................................................................................................................ 371
  16. 16. Performance ..................................................................................................................................... 372Insertion sort ........................................................................................................................................ 372 Advantages ....................................................................................................................................... 372 Algorithm ......................................................................................................................................... 372 Implementation................................................................................................................................ 373 Example ............................................................................................................................................ 373 Analysis ............................................................................................................................................ 373 Performance ..................................................................................................................................... 374 Comparisons to Other Sorting Algorithms ..................................................................................... 374Shell sort ............................................................................................................................................... 375 Implementation................................................................................................................................ 375 Analysis ............................................................................................................................................ 376 Performance ..................................................................................................................................... 376Merge sort ............................................................................................................................................ 376 Important Notes ............................................................................................................................... 376 Implementation................................................................................................................................ 376 Analysis ............................................................................................................................................ 378 Performance ..................................................................................................................................... 378Heapsort ............................................................................................................................................... 378 Performance ..................................................................................................................................... 378Quicksort .............................................................................................................................................. 379 Algorithm ......................................................................................................................................... 379 Implementation................................................................................................................................ 379 Analysis ............................................................................................................................................ 380 Performance ..................................................................................................................................... 381 Randomized Quick sort ................................................................................................................... 382Tree Sort ............................................................................................................................................... 382 Performance ..................................................................................................................................... 382Comparison of Sorting Algorithms ..................................................................................................... 382Linear Sorting Algorithms ................................................................................................................... 383Counting Sort ....................................................................................................................................... 383Bucket sort [or Bin Sort] ...................................................................................................................... 384Radix sort.............................................................................................................................................. 385
  17. 17. Topological Sort ................................................................................................................................... 386 External Sorting ................................................................................................................................... 386 Problems on Sorting ............................................................................................................................ 387Chapter 12 Searching ................................................................................................................ 403 What is Searching? .............................................................................................................................. 403 Why Searching? ................................................................................................................................... 403 Types of Searching ............................................................................................................................... 403 Unordered Linear Search..................................................................................................................... 403 Sorted/Ordered Linear Search ............................................................................................................. 404 Binary Search ....................................................................................................................................... 404 Comparing Basic Searching Algorithms ............................................................................................. 406 Symbol Tables and Hashing ................................................................................................................ 406 String Searching Algorithms ............................................................................................................... 406 Problems on Searching ........................................................................................................................ 406Chapter 13 Selection Algorithms [Medians] ............................................................................ 438 What are Selection Algorithms? ......................................................................................................... 438 Selection by Sorting ............................................................................................................................. 438 Partition-based Selection Algorithm .................................................................................................. 438 Linear Selection algorithm - Median of Medians algorithm ............................................................. 438 Finding the k Smallest Elements in Sorted Order .............................................................................. 439 Problems on Selection Algorithms ..................................................................................................... 439Chapter 14 Symbol Tables ........................................................................................................ 451 Introduction ......................................................................................................................................... 451 What are Symbol Tables? .................................................................................................................... 451 Symbol Table Implementations .......................................................................................................... 452 Unordered Array Implementation .................................................................................................. 452 Ordered [Sorted] Array Implementation........................................................................................ 452 Unordered Linked List Implementation ......................................................................................... 452 Ordered Linked List Implementation ............................................................................................. 452 Binary Search Trees Implementation ............................................................................................. 452 Balanced Binary Search Trees Implementation ............................................................................. 452 Ternary Search Implementation ..................................................................................................... 453 Hashing Implementation ................................................................................................................. 453
  18. 18. Comparison of Symbol Table Implementations ................................................................................. 453Chapter 15 Hashing ................................................................................................................... 454 What is Hashing? ................................................................................................................................. 454 Why Hashing?...................................................................................................................................... 454 HashTable ADT.................................................................................................................................... 454 Understanding Hashing ....................................................................................................................... 454 If Arrays Are There Why Hashing? ................................................................................................ 455 Components in Hashing ...................................................................................................................... 456 Hash Table ............................................................................................................................................ 456 Hash Function ...................................................................................................................................... 457 How to Choose Hash Function? .......................................................................................................... 457 Characteristics of Good Hash Functions......................................................................................... 457 Load Factor ........................................................................................................................................... 457 Collisions .............................................................................................................................................. 457 Collision Resolution Techniques......................................................................................................... 458 Separate Chaining ................................................................................................................................ 458 Open Addressing .................................................................................................................................. 458 Linear Probing ................................................................................................................................. 458 Quadratic Probing............................................................................................................................ 459 Double Hashing................................................................................................................................ 460 Comparison of Collision Resolution Techniques ............................................................................... 460 Comparisons: Linear Probing vs. Double Hashing......................................................................... 460 Comparisons: Open Addressing vs. Separate Chaining ................................................................. 460 Comparisons: Open Addressing methods ...................................................................................... 461 How Hashing Gets O(1) Complexity?................................................................................................. 461 Hashing Techniques ............................................................................................................................ 461 Static Hashing .................................................................................................................................. 461 Dynamic Hashing............................................................................................................................. 462 Problems for which Hash Tables are not Suitable ............................................................................. 462 Problems on Hashing ........................................................................................................................... 462Chapter 16 String Algorithms................................................................................................... 475 Introduction ......................................................................................................................................... 475 String Matching Algorithms................................................................................................................ 475
  19. 19. Brute Force Method ............................................................................................................................. 476Robin-Karp String Matching Algorithm ............................................................................................ 476 Selecting Hash Function .................................................................................................................. 476 Step by Step explanation.................................................................................................................. 477String Matching with Finite Automata .............................................................................................. 478 Finite Automata ............................................................................................................................... 478 How Finite Automata Works? ........................................................................................................ 478 Important Notes for Constructing the Finite Automata ................................................................ 478 Matching Algorithm ........................................................................................................................ 479KMP Algorithm ................................................................................................................................... 479 Filling Prefix Table .......................................................................................................................... 480 Matching Algorithm ........................................................................................................................ 481Boyce-Moore Algorithm ..................................................................................................................... 484Data structures for Storing Strings ...................................................................................................... 484Hash Tables for Strings ........................................................................................................................ 485Binary Search Trees for Strings ........................................................................................................... 485 Issues with Binary Search Tree Representation ............................................................................. 485Tries ...................................................................................................................................................... 485 What is a Trie? ................................................................................................................................. 485 Why Tries? ....................................................................................................................................... 486 Inserting a String in Trie ................................................................................................................. 486 Searching a String in Trie ................................................................................................................ 487 Issues with Tries Representation .................................................................................................... 487Ternary Search Trees ........................................................................................................................... 488 Ternary Search Trees Declaration................................................................................................... 488 Inserting strings in Ternary Search Tree ........................................................................................ 488 Searching in Ternary Search Tree ................................................................................................... 491 Displaying All Words of Ternary Search Tree ............................................................................... 492 Finding Length of Largest Word in TST ......................................................................................... 492Comparing BSTs, Tries and TSTs ........................................................................................................ 493Suffix Trees ........................................................................................................................................... 493 Prefix and Suffix............................................................................................................................... 493 Observation ...................................................................................................................................... 493
  20. 20. What is a Suffix Tree? ...................................................................................................................... 494 The Construction of Suffix Trees .................................................................................................... 494 Applications of Suffix Trees............................................................................................................. 497 Problems on Strings ............................................................................................................................. 497Chapter 17 Complexity Classes................................................................................................. 506 Introduction ......................................................................................................................................... 506 Polynomial/exponential time .............................................................................................................. 506 What is Decision Problem? ................................................................................................................. 507 Decision Procedure .............................................................................................................................. 507 What is a Complexity Class? ............................................................................................................... 507 Types of Complexity Classes ............................................................................................................... 507 P Class ............................................................................................................................................... 507 NP Class ............................................................................................................................................ 507 Co-NP Class ...................................................................................................................................... 508 Relationship between P, NP and Co-NP ........................................................................................ 508 NP-hard Class ................................................................................................................................... 508 NP-complete Class ........................................................................................................................... 509 Relationship between P, NP Co-NP, NP-Hard and NP-Complete ............................................... 509 Does P==NP? .................................................................................................................................... 510 Reductions ............................................................................................................................................ 510 Important NP-Complete Problems (Reductions) ........................................................................... 511 Problems on Complexity Classes......................................................................................................... 513Chapter 18 Miscellaneous Concepts ......................................................................................... 516 Introduction ......................................................................................................................................... 516 Hacks on Bitwise Programming .......................................................................................................... 516 Bitwise AND..................................................................................................................................... 516 Bitwise OR........................................................................................................................................ 516 Bitwise Exclusive-OR ...................................................................................................................... 517 Bitwise Left Shift.............................................................................................................................. 517 Bitwise Right Shift ........................................................................................................................... 517 Bitwise Complement........................................................................................................................ 517 Checking whether K-th bit is set or not ......................................................................................... 517 Setting K-th bit................................................................................................................................. 518
  21. 21. Clearing K-th bit .............................................................................................................................. 518Toggling K-th bit.............................................................................................................................. 518Toggling Rightmost One bit ............................................................................................................ 519Isolating Rightmost One bit ............................................................................................................ 519Isolating Rightmost Zero bit ........................................................................................................... 519Checking Whether Number is Power of 2 not .............................................................................. 519Multiplying Number by Power of 2................................................................................................ 520Dividing Number by Power of 2 ..................................................................................................... 520Finding Modulo of a Given Number............................................................................................... 520Reversing the Binary Number ......................................................................................................... 521Counting Number of One’s in number ........................................................................................... 521Creating for Mask for Trailing Zero’s ............................................................................................. 522
  22. 22. DATA STRUCTURES AND ALGORITHMS MADE EASY
  23. 23. ©www.CareerMonk.comData Structures and Algorithms Made Easy Chapter 2 ANALYSIS OF ALGORITHMSIntroductionThe objective of this chapter is to explain the importance of analysis of algorithms, their notations,relationships and solving as many problems as possible. We first concentrate on understanding theimportance of analysis and then slowly move towards analyzing the algorithms with different notations andfinally, the problems. After completion of this chapter you should be able to find the complexity of anygiven algorithm (especially recursive functions).What is an Algorithm?Just to understand better, let us consider the problem of preparing an omelet. For preparing omelet, generalsteps which we follow are: 1) Get the frying pan. 2) Get the oil. a. Do we have oil? i. If yes, put it in the pan. ii. If no, do we want to buy oil? 1. If yes, then go out and buy. 2. If no, we can terminate. 3) Turn on the stove, etc..What we are doing is, for a given problem (preparing an omelet), giving step by step procedure for solvingit. Formal definition of an algorithm can be given as: An algorithm is the step-by-step instructions to a given problem.One important note to remember while writing the algorithms is: we do not have to prove each step of thealgorithm.Why Analysis of Algorithms?If we want to go from city to city There can be many ways of doing this: by flight, by bus, by trainand also by cycle. Depending on the availability and convenience we choose the one which suits us.Similarly, in computer science there can be multiple algorithms exist for solving the same problem (for 57 Analysis of Algorithms | Introduction
  24. 24. ©www.CareerMonk.comData Structures and Algorithms Made Easyexample, sorting problem has lot of algorithms like insertion sort, selection sort, quick sort and many more).Algorithm analysis helps us determining which of them is efficient in terms of time and space consumed.Goal of Analysis of Algorithms?The goal of is to compare algorithms (or solutions) mainly in terms of runningtime but also in terms of other factors (e.g., memory, developers effort etc.)What is Running Time Analysis?It is the process of determining how processing time increases as the size of the problem (input size)increases. Input size is number of elements in the input and depending on the problem type the input maybe of different types. In general, we encounter the following types of inputs.  Size of an array  Polynomial degree  Number of elements in a matrix  Number of bits in binary representation of the input  Vertices and edges in a graphHow to Compare Algorithms?To compare algorithms, let us define some .Execution times? as execution times are specific to a particular computer.Number of statements executed? since the number of statements varies with theprogramming language as well as the style of the individual programmer.Ideal Solution?Let us assume that we expressed running time of given algorithm as a function of the input size (i.e., ). We can compare these different functions corresponding to running times and this kind ofcomparison is independent of machine time, programming style, etc..What is Rate of Growth?The rate at which the running time increases as a function of input is called . Let us assumethat you went to a shop for buying a car and a cycle. If your friend sees you there and asks what you arebuying then in general we say This is because cost of car is too big compared to cost of cycle (approximating the cost of cycle to cost of car). Total Cost = cost_of_car + cost_of_cycle Total Cost cost_of_car (approximation) 58 Analysis of Algorithms | Goal of Analysis of Algorithms?
  25. 25. ©www.CareerMonk.comData Structures and Algorithms Made EasyFor the above example, we can represent the cost of car and cost of cycle in terms of function and for agiven function we ignore the low order terms that are relatively insignificant (for large value of input size, ). As an example in the below case, , , and are the individual costs of some function andwe approximate it to . Since, is the highest rate of growth.Commonly used Rate of GrowthsBelow diagram shows the relationship between different rates of growth. D e c r e a s i n g R a t e s O f G r o w t √ h 59 Analysis of Algorithms | Commonly used Rate of Growths
  26. 26. ©www.CareerMonk.comData Structures and Algorithms Made EasyBelow is the list of rate of growths which come across in remaining chapters. Time complexity Name Example Constant Adding an element to the front of a linked list Logarithmic Finding an element in a sorted array Linear Finding an element in an unsorted array Linear Logarithmic Sorting n items by „divide-and-conquer‟-Mergesort Quadratic Shortest path between two nodes in a graph Cubic Matrix Multiplication Exponential The Towers of Hanoi problemTypes of AnalysisIf we have an algorithm for a problem and want to know on what inputs the algorithm is taking less time(performing well) and on what inputs the algorithm is taking huge time.We have already seen that an algorithm can be represented in the form of an expression. That means werepresent the algorithm with multiple expressions: one for case where it is taking the less time and other forcase where it is taking the more time. In general the first case is called the best case and second case iscalled the worst case for the algorithm.To analyze an algorithm we need some kind of syntax and that forms the base for asymptoticanalysis/notation. There are three types of analysis: Worst case o Defines the input for which the algorithm takes huge time. o Input is the one for which the algorithm runs the slower. Best case o Defines the input for which the algorithm takes lowest time. o Input is the one for which the algorithm runs the fastest. Average case o Provides a prediction about the running time of the algorithm o Assumes that the input is randomFor a given algorithm, we can represent best case, worst case, and average case analysis in the form ofexpressions. As an example, let be the function which represents the given algorithm. , for worst case for best case 60 Analysis of Algorithms | Types of Analysis
  27. 27. ©www.CareerMonk.comData Structures and Algorithms Made EasySimilarly, for average case too. The expression defines the inputs with which the algorithm takes theaverage running time (or memory).Asymptotic Notation?Having the expressions for best case, average case and worst case, for all the three cases we need to identifythe upper bound, lower bounds. In order to represent these upper bound and lower bounds we need somesyntax and that is the subject of following discussion. Let us assume that the given algorithm is representedin the form of function .Big-O NotationThis notation gives the upper bound of the given function. Generally we represent it as . That means, at larger values of , the upper bound of is .For example, if is the given algorithm, then is . That meansgives the maximum rate of growth for at larger values of .Let us see the notation with little more detail. notation defined as there existpositive constants and such that for all . is an asymptotic tight upperbound for . Our objective is to give smallest rate of growth which is greater than or equal to givenalgorithms rate of growth .In general, we discard lower values of . That means the rate of growth at lower values of is notimportant. In the below figure, is the point from which we need to consider the rate of growths for agiven algorithm. Below the rate of growths could be different. Rate of Growth Input Size,Big-O Visualization is the set of functions with smaller or same order of growth as For example, includes etc..Note: Analyze the algorithms at larger values of only. What this means is, below we do not care forrate of growth. 61 Analysis of Algorithms | Asymptotic Notation?
  28. 28. ©www.CareerMonk.comData Structures and Algorithms Made Easy , , . .Big-O ExamplesExample-1 Find upper bound forSolution: for all ∴ = with c = 4 andExample-2 Find upper bound forSolution: for all ∴ = with andExample-3 Find upper bound forSolution: for all ∴ = with andExample-4 Find upper bound forSolution: for all ∴ = with andExample-5 Find upper bound forSolution: for all ∴ = with andExample-6 Find upper bound forSolution: for all ∴ with and 62 Analysis of Algorithms | Big-O Notation
  29. 29. ©www.CareerMonk.comData Structures and Algorithms Made EasyNo Uniqueness?There is no unique set of values for and in proving the asymptotic bounds. Let us consider, For this function there are multiple and values possible.Solution1: for all and is a solution.Solution2: for all and is also asolution.Omega-Ω NotationSimilar to discussion, this notation gives the tighter lower bound of the given algorithm and we representit as . That means, at larger values of , the tighter lower bound of isFor example, if , isThe notation can be defined as there exist positive constants and such that for all is an asymptotic tight lower bound for Our objective is togive largest rate of growth which is less than or equal to given algorithms rate of growth . Rate of Growth Input Size,Ω ExamplesExample-1 Find lower bound forSolution: Such that: and =1 ∴ with and =1Example-2 ProveSolution: Such that: 63 Analysis of Algorithms | Omega-Ω Notation
  30. 30. ©www.CareerMonk.comData Structures and Algorithms Made Easy – Since is positive – Contradiction: cannot be smaller than a constantExample-3Theta-θ NotationThis notation decides whether the upper and lower bounds of a given function (algorithm) are same or not.The average running time of algorithm is always between lower bound and upper bound. If the upperbound ( ) and lower bound ( ) gives the same result then notation will also have the same rate ofgrowth. As an example, let us assume that is the expression. Then, its tight upper bound is . The rate of growth in best case is . In this case, rate of growths in best case andworst are same. As a result, the average case will also be same. For a given function (algorithm), if the rateof growths (bounds) for and are not same then the rate of growth case may not be same. Rate of Growth Input Size,Now consider the definition of notation. It is defined as there exist positive constants and such that for all . is an asymptotic tight boundfor is the set of functions with the same order of growth asΘ ExamplesExample-1 Find θ bound forSolution: , for all, ∴ with and =1Example-2 ProveSolution: only holds for: ∴ 64 Analysis of Algorithms | Theta-θ Notation
  31. 31. ©www.CareerMonk.comData Structures and Algorithms Made EasyExample-3 Prove ≠Solution: ≤6 ≤ only holds for: ∴Example-4 ProveSolution: – ImpossibleImportant NotesFor analysis (best case, worst case and average) we try to give upper bound ( ) and lower bound ( ) andaverage running time . From the above examples, it should also be clear that, for a given function(algorithm) getting upper bound ( ) and lower bound ( ) and average running time ( ) may not be possiblealways. For example, if we are discussing the best case of an algorithm, then we try to give upper bound ( )and lower bound ( ) and average running time ( ).In the remaining chapters we generally concentrate on upper bound ( ) because knowing lower bound ( )of an algorithm is of no practical importance and we use notation if upper bound ( ) and lower bound ( )are same.Why is it called Asymptotic Analysis?From the above discussion (for all the three notations: worst case, best case and average case), we can easilyunderstand that, in every case for a given function we are trying to find other function whichapproximates at higher values of . That means, is also a curve which approximates athigher values of . In mathematics we call such curve as . In other terms, is theasymptotic curve for For this reason, we call algorithm analysis as .Guidelines for Asymptotic Analysis?There are some general rules to help us in determining the running time of an algorithm. Below are few ofthem.1) Loops: The running time of a loop is, at most, the running time of the statements inside the loop (including tests) multiplied by the number of iterations. // executes times for (i=1; i<=n; i++) { m = m + 2; // constant time, c } 65 Analysis of Algorithms | Why is it called Asymptotic Analysis?
  32. 32. ©www.CareerMonk.comData Structures and Algorithms Made Easy Total time = a constant2) Nested loops: Analyze from inside out. Total running time is the product of the sizes of all the loops. //outer loop executed n times for (i=1; i<=n; i++) { // inner loop executed n times for (j=1; j<=n; j++) { k = k+1; //constant time } } Total time3) Consecutive statements: Add the time complexities of each statement. x = x +1; //constant time // executed n times for (i=1; i<=n; i++) { m = m + 2; //constant time } //outer loop executed n times for (i=1; i<=n; i++) { //inner loop executed n times for (j=1; j<=n; j++) { k = k+1; //constant time } } Total time4) If-then-else statements: Worst-case running time: the test, plus the part or the part (whichever is the larger). //test: constant if (length ( ) != otherStack. length ( ) ) { return false; //then part: constant 66 Analysis of Algorithms | Guidelines for Asymptotic Analysis?

×