Algorithms Complexity and Data                              Structures Efficiency           Computational Complexity, Choo...
Table of Contents1.   Algorithms Complexity and Asymptotic     Notation      Time and Memory Complexity      Mean, Avera...
Why Data Structures are                             Important? Data structures and algorithms                            ...
Algorithms Complexity     Asymtotic Notation
Algorithm Analysis Why we should analyze    algorithms?  Predict the resources that the algorithm   requires    Computa...
Algorithmic Complexity What to measure?  CPU Time  Memory  Number of steps  Number of particular operations    Numbe...
Time Complexity Worst-case  An upper bound on the running time for any   input of given size Average-case  Assume all ...
Time Complexity – Example Sequential search in a list   of size n   Worst-case:    n comparisons                       ...
Algorithms Complexity   Algorithm complexity is rough estimation of the    number of steps performed by given computation...
Asymptotic Notation: Definition   Asymptotic upper bound     O-notation (Big O notation)   For given function g(n), we ...
Typical ComplexitiesComplexity Notation             Description                     Constant number of                    ...
Typical Complexities (2)Complexity Notation             Description                       Number of operations            ...
Time Complexity and SpeedComplexity     10     20    50    100 1 000 10 000 100 000   O(1)        <1s   <1s   <1s    <1s  ...
Time and Memory Complexity   Complexity can be expressed as formula on    multiple variables, e.g.     Algorithm filling...
The Hidden Constant Sometime a linearalgorithm could be slower than quadratic algorithm  The hidden constant should not ...
Polynomial Algorithms A polynomial-time algorithm  is one whose worst-case time complexity is bounded above by a polynomi...
Analyzing Complexity   of Algorithms       Examples
Complexity Examples     int FindMaxElement(int[] array)     {         int max = array[0];         for (int i=0; i<array.le...
Complexity Examples (2)    long FindInversions(int[] array)    {        long inversions = 0;        for (int i=0; i<array....
Complexity Examples (3)    decimal Sum3(int n)    {        decimal sum = 0;        for (int a=0; a<n; a++)            for ...
Complexity Examples (4)    long SumMN(int n, int m)    {        long sum = 0;        for (int x=0; x<n; x++)            fo...
Complexity Examples (5)    long SumMN(int n, int m)    {        long sum = 0;        for (int x=0; x<n; x++)            fo...
Complexity Examples (6)    decimal Calculation(int n)    {        decimal result = 0;        for (int i = 0; i < (1<<n); i...
Complexity Examples (7)     decimal Factorial(int n)     {         if (n==0)             return 1;         else           ...
Complexity Examples (8)  decimal Fibonacci(int n)  {      if (n == 0)          return 1;      else if (n == 1)          re...
Comparing Data Structures          Examples
Data Structures Efficiency                                          Get-by- Data Structure        Add    Find Delete      ...
Data Structures Efficiency (2)                                             Get-by- Data Structure      Add      Find    De...
Choosing Data Structure Arrays   (T[])   Use when fixed number of elements should be    processed by index Resizable ar...
Choosing Data Structure (2)   Stacks (Stack<T>)     Use to implement LIFO (last-in-first-out) behavior     List<T> coul...
Choosing Data Structure (3)   Balanced search tree based dictionary    (SortedDictionary<K,T>)     Use when key-value pa...
Summary   Algorithm complexity is rough estimation of the    number of steps performed by given computation     Complexi...
Algorithms Complexity and Data                                                                                            ...
Exercises1.   A text file students.txt holds information about     students and their courses in the following format:    ...
Exercises (2)2.   A large trade company has millions of articles, each     described by barcode, vendor, title and price. ...
Exercises (3)5.   A text file phones.txt holds information about     people, their town and phone number:      Mimi Shmatk...
Free Trainings @ Telerik Academy Fundamentals of C# Programming    Course       csharpfundamentals.telerik.com   Teleri...
Upcoming SlideShare
Loading in...5
×

24. Algorithms Complexity and Data Structures Efficiency - C# Fundamentals

5,779

Published on

This article discuss the Algorithms Complexity and Data Structures Efficiency.
Understanding the Algorithms Complexity and Asyptotic notation.
Time and Memory Complexity and few Fundamental Data Structures – Comparisons.
Telerik Software Academy: http://www.academy.telerik.com
The website and all video materials are in Bulgarian

Algorithms Complexity and Asymptotic Notation
Time and Memory Complexity
Mean, Average and Worst Case
Fundamental Data Structures – Comparison
Arrays vs. Lists vs. Trees vs. Hash-Tables
Choosing Proper Data Structure

C# Programming Fundamentals Course @ Telerik Academy
http://academy.telerik.com

Published in: Education, Technology
2 Comments
15 Likes
Statistics
Notes
No Downloads
Views
Total Views
5,779
On Slideshare
0
From Embeds
0
Number of Embeds
5
Actions
Shares
0
Downloads
365
Comments
2
Likes
15
Embeds 0
No embeds

No notes for slide

24. Algorithms Complexity and Data Structures Efficiency - C# Fundamentals

  1. 1. Algorithms Complexity and Data Structures Efficiency Computational Complexity, Choosing Data StructuresSvetlin NakovManager Technical Trainerhttp://www.nakov.com/Telerik Software Academyhttp://academy.telerik.com/
  2. 2. Table of Contents1. Algorithms Complexity and Asymptotic Notation  Time and Memory Complexity  Mean, Average and Worst Case2. Fundamental Data Structures – Comparison  Arrays vs. Lists vs. Trees vs. Hash-Tables3. Choosing Proper Data Structure 2
  3. 3. Why Data Structures are Important? Data structures and algorithms are the foundation of computer programming Algorithmic thinking, problem solving and data structures are vital for software engineers  All .NET developers should know when to use T[], LinkedList<T>, List<T>, Stack<T>, Queue<T>, Dictionary<K,T>, HashSet<T>, SortedDictionary<K,T> and SortedSet<T> Computational complexity is important for algorithm design and efficient programming 3
  4. 4. Algorithms Complexity Asymtotic Notation
  5. 5. Algorithm Analysis Why we should analyze algorithms?  Predict the resources that the algorithm requires  Computational time (CPU consumption)  Memory space (RAM consumption)  Communication bandwidth consumption  The running time of an algorithm is:  The total number of primitive operations executed (machine independent steps)  Also known as algorithm complexity 5
  6. 6. Algorithmic Complexity What to measure?  CPU Time  Memory  Number of steps  Number of particular operations  Number of disk operations  Number of network packets  Asymptotic complexity 6
  7. 7. Time Complexity Worst-case  An upper bound on the running time for any input of given size Average-case  Assume all inputs of a given size are equally likely Best-case  The lower bound on the running time 7
  8. 8. Time Complexity – Example Sequential search in a list of size n  Worst-case:  n comparisons … … … … … … …  Best-case:  1 comparison n  Average-case:  n/2 comparisons The algorithm runs in linear time  Linear number of operations 8
  9. 9. Algorithms Complexity Algorithm complexity is rough estimation of the number of steps performed by given computation depending on the size of the input data  Measured through asymptotic notation  O(g) where g is a function of the input data size  Examples:  Linear complexity O(n) – all elements are processed once (or constant number of times)  Quadratic complexity O(n2) – each of the elements is processed n times 9
  10. 10. Asymptotic Notation: Definition Asymptotic upper bound  O-notation (Big O notation) For given function g(n), we denote by O(g(n)) the set of functions that are different than g(n) by a constant O(g(n)) = {f(n): there exist positive constants c and n0 such that f(n) <= c*g(n) for all n >= n0} Examples:  3 * n2 + n/2 + 12 ∈ O(n2)  4*n*log2(3*n+1) + 2*n-1 ∈ O(n * log n) 10
  11. 11. Typical ComplexitiesComplexity Notation Description Constant number of operations, not depending on constant O(1) the input data size, e.g. n = 1 000 000  1-2 operations Number of operations propor- tional of log2(n) where n is thelogarithmic O(log n) size of the input data, e.g. n = 1 000 000 000  30 operations Number of operations proportional to the input data linear O(n) size, e.g. n = 10 000  5 000 operations 11
  12. 12. Typical Complexities (2)Complexity Notation Description Number of operations proportional to the square of quadratic O(n2) the size of the input data, e.g. n = 500  250 000 operations Number of operations propor- tional to the cube of the size cubic O(n3) of the input data, e.g. n = 200  8 000 000 operations O(2n), Exponential number ofexponential O(kn), operations, fast growing, e.g. O(n!) n = 20  1 048 576 operations 12
  13. 13. Time Complexity and SpeedComplexity 10 20 50 100 1 000 10 000 100 000 O(1) <1s <1s <1s <1s <1s <1s <1s O(log(n)) <1s <1s <1s <1s <1s <1s <1s O(n) <1s <1s <1s <1s <1s <1s <1sO(n*log(n)) <1s <1s <1s <1s <1s <1s <1s O(n2) <1s <1s <1s <1s <1s 2s 3-4 min O(n3) <1s <1s <1s <1s 20 s 5 hours 231 days 260 O(2n) <1s <1s hangs hangs hangs hangs days O(n!) <1s hangs hangs hangs hangs hangs hangs O(nn) 3-4 min hangs hangs hangs hangs hangs hangs 13
  14. 14. Time and Memory Complexity Complexity can be expressed as formula on multiple variables, e.g.  Algorithm filling a matrix of size n * m with natural numbers 1, 2, … will run in O(n*m)  DFS traversal of graph with n vertices and m edges will run in O(n + m) Memory consumption should also be considered, for example:  Running time O(n), memory requirement O(n2)  n = 50 000  OutOfMemoryException 14
  15. 15. The Hidden Constant Sometime a linearalgorithm could be slower than quadratic algorithm  The hidden constant should not always be ignored Example:  Algorithm A makes: 100*n steps  O(n)  Algorithm B makes: n*n/2 steps  O(n2)  For n < 200 algorithm B is faster 15
  16. 16. Polynomial Algorithms A polynomial-time algorithm is one whose worst-case time complexity is bounded above by a polynomial function of its input size W(n) ∈ O(p(n)) Example of worst-case time complexity  Polynomial-time: log n, 2n, 3n3 + 4n, 2 * n log n  Non polynomial-time : 2n, 3n, nk, n! Non-polynomial algorithms dont work for large input data sets 16
  17. 17. Analyzing Complexity of Algorithms Examples
  18. 18. Complexity Examples int FindMaxElement(int[] array) { int max = array[0]; for (int i=0; i<array.length; i++) { if (array[i] > max) { max = array[i]; } } return max; } Runs in O(n) where n is the size of the array The number of elementary steps is ~n
  19. 19. Complexity Examples (2) long FindInversions(int[] array) { long inversions = 0; for (int i=0; i<array.Length; i++) for (int j = i+1; j<array.Length; i++) if (array[i] > array[j]) inversions++; return inversions; } Runs in O(n2) where n is the size of the array The number of elementary steps is ~ n*(n+1) / 2
  20. 20. Complexity Examples (3) decimal Sum3(int n) { decimal sum = 0; for (int a=0; a<n; a++) for (int b=0; b<n; b++) for (int c=0; c<n; c++) sum += a*b*c; return sum; } Runs in cubic time O(n3) The number of elementary steps is ~ n3
  21. 21. Complexity Examples (4) long SumMN(int n, int m) { long sum = 0; for (int x=0; x<n; x++) for (int y=0; y<m; y++) sum += x*y; return sum; } Runs in quadratic time O(n*m) The number of elementary steps is ~ n*m
  22. 22. Complexity Examples (5) long SumMN(int n, int m) { long sum = 0; for (int x=0; x<n; x++) for (int y=0; y<m; y++) if (x==y) for (int i=0; i<n; i++) sum += i*x*y; return sum; } Runs in quadratic time O(n*m) The number of elementary steps is ~ n*m + min(m,n)*n
  23. 23. Complexity Examples (6) decimal Calculation(int n) { decimal result = 0; for (int i = 0; i < (1<<n); i++) result += i; return result; } Runs in exponential time O(2n) The number of elementary steps is ~ 2n
  24. 24. Complexity Examples (7) decimal Factorial(int n) { if (n==0) return 1; else return n * Factorial(n-1); } Runs in linear time O(n) The number of elementary steps is ~n
  25. 25. Complexity Examples (8) decimal Fibonacci(int n) { if (n == 0) return 1; else if (n == 1) return 1; else return Fibonacci(n-1) + Fibonacci(n-2); } Runs in exponential time O(2n) The number of elementary steps is ~ Fib(n+1) where Fib(k) is the k-th Fibonaccis number
  26. 26. Comparing Data Structures Examples
  27. 27. Data Structures Efficiency Get-by- Data Structure Add Find Delete index Array (T[]) O(n) O(n) O(n) O(1) Linked list O(1) O(n) O(n) O(n)(LinkedList<T>)Resizable array list O(1) O(n) O(n) O(1) (List<T>)Stack (Stack<T>) O(1) - O(1) -Queue (Queue<T>) O(1) - O(1) - 27
  28. 28. Data Structures Efficiency (2) Get-by- Data Structure Add Find Delete index Hash table O(1) O(1) O(1) -(Dictionary<K,T>) Tree-baseddictionary (Sorted O(log n) O(log n) O(log n) -Dictionary<K,T>) Hash table based O(1) O(1) O(1) - set (HashSet<T>) Tree based set O(log n) O(log n) O(log n) - (SortedSet<T>) 28
  29. 29. Choosing Data Structure Arrays (T[])  Use when fixed number of elements should be processed by index Resizable array lists (List<T>)  Use when elements should be added and processed by index Linked lists (LinkedList<T>)  Use when elements should be added at the both sides of the list  Otherwise use resizable array list (List<T>) 29
  30. 30. Choosing Data Structure (2) Stacks (Stack<T>)  Use to implement LIFO (last-in-first-out) behavior  List<T> could also work well Queues (Queue<T>)  Use to implement FIFO (first-in-first-out) behavior  LinkedList<T> could also work well Hash table based dictionary (Dictionary<K,T>)  Use when key-value pairs should be added fast and searched fast by key  Elements in a hash table have no particular order 30
  31. 31. Choosing Data Structure (3) Balanced search tree based dictionary (SortedDictionary<K,T>)  Use when key-value pairs should be added fast, searched fast by key and enumerated sorted by key Hash table based set (HashSet<T>)  Use to keep a group of unique values, to add and check belonging to the set fast  Elements are in no particular order Search tree based set (SortedSet<T>)  Use to keep a group of ordered unique values 31
  32. 32. Summary Algorithm complexity is rough estimation of the number of steps performed by given computation  Complexity can be logarithmic, linear, n log n, square, cubic, exponential, etc.  Allows to estimating the speed of given code before its execution Different data structures have different efficiency on different operations  The fastest add / find / delete structure is the hash table – O(1) for all these operations 32
  33. 33. Algorithms Complexity and Data Structures Efficiency курсове и уроци по програмиране, уеб дизайн – безплатно BG Coder - онлайн състезателна система - online judge курсове и уроци по програмиране – Телерик академия форум програмиране, форум уеб дизайн уроци по програмиране и уеб дизайн за ученици ASP.NET курс - уеб програмиране, бази данни, C#, .NET, ASP.NET http://academy.telerik.com програмиране за деца – безплатни курсове и уроци ASP.NET MVC курс – HTML, SQL, C#, .NET, ASP.NET MVC безплатен SEO курс - оптимизация за търсачки алго академия – състезателно програмиране, състезаниякурсове и уроци по програмиране, книги – безплатно от Наков курс мобилни приложения с iPhone, Android, WP7, PhoneGap уроци по уеб дизайн, HTML, CSS, JavaScript, Photoshop Дончо Минков - сайт за програмиране free C# book, безплатна книга C#, книга Java, книга C# Николай Костов - блог за програмиране безплатен курс "Качествен програмен код" безплатен курс "Разработка на софтуер в cloud среда" C# курс, програмиране, безплатно
  34. 34. Exercises1. A text file students.txt holds information about students and their courses in the following format: Kiril | Ivanov | C# Stefka | Nikolova | SQL Stela | Mineva | Java Milena | Petrova | C# Ivan | Grigorov | C# Ivan | Kolev | SQL Using SortedDictionary<K,T> print the courses in alphabetical order and for each of them prints the students ordered by family and then by name: C#: Ivan Grigorov, Kiril Ivanov, Milena Petrova Java: Stela Mineva SQL: Ivan Kolev, Stefka Nikolova 34
  35. 35. Exercises (2)2. A large trade company has millions of articles, each described by barcode, vendor, title and price. Implement a data structure to store them that allows fast retrieval of all articles in given price range [x…y]. Hint: use OrderedMultiDictionary<K,T> from Wintellects Power Collections for .NET.3. Implement a data structure PriorityQueue<T> that provides a fast way to execute the following operations: add element; extract the smallest element.4. Implement a class BiDictionary<K1,K2,T> that allows adding triples {key1, key2, value} and fast search by key1, key2 or by both key1 and key2. Note: multiple values can be stored for given key. 35
  36. 36. Exercises (3)5. A text file phones.txt holds information about people, their town and phone number: Mimi Shmatkata | Plovdiv | 0888 12 34 56 Kireto | Varna | 052 23 45 67 Daniela Ivanova Petrova | Karnobat | 0899 999 888 Bat Gancho | Sofia | 02 946 946 946 Duplicates can occur in people names, towns and phone numbers. Write a program to execute a sequence of commands from a file commands.txt:  find(name) – display all matching records by given name (first, middle, last or nickname)  find(name, town) – display all matching records by given name and town 36
  37. 37. Free Trainings @ Telerik Academy Fundamentals of C# Programming Course  csharpfundamentals.telerik.com Telerik Software Academy  academy.telerik.com Telerik Academy @ Facebook  facebook.com/TelerikAcademy Telerik Software Academy Forums  forums.academy.telerik.com
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×