Kk20503 1 introduction

  • 860 views
Uploaded on

 

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
    Be the first to like this
No Downloads

Views

Total Views
860
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
76
Comments
0
Likes
0

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Dr James Mountstephens
  • 2. Introductions
    • I am Dr James Mountstephens and I will be teaching Algorithm Analysis (AA) this semester
    • Who are you?
    • I will be teaching in English
      • Please tell me if you don’t understand what I am saying
    • Some rules for lectures
      • Please be punctual.
        • Students more than 10 minutes late will not be allowed to enter the lecture or tutorial
      • Please, no talking .
      • Please ask questions. I will be asking you questions.
      • Please think about the material!
  • 3. About The Course
    • AA is a crucial and useful course for computer scientists
    • AA is a difficult course
      • I am going to assume that you are intelligent people.
      • I will not dumb the subject down and I will discuss complex topics. Again, please ask questions.
      • You will fail if you are lazy but each of you can pass if you work hard and really think about the material
      • The first two lectures will probably be the hardest …!
  • 4. Course Delivery
    • 14 Lectures and 14 Tutorials
    • Quizzes (5%)
      • You will be in groups, to be assigned soon
    • Group Assignment (15%), Individual Assignment (10%)
    • Midterm (20%) and Final Exam (50%)
      • All students, please sign up to Opencommittee
      • I will post lecture slides, assignments etc.
      • I will make announcements so please check regularly
      • http://www.opencommittee.com
      • Join the committee for this course ( code 102923 )
      • USE YOUR PROPER NAME!
  • 5. Lectures
    • Introduction
    • Fundamentals of the Analysis of Algorithm Efficiency
    • Brute Force
    • Divide and Conquer
    • Decrease and Conquer
    • Transform and Conquer I
    • Transform and Conquer II
    • Space and Time Tradeoffs
    • Dynamic Programming
    • Greedy Technique
    • Iterative Improvement
    • Limitations of Algorithmic Power
    • Coping with the Limitations of Algorithmic Power
    • Revision
  • 6. The Textbook
    • Please read the textbook
      • “ Introduction to The Design and Analysis of Algorithms” by Anany Levitin (Second Edition)
      • Expensive but please buy it and read it
  • 7. Dr James Mountstephens 1: Introduction
  • 8. Contents
    • What is an Algorithm?
    • Fundamentals of Algorithmic Problem Solving
    • Important Problem Types
    • Fundamental Data Structures
  • 9. What is an Algorithm?
    • An algorithm is a sequence of unambiguous instructions for solving a problem in a finite time
      • Algorithmic problem solving means obtaining a required output for any legitimate input
    “ computer” problem algorithm input output solution
  • 10. What is an Algorithm?
    • Algorithms are the core of Computer Science
      • Computers are algorithmic machines
      • EVERY piece of software that you use is an instance of an algorithm
        • Take time to think about that…Google, Facebook, MS Word etc. are all algorithms.
    • Every Computer Scientist/Software Engineer/IT person should have a toolkit of known algorithms
      • For practically doing your job!
      • Provide a framework for designing and analysing algorithms for new problems
  • 11. What is an Algorithm?
    • An algorithm is a sequence of unambiguous instructions for solving a problem in a finite time
      • Solving a problem
      • Sequence
      • Instructions
      • Non-ambiguity
      • Finite time
      • The range of inputs must be carefully specified
    • We can consider algorithms to be procedural solutions to problems
  • 12. What is an Algorithm?
    • Algorithms are not computer programs
      • Programs represent and implement algorithms
    • The same algorithm may be represented in several different ways
      • Natural language, Programming language, Pseudocode, flowcharts…
    • Several different algorithms for solving the same problem may exist
      • Each may be based on different design ideas and have very different performance
  • 13. Example Problem: Greatest Common Divisor
    • GCD of two integers m,n is the largest positive integer that divides them both with no remainder
      • m,n are nonnegative and not both zero
    • Examples:
      • gcd(60,24) = 12
      • gcd(60,0) = 60
      • gcd(57, 13) = 1
    • GCD is used in number theory, solution of equations and cryptography (RSA algorithm)
  • 14. Example: Euclid’s Algorithm for GCD
    • Euclid’s Algorithm for the GCD of two numbers is the earliest recorded algorithm (c. 300 BC)
    • It is based on repeated application of the following equality until the second number becomes 0, leaving the first number as the answer
    • gcd( m,n ) = gcd( n, m mod n )
    • Examples
      • gcd(60,24) = gcd(24,12) = gcd(12,0) = 12
      • gcd(57,13) = gcd(13,5) = gcd(5,3) = gcd(3,2) = gcd(2,1) = gcd(1,0) = 1
    m mod n is the remainder of m/n
  • 15. Two Descriptions of Euclid’s algorithm
    • while n ≠ 0 do
    • r ← m mod n
    • m← n
    • n ← r
    • return m
    Step 1 If n = 0, return m and stop; otherwise go to Step 2 Step 2 Divide m by n and assign the value of the remainder to r Step 3 Assign the value of n to m and the value of r to n. Go to Step 1.
    • Remember, the same algorithm may be represented in several different ways …
  • 16. Other Algorithms for GCD
    • Consecutive Integer Checking Algorithm
    • Step 1 Assign the value of min{m,n} to t
    • Step 2 Divide m by t. If the remainder is 0, go to Step 3; otherwise, go to Step 4
    • Step 3 Divide n by t. If the remainder is 0, return t and stop; otherwise, go to Step 4
    • Step 4 Decrease t by 1 and go to Step 2
    • Remember, several different algorithms for solving the same problem may exist
      • Each may be based on different design ideas and have very different performance…
    • Consecutive Integer Checking Algorithm for GCD
  • 17. Other Algorithms for GCD
    • Consecutive Integer Checking Algorithm
    • Step 1 Assign the value of min{m,n} to t
    • Step 2 Divide m by t. If the remainder is 0, go to Step 3; otherwise, go to Step 4
    • Step 3 Divide n by t. If the remainder is 0, return t and stop; otherwise, go to Step 4
    • Step 4 Decrease t by 1 and go to Step 2
    • Consecutive Integer Checking Algorithm example: gcd(60,24)
      • t = min(60,24) = 24
      • 60/24 = 2 remainder 12, t = 23
      • 60/23 = 2 remainder 14, t = 22
      • 60/22 = 2 remainder 16, t =21
      • 60/21 = 2 remainder 18, t=20
      • 60/20 = 3 remainder 0
      • 24/20 = 1 remainder 4, t=19
      • … ..(t = 19-16 not shown)… t=15
      • 60/15 = 4 remainder 0
      • 24/15 = 1 remainder 9
      • … ..(t = 14-13 not shown)… t = 12
      • 60/12 = 5 remainder 0
      • 24/12 = 2 remainder 0
      • gcd(24,12) = 12
    • How does the performance of this algorithm compare to Euclid’s?
      • It is clearly worse for this pair of numbers, at least
      • Soon we shall see that Euclid’s is O(log n) and this is O(n)
  • 18. Other Algorithms for GCD
    • “ Middle-school” procedure
    • Step 1 Find the prime factorisation of m
    • Step 2 Find the prime factorisation of n
    • Step 3 Find all the common prime factors
    • Step 4 Compute the product of all the common prime factors and return it as gcd (m,n )
    • Is this an algorithm?
      • Consider the non-ambiguity requirement…
    48 180 gcd = 12
  • 19. Other Algorithms for GCD
    • We need an algorithm for prime factorisation to make the middle-school procedure into an algorithm
    • We just repeatedly divide n by the prime numbers less than it, with no remainder, until the quotient becomes 1
    • But to do this prime factorisation, we need an algorithm to find all the prime numbers, up to n…
    • This can be done with the “Sieve of Eratosthenes”
  • 20. Other Algorithms for GCD
    • “ Middle-school” procedure
    • Step 1 Find the prime factorisation of m
    • Step 2 Find the prime factorisation of n
    • Step 3 Find all the common prime factors
    • Step 4 Compute the product of all the common prime factors and return it as gcd (m,n )
    Prime Factorisation Input: Integer x ≥ 2, Output: List F of prime factors of x P ← Sieve(x) while n > 1 do while n mod P[i] = 0 do F ← F + P[i] x ← x / P[i] i ← i + 1 Sieve of Eratosthenes Input: Integer x ≥ 2 Output: List of primes less than or equal to x for p ← 2 to x do A [ p ] ← p for p ← 2 to x do if A [ p ]  0 j ← p * p while j ≤ x do A [ j ] ← 0 j ← j + p
    • How does the performance of this algorithm compare to Euclid’s and Consecutive Integer Checking?
    • Note also that an algorithm will be required to find all common prime factors too…
  • 21. Fundamentals of Algorithmic Problem Solving
    • Understand the Problem
    • Ascertain the Capabilities of the Computational Device
    • Choosing between Exact and Approximate Problem Solving
    • Deciding on Appropriate Data Structures
    • Algorithm Design Techniques
    • Methods of Specifying an Algorithm
    • Proving an Algorithm’s Correctness
    • Analysing an Algorithm
    • Coding an Algorithm
    • Please read this section in the textbook in much more detail…
  • 22. Algorithm Design Techniques
    • An algorithm design technique (or “strategy” or “paradigm”) is…
      • a general approach to solving problems algorithmically that is applicable to a wide range of problems from different areas of computing
    • This course is organised by design techniques
      • See the lecture topics from earlier…
      • Usually, AA courses are organised by problem type not algorithm design technique
  • 23. Analysing an Algorithm
    • Analysis is essential. We want to know that our algorithms have good characteristics:
      • Good Time Efficiency ( or Complexity )
      • Good Space Efficiency ( or Complexity )
      • Simplicity
      • Generality
    • Analysis allows comparison of algorithms and allows us to know if they can be used practically or not
    • We will study a framework for analysis in the next lecture
  • 24. Important Problem Types
    • Sorting
    • Searching
    • String Processing
    • Graph Problems
    • Combinatorial Problems
    • Geometric Problems
    • Numerical Problems
  • 25. Sorting Problems
    • Problem: rearrange the items of a given list in ascending order .
      • Input: A sequence of n items <a 1 , a 2 , …, a n >
      • Output: A reordering <a * 1 , a * 2 , …, a * n > of the input sequence such that a * 1 ≤ a * 2 ≤ … ≤ a * n.
    • Importance of sorting
      • Can help searching tremendously
      • Algorithms often use sorting as a key subroutine.
    • Sorting key
      • A specially chosen piece of information used to guide sorting. E.g., sort student records by names.
  • 26. Sorting Problems
    • Examples of sorting algorithms
      • Selection Sort
      • Bubble sort
      • Insertion sort
      • Merge sort
      • Heap sort …and others
    • We usually evaluate sorting algorithm complexity by the number of key comparisons .
    • Two important properties
      • Stability : A sorting algorithm is called stable if it preserves the relative order of any two equal elements in its input.
      • In place : A sorting algorithm is in place if it does not require extra memory, except, possibly for a few memory units.
  • 27. Sorting Problems SelectionSort(A[0..n-1]) //Input: An array A[0..n-1] of orderable elements //Output: Array A[0..n-1] sorted in ascending order for i ← 0 to n – 2 do min ← i for j ← i + 1 to n – 1 do if A[j] < A[min] min ← j swap A[i] and A[min]
    • An example of a simple (and often low-performance) sorting algorithm is Selection Sort
  • 28. Searching Problems
    • Problem: find a given value, called a search key, in a given set.
      • Eg ID, ref number, name, etc
    • Search is a hugely important problem
      • Retrieving data from database storage
        • Student records, company accounts etc.
    • And more general search…
      • Searching the web for information
      • Searching to solve problems in AI
  • 29. Searching Problems
    • Examples of searching algorithms
      • Sequential search (Time O(n))
      • Binary search (Time: O(log n))
    Binary Search // Input: sorted array a_i < … < a_j and key x; m  (i+j)/2; while i < j and x != a_m do if x < a_m then j  m-1 else i  m+1; if x = a_m then output a_m;
  • 30. String Processing Problems
    • A string is a sequence of characters from an alphabet.
      • “ This is a string” is a string
    • Text strings: letters, numbers, and special characters.
    • String matching: searching for a given word/pattern in a text.
    • Examples
      • searching for a word or phrase on WWW or in a Word document
      • searching for a particular sequence of {A,C,G,T} in a reference genomic sequence
  • 31. Graph Problems
    • (Informally) A graph is a collection of points called vertices , some of which are connected by line segments called edges .
    • Graphs are extremely important for modeling real-life problems
      • Modeling WWW
      • Social relationships
      • Communication networks
      • Project scheduling …
    • Examples of graph algorithms
      • Graph traversal algorithms
      • Shortest-path algorithms
      • Topological sorting
    1 2 3 4
  • 32. Graph Problems
    • Example: Travelling Salesman Problem
      • For a set of cities linked by roads, find the shortest route that visits them all only once.
      • Important to
        • Businesses and route planning
        • Circuit board and VLSI design and manufacture
        • X Ray crystallography
        • Genetic engineering
    • TSP, like many other graph problems, is extremely difficult and no general efficient algorithms are known
      • In fact, they may not exist…
  • 33. Combinatorial Problems
    • Finding a Combinatorial Object that satisfies certain constraints or has certain properties
      • Sets, subsets
      • Permutations
        • Set P of all ordered sequences of length r from a set S of n elements
        • Eg. for S = {1,2,3} and r=3, P = {(1,2,3), (1,3,2), (2,1,3),(2,3,1), (3,1,2), (3,2,1)}
        • Eg. for S = {1,2,3} and r=2, P = {(1,2), (1,3), (2,1),(2,3), (3,1), (3,2)}
        • The number of permutations is n P r = n!/(n-r)!
      • Combinations
        • Set C of all unordered sequences of length r from a set S of n elements
        • Eg. for S = {1,2,3} and r=2, C = {(1,2), (1,3), (2,3),}
        • Eg. for S = {1,2,3} and r=3, C = {(1,2,3)}
        • The number of combinations is n C r = n!/(n-r)! r!
    • Combinatorial problems are among the hardest known
  • 34. Combinatorial Problems
    • Example: The Knapsack Problem
      • Given a set of items, each with a weight and a value, determine the most valuable subset of items with total weight less than or equal to a given limit
  • 35. Geometric Problems
    • Problems dealing with Geometric Objects such as points, lines, polygons
    • Examples
      • Finding closest pair of points
      • Finding convex hull of a set of points
  • 36. Numerical Problems
    • Problems dealing with mathematical objects
      • Numbers, Equations, Matrices…
    • Examples
      • Efficient arithmetic
      • Evaluating functions
      • Solving systems of linear equations with Gaussian Elimination
      • Evaulating Definite Integrals
      • Optimising a function subject to constraints
  • 37. Fundamental Data Structures
    • Linear Data Structures
      • Array
      • String
      • Linked list
      • Stack
      • Queue
        • Priority queue/heap
    • Graph
      • General Graph
      • Tree
        • Binary Tree
    • Set and Dictionary
  • 38. Linear Data Structures
    • Arrays
      • A sequence of n items of the same data type that are stored contiguously in computer memory and made accessible by specifying a value of the array’s index.
    • Array properties
      • fixed length (need preliminary reservation of memory)
      • contiguous memory locations
      • direct access
      • Operations of insert/delete
      • Often used to implement Lists and Strings
  • 39. Linear Data Structures
    • Linked List
      • A sequence of zero or more nodes each containing two kinds of information: some data and one or more links called pointers to other nodes of the linked list.
      • Singly linked list (next pointer)
      • Doubly linked list (next + previous pointers)
    • Linked List Properties
      • dynamic length
      • arbitrary memory locations
      • access by following links
      • Operations of Insert/delete
  • 40. Stacks and Queues
    • Stacks
      • A stack of plates
        • insertion/deletion can be done only at the top.
        • LIFO
      • Two operations (push and pop)
    • Queues
      • A queue of customers waiting for services
        • Insertion/enqueue from the rear and deletion/dequeue from the front.
        • FIFO
      • Two operations (enqueue and dequeue)
  • 41. Priority Queues and Heaps
    • Priority queues (implemented using heaps)
      • A data structure for maintaining a set of elements, each associated with a key/priority, with the following operations
    • Operations
      • Finding the element with the highest priority
      • Deleting the element with the highest priority
      • Inserting a new element
    9 6 8 5 2 3 9 6 5 8 2 3
  • 42. Graphs
    • Formal definition
      • A graph G = <V, E> is defined by a pair of sets: a finite set V of items called vertices and a set E of vertex pairs called edges.
    • Undirected and directed graphs (digraphs).
      • What’s the maximum number of edges in an undirected graph with |V| vertices?
    • Complete graphs
      • A graph with every pair of its vertices connected by an edge is called complete, K |V|
    • Dense and sparse graphs
      • Dense graphs have edges between most nodes
      • Sparse graphs have few edges between nodes
    Complete, undirected graph Incomplete, directed graph Both graphs here are dense 1 2 3 4 1 2 3 4
  • 43. Graph Representation
    • Nodes are said to be adjacent if an edge exists between them
    • Adjacency Matrix
      • n x n boolean matrix if number of nodes |V| is n.
      • The element on the ith row and jth column is 1 if there’s an edge from ith vertex to the jth vertex; otherwise 0.
      • The adjacency matrix of an undirected graph is symmetric.
    • Adjacency Linked Lists
      • A collection of linked lists, one for each vertex, that contain all the vertices adjacent to the list’s vertex.
    • Which data structure would you use for dense and sparse graphs?
    0 1 1 1 0 0 0 1 0 0 0 1 0 0 0 0 Incomplete, directed graph G Adjacency Matrix for G Adjacency Linked List for G 2 3 4 4 4 1 2 3 4
  • 44. Weighted Graphs
    • Weighted graphs
      • Graphs or digraphs with numbers assigned to the edges.
    6 8 5 7 9 1 2 3 4
  • 45. Graph Properties -- Paths and Connectivity
    • Paths
      • A path from vertex u to v of a graph G is defined as a sequence of adjacent vertices that starts with u and ends with v.
      • Simple paths: All edges of a path are distinct.
      • Path lengths: the number of edges, or the number of vertices – 1.
    Simple Path from 4 to 3 (4,1,2,3) Path Length 3 Path from 4 to 3 (4,1,2,5,4,1,2,3) Path Length 7 1 2 4 5 3 1 2 4 5 3
  • 46. Graph Properties -- Paths and Connectivity
    • Connected graphs
      • A graph is said to be connected if for every pair of its vertices u and v there is a path from u to v.
    • Connected component
      • The maximum connected subgraph of a given graph .
    Connected Graph G1 Non-Connected Graph G2 1 2 4 5 3 1 2 4 5 3
  • 47. Graph Properties -- Acyclicity
    • Cycle
      • A simple path of a positive length that starts and ends a the same vertex.
    • Acyclic graph
      • A graph without cycles
      • DAG (Directed Acyclic Graph)
    Directed Acyclic Graph Cycle (1,2,5,4,1) 1 2 3 4 1 2 4 5 3
  • 48. Trees
    • Trees
      • A tree (or free tree) is a connected acyclic graph.
      • Forest: a graph that has no cycles but is not necessarily connected.
    • Properties of trees
      • |E| = |V| - 1
      • For every two vertices in a tree there always exists exactly one simple path from one of these vertices to the other….
    Free Tree Free Tree that is also a forest 1 3 2 4 5 1 3 2 4 5 6 7
  • 49. Rooted Trees
    • This “single path” property makes it possible to select an arbitrary vertex in a free tree and consider it as the root of the so-called rooted tree .
    • A root then defines levels in the tree.
      • Number of edges from the root
    Free Tree Rooted Tree with 2 levels 1 3 2 4 5 1 3 2 4 5
  • 50. Rooted Trees
    • Ancestors
      • For any vertex v in a tree T , all the vertices on the simple path from the root to that vertex are called ancestors.
    • Descendants
      • All the vertices for which a vertex v is an ancestor are said to be descendants of v .
    • Parent, child and siblings
      • If (u, v) is the last edge of the simple path from the root to vertex v , u is said to be the parent of v and v is called a child of u .
      • Vertices that have the same parent are called siblings.
    • Leaves
      • A vertex without children is called a leaf.
    • Subtree
      • A vertex v with all its descendants is called the subtree of T rooted at v.
    • Examples:
    • 1 is an ancestor of 6 and 7
    • 8 is a descendant of 5
    • 1 is parent (and ancestor) of 6 and 7
    • 8 and 9 are children (and descendants) of 5
    • 6 and 7 are siblings
    • 8 and 9 are siblings
    • 4,6,7,8,9 are leaves
    • 5 is the subtree consisting of 5,8,9
    1 3 2 4 5 7 6 8 9
  • 51. Rooted Trees
    • Depth of a vertex
      • The length of the simple path from the root to the vertex.
    • Height of a tree
      • The length of the longest simple path from the root to a leaf.
    • Examples:
    • Vertex 2 has depth d=2
    • Vertex 7 has depth d=3
    • The tree itself has height h=3
    1 3 2 4 5 7 6 8 9
  • 52. Ordered Trees
    • Ordered trees
      • An ordered tree is a rooted tree in which all the children of each vertex are ordered.
    • Binary trees
      • A binary tree is an ordered tree in which every vertex has no more than two children and each children is designated s either a left child or a right child of its parent.
    Binary Tree L R L 9 6 8 5 2 3
  • 53. Ordered Trees
    • Binary search trees
      • Each vertex is assigned a number.
      • A number assigned to each parental vertex is larger than all the numbers in its left subtree and smaller than all the numbers in its right subtree.
      •  log 2 n   h  n – 1, where h is the height of a binary tree and n the size.
    6 3 9 2 5 8
  • 54. Sets and Dictionaries
    • I will assume you already know what a set is!
      • Elements, subsets, union, intersection etc..
    • Dictionaries are Sets with 3 operations:
      • Searching for a given element
      • Inserting a new element
      • Deleting an existing element
    • Please read the textbook for more detail…