Kk20503 1 introduction
Upcoming SlideShare
Loading in...5
×
 

Kk20503 1 introduction

on

  • 1,032 views

 

Statistics

Views

Total Views
1,032
Views on SlideShare
1,032
Embed Views
0

Actions

Likes
0
Downloads
74
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Kk20503 1 introduction Kk20503 1 introduction Presentation Transcript

  • Dr James Mountstephens
  • Introductions
    • I am Dr James Mountstephens and I will be teaching Algorithm Analysis (AA) this semester
    • Who are you?
    • I will be teaching in English
      • Please tell me if you don’t understand what I am saying
    • Some rules for lectures
      • Please be punctual.
        • Students more than 10 minutes late will not be allowed to enter the lecture or tutorial
      • Please, no talking .
      • Please ask questions. I will be asking you questions.
      • Please think about the material!
  • About The Course
    • AA is a crucial and useful course for computer scientists
    • AA is a difficult course
      • I am going to assume that you are intelligent people.
      • I will not dumb the subject down and I will discuss complex topics. Again, please ask questions.
      • You will fail if you are lazy but each of you can pass if you work hard and really think about the material
      • The first two lectures will probably be the hardest …!
  • Course Delivery
    • 14 Lectures and 14 Tutorials
    • Quizzes (5%)
      • You will be in groups, to be assigned soon
    • Group Assignment (15%), Individual Assignment (10%)
    • Midterm (20%) and Final Exam (50%)
      • All students, please sign up to Opencommittee
      • I will post lecture slides, assignments etc.
      • I will make announcements so please check regularly
      • http://www.opencommittee.com
      • Join the committee for this course ( code 102923 )
      • USE YOUR PROPER NAME!
  • Lectures
    • Introduction
    • Fundamentals of the Analysis of Algorithm Efficiency
    • Brute Force
    • Divide and Conquer
    • Decrease and Conquer
    • Transform and Conquer I
    • Transform and Conquer II
    • Space and Time Tradeoffs
    • Dynamic Programming
    • Greedy Technique
    • Iterative Improvement
    • Limitations of Algorithmic Power
    • Coping with the Limitations of Algorithmic Power
    • Revision
  • The Textbook
    • Please read the textbook
      • “ Introduction to The Design and Analysis of Algorithms” by Anany Levitin (Second Edition)
      • Expensive but please buy it and read it
  • Dr James Mountstephens 1: Introduction
  • Contents
    • What is an Algorithm?
    • Fundamentals of Algorithmic Problem Solving
    • Important Problem Types
    • Fundamental Data Structures
  • What is an Algorithm?
    • An algorithm is a sequence of unambiguous instructions for solving a problem in a finite time
      • Algorithmic problem solving means obtaining a required output for any legitimate input
    “ computer” problem algorithm input output solution
  • What is an Algorithm?
    • Algorithms are the core of Computer Science
      • Computers are algorithmic machines
      • EVERY piece of software that you use is an instance of an algorithm
        • Take time to think about that…Google, Facebook, MS Word etc. are all algorithms.
    • Every Computer Scientist/Software Engineer/IT person should have a toolkit of known algorithms
      • For practically doing your job!
      • Provide a framework for designing and analysing algorithms for new problems
  • What is an Algorithm?
    • An algorithm is a sequence of unambiguous instructions for solving a problem in a finite time
      • Solving a problem
      • Sequence
      • Instructions
      • Non-ambiguity
      • Finite time
      • The range of inputs must be carefully specified
    • We can consider algorithms to be procedural solutions to problems
  • What is an Algorithm?
    • Algorithms are not computer programs
      • Programs represent and implement algorithms
    • The same algorithm may be represented in several different ways
      • Natural language, Programming language, Pseudocode, flowcharts…
    • Several different algorithms for solving the same problem may exist
      • Each may be based on different design ideas and have very different performance
  • Example Problem: Greatest Common Divisor
    • GCD of two integers m,n is the largest positive integer that divides them both with no remainder
      • m,n are nonnegative and not both zero
    • Examples:
      • gcd(60,24) = 12
      • gcd(60,0) = 60
      • gcd(57, 13) = 1
    • GCD is used in number theory, solution of equations and cryptography (RSA algorithm)
  • Example: Euclid’s Algorithm for GCD
    • Euclid’s Algorithm for the GCD of two numbers is the earliest recorded algorithm (c. 300 BC)
    • It is based on repeated application of the following equality until the second number becomes 0, leaving the first number as the answer
    • gcd( m,n ) = gcd( n, m mod n )
    • Examples
      • gcd(60,24) = gcd(24,12) = gcd(12,0) = 12
      • gcd(57,13) = gcd(13,5) = gcd(5,3) = gcd(3,2) = gcd(2,1) = gcd(1,0) = 1
    m mod n is the remainder of m/n
  • Two Descriptions of Euclid’s algorithm
    • while n ≠ 0 do
    • r ← m mod n
    • m← n
    • n ← r
    • return m
    Step 1 If n = 0, return m and stop; otherwise go to Step 2 Step 2 Divide m by n and assign the value of the remainder to r Step 3 Assign the value of n to m and the value of r to n. Go to Step 1.
    • Remember, the same algorithm may be represented in several different ways …
  • Other Algorithms for GCD
    • Consecutive Integer Checking Algorithm
    • Step 1 Assign the value of min{m,n} to t
    • Step 2 Divide m by t. If the remainder is 0, go to Step 3; otherwise, go to Step 4
    • Step 3 Divide n by t. If the remainder is 0, return t and stop; otherwise, go to Step 4
    • Step 4 Decrease t by 1 and go to Step 2
    • Remember, several different algorithms for solving the same problem may exist
      • Each may be based on different design ideas and have very different performance…
    • Consecutive Integer Checking Algorithm for GCD
  • Other Algorithms for GCD
    • Consecutive Integer Checking Algorithm
    • Step 1 Assign the value of min{m,n} to t
    • Step 2 Divide m by t. If the remainder is 0, go to Step 3; otherwise, go to Step 4
    • Step 3 Divide n by t. If the remainder is 0, return t and stop; otherwise, go to Step 4
    • Step 4 Decrease t by 1 and go to Step 2
    • Consecutive Integer Checking Algorithm example: gcd(60,24)
      • t = min(60,24) = 24
      • 60/24 = 2 remainder 12, t = 23
      • 60/23 = 2 remainder 14, t = 22
      • 60/22 = 2 remainder 16, t =21
      • 60/21 = 2 remainder 18, t=20
      • 60/20 = 3 remainder 0
      • 24/20 = 1 remainder 4, t=19
      • … ..(t = 19-16 not shown)… t=15
      • 60/15 = 4 remainder 0
      • 24/15 = 1 remainder 9
      • … ..(t = 14-13 not shown)… t = 12
      • 60/12 = 5 remainder 0
      • 24/12 = 2 remainder 0
      • gcd(24,12) = 12
    • How does the performance of this algorithm compare to Euclid’s?
      • It is clearly worse for this pair of numbers, at least
      • Soon we shall see that Euclid’s is O(log n) and this is O(n)
  • Other Algorithms for GCD
    • “ Middle-school” procedure
    • Step 1 Find the prime factorisation of m
    • Step 2 Find the prime factorisation of n
    • Step 3 Find all the common prime factors
    • Step 4 Compute the product of all the common prime factors and return it as gcd (m,n )
    • Is this an algorithm?
      • Consider the non-ambiguity requirement…
    48 180 gcd = 12
  • Other Algorithms for GCD
    • We need an algorithm for prime factorisation to make the middle-school procedure into an algorithm
    • We just repeatedly divide n by the prime numbers less than it, with no remainder, until the quotient becomes 1
    • But to do this prime factorisation, we need an algorithm to find all the prime numbers, up to n…
    • This can be done with the “Sieve of Eratosthenes”
  • Other Algorithms for GCD
    • “ Middle-school” procedure
    • Step 1 Find the prime factorisation of m
    • Step 2 Find the prime factorisation of n
    • Step 3 Find all the common prime factors
    • Step 4 Compute the product of all the common prime factors and return it as gcd (m,n )
    Prime Factorisation Input: Integer x ≥ 2, Output: List F of prime factors of x P ← Sieve(x) while n > 1 do while n mod P[i] = 0 do F ← F + P[i] x ← x / P[i] i ← i + 1 Sieve of Eratosthenes Input: Integer x ≥ 2 Output: List of primes less than or equal to x for p ← 2 to x do A [ p ] ← p for p ← 2 to x do if A [ p ]  0 j ← p * p while j ≤ x do A [ j ] ← 0 j ← j + p
    • How does the performance of this algorithm compare to Euclid’s and Consecutive Integer Checking?
    • Note also that an algorithm will be required to find all common prime factors too…
  • Fundamentals of Algorithmic Problem Solving
    • Understand the Problem
    • Ascertain the Capabilities of the Computational Device
    • Choosing between Exact and Approximate Problem Solving
    • Deciding on Appropriate Data Structures
    • Algorithm Design Techniques
    • Methods of Specifying an Algorithm
    • Proving an Algorithm’s Correctness
    • Analysing an Algorithm
    • Coding an Algorithm
    • Please read this section in the textbook in much more detail…
  • Algorithm Design Techniques
    • An algorithm design technique (or “strategy” or “paradigm”) is…
      • a general approach to solving problems algorithmically that is applicable to a wide range of problems from different areas of computing
    • This course is organised by design techniques
      • See the lecture topics from earlier…
      • Usually, AA courses are organised by problem type not algorithm design technique
  • Analysing an Algorithm
    • Analysis is essential. We want to know that our algorithms have good characteristics:
      • Good Time Efficiency ( or Complexity )
      • Good Space Efficiency ( or Complexity )
      • Simplicity
      • Generality
    • Analysis allows comparison of algorithms and allows us to know if they can be used practically or not
    • We will study a framework for analysis in the next lecture
  • Important Problem Types
    • Sorting
    • Searching
    • String Processing
    • Graph Problems
    • Combinatorial Problems
    • Geometric Problems
    • Numerical Problems
  • Sorting Problems
    • Problem: rearrange the items of a given list in ascending order .
      • Input: A sequence of n items <a 1 , a 2 , …, a n >
      • Output: A reordering <a * 1 , a * 2 , …, a * n > of the input sequence such that a * 1 ≤ a * 2 ≤ … ≤ a * n.
    • Importance of sorting
      • Can help searching tremendously
      • Algorithms often use sorting as a key subroutine.
    • Sorting key
      • A specially chosen piece of information used to guide sorting. E.g., sort student records by names.
  • Sorting Problems
    • Examples of sorting algorithms
      • Selection Sort
      • Bubble sort
      • Insertion sort
      • Merge sort
      • Heap sort …and others
    • We usually evaluate sorting algorithm complexity by the number of key comparisons .
    • Two important properties
      • Stability : A sorting algorithm is called stable if it preserves the relative order of any two equal elements in its input.
      • In place : A sorting algorithm is in place if it does not require extra memory, except, possibly for a few memory units.
  • Sorting Problems SelectionSort(A[0..n-1]) //Input: An array A[0..n-1] of orderable elements //Output: Array A[0..n-1] sorted in ascending order for i ← 0 to n – 2 do min ← i for j ← i + 1 to n – 1 do if A[j] < A[min] min ← j swap A[i] and A[min]
    • An example of a simple (and often low-performance) sorting algorithm is Selection Sort
  • Searching Problems
    • Problem: find a given value, called a search key, in a given set.
      • Eg ID, ref number, name, etc
    • Search is a hugely important problem
      • Retrieving data from database storage
        • Student records, company accounts etc.
    • And more general search…
      • Searching the web for information
      • Searching to solve problems in AI
  • Searching Problems
    • Examples of searching algorithms
      • Sequential search (Time O(n))
      • Binary search (Time: O(log n))
    Binary Search // Input: sorted array a_i < … < a_j and key x; m  (i+j)/2; while i < j and x != a_m do if x < a_m then j  m-1 else i  m+1; if x = a_m then output a_m;
  • String Processing Problems
    • A string is a sequence of characters from an alphabet.
      • “ This is a string” is a string
    • Text strings: letters, numbers, and special characters.
    • String matching: searching for a given word/pattern in a text.
    • Examples
      • searching for a word or phrase on WWW or in a Word document
      • searching for a particular sequence of {A,C,G,T} in a reference genomic sequence
  • Graph Problems
    • (Informally) A graph is a collection of points called vertices , some of which are connected by line segments called edges .
    • Graphs are extremely important for modeling real-life problems
      • Modeling WWW
      • Social relationships
      • Communication networks
      • Project scheduling …
    • Examples of graph algorithms
      • Graph traversal algorithms
      • Shortest-path algorithms
      • Topological sorting
    1 2 3 4
  • Graph Problems
    • Example: Travelling Salesman Problem
      • For a set of cities linked by roads, find the shortest route that visits them all only once.
      • Important to
        • Businesses and route planning
        • Circuit board and VLSI design and manufacture
        • X Ray crystallography
        • Genetic engineering
    • TSP, like many other graph problems, is extremely difficult and no general efficient algorithms are known
      • In fact, they may not exist…
  • Combinatorial Problems
    • Finding a Combinatorial Object that satisfies certain constraints or has certain properties
      • Sets, subsets
      • Permutations
        • Set P of all ordered sequences of length r from a set S of n elements
        • Eg. for S = {1,2,3} and r=3, P = {(1,2,3), (1,3,2), (2,1,3),(2,3,1), (3,1,2), (3,2,1)}
        • Eg. for S = {1,2,3} and r=2, P = {(1,2), (1,3), (2,1),(2,3), (3,1), (3,2)}
        • The number of permutations is n P r = n!/(n-r)!
      • Combinations
        • Set C of all unordered sequences of length r from a set S of n elements
        • Eg. for S = {1,2,3} and r=2, C = {(1,2), (1,3), (2,3),}
        • Eg. for S = {1,2,3} and r=3, C = {(1,2,3)}
        • The number of combinations is n C r = n!/(n-r)! r!
    • Combinatorial problems are among the hardest known
  • Combinatorial Problems
    • Example: The Knapsack Problem
      • Given a set of items, each with a weight and a value, determine the most valuable subset of items with total weight less than or equal to a given limit
  • Geometric Problems
    • Problems dealing with Geometric Objects such as points, lines, polygons
    • Examples
      • Finding closest pair of points
      • Finding convex hull of a set of points
  • Numerical Problems
    • Problems dealing with mathematical objects
      • Numbers, Equations, Matrices…
    • Examples
      • Efficient arithmetic
      • Evaluating functions
      • Solving systems of linear equations with Gaussian Elimination
      • Evaulating Definite Integrals
      • Optimising a function subject to constraints
  • Fundamental Data Structures
    • Linear Data Structures
      • Array
      • String
      • Linked list
      • Stack
      • Queue
        • Priority queue/heap
    • Graph
      • General Graph
      • Tree
        • Binary Tree
    • Set and Dictionary
  • Linear Data Structures
    • Arrays
      • A sequence of n items of the same data type that are stored contiguously in computer memory and made accessible by specifying a value of the array’s index.
    • Array properties
      • fixed length (need preliminary reservation of memory)
      • contiguous memory locations
      • direct access
      • Operations of insert/delete
      • Often used to implement Lists and Strings
  • Linear Data Structures
    • Linked List
      • A sequence of zero or more nodes each containing two kinds of information: some data and one or more links called pointers to other nodes of the linked list.
      • Singly linked list (next pointer)
      • Doubly linked list (next + previous pointers)
    • Linked List Properties
      • dynamic length
      • arbitrary memory locations
      • access by following links
      • Operations of Insert/delete
  • Stacks and Queues
    • Stacks
      • A stack of plates
        • insertion/deletion can be done only at the top.
        • LIFO
      • Two operations (push and pop)
    • Queues
      • A queue of customers waiting for services
        • Insertion/enqueue from the rear and deletion/dequeue from the front.
        • FIFO
      • Two operations (enqueue and dequeue)
  • Priority Queues and Heaps
    • Priority queues (implemented using heaps)
      • A data structure for maintaining a set of elements, each associated with a key/priority, with the following operations
    • Operations
      • Finding the element with the highest priority
      • Deleting the element with the highest priority
      • Inserting a new element
    9 6 8 5 2 3 9 6 5 8 2 3
  • Graphs
    • Formal definition
      • A graph G = <V, E> is defined by a pair of sets: a finite set V of items called vertices and a set E of vertex pairs called edges.
    • Undirected and directed graphs (digraphs).
      • What’s the maximum number of edges in an undirected graph with |V| vertices?
    • Complete graphs
      • A graph with every pair of its vertices connected by an edge is called complete, K |V|
    • Dense and sparse graphs
      • Dense graphs have edges between most nodes
      • Sparse graphs have few edges between nodes
    Complete, undirected graph Incomplete, directed graph Both graphs here are dense 1 2 3 4 1 2 3 4
  • Graph Representation
    • Nodes are said to be adjacent if an edge exists between them
    • Adjacency Matrix
      • n x n boolean matrix if number of nodes |V| is n.
      • The element on the ith row and jth column is 1 if there’s an edge from ith vertex to the jth vertex; otherwise 0.
      • The adjacency matrix of an undirected graph is symmetric.
    • Adjacency Linked Lists
      • A collection of linked lists, one for each vertex, that contain all the vertices adjacent to the list’s vertex.
    • Which data structure would you use for dense and sparse graphs?
    0 1 1 1 0 0 0 1 0 0 0 1 0 0 0 0 Incomplete, directed graph G Adjacency Matrix for G Adjacency Linked List for G 2 3 4 4 4 1 2 3 4
  • Weighted Graphs
    • Weighted graphs
      • Graphs or digraphs with numbers assigned to the edges.
    6 8 5 7 9 1 2 3 4
  • Graph Properties -- Paths and Connectivity
    • Paths
      • A path from vertex u to v of a graph G is defined as a sequence of adjacent vertices that starts with u and ends with v.
      • Simple paths: All edges of a path are distinct.
      • Path lengths: the number of edges, or the number of vertices – 1.
    Simple Path from 4 to 3 (4,1,2,3) Path Length 3 Path from 4 to 3 (4,1,2,5,4,1,2,3) Path Length 7 1 2 4 5 3 1 2 4 5 3
  • Graph Properties -- Paths and Connectivity
    • Connected graphs
      • A graph is said to be connected if for every pair of its vertices u and v there is a path from u to v.
    • Connected component
      • The maximum connected subgraph of a given graph .
    Connected Graph G1 Non-Connected Graph G2 1 2 4 5 3 1 2 4 5 3
  • Graph Properties -- Acyclicity
    • Cycle
      • A simple path of a positive length that starts and ends a the same vertex.
    • Acyclic graph
      • A graph without cycles
      • DAG (Directed Acyclic Graph)
    Directed Acyclic Graph Cycle (1,2,5,4,1) 1 2 3 4 1 2 4 5 3
  • Trees
    • Trees
      • A tree (or free tree) is a connected acyclic graph.
      • Forest: a graph that has no cycles but is not necessarily connected.
    • Properties of trees
      • |E| = |V| - 1
      • For every two vertices in a tree there always exists exactly one simple path from one of these vertices to the other….
    Free Tree Free Tree that is also a forest 1 3 2 4 5 1 3 2 4 5 6 7
  • Rooted Trees
    • This “single path” property makes it possible to select an arbitrary vertex in a free tree and consider it as the root of the so-called rooted tree .
    • A root then defines levels in the tree.
      • Number of edges from the root
    Free Tree Rooted Tree with 2 levels 1 3 2 4 5 1 3 2 4 5
  • Rooted Trees
    • Ancestors
      • For any vertex v in a tree T , all the vertices on the simple path from the root to that vertex are called ancestors.
    • Descendants
      • All the vertices for which a vertex v is an ancestor are said to be descendants of v .
    • Parent, child and siblings
      • If (u, v) is the last edge of the simple path from the root to vertex v , u is said to be the parent of v and v is called a child of u .
      • Vertices that have the same parent are called siblings.
    • Leaves
      • A vertex without children is called a leaf.
    • Subtree
      • A vertex v with all its descendants is called the subtree of T rooted at v.
    • Examples:
    • 1 is an ancestor of 6 and 7
    • 8 is a descendant of 5
    • 1 is parent (and ancestor) of 6 and 7
    • 8 and 9 are children (and descendants) of 5
    • 6 and 7 are siblings
    • 8 and 9 are siblings
    • 4,6,7,8,9 are leaves
    • 5 is the subtree consisting of 5,8,9
    1 3 2 4 5 7 6 8 9
  • Rooted Trees
    • Depth of a vertex
      • The length of the simple path from the root to the vertex.
    • Height of a tree
      • The length of the longest simple path from the root to a leaf.
    • Examples:
    • Vertex 2 has depth d=2
    • Vertex 7 has depth d=3
    • The tree itself has height h=3
    1 3 2 4 5 7 6 8 9
  • Ordered Trees
    • Ordered trees
      • An ordered tree is a rooted tree in which all the children of each vertex are ordered.
    • Binary trees
      • A binary tree is an ordered tree in which every vertex has no more than two children and each children is designated s either a left child or a right child of its parent.
    Binary Tree L R L 9 6 8 5 2 3
  • Ordered Trees
    • Binary search trees
      • Each vertex is assigned a number.
      • A number assigned to each parental vertex is larger than all the numbers in its left subtree and smaller than all the numbers in its right subtree.
      •  log 2 n   h  n – 1, where h is the height of a binary tree and n the size.
    6 3 9 2 5 8
  • Sets and Dictionaries
    • I will assume you already know what a set is!
      • Elements, subsets, union, intersection etc..
    • Dictionaries are Sets with 3 operations:
      • Searching for a given element
      • Inserting a new element
      • Deleting an existing element
    • Please read the textbook for more detail…