Course: Intro to Computer Science (Malmö Högskola):
A overview of computability and complexity (for non-mathematicians). definition of algorithm, turing machines, lambda, calculus and concepts of complexity
The document discusses two types of knapsack problems - the 0-1 knapsack problem and the fractional knapsack problem. The 0-1 knapsack problem uses dynamic programming to determine how to fill a knapsack to maximize the total value of items without exceeding the knapsack's weight limit, where each item is either fully included or not included. The fractional knapsack problem allows partial inclusion of items and can be solved greedily by always including a fraction of the highest value per unit weight item until the knapsack is full.
1. An algorithm is a sequence of unambiguous instructions to solve a problem and obtain an output for any valid input in a finite amount of time. Pseudocode is used to describe algorithms using a natural language format.
2. Analyzing algorithm efficiency involves determining the theoretical and empirical time complexity by counting the number of basic operations performed relative to the input size. Common measures are best-case, worst-case, average-case, and amortized analysis.
3. Important problem types for algorithms include sorting, searching, string processing, graphs, combinatorics, geometry, and numerical problems. Fundamental algorithms are analyzed for correctness and time/space complexity.
This document discusses NP-complete problems and their properties. Some key points:
- NP-complete problems have an exponential upper bound on runtime but only a polynomial lower bound, making them appear intractable. However, their intractability cannot be proven.
- NP-complete problems are reducible to each other in polynomial time. Solving one would solve all NP-complete problems.
- NP refers to problems that can be verified in polynomial time. P refers to problems that can be solved in polynomial time.
- A problem is NP-complete if it is in NP and all other NP problems can be reduced to it in polynomial time. Proving a problem is NP-complete involves showing
This document provides an overview of dynamic programming. It begins by explaining that dynamic programming is a technique for solving optimization problems by breaking them down into overlapping subproblems and storing the results of solved subproblems in a table to avoid recomputing them. It then provides examples of problems that can be solved using dynamic programming, including Fibonacci numbers, binomial coefficients, shortest paths, and optimal binary search trees. The key aspects of dynamic programming algorithms, including defining subproblems and combining their solutions, are also outlined.
The document discusses the divide and conquer algorithm design paradigm. It begins by defining divide and conquer as recursively breaking down a problem into smaller sub-problems, solving the sub-problems, and then combining the solutions to solve the original problem. Some examples of problems that can be solved using divide and conquer include binary search, quicksort, merge sort, and the fast Fourier transform algorithm. The document then discusses control abstraction, efficiency analysis, and uses divide and conquer to provide algorithms for large integer multiplication and merge sort. It concludes by defining the convex hull problem and providing an example input and output.
Inter-process communication (IPC) allows processes to communicate and synchronize actions. There are two main models - shared memory, where processes directly read/write shared memory, and message passing, where processes communicate by sending and receiving messages. Critical sections are parts of code that access shared resources and must be mutually exclusive to avoid race conditions. Semaphores can be used to achieve mutual exclusion, with operations P() and V() that decrement or increment the semaphore value to control access to the critical section. For example, in the producer-consumer problem semaphores can suspend producers if the buffer is full and consumers if empty, allowing only one process at a time in the critical section.
The document discusses two types of knapsack problems - the 0-1 knapsack problem and the fractional knapsack problem. The 0-1 knapsack problem uses dynamic programming to determine how to fill a knapsack to maximize the total value of items without exceeding the knapsack's weight limit, where each item is either fully included or not included. The fractional knapsack problem allows partial inclusion of items and can be solved greedily by always including a fraction of the highest value per unit weight item until the knapsack is full.
1. An algorithm is a sequence of unambiguous instructions to solve a problem and obtain an output for any valid input in a finite amount of time. Pseudocode is used to describe algorithms using a natural language format.
2. Analyzing algorithm efficiency involves determining the theoretical and empirical time complexity by counting the number of basic operations performed relative to the input size. Common measures are best-case, worst-case, average-case, and amortized analysis.
3. Important problem types for algorithms include sorting, searching, string processing, graphs, combinatorics, geometry, and numerical problems. Fundamental algorithms are analyzed for correctness and time/space complexity.
This document discusses NP-complete problems and their properties. Some key points:
- NP-complete problems have an exponential upper bound on runtime but only a polynomial lower bound, making them appear intractable. However, their intractability cannot be proven.
- NP-complete problems are reducible to each other in polynomial time. Solving one would solve all NP-complete problems.
- NP refers to problems that can be verified in polynomial time. P refers to problems that can be solved in polynomial time.
- A problem is NP-complete if it is in NP and all other NP problems can be reduced to it in polynomial time. Proving a problem is NP-complete involves showing
This document provides an overview of dynamic programming. It begins by explaining that dynamic programming is a technique for solving optimization problems by breaking them down into overlapping subproblems and storing the results of solved subproblems in a table to avoid recomputing them. It then provides examples of problems that can be solved using dynamic programming, including Fibonacci numbers, binomial coefficients, shortest paths, and optimal binary search trees. The key aspects of dynamic programming algorithms, including defining subproblems and combining their solutions, are also outlined.
The document discusses the divide and conquer algorithm design paradigm. It begins by defining divide and conquer as recursively breaking down a problem into smaller sub-problems, solving the sub-problems, and then combining the solutions to solve the original problem. Some examples of problems that can be solved using divide and conquer include binary search, quicksort, merge sort, and the fast Fourier transform algorithm. The document then discusses control abstraction, efficiency analysis, and uses divide and conquer to provide algorithms for large integer multiplication and merge sort. It concludes by defining the convex hull problem and providing an example input and output.
Inter-process communication (IPC) allows processes to communicate and synchronize actions. There are two main models - shared memory, where processes directly read/write shared memory, and message passing, where processes communicate by sending and receiving messages. Critical sections are parts of code that access shared resources and must be mutually exclusive to avoid race conditions. Semaphores can be used to achieve mutual exclusion, with operations P() and V() that decrement or increment the semaphore value to control access to the critical section. For example, in the producer-consumer problem semaphores can suspend producers if the buffer is full and consumers if empty, allowing only one process at a time in the critical section.
The document discusses the divide and conquer algorithm design technique. It begins by explaining the basic approach of divide and conquer which is to (1) divide the problem into subproblems, (2) conquer the subproblems by solving them recursively, and (3) combine the solutions to the subproblems into a solution for the original problem. It then provides merge sort as a specific example of a divide and conquer algorithm for sorting a sequence. It explains that merge sort divides the sequence in half recursively until individual elements remain, then combines the sorted halves back together to produce the fully sorted sequence.
it contains the detail information about Dynamic programming, Knapsack problem, Forward / backward knapsack, Optimal Binary Search Tree (OBST), Traveling sales person problem(TSP) using dynamic programming
The document discusses the knapsack problem, which involves selecting a subset of items that fit within a knapsack of limited capacity to maximize the total value. There are two versions - the 0-1 knapsack problem where items can only be selected entirely or not at all, and the fractional knapsack problem where items can be partially selected. Solutions include brute force, greedy algorithms, and dynamic programming. Dynamic programming builds up the optimal solution by considering all sub-problems.
This document discusses automata theory and focuses on grammars, languages, and finite state machines. It defines key terminology like alphabets, strings, languages, and regular expressions. It explains Chomsky's hierarchy of formal languages from type-3 regular languages to type-0 recursively enumerable languages. The document also discusses finite state automata (FSA), deterministic finite automata (DFA), non-deterministic finite automata (NFA), context-free grammars, pushdown automata, and Turing machines. Examples of grammars, languages, and finite state machines are provided to illustrate these concepts.
The document discusses Big O notation, which is used to classify algorithms based on how their running time scales with input size. It provides examples of common Big O notations like O(1), O(log n), O(n), O(n^2), and O(n!). The document also explains that Big O looks only at the fastest growing term as input size increases. Well-chosen data structures can help reduce an algorithm's Big O complexity. For example, searching a sorted list is O(log n) rather than O(n) for an unsorted list.
this is a briefer overview about the Big O Notation. Big O Notaion are useful to check the Effeciency of an algorithm and to check its limitation at higher value. with big o notation some examples are also shown about its cases and some functions in c++ are also described.
The document discusses the Travelling Salesman Problem (TSP), which aims to find the shortest route to visit each city in a list exactly once and return to the origin city. It describes TSP as an NP-hard problem, belonging to the complexity class NP-complete. The document provides background on TSP, explaining it cannot be solved in polynomial time using techniques like linear programming. While an efficient solution to the general TSP has not been found, there are approximation algorithms that provide near-optimal solutions.
BackTracking Algorithm: Technique and ExamplesFahim Ferdous
This slides gives a strong overview of backtracking algorithm. How it came and general approaches of the techniques. Also some well-known problem and solution of backtracking algorithm.
Graph coloring is an important concept in graph theory. It is a special kind of problem in which we have assign colors to certain elements of the graph along with certain constraints. Suppose we are given K colors, we have to color the vertices in such a way that no two adjacent vertices of the graph have the same color, this is known as vertex coloring, similarly we have edge coloring and face coloring. The coloring problem has a huge number of applications in modern computer science such as making schedule of time table , Sudoku, Bipartite graphs , Map coloring, data mining, networking. In this paper we are going to focus on certain applications like Final exam timetabling, Aircraft Scheduling, guarding an art gallery.
Now we have learnt the basics in logic.
We are going to apply the logical rules in proving mathematical theorems.
1-Direct proof
2-Contrapositive
3-Proof by contradiction
4-Proof by cases
The document discusses shortest path problems and algorithms. It defines the shortest path problem as finding the minimum weight path between two vertices in a weighted graph. It presents the Bellman-Ford algorithm, which can handle graphs with negative edge weights but detects negative cycles. It also presents Dijkstra's algorithm, which only works for graphs without negative edge weights. Key steps of the algorithms include initialization, relaxation of edges to update distance estimates, and ensuring the shortest path property is satisfied.
This document provides an introduction to formal logic and mathematical logic. It defines key concepts like logic, propositions, statements, truth values, and logical connectives. It also differentiates between propositions and non-propositions, and statements and non-statements. Examples are given of propositions and non-propositions. The basics of propositional logic and how to determine the truth values of expressions are explained.
The document discusses Turing machines and their properties. It introduces the Church-Turing thesis that any problem that can be solved by an algorithm can be modeled by a Turing machine. It then describes different types of Turing machines, such as multi-track, nondeterministic, two-way, multi-tape, and multidimensional Turing machines. The document provides examples of Turing machines that accept specific languages and evaluate mathematical functions through their transition tables and diagrams.
This document discusses the P versus NP problem in complexity theory. The P class contains problems that can be solved quickly by algorithms, while NP contains problems that can be verified quickly given a proposed solution. It is unknown whether NP problems can also be solved quickly (meaning P=NP), or if finding solutions to NP problems requires exponential time (meaning P≠NP). Solving this problem could have significant impacts on fields like cryptography, optimization, and theorem proving. However, despite extensive research, the P versus NP problem remains unsolved.
The document discusses the theory of NP-completeness. It begins by defining the complexity classes P, NP, NP-hard, and NP-complete. It then explains the concepts of reduction and how none of the NP-complete problems can be solved in polynomial time deterministically. The document provides examples of NP-complete problems like satisfiability (SAT), vertex cover, and the traveling salesman problem. It shows how nondeterministic algorithms can solve these problems and how they can be transformed into SAT instances. Finally, it proves that SAT is the first NP-complete problem by showing it is in NP and NP-hard.
The document describes external sorting techniques used when data is too large to fit in main memory. It discusses two-way sorting which uses two tape drive pairs to alternately write sorted runs. It also covers multi-way merging which merges multiple runs simultaneously using a heap. The techniques can improve performance over standard internal sorting.
The document discusses approximation algorithms for NP-complete problems. It introduces the idea of finding near-optimal solutions in polynomial time for problems where optimal solutions cannot be found efficiently. It provides examples of the vertex cover problem and set cover problem, describing greedy approximation algorithms that provide performance guarantees for finding near-optimal solutions for these problems. The document also discusses some open questions around whether these approximation ratios can be improved.
This document provides an overview of Turing machines. It introduces Turing machines as a simple mathematical model of a computer proposed by Alan Turing in 1936. A Turing machine consists of a tape divided into cells, a read/write head, finite states, and a transition function. The transition function defines how the machine moves between states and reads/writes symbols on the tape based on its current state and tape symbol. Turing machines can accept languages and compute functions. Variations include multi-tape, non-deterministic, multi-head, offline, multi-dimensional, and stationary-head Turing machines. The properties of Turing machines include their ability to recognize any language generated by a phrase-structure grammar and the Church
We will discuss the following: Graph, Directed vs Undirected Graph, Acyclic vs Cyclic Graph, Backedge, Search vs Traversal, Breadth First Traversal, Depth First Traversal, Detect Cycle in a Directed Graph.
Merge sort is a sorting technique based on divide and conquer technique. With worst-case time complexity being Ο(n log n), it is one of the most respected algorithms.
Merge sort first divides the array into equal halves and then combines them in a sorted manner.
Domain Specific Languages: An introduction (DSLs)Pedro Silva
Domain Specific Languages (DSLs) are special-purpose programming languages developed for a specific domain.
Some of its most interesting benefits include:
● increasing productivity
○ by reducing
■ the lines of code that have to be written manually
■ the number of coding errors
● (due to automatic domain restrictions)
● test generation
● formal verification
(Check my books at https://beacons.ai/tagido)
This document discusses Domain Specific Languages (DSLs) and their use in JavaScript. It begins by defining DSLs and providing examples of domain-specific lexicons. It then covers internal and external DSLs, and examples of JavaScript DSLs like JSSpec (for testing) and ActiveRecord.js (for object-relational mapping). The document concludes by discussing lessons learned from building DSLs in JavaScript, such as its lack of namespaces being a challenge but prototypal inheritance and object literals providing opportunities.
The document discusses the divide and conquer algorithm design technique. It begins by explaining the basic approach of divide and conquer which is to (1) divide the problem into subproblems, (2) conquer the subproblems by solving them recursively, and (3) combine the solutions to the subproblems into a solution for the original problem. It then provides merge sort as a specific example of a divide and conquer algorithm for sorting a sequence. It explains that merge sort divides the sequence in half recursively until individual elements remain, then combines the sorted halves back together to produce the fully sorted sequence.
it contains the detail information about Dynamic programming, Knapsack problem, Forward / backward knapsack, Optimal Binary Search Tree (OBST), Traveling sales person problem(TSP) using dynamic programming
The document discusses the knapsack problem, which involves selecting a subset of items that fit within a knapsack of limited capacity to maximize the total value. There are two versions - the 0-1 knapsack problem where items can only be selected entirely or not at all, and the fractional knapsack problem where items can be partially selected. Solutions include brute force, greedy algorithms, and dynamic programming. Dynamic programming builds up the optimal solution by considering all sub-problems.
This document discusses automata theory and focuses on grammars, languages, and finite state machines. It defines key terminology like alphabets, strings, languages, and regular expressions. It explains Chomsky's hierarchy of formal languages from type-3 regular languages to type-0 recursively enumerable languages. The document also discusses finite state automata (FSA), deterministic finite automata (DFA), non-deterministic finite automata (NFA), context-free grammars, pushdown automata, and Turing machines. Examples of grammars, languages, and finite state machines are provided to illustrate these concepts.
The document discusses Big O notation, which is used to classify algorithms based on how their running time scales with input size. It provides examples of common Big O notations like O(1), O(log n), O(n), O(n^2), and O(n!). The document also explains that Big O looks only at the fastest growing term as input size increases. Well-chosen data structures can help reduce an algorithm's Big O complexity. For example, searching a sorted list is O(log n) rather than O(n) for an unsorted list.
this is a briefer overview about the Big O Notation. Big O Notaion are useful to check the Effeciency of an algorithm and to check its limitation at higher value. with big o notation some examples are also shown about its cases and some functions in c++ are also described.
The document discusses the Travelling Salesman Problem (TSP), which aims to find the shortest route to visit each city in a list exactly once and return to the origin city. It describes TSP as an NP-hard problem, belonging to the complexity class NP-complete. The document provides background on TSP, explaining it cannot be solved in polynomial time using techniques like linear programming. While an efficient solution to the general TSP has not been found, there are approximation algorithms that provide near-optimal solutions.
BackTracking Algorithm: Technique and ExamplesFahim Ferdous
This slides gives a strong overview of backtracking algorithm. How it came and general approaches of the techniques. Also some well-known problem and solution of backtracking algorithm.
Graph coloring is an important concept in graph theory. It is a special kind of problem in which we have assign colors to certain elements of the graph along with certain constraints. Suppose we are given K colors, we have to color the vertices in such a way that no two adjacent vertices of the graph have the same color, this is known as vertex coloring, similarly we have edge coloring and face coloring. The coloring problem has a huge number of applications in modern computer science such as making schedule of time table , Sudoku, Bipartite graphs , Map coloring, data mining, networking. In this paper we are going to focus on certain applications like Final exam timetabling, Aircraft Scheduling, guarding an art gallery.
Now we have learnt the basics in logic.
We are going to apply the logical rules in proving mathematical theorems.
1-Direct proof
2-Contrapositive
3-Proof by contradiction
4-Proof by cases
The document discusses shortest path problems and algorithms. It defines the shortest path problem as finding the minimum weight path between two vertices in a weighted graph. It presents the Bellman-Ford algorithm, which can handle graphs with negative edge weights but detects negative cycles. It also presents Dijkstra's algorithm, which only works for graphs without negative edge weights. Key steps of the algorithms include initialization, relaxation of edges to update distance estimates, and ensuring the shortest path property is satisfied.
This document provides an introduction to formal logic and mathematical logic. It defines key concepts like logic, propositions, statements, truth values, and logical connectives. It also differentiates between propositions and non-propositions, and statements and non-statements. Examples are given of propositions and non-propositions. The basics of propositional logic and how to determine the truth values of expressions are explained.
The document discusses Turing machines and their properties. It introduces the Church-Turing thesis that any problem that can be solved by an algorithm can be modeled by a Turing machine. It then describes different types of Turing machines, such as multi-track, nondeterministic, two-way, multi-tape, and multidimensional Turing machines. The document provides examples of Turing machines that accept specific languages and evaluate mathematical functions through their transition tables and diagrams.
This document discusses the P versus NP problem in complexity theory. The P class contains problems that can be solved quickly by algorithms, while NP contains problems that can be verified quickly given a proposed solution. It is unknown whether NP problems can also be solved quickly (meaning P=NP), or if finding solutions to NP problems requires exponential time (meaning P≠NP). Solving this problem could have significant impacts on fields like cryptography, optimization, and theorem proving. However, despite extensive research, the P versus NP problem remains unsolved.
The document discusses the theory of NP-completeness. It begins by defining the complexity classes P, NP, NP-hard, and NP-complete. It then explains the concepts of reduction and how none of the NP-complete problems can be solved in polynomial time deterministically. The document provides examples of NP-complete problems like satisfiability (SAT), vertex cover, and the traveling salesman problem. It shows how nondeterministic algorithms can solve these problems and how they can be transformed into SAT instances. Finally, it proves that SAT is the first NP-complete problem by showing it is in NP and NP-hard.
The document describes external sorting techniques used when data is too large to fit in main memory. It discusses two-way sorting which uses two tape drive pairs to alternately write sorted runs. It also covers multi-way merging which merges multiple runs simultaneously using a heap. The techniques can improve performance over standard internal sorting.
The document discusses approximation algorithms for NP-complete problems. It introduces the idea of finding near-optimal solutions in polynomial time for problems where optimal solutions cannot be found efficiently. It provides examples of the vertex cover problem and set cover problem, describing greedy approximation algorithms that provide performance guarantees for finding near-optimal solutions for these problems. The document also discusses some open questions around whether these approximation ratios can be improved.
This document provides an overview of Turing machines. It introduces Turing machines as a simple mathematical model of a computer proposed by Alan Turing in 1936. A Turing machine consists of a tape divided into cells, a read/write head, finite states, and a transition function. The transition function defines how the machine moves between states and reads/writes symbols on the tape based on its current state and tape symbol. Turing machines can accept languages and compute functions. Variations include multi-tape, non-deterministic, multi-head, offline, multi-dimensional, and stationary-head Turing machines. The properties of Turing machines include their ability to recognize any language generated by a phrase-structure grammar and the Church
We will discuss the following: Graph, Directed vs Undirected Graph, Acyclic vs Cyclic Graph, Backedge, Search vs Traversal, Breadth First Traversal, Depth First Traversal, Detect Cycle in a Directed Graph.
Merge sort is a sorting technique based on divide and conquer technique. With worst-case time complexity being Ο(n log n), it is one of the most respected algorithms.
Merge sort first divides the array into equal halves and then combines them in a sorted manner.
Domain Specific Languages: An introduction (DSLs)Pedro Silva
Domain Specific Languages (DSLs) are special-purpose programming languages developed for a specific domain.
Some of its most interesting benefits include:
● increasing productivity
○ by reducing
■ the lines of code that have to be written manually
■ the number of coding errors
● (due to automatic domain restrictions)
● test generation
● formal verification
(Check my books at https://beacons.ai/tagido)
This document discusses Domain Specific Languages (DSLs) and their use in JavaScript. It begins by defining DSLs and providing examples of domain-specific lexicons. It then covers internal and external DSLs, and examples of JavaScript DSLs like JSSpec (for testing) and ActiveRecord.js (for object-relational mapping). The document concludes by discussing lessons learned from building DSLs in JavaScript, such as its lack of namespaces being a challenge but prototypal inheritance and object literals providing opportunities.
Internal DSLs For Automated Functional TestingJohn Sonmez
Automated functional testing is one of the hardest things to get right on a project. Many people with high hopes set out to develop fully automated regression tests, only to be caught up in the tangle of fragile tests which are always broken. In this session I will present a clean way to build a framework using free tools to develop an internal Domain Specific Language custom to the application being testing. I will show the benefits of this approach versus the record and modify methods of most automated testing tools. In addition, I will talk about a real working practical example I have developed for the Dept of Health in Welfare in testing their new case management system.
Learn how to practice configuration as code by using the Job DSL plugin for Jenkins. Find out how to organize Job DSL scripts and apply code reuse and refactoring to your Jenkins configuration. This talk will cover advanced techniques for large scale installations and show how to extend the Job DSL for your favorite plugins.
The document provides an overview of topics in discrete mathematics including logic, sets, and functions. It outlines the following content: introduction to logic and logical operators; propositions and logical equivalences; predicates and quantifiers; sets and set operations; and functions. For each topic, it provides definitions, examples, and truth tables to illustrate key concepts in propositional and predicate logic, and sets. It also includes examples, explanations and review questions to help explain the material.
Knowledge representation and Predicate logicAmey Kerkar
1. The document discusses knowledge representation and predicate logic.
2. It explains that knowledge representation involves representing facts through internal representations that can then be manipulated to derive new knowledge. Predicate logic allows representing objects and relationships between them using predicates, quantifiers, and logical connectives.
3. Several examples are provided to demonstrate representing simple facts about individuals as predicates and using quantifiers like "forall" and "there exists" to represent generalized statements.
The document discusses algorithms and their properties. It defines an algorithm as a finite sequence of steps to solve a specific problem. Algorithms must have a defined input and output, and can solve the same problem in different ways. Common algorithm types include recursive, dynamic programming, backtracking, divide and conquer, greedy, brute force, and heuristic algorithms. Efficiency is measured by time and space complexity. Variables are introduced as a way to store input, intermediate results, and output values in algorithms.
This document provides an overview of algorithms. It begins by discussing the origins and evolution of the term "algorithm" from its Arabic roots in the 9th century to its modern meaning of a well-defined computational procedure. The document then defines algorithms and their key characteristics such as being precise, unambiguous, and terminating after a finite number of steps. Common algorithm design techniques like divide-and-conquer, greedy algorithms, and dynamic programming are introduced. Examples of merge sort and finding the maximum element in a list are used to illustrate algorithm concepts.
The document discusses algorithms and their analysis. It covers the following key points:
1) The course objectives are to develop problem solving abilities using mathematical theories, apply algorithmic strategies to solve problems, analyze algorithm performance, and study time/space efficient algorithms.
2) The course outcomes are for students to formulate problems, analyze asymptotic performance, decide algorithmic strategies, find optimal solutions, and analyze sorting/scheduling algorithms.
3) Topics covered include classical algorithms, designing algorithms, proving correctness, and analyzing performance.
This document discusses time and space complexity analysis of algorithms. It defines key concepts like computational problems, algorithms, inputs, outputs, and properties of good algorithms. It then explains space complexity and time complexity, and provides examples of typical time functions like constant, logarithmic, linear, quadratic, and exponential. An example C program for matrix multiplication is provided, with its time complexity analyzed as O(n^2) + O(n^3).
This document provides an overview of a course on algorithms and data structures. It outlines the course topics that will be covered over 15 weeks of lectures. These include data types, arrays, matrices, pointers, linked lists, stacks, queues, trees, graphs, sorting, and searching algorithms. Evaluation will be based on assignments, quizzes, projects, sessionals, and a final exam. The goal is for students to understand different algorithm techniques, apply suitable data structures to problems, and gain experience with classical algorithm problems.
Chapter1.1 Introduction to design and analysis of algorithm.pptTekle12
This document discusses the design and analysis of algorithms. It begins with defining what an algorithm is - a well-defined computational procedure that takes inputs and produces outputs. It describes analyzing algorithms to determine their efficiency and comparing different algorithms that solve the same problem. The document outlines steps for designing algorithms, including understanding the problem, deciding a solution approach, designing the algorithm, proving correctness, and analyzing and coding it. It discusses using mathematical techniques like asymptotic analysis and Big O notation to analyze algorithms independently of implementations or inputs. The importance of analysis is also covered.
This document discusses the design and analysis of algorithms. It begins with defining what an algorithm is - a well-defined computational procedure that takes inputs and produces outputs. It describes analyzing algorithms to determine their efficiency and comparing different algorithms that solve the same problem. The document outlines steps for designing algorithms, including understanding the problem, deciding a solution approach, designing the algorithm, proving correctness, and analyzing and coding it. It discusses using mathematical techniques like asymptotic analysis and Big O notation to analyze algorithms independently of implementations or data. The importance of analyzing algorithms and techniques like divide-and-conquer are also covered.
These lecture notes cover algorithms and their analysis over 4 modules. Module I introduces algorithms, their properties, analysis of complexity and asymptotic notations. It covers analysis of sorting algorithms like merge sort, quicksort and binary search. Module II covers dynamic programming and greedy algorithms. Module III covers graph algorithms like BFS, DFS and minimum spanning trees. Module IV covers advanced topics like fast Fourier transform, string matching, NP-completeness and approximation algorithms.
These lecture notes cover the design and analysis of algorithms over 4 modules. Module I introduces algorithms, their characteristics, expectations and analysis. It discusses asymptotic analysis using big O, Ω and Θ notations to analyze the growth of algorithms like insertion sort, which has a worst case running time of Θ(n2). Subsequent modules cover dynamic programming, greedy algorithms, graphs, and NP-completeness. The notes provide an overview of key algorithm design and analysis topics.
Design & Analysis of Algorithms Lecture NotesFellowBuddy.com
FellowBuddy.com is an innovative platform that brings students together to share notes, exam papers, study guides, project reports and presentation for upcoming exams.
We connect Students who have an understanding of course material with Students who need help.
Benefits:-
# Students can catch up on notes they missed because of an absence.
# Underachievers can find peer developed notes that break down lecture and study material in a way that they can understand
# Students can earn better grades, save time and study effectively
Our Vision & Mission – Simplifying Students Life
Our Belief – “The great breakthrough in your life comes when you realize it, that you can learn anything you need to learn; to accomplish any goal that you have set for yourself. This means there are no limits on what you can be, have or do.”
Like Us - https://www.facebook.com/FellowBuddycom
This document provides an introduction to the CS-701 Advanced Analysis of Algorithms course. It includes the course objectives, which are to design and analyze modern algorithms and evaluate their efficiency. The instructor and contact information are provided. The document outlines the course contents, including topics like sorting algorithms, graph algorithms, and complexity theory. It also discusses what algorithms are, how to represent them, and examples of algorithm applications. Common algorithm design techniques like greedy algorithms and heuristics are introduced.
This document introduces the topic of algorithms for a computer science course. It defines an algorithm as a step-by-step method for solving a problem and notes key properties like being unambiguous, terminating, and giving the correct output. The document gives examples of algorithms for finding the maximum of three numbers and computing greatest common divisors. It outlines some common algorithm design approaches and topics that will be covered in the course like graph algorithms and sorting.
Data Analysis and Algorithms Lecture 1: IntroductionTayyabSattar5
This document outlines a course on design and analysis of algorithms. It covers topics like algorithm complexity analysis using growth functions, classic algorithm problems like the traveling salesperson problem, and algorithm design techniques like divide-and-conquer, greedy algorithms, and dynamic programming. Example algorithms and problems are provided for each topic. Reference books on algorithms are also listed.
Systems in the small - Introduction to AlgorithmsDavid Millard
A brief presentation to introduce new IT students to the concept of algorithms - including some short group exercises based around search and sorting lists
The document discusses asymptotic analysis of algorithms. It explains that the correctness of an algorithm is not enough, and efficiency must also be considered. The efficiency of an algorithm is measured by its time and space complexity. Time complexity is how long it takes to execute, while space complexity refers to memory usage. Basic operations like addition and comparison are assumed to take constant time. The document analyzes several algorithms to determine their time complexity using Θ notation. It also discusses analyzing nested loops and determining the dominant term to classify complexity.
This document contains lecture notes on the design and analysis of algorithms. It covers topics like algorithm definition, complexity analysis, divide and conquer algorithms, greedy algorithms, dynamic programming, and NP-complete problems. The notes provide examples of algorithms like selection sort, towers of Hanoi, and generating permutations. Pseudocode is used to describe algorithms precisely yet readably.
The document provides an introduction to algorithms and their analysis. It discusses the definition of an algorithm, their characteristics, types and examples. It also covers the analysis of algorithms including best, average and worst case analysis. Common asymptotic notations like Big-O, Omega and Theta notations are introduced to analyze the time complexity of algorithms. Examples of analyzing for loops and other control statements are provided.
This document provides an overview of the theory of computation. It discusses the following key points:
1. Computation involves executing programs on computers to perform input-output transformations. Programs are algorithms expressed in programming languages.
2. The theory of computation classifies problems by their computability and complexity. It studies models of computation like Turing machines.
3. The theory has three main components: computability theory, complexity theory, and automata theory/formal languages.
4. Proofs in the theory use techniques like mathematical induction to show properties are true for all cases.
This document provides an overview of data structures and algorithms analysis. It discusses big-O notation and how it is used to analyze computational complexity and asymptotic complexity of algorithms. Various growth functions like O(n), O(n^2), O(log n) are explained. Experimental and theoretical analysis methods are described and limitations of experimental analysis are highlighted. Key aspects like analyzing loop executions and nested loops are covered. The document also provides examples of analyzing algorithms and comparing their efficiency using big-O notation.
JThermodynamicsCloud is software service for the chemical, or more specifically, the combustion research
domain. JThermodynamicsCloud service can be said to be an model driven application, where the ontology
is a platform independent model of the data and operational structures. The ontology, as used by the service,
has three distinct purposes: documentation, data structure definition and operational definitions. One goal of
the ontology is to place as much of the design and domain specific structures in the ontology rather than in
the application code. The application code interprets the ontology in the backend. The primary purpose of
the JThermodynamicsCloud is to perform thermdynamic calculations and manage the data needed to make
those calculations. The calculation itself is highly dependent on the varied types of molecular data found in
the database The complete service is a system with three interacting components, a user interface using
Angular, a (RESTful) backend written in JAVA (with the JENA API interpreting the ontology) and the
Google Firestore noSQL document database and Firebase storage. The service uses these three components
to make calculations for thermodynamic quantities based on molecular species structure. These different
platforms are united through the ontology.
JThermodynamicsCloud is software service for the chemical, or more specifically, the combustion research
domain. JThermodynamicsCloud service can be said to be an model driven application, where the ontology
is a platform independent model of the data and operational structures. The ontology, as used by the service,
has three distinct purposes: documentation, data structure definition and operational definitions. One goal of
the ontology is to place as much of the design and domain specific structures in the ontology rather than in
the application code. The application code interprets the ontology in the backend. The primary purpose of
the JThermodynamicsCloud is to perform thermdynamic calculations and manage the data needed to make
those calculations. The calculation itself is highly dependent on the varied types of molecular data found in
the database The complete service is a system with three interacting components, a user interface using
Angular, a (RESTful) backend written in JAVA (with the JENA API interpreting the ontology) and the
Google Firestore noSQL document database and Firebase storage. The service uses these three components
to make calculations for thermodynamic quantities based on molecular species structure. These different
platforms are united through the ontology.
JThermodynamicsCloud is software service for the chemical, or more specifically, the combustion research
domain. JThermodynamicsCloud service can be said to be an model driven application, where the ontology
is a platform independent model of the data and operational structures. The ontology, as used by the service,
has three distinct purposes: documentation, data structure definition and operational definitions. One goal of
the ontology is to place as much of the design and domain specific structures in the ontology rather than in
the application code. The application code interprets the ontology in the backend. The primary purpose of
the JThermodynamicsCloud is to perform thermdynamic calculations and manage the data needed to make
those calculations. The calculation itself is highly dependent on the varied types of molecular data found in
the database The complete service is a system with three interacting components, a user interface using
Angular, a (RESTful) backend written in JAVA (with the JENA API interpreting the ontology) and the
Google Firestore noSQL document database and Firebase storage. The service uses these three components
to make calculations for thermodynamic quantities based on molecular species structure. These different
platforms are united through the ontology.
ChemConnect: Poster for European Combustion Meeting 2017Edward Blurock
ChemConnect is a freely available combustion database that organizes experimental and modeling chemical data into a searchable network of interconnected concepts. It goes beyond traditional repositories by parsing data sets, extracting fine-grained information, and building relationships between data using semantic techniques. This enriches the data and allows connections to be made between otherwise independent sources. The goal is to provide a collaborative platform for researchers to disseminate and exchange both published and preliminary combustion data.
EU COST Action CM1404: WG€ - Efficient Data ExchangeEdward Blurock
This talk discusses the topic of data exchange within the combustion community. This is a summary of a task force on data exchange within the WG4 working group, Standard definition for data collection and mining toward a virtual chemistry of Smart Energy Carriers, within the SMARTCATS EU COST Action CM1404
ChemConnect: Characterizing CombusAon KineAc Data with ontologies and meta-‐...Edward Blurock
ChemConnect is a database that interconnects fine-grained information extracted from chemical kinetic and thermodynamic sources such as
CHEMKIN mechanism files, NASA polynomial files, and even the information behind automatic generation files.
The key to the interconnection is the Resource Description Framework (RDF) from Semantic Web technologies. The RDF is a triplet where an object item (first) is associated through a descriptor (second) to a subject item.
In this way the information of the object is connected (through the descriptor) to the subject.
In ChemConnect the object is word (text) and the subject can be text or a database item. The search mechanism within ChemConnect uses the object and subject text as search strings.
The presentation also contains an brief introduction to cloud computing.
This was presented at the COST Action 1404 SMARTCATS workshop on Databases and Systems Use Cases (http//http://www.smartcats.eu/wg4ws1dp/)
Poster: Characterizing Ignition behavior through morphing to generic curvesEdward Blurock
The qualitative notion that ignition processes have similar behavior, even over an extensive range of starting conditions, is quantitatively demonstrated through the production of a single ’generic’ ignition curve. The key to the production of the generic curve is the recognition that the basic shapes of the species and temperature profiles occurring in the ignition process differ only in their ’timing’. By ’morphing’ the time scale, the profile shapes can be made to align. From the aligned profile shapes a generic or ’average’ profile can be derived. Synchronizing chemical events modifies the ignition progress times. In addition to fixing the ignition time to have the progress value of one, intermediate ignition events (such as selected profile maxima or inflection points) that occur before ignition are also aligned to have specific ’normalized’ times.
The goal of the Very Open Data Project is to provide a software-technical foundation for this exchange of data, more specifically to provide an open database platform for data from the raw data coming from experimental measurements or models through intermediate manipulations to finally published results. The sheer expanse of the amount data involved creates some unique software-technical challenges. One of these challenges is addressed in the part of the study presented here, namely to characterize scientific data (with the initial focus being detailed chemistry data from the combustion kinetic community), so that efficient searches can be made. A formalization of this characterization comes in the form of schemas of descriptions of tags and keywords describing data and ontologies describing the relationship between data types and the relationship between the characterizations themselves. These will be translated to meta-data tags connected to the data points within a non-relational data of data for the community.
The focus of the initial work will be on data and its accessibility. As the project progresses, the emphasis will shift on not only having available data accessible for the community, but that the community itself will be able to, with emphasis on minimal effort, will be able contribute their own data. This will involve, for example, the concepts of the ‘electronic lab notebook’ and the existence and availability of extensive concept extraction tools, primarily from the chemical informatics field.
This describes a tabulation method based on computing, retaining and accessing a large, on the order of millions, number of individual kinetic time step calculations and approximations. It is essentially an extension of Pope’s In Situ Adaptive Tabulation
(ISAT) method. The primary differences lie in that not all configurations need be stored in memory and that a polynomial approximation is only calculated when enough points have accumulated within a localized area to be able to calculate the
polynomial approximation. The latter increases efficiency because no extra points are evaluated to form an approximation (as is done in ISAT). The speed up is expected to be that of ISAT.
Characterization Ignition Behavior through Morphing to Generic Ignition CurvesEdward Blurock
Presented at the International Conference of Chemical Kinetics, Ghent, Belgium, July, 2015
The qualitative notion that ignition processes have similar behavior, even over an extensive range of starting conditions, is quantitatively demonstrated through the production of a single ’generic’ ignition curve. The key to the production of the generic curve is the recognition that the basic shapes of the species and temperature profiles occurring in the ignition process differ only in their ’timing’. By ’morphing’ the time scale, the profile shapes can be made to align. From the aligned profile shapes, a generic or ’average’ profile can be derived. Synchronizing chemical events modifies the ignition progress times. In addition to fixing the ignition time to have the progress value of one, intermediate ignition events (such as selected profile maxima or inflection points) that occur before ignition are also aligned to have specific ’normalized’ times. With this additional synchronization, a single generic curve, derived from the average of the morphed curves, can be derived. This generic curve represents a kinetic modelers intuitive notion of the mechanism of the process.
The document discusses programming paradigms and introduces imperative programming. It defines imperative programming as a paradigm that describes computation in terms of statements that change a program's state. Imperative programming uses commands to update variables in storage and defines sequences of commands for the computer to perform. The document contrasts structured and unstructured programming and discusses concepts like l-values, r-values, iteration, and goto statements as they relate to imperative programming.
Course: Programming Languages and Paradigms:
A brief introduction to imperative programming principles: history, von neumann, BNF, variables (r-values, l-values), modifiable data structures, order of evaluation, static and dynamic scopes, referencing environments, call by value, control flow (sequencing, selection, iteration), ...
Course: Programming Languages and Paradigms:
This introduces concepts related to programming languate design: abstraction, a bit of history, the syntax, semantics and pragmatics of programming languages, languages as abstraction, thought shaper, simplifier and law enforcer.program verification, denotational and operational semantics
Course: Intro to Computer Science (Malmö Högskola):
knowledge representation and abstraction, decision making, generalization, data acquistion (abstraction), machine learning, similarity
another version of abstraction
Course: Intro to Computer Science (Malmö Högskola):
A very general overview of computer science from machine, operating systems, networks, applications...
How to Fix the Import Error in the Odoo 17Celine George
An import error occurs when a program fails to import a module or library, disrupting its execution. In languages like Python, this issue arises when the specified module cannot be found or accessed, hindering the program's functionality. Resolving import errors is crucial for maintaining smooth software operation and uninterrupted development processes.
Strategies for Effective Upskilling is a presentation by Chinwendu Peace in a Your Skill Boost Masterclass organisation by the Excellence Foundation for South Sudan on 08th and 09th June 2024 from 1 PM to 3 PM on each day.
This presentation was provided by Steph Pollock of The American Psychological Association’s Journals Program, and Damita Snow, of The American Society of Civil Engineers (ASCE), for the initial session of NISO's 2024 Training Series "DEIA in the Scholarly Landscape." Session One: 'Setting Expectations: a DEIA Primer,' was held June 6, 2024.
How to Manage Your Lost Opportunities in Odoo 17 CRMCeline George
Odoo 17 CRM allows us to track why we lose sales opportunities with "Lost Reasons." This helps analyze our sales process and identify areas for improvement. Here's how to configure lost reasons in Odoo 17 CRM
A workshop hosted by the South African Journal of Science aimed at postgraduate students and early career researchers with little or no experience in writing and publishing journal articles.
Introduction to AI for Nonprofits with Tapp NetworkTechSoup
Dive into the world of AI! Experts Jon Hill and Tareq Monaur will guide you through AI's role in enhancing nonprofit websites and basic marketing strategies, making it easy to understand and apply.
Thinking of getting a dog? Be aware that breeds like Pit Bulls, Rottweilers, and German Shepherds can be loyal and dangerous. Proper training and socialization are crucial to preventing aggressive behaviors. Ensure safety by understanding their needs and always supervising interactions. Stay safe, and enjoy your furry friends!
A review of the growth of the Israel Genealogy Research Association Database Collection for the last 12 months. Our collection is now passed the 3 million mark and still growing. See which archives have contributed the most. See the different types of records we have, and which years have had records added. You can also see what we have for the future.
How to Add Chatter in the odoo 17 ERP ModuleCeline George
In Odoo, the chatter is like a chat tool that helps you work together on records. You can leave notes and track things, making it easier to talk with your team and partners. Inside chatter, all communication history, activity, and changes will be displayed.
The simplified electron and muon model, Oscillating Spacetime: The Foundation...RitikBhardwaj56
Discover the Simplified Electron and Muon Model: A New Wave-Based Approach to Understanding Particles delves into a groundbreaking theory that presents electrons and muons as rotating soliton waves within oscillating spacetime. Geared towards students, researchers, and science buffs, this book breaks down complex ideas into simple explanations. It covers topics such as electron waves, temporal dynamics, and the implications of this model on particle physics. With clear illustrations and easy-to-follow explanations, readers will gain a new outlook on the universe's fundamental nature.
2. Algorithm
• You have a task
• You want to formulate an algorithm to
perform the task
• Three basic questions:
– What Is an algorithm?
– Will an algorithm be able to compute the task?
– How long will the algorithm take?
• how complex is the algorithm?
3. Three questions
about algorithms
• These are actually non trivial questions
– What Is an algorithm?
– Will an algorithm be able to compute the task?
– How long will the algorithm take?
• how complex is the algorithm?
• Important to mathematically formalize these
questions
– to create better algorithms
– To know where the limitations of algorithms lie
4. Questions
• What Is an algorithm?
– To make an algorithm, you have to know “exactly” what it
is.. Its components and how to put these components
together.
– To ask questions, you have to know what you are asking
questions about
• Will an algorithm be able to compute the task?
– Is the task I want impossible?
– Is it possible to formulate an impossible task
– What is the consequences for other algorithms
• How complex is the algorithm?
• Will it take “too much” time? (for cryptography, this is a good
thing)
• Can I make a faster algorithm for a given type of task?
5. DEFINITION
What Is an algorithm?
To make an algorithm, you have to know “exactly” what it is.. Its components
and how to put these components together.
To ask questions, you have to know what you are asking questions about
6. Algorithm Definition
• Part of the research is to formulate an exact
workable definition of an algorithm
• There are several/many definitions
– The do have common characteristics
7. Algorithm Definition
• A logical sequence of steps for solving a
problem, …
» From http://Dictionary.msn.com
• Dale and Lewis:
» a plan of solution for a problem
» Algorithm – An unambiguous (and precise) set of steps
for solving a problem (or sub-problem) in a finite amount
of time using a finite amount of data.
8. Algorithm Definition
• Shackelford, Russell L. in Introduction to
Computing and Algorithms –
– “An algorithm is a specification of a behavioral
process. It consists of a finite set of instructions
that govern behavior step-by-step.”
9. Definition
• An algorithm is a finite sequence of step by
step, discrete, unambiguous instructions for
solving a particular problem
– has input data, and is expected to produce output
data
– each instruction can be carried out in a finite
amount of time in a deterministic way
10. Keywords
• Notice the term finite.
– Algorithms should lead to an eventual solution.
– The algorithm should stop (halt).
• Sequence of steps
– Each step should do one logical action.
11. COMPUTABILITY
Will an algorithm be able to compute the task?
Is the task I want impossible?
Is it possible to formulate an impossible task
What is the consequences for other algorithms
12. Computability vs Complexity
• Computability refers to whether of not in
principle it is possible to evaluate f(n) by
following a set of instructions.
• We are not for the moment worried if this
computation requires 1,000,000 or more
consecutive steps.
• The latter refers to complexity which we will
return to later.
13. Ensheidungsproblem
Decision problem
Hilbert 1928
The Entscheidungsproblem asks for
an algorithm
that takes as input a statement of a first-order logic
and
answers "Yes" or "No"
according to whether the statement is universally valid
(valid in every structure satisfying the axioms).
An algorithm to ask whether something is true
Is it computable
14. Ensheidungsproblem
Decision problem
Hilbert 1928
• Before the question could be answered, the
notion of "algorithm" had to be formally
defined.
– This was done by Alonzo Church in 1936 with the
concept of "effective calculability" based on his λ
calculus
– and by Alan Turing in the same year with his
concept of Turing machines.
– It was recognized immediately by Turing that
these are equivalent models of computation.
15. 15
Alan Turing
(1912 – 1954)
Alonzo Church
(1903-1995)
Turing Machine Lambda calculus
Two mathematical ways to ask questions about
“computability”
16. Equivalent Computers
z z zz z z z
1
Start
HALT
), X, L
2:
look
for (
#, 1, -
), #, R
(, #, L
(, X, R
#, 0, -
Finite State Machine
...
Turing Machine
term = variable
| term term
| (term)
| variable . term
y. M v. (M [y v])
where v does not occur in M.
(x. M)N M [ x N ]
Lambda Calculus
17. Alan Turing, 1936
• "On computable numbers, with an application
to the Entscheidungsproblem [decision
problem]”
• addressed a previously unsolved
mathematical problem posed by the German
mathematician David Hilbert in 1928:
18. Turing, 1936
Is there, in principal,
any definite mechanical method or process
by which
all mathematical questions
could be decided?
The Decision Problem
19. Turing, 1936
• To answer the question (in the negative),
• Turing proposed a simple abstract computing
machine,
• modeled after a mathematician with a pencil,
an eraser, and a stack of pieces of paper
• He asserted that any function is a computable
function if it is computable by one of these
abstract machines.
20. Turing, 1936
• He then investigated whether there were
some mathematical problems which could not
be solved by any such abstract machine.
• Such abstract computing machines are now
called "Turing machines".
• One particular Turing machine, called a
"Universal Turing Machine", served as the
model for designing actual programmable
computers.
21. Why a Turing Machine
Because Turing machines are very simple
compared with computers in practical use,
conceptually easier
to prove impossibility results
for Turing machines.
These impossibility results
apply as well to all known computers
22. Turing, 1936
• Startling result of Turing's 1936 paper
• assertion that there are well-defined
problems that cannot be solved by any
computational procedure.
23. What is Computable?
Computation is usually modelled as a mapping from inputs to outputs,
carried out by a formal “machine,” or program, which processes its
input in a sequence of steps.
An “effectively computable” function is one that can be computed in
a finite amount of time using finite resources.
…………
…………
…
input
yes
no
output
Effectively
computable
function
24. Unsolvable
• problems are formulated as functions, we call
unsolvable functions are
• noncomputable;
• Problems formulated as predicates,
• they are called
• undecidable.
• Using Turing's concept of the abstract machine,
we would say that a function is noncomputable if
there exists no Turing machine that could
compute it.
26. -Dept. of CSIE, CCU, Taiwan- 26
............
Read-Write head
No boundaries -- infinite length
The head moves Left or Right
The
tape
OR
27. -Dept. of CSIE, CCU, Taiwan- 27
............
Read-Write head
The head at each time step:
1. Reads a symbol
2. Writes a symbol
3. Moves Left or Right
28. -Dept. of CSIE, CCU, Taiwan- 28
............
Example:
Time 0
............
Time 1
a a cb
a b k c
1. Reads a
2. Writes k
3. Moves Left
a
29. -Dept. of CSIE, CCU, Taiwan- 29
............
Time 1
a b k c
............
Time 2
a k cf
1. Reads b
2. Writes f
3. Moves Right
b
30. Add 1 to a sequence of ones
If 1, then move right If 1, then move left
Loop
If 0, replace with 1
Start: 0 0 0 1 1 1 1 0 0 0 4 ones in a row
End: 0 0 0 1 1 1 1 1 0 0 5 ones in a row
31. So what?
You can add a 1 to a sequence of ones
From a set of a few (a manageable number) of primitive operations
If a number is represented by a sequence of ones
then you have the x+1 operator, i.e addition
This is a/(the most) basic mathematical operator
Powerful enough to represent complex mathematics
You can derive other (all) mathematic operations and objects
32. Powerful “language”
From a set of a few (a manageable number) of
primitive operations
Easier enough
To formulate
Proofs about algorithms
Powerful enough
To
represent complex
mathematics
All the ingredients are there Only have to deal with a few basic operations
Not worried about complexity…. Worried about, for example, “can” we compute?
33. -Dept. of CSIE, CCU, Taiwan- 33
The Input String
............
Blank symbol
head
a b ca
Head starts at the leftmost position
of the input string
Input string
34. -Dept. of CSIE, CCU, Taiwan- 34
States & Transitions
1q 2qLba ,
Read Write Move Left
1q 2qRba ,
Move Right
35. -Dept. of CSIE, CCU, Taiwan- 35
Turing machine for the language }{ nn
ba
0q 1q 2q3q
Rxa ,
Raa ,
Ryy ,
Lyb ,
Laa ,
Lyy ,
Rxx ,
Ryy ,
Ryy ,
4q
L,
49. -Dept. of CSIE, CCU, Taiwan- 49
0q 1q 2q3q
Rxa ,
Raa ,
Ryy ,
Lyb ,
Laa ,
Lyy ,
Rxx ,
Ryy ,
Ryy ,
4q
L,
yx
4q
x y
Halt & Accept
Time 13
50. What is Computable?
Computation is usually modelled as a mapping from inputs to outputs,
carried out by a formal “machine,” or program, which processes its
input in a sequence of steps.
An “effectively computable” function is one that can be computed in
a finite amount of time using finite resources.
…………
…………
…
input
yes
no
output
Effectively
computable
function
51. Uncomputable
Using Turing's concept of the abstract machine,
we would say that a function
is
noncomputable
if there exists
no Turing machine that could compute it.
52. Computability
Function f is computable
if some program P computes it:
For any input x,
the computation P(x)
halts
with output f(x)
Finite number of steps
53. Algorithm for 𝚷
• To calculate the area of a circle,
• we need the value of 𝚷
• Can we find an algorithm to compute 𝚷?
54. 𝚷
• 3.14159265358979323846264338327950288419716939937510
• 58209749445923078164062862089986280348253421170679
• 82148086513282306647093844609550582231725359408128
• 48111745028410270193852110555964462294895493038196
• 44288109756659334461284756482337867831652712019091
• 45648566923460348610454326648213393607260249141273
• 72458700660631558817488152092096282925409171536436
• 78925903600113305305488204665213841469519415116094
• 33057270365759591953092186117381932611793105118548
• 07446237996274956735188575272489122793818301194912
• 98336733624406566430860213949463952247371907021798
• 60943702770539217176293176752384674818467669405132
• 00056812714526356082778577134275778960917363717872
• 14684409012249534301465495853710507922796892589235
• 42019956112129021960864034418159813629774771309960
• 51870721134999999837297804995105973173281609631859
• 50244594553469083026425223082533446850352619311881
• 71010003137838752886587533208381420617177669147303
• 59825349042875546873115956286388235378759375195778
• 18577805321712268066130019278766111959092164201989…….
The first 1000 digits of 𝚷
Goes on as long as you want…
Π 10,000 digits
Π 100,000 digits
55. Algorithm for 𝚷
An algorithm (in general)
is
a finite sequence
of step by step, discrete, unambiguous
instructions for solving a particular problem
Problem
𝚷 is a infinite sequence of numbers…
cannot calculate with a finite number of steps
But there are algorithms????
56. Calculating 𝚷
• We can’t
– Calculate the EXACT value of 𝚷
• However, we can
– Calculate 𝚷 to whatever accuracy we desire
– Finite number of digits, finite number of steps
57. Probabilistic Algorithms
Calculate pi with a dart board
Area of square
d2
Area of Circle:
P
d
2
æ
è
ç
ö
ø
÷
2
prob =
circle
square
=
p
d
2
æ
è
ç
ö
ø
÷
2
d2
=
p
4
Probability dart will be in circle
d
number darts in circle
divided by
number of darts in total
times
Is π
Monte Carlo Method
Always Gives an answer
But not necessarily Correct
The probability of correctness goes up with time
58. Calculate
All do not stop (halt)…..
They can be an algorithm if only an approximation is needed
59. Halting function
• Decide whether program halts on input
– Given program P and input x to P,
Halt (P,x) =
Fact: There is no algorithm for Halt
yes if P(x) halts
no otherwise
Clarifications
Assume program P requires one string input x
Write P(x) for output of P when run in input x
Program P is string input to Halt
60. Unsolvability of the halting problem
• Suppose P solves variant of halting problem
– On input Q, assume
P(Q) =
• Build program D
– D(Q) =
• Does this make sense? What can D(D) do?
– If D(D) halts, then D(D) runs forever.
– If D(D) runs forever, then D(D) halts.
– CONTRADICTION: program P must not exist.
yes if Q(Q) halts
no otherwise
run forever if Q(Q) halts
halt if Q(Q) runs forever
62. Equivalent Computers
z z zz z z z
1
Start
HALT
), X, L
2:
look
for (
#, 1, -
), #, R
(, #, L
(, X, R
#, 0, -
Finite State Machine
...
Turing Machine
term = variable
| term term
| (term)
| variable . term
y. M v. (M [y v])
where v does not occur in M.
(x. M)N M [ x N ]
Lambda Calculus
63. What is Calculus?
• In High School:
d/dx xn = nxn-1 [Power Rule]
d/dx (f + g) = d/dx f + d/dx g [Sum Rule]
Calculus is a branch of mathematics that
deals with limits and the differentiation and
integration of functions of one or more
variables...
64. Real Definition
• A calculus is just a bunch of rules for
manipulating symbols.
• People can give meaning to those symbols,
but that’s not part of the calculus.
• Differential calculus is a bunch of rules for
manipulating symbols. There is an
interpretation of those symbols corresponds
with physics, slopes, etc.
67. Why?
• Once we have precise and formal rules for
manipulating symbols, we can use it to reason
with.
• Since we can interpret the symbols as
representing computations, we can use it to
reason about programs.
68. Universal Computer
• Lambda Calculus can simulate a Turing Machine
– Everything a Turing Machine can compute, Lambda
Calculus can compute also
• Turing Machine can simulate Lambda Calculus
– Everything Lambda Calculus can compute, a Turing
Machine can compute also
• Church-Turing Thesis: this is true for any other
mechanical computer also
69. Normal Steps
• Turing machine:
– Read one square on tape, follow one FSM
transition rule, write one square on tape, move
tape head one square
• Lambda calculus:
– One beta reduction
• Your PC:
– Execute one instruction (?)
• What one instruction does varies
70. -Reduction
(the source of all computation)
(x. M)N
This is the function definition
M is an expression with the variable x
N is used at the value of x
N substitutes x in the expression M
71. (x. x)a = a
-Reduction
(the source of all computation)
Identity operation
Function gives the argument as output
(x. xx)a = aa
(y.(x. xy)a)b =(y.ay)b=ab
(y.(x. xy) z.zy)b =(y.(z.zy)y)b
= (y. yy)b
=bb
72. -calculus
Useful and expression enough that a programming
language was modeled after it
LISP
was developed from -calculus
not the other way round.)
This was the “first” general language of
Artificial intelligence
Also
The “first” functional programming language
73. Complexity
Algorithm:
An algorithm is a finite sequence of step by step, discrete,
unambiguous instructions for solving a particular problem
Computable:
Does an algorithm exist??
Complexity:
The algorithm exists
How “complex” is it?
How many “descrete steps” does it have?
how much does each step cost?
How does complexity depend on size of input?
74. Vector/Array of size n
Efficient for the nth element
A single arithmetic calculation
Complexity does not increase as the vector gets bigger
O(1)
76 8 9 10 11 120 1 2 3 4 5
Pos(v[0]) Pos(v[0]) + 5
Find ith element of vector
This is exactly an example for what a vector is designed for
75. Cost/Complexity
Compute: Pos(v[0]) + n
Find the nth element
Cost:
C1: Cost of determining the address of the first element Pos(v[0])
Does not depend on the size of the array of numbers
C2: Cost of adding n to this address… cost of addition operation
Does not depend on the size of the array of numbers
Total cost: C1 + C2
76. Cost/Complexity
Compute: Pos(v[0]) + nFind the nth element
Cost: Does not depend on the size of the array of numbers
Total cost = T = C1 + C2
N: size of array
T:TotalCost
Cost(n) = T a constant amount
In the O notation
O(1)
The constant factor is not so important
It is only important that there in no
dependence on the size of the array
77. Vector/Array of size n
Insert before ith element in vector
76 8 9 10 11 120 1 2 3 4 5
1. Allocate vector of size n+1
2. Copy element 0 to i-1 to places 0 to i-1 and
copy elements i to n-1 to places i+1 to n
76 8 9 10 11 120 1 2 3 4 5
1 operation
n operations
3. Set in element
76 8 9 10 11 120 1 2 3 4 5 13
1 operation
Vectors are not designed to be used with insertion operations
As the vector gets larger, the insertion takes more time/operations
O(n)
78. Cost of insertion
• C1: Allocate array size n+1
• C2: Copy I elements from old to new array
• C2=(i)*c (c: the cost to copy an element)
• C3: Copy the element to insert
• C3= c (c: the cost to copy an element)
• C4: Copy n-I elements from old to new array
• C4=(n-i)*c
Insert a new element at position I in array of length n
Total Cost = T = C1 + C2 + C3 + C4
= C1 + i*c + c + (n-i)*c
= C1 + (n+1)*c
Total Cost = T = C1 + (n+1)*c
79. Cost of insertion
Insert a new element at position I in array of length n
Total Cost = T(n) = C1 + (n+1)*c
Cost: Depends on the size of the array of numbers
N: size of array
T:TotalCost
Cost(n) = C1 + (n+1)*c
In the O notation
O(n)
The constant factor is not so important
It is only important that there in no
dependence on the size of the array
80. Complexity
O(1) Time/operation complexity does not increase
with the size of the problem
O(n) Time/operation complexity does increases linearly
with the size of the problem
Find ith element of vector
Insert before ith element in vector
N: size of array
T:TotalCost
N: size of array
T:TotalCost
82. Bubble Sort: Idea
• Idea: bubble in water.
– Bubble in water moves upward. Why?
• How?
– When a bubble moves upward, the water
from above will move downward to fill in
the space left by the bubble.
83. Bubble Sort Example
9, 6, 2, 12, 11, 9, 3, 7
6, 9, 2, 12, 11, 9, 3, 7
6, 2, 9, 12, 11, 9, 3, 7
6, 2, 9, 12, 11, 9, 3, 7
6, 2, 9, 11, 12, 9, 3, 7
6, 2, 9, 11, 9, 12, 3, 7
6, 2, 9, 11, 9, 3, 12, 7
6, 2, 9, 11, 9, 3, 7, 12The 12 is greater than the 7 so they are exchanged.
The 12 is greater than the 3 so they are exchanged.
The twelve is greater than the 9 so they are exchanged
The 12 is larger than the 11 so they are exchanged.
In the third comparison, the 9 is not larger than the 12 so no
exchange is made. We move on to compare the next pair without
any change to the list.
Now the next pair of numbers are compared. Again the 9 is the
larger and so this pair is also exchanged.
Bubblesort compares the numbers in pairs from left to right
exchanging when necessary. Here the first number is compared
to the second and as it is larger they are exchanged.
The end of the list has been reached so this is the end of the first pass. The
twelve at the end of the list must be largest number in the list and so is now in
the correct position. We now start a new pass from left to right.
84. Bubble Sort Example
6, 2, 9, 11, 9, 3, 7, 122, 6, 9, 11, 9, 3, 7, 122, 6, 9, 9, 11, 3, 7, 122, 6, 9, 9, 3, 11, 7, 122, 6, 9, 9, 3, 7, 11, 12
6, 2, 9, 11, 9, 3, 7, 12
Notice that this time we do not have to compare the last two
numbers as we know the 12 is in position. This pass therefore only
requires 6 comparisons.
First Pass
Second Pass
85. Bubble Sort Example
2, 6, 9, 9, 3, 7, 11, 122, 6, 9, 3, 9, 7, 11, 122, 6, 9, 3, 7, 9, 11, 12
6, 2, 9, 11, 9, 3, 7, 12
2, 6, 9, 9, 3, 7, 11, 12
Second Pass
First Pass
Third Pass
This time the 11 and 12 are in position. This pass therefore only
requires 5 comparisons.
86. Bubble Sort Example
2, 6, 9, 3, 7, 9, 11, 122, 6, 3, 9, 7, 9, 11, 122, 6, 3, 7, 9, 9, 11, 12
6, 2, 9, 11, 9, 3, 7, 12
2, 6, 9, 9, 3, 7, 11, 12
Second Pass
First Pass
Third Pass
Each pass requires fewer comparisons. This time only 4 are needed.
2, 6, 9, 3, 7, 9, 11, 12Fourth Pass
87. Bubble Sort Example
2, 6, 3, 7, 9, 9, 11, 122, 3, 6, 7, 9, 9, 11, 12
6, 2, 9, 11, 9, 3, 7, 12
2, 6, 9, 9, 3, 7, 11, 12
Second Pass
First Pass
Third Pass
The list is now sorted but the algorithm does not know this until it
completes a pass with no exchanges.
2, 6, 9, 3, 7, 9, 11, 12Fourth Pass
2, 6, 3, 7, 9, 9, 11, 12Fifth Pass
88. Bubble Sort Example
2, 3, 6, 7, 9, 9, 11, 12
6, 2, 9, 11, 9, 3, 7, 12
2, 6, 9, 9, 3, 7, 11, 12
Second Pass
First Pass
Third Pass
2, 6, 9, 3, 7, 9, 11, 12Fourth Pass
2, 6, 3, 7, 9, 9, 11, 12Fifth Pass
Sixth Pass
2, 3, 6, 7, 9, 9, 11, 12
This pass no exchanges are made so the algorithm knows the list is
sorted. It can therefore save time by not doing the final pass. With
other lists this check could save much more work.
89. Bubble Sort: Example
• Notice that at least one element will be
in the correct position each iteration.
40 2 1 43 3 65 0 -1 58 3 42 4
652 1 40 3 43 0 -1 58 3 42 4
65581 2 3 40 0 -1 43 3 42 4
1 2 3 400 65-1 43 583 42 4
1
2
3
4
91. Bubble Sort Complexity
1 2 3 4 5 n-1 comparisons Total Cost= (n-1) * c
Even the lowest cost is a function of n
Cost = (n-1)*c = -c + n*c
O(n)
Remember: not so interested in the constant factors
N: size of array
T:TotalCost
92. Bubble Sort Complexity
5 4 3 2 1 Worse case: exactly opposite
4 5 3 2 1 Compare (c ) and swap (s)
4 3 5 2 1 Compare (c ) and swap (s)
4 3 2 5 1 Compare (c ) and swap (s)
4 3 2 1 5 Compare (c ) and swap (s)
3 2 1 4 5
3 4 2 1 5 Compare (c ) and swap (s)
3 2 4 1 5 Compare (c ) and swap (s)
Compare (c ) and swap (s)
1 2 3 4 5 Compare (c ) and swap (s)
Move 5 to place
4* (c+s)
Move 5 to place
3* (c+s)
Move 5 to place
1* (c+s)
2 3 1 4 5 Compare (c ) and swap (s)
2 1 3 4 5 Compare (c ) and swap (s)
Move 5 to place
2* (c+s)
93. Bubble Sort Complexity
Total Cost:
5*(c+s) + 4*(c+s) + 3*(c+s) + 2*(c+s) 1*(c+s)
(5+4+3+2+1)*(c+s)
15*(c+s)
3*5*(c+s)
In general the sum of numbers from 1 to n is:
If n odd: n*(n+1)/2
If n even n*n/2
94. Bubble Sort Complexity
Size of array to sort
Cost:
In general the sum of numbers
from 1 to n is:
If n odd: n*(n+1)/2= n2/2 + n/2
If n even n*n/2 = n2/2
Approx: n2/2
For a large n….
n2 is much much larger than n
96. Tractability
• Some problems are intractable:
as they grow large, we are unable to solve them
in reasonable time
• What constitutes reasonable time?
– Standard working definition: polynomial time
– On an input of size n the worst-case running time is
O(nk) for some constant k
– O(n2), O(n3), O(1), O(n lg n), O(2n), O(nn), O(n!)
– Polynomial time: O(n2), O(n3), O(1), O(n lg n)
– Not in polynomial time: O(2n), O(nn), O(n!)
97. Polynomial-Time Algorithms
• Are some problems solvable in polynomial time?
– Of course: many algorithms we’ve studied provide
polynomial-time solutions to some problems
• Are all problems solvable in polynomial time?
– No: Turing’s “Halting Problem” is not solvable by any
computer, no matter how much time is given
• Most problems that do not yield polynomial-time
algorithms are either optimization or decision
problems.
98. Optimization/Decision Problems
• An optimization problem tries to find an optimal solution
• A decision problem tries to answer a yes/no question
• Many problems will have decision and optimization versions
– Eg: Traveling salesman problem
• optimization: find hamiltonian cycle of minimum weight
• decision: is there a hamiltonian cycle of weight k
• Some problems are decidable, but intractable:
as they grow large, we are unable to solve them in reasonable
time
– Is there a polynomial-time algorithm that solves the problem?
99. The Class P
P: the class of decision problems that have polynomial-time
deterministic algorithms.
– That is, they are solvable in O(p(n)), where p(n) is a polynomial on n
– A deterministic algorithm is (essentially) one that always computes the
correct answer
Why polynomial?
– if not, very inefficient
– nice closure properties
• the sum and composition of two polynomials are always polynomials too
101. The class NP
NP: the class of decision problems that are solvable in polynomial
time on a nondeterministic machine (or with a nondeterministic
algorithm)
• (A determinstic computer is what we know)
• A nondeterministic computer is one that can “guess” the right
answer or solution
– Think of a nondeterministic computer as a parallel machine that can
freely spawn an infinite number of processes
• Thus NP can also be thought of as the class of problems
– whose solutions can be verified in polynomial time
• Note that NP stands for “Nondeterministic Polynomial-time”
102. Review: P And NP Summary
• P = set of problems that can be solved in polynomial
time
– Examples: Fractional Knapsack, …
• NP = set of problems for which a solution can be
verified in polynomial time
– Examples: Fractional Knapsack,…, Hamiltonian Cycle,
CNF SAT, 3-CNF SAT
• Clearly P NP
• Open question: Does P = NP?
– Most suspect not
– An August 2010 claim of proof that P ≠ NP, by Vinay
Deolalikar, researcher at HP Labs, Palo Alto, has flaws
103. NP-complete problems
• A decision problem D is NP-complete iff
1. D NP
2. every problem in NP is polynomial-time reducible
to D
• Cook’s theorem (1971): CNF-sat is NP-
complete
104. NP-Complete
“NP-Complete” comes from:
– Nondeterministic Polynomial
– Complete - “Solve one, Solve them all”
There are more NP-Complete problems than
provably intractable problems.