The document discusses the characteristics of algorithms and the concept of mathematical expectation in average case analysis. It then provides the pseudocode for the MaxMin algorithm and discusses the greedy knapsack algorithm and the travelling salesman problem. Finally, it explains the sum of subsets problem, describing two formulations and how the solution space can be organized into trees.

1. 1
Fundamentals of Algorithms
BT0080 Part-1
By Milan K Antony

2. 2
1.Write the different characteristics of an algorithm.
An algorithm is a logical sequence of discrete steps that describe a
complete solution to a given problem in a finite amount of time.It is
the set of rules that define how a particular problem can be solved in
finite number of steps.we can identify five important characteristics
of algorithms.
Algorithms are well-ordered
Since an algorithm is a collection of operations or instructions, we
must know the correct order in which to execute the instructions. If
the order is unclear, we may perform the wrong instruction or we
may be uncertain which instruction should be performed next. This
characteristic is especially important for computers. A computer can
only execute an algorithm if it knows the exact order of steps to
perform.
Algorithms have unambiguous operations
Each operation in an algorithm must be sufficiently clear so that it
does not need to be simplified. Given a list of numbers, you can
easily order them from largest to smallest with the simple instruction
"Sort these numbers." A computer, however, needs more detail to
sort numbers. It must be told to search for the smallest number,
how to find the smallest number, how to compare numbers together,
etc. The operation "Sort these numbers" is ambiguous to a computer
because the computer has no basic operations for sorting. Basic

3. 3
operations used for writing algorithms are known as primitive
operations or primitives. When an algorithm is written in computer
primitives, then the algorithm is unambiguous and the computer can
execute it.
Algorithms have effectively computable operations
Each operation in an algorithm must be doable, that is, the operation
must be something that is possible to do. Suppose you were given an
algorithm for planting a garden where the first step instructed you to
remove all large stones from the soil. This instruction may not be
doable if there is a four ton rock buried just below ground level.
For computers, many mathematical operations such as division by zero
or finding the square root of a negative number are also impossible.
These operations are not effectively computable so they cannot be
used in writing algorithms.
Algorithms produce a result
In our simple definition of an algorithm, we stated that an algorithm
is a set of instructions for solving a problem. Unless an algorithm
produces some result, we can never be certain whether our solution is
correct. Have you ever given a command to a computer and
discovered that nothing changed? What was your response? You
probably thought that the computer was malfunctioning because your
command did not produce any type of result. Without some visible
change, you have no way of determining the effect of your command.
The same is true with algorithms. Only algorithms which produce results
can be verified as either right or wrong.
Algorithms halt in a finite amount of time
Algorithms should be composed of a finite number of operations and
they should complete their execution in a finite amount of time.
Suppose we wanted to write an algorithm to print all the integers
greater than 1. Our steps might look something like this:

4. 4
1. Print the number 2.
2. Print the number 3.
3. Print the number 4.
.
.
.
While our algorithm seems to be pretty clear, we have two problems.
First, the algorithm must have an infinite number of steps because
there are an infinite number of integers greater than one. Second,
the algorithm will run forever trying to count to infinity. These
problems violate our definition that an algorithm must halt in a finite
amount of time. Every algorithm must reach some operation that tells
it to stop.
2. Explain the concept of Mathematical expectation in average case
analysis.
The concept of mathematical expectation is needed in average –
case
analysis of algorithms. In order to understand the concept better,
consider
the following example.
Example: Suppose, the students of MCA, who completed all the courses
in
the year 2005, had the following distribution of marks:
Range of marks Percentage of students who
scored in the range
0% to 20% 08
20% to 40% 20

5. 5
40% to 60% 57
60% to 80% 09
80% to 100% 06
If a student is picked up randomly from the set of students under
consideration, what is the % of marks expected of such a student?
After
scanning the table given above, we intuitively expect the student to
score
around the 40% to 60% class, because, more than half of the students
have
scored marks in and around this class.
Assuming that marks within a class are uniformly scored by the
students in
the class, the above table may be approximated by the following more
compact table.
% marks Percentage of students
scoring the marks
10* 08
30 20
50 57
70 09
90 06
As explained earlier, we expect a student picked up randomly, to
score
around 50% because more than half of the students have scored marks
around 50%

6. 6
This informal idea of expectation may be formalized by giving to each
percentage of marks, weight in proportion to the number of students
scoring
the particular percentage of marks in the above table.
Thus, we assign weight (8/100) to the score 10% ( 8, out of 100
students score on the average 10% marks);[20/100] to the score 30%
and so on.
Thus,
Expected % of marks
=10*8/100+30*20/100+50*57/100+70*9/100+90*6/100=47
The final calculation of expected marks of 47 is roughly equal to our
intuition
of the expected marks, according to our intuition, to be around 50.
We generalize and formalize these ideas in the form of the following
definition.
3. Write the MaxMin algorithm.
1. Algorithm MaxMin (i,j, max, min)
2. // a[1:n] is a global array Parameter i and j are integers,
3. // 1  i  j < n. The effect is to set max and min to the
4. // largest and smallest values in a [i, j], respectively.
5. {
6. if (i=j) then max:=min:=a[i]; //small(P)

7. 7
7. else if (i = j – 1) then //Another case of Small (P)
8. {
9. if a [i] < a[j] then
10. {
11. max:=a[j]; min:=a[i];
12. }
13. else
14. {
15. max:=a[i]; min:=a[j]
16. }
17. }
18. else
19. {
20. // If P is not small, divide P into subproblems
21. // Find where to split the set
22. mid:= (i+j)/2;
23. //Solve the subproblems
24. Maxmin (i, mid, max, min);
25. MaxMin (mid+i, j, max1, min1);
26. // Combine the solutions
27. if (max<max 1) then max:=max1;
28. if(min>min1) then min:=min1;
29. }
30. }
4.Prove that “ If then, Greedy knapsack algorithm
generates an optimal solution to the given instance of the
knapsack problem”
Let x = (x1, …….,xn) be the solution generated by Greedy

8. 8
knapsack
algorithm. If all the xi equal one, then clearly the solution is optimal.
So, let j
be the least index such that xj 1. From the algorithm it follows that
xi=1 for
1 i<j, xi = 0 for j<i n and 0 xj<1. Let y=(y1, ……, yn) be an optimal
solution
we can assume that wjyi = m. Let k be the least index such that yk
xk.
Clearly, such a k must exist. It also follows that yk<xk. To see this,
consider
the three possibilities k<j, k=j or k>j.
1. If k<j, then xk=1, But, yk xk, and so yk < xk.
2. k=j, then since wiyi = m and yi = xi for 1 i<j. it follow that either
yk<xk
or wiyi = m
3. k > j, then wiyi = m, and this is not possible.
Now, suppose we increase yk to xk and decrease as many of (yk+1,
……., yn)
as necessary so that the total capacity used is still on. This results in
a new
solution z=(z1, …….., zn) with zi=xi, 1 i k and k<i nwi (yi - zi) =
wk(zk-yk),
Then, for z we have
if

9. 9
these sums are equal, then either z=x and x is optimal, or z x. In
the latter case,
repeated use of the above argument will either show that y is not
optimal or
transform y into x and thus show that x too is optimal.
Greedy algorithm for sequencing unit time jobs with deadlines and
profits
For JS, there are two possible parameters in terms of which its
complexity
can be measured. We can use n, number of jobs and s, the number
of jobs
included in the solution J. The while loop of line 15 in Algorithm 4.4
is
iterated at most K times. Each iteration takes (1) time. If the
condition; of
line 16 is true, then lines 19 and 20 are executed. These lines require
k-r)
time to insert job i. Hence, the total time for each iteration of the
for loop of line 10 is (k). This loop is iterated n-1 times. If s is
the final value of k, that is, S is the number of jobs in the final
solution, then the total time needed by algorithm JS is (Sn). Since
Sn, the worst-case time, as a function of n alone is (n2). If we
consider the job set pi=di=n — i + 1, 1 i n, then algorithm JS takes
(n2) time to determine J. Hence. The worst-case computing time
for JS is (n2) In addition to the space needed for d, JS needs
(S) amount of space for J.
5. Explain the concept of travelling salesman problem.

10. 10
A tour of G is a directed simple cycle that includes every vertex
in V. The cost of the tour is the sum of the cost of the edges on
the tour. The travelling salesperson problem is to find a tour of
minimum cost.
Let G=(V,E) be a directed graph with edges cost cij. The variable cij
is
defined such that cij = 0, cij> o for all i and j and cij = if < i,j> E.
Let lVl = n
and assume n>1. A tour is a simple path that starts and ends at
vertex 1.
Every tour consists of an edge < 1, k> for some k V — {1} and a path
from
vertex k to vertex 1. The path from vertex k to vertex 1 goes
through each
vertex in V-{1,k} exactly once. It easy to see that if the tour is
optimal, then
the path from k to 1 must be a shortest k to 1 path going through all
vertices
in V– {1,k). Hence, the principle of optimality holds. Let g (i,s) be
the length
of a shortest path starting at vertex i, going through all vertices in
S, and
terminating at vertex 1. The function g(1, V-{1}) is the length of an
optimal
salesperson tour. From the principal of optimality if follows that
g(1,V {1}) min { c1k g(k,V { j ,k })}
2 k n
……….5.5

11. 11
Generalizing (5.5), we obtain (for i s)
g( i ,S) min{ cij g( j ,S { j })}
j S
…………..5.6
Equation 5.5 can be solved for g (1, V- {1}) if we know g (k, V-
{1,k}) for all
choices of k. The g values can be obtained by using (5.6). Clearly, g
(i, ) =
Ci1, i 1 n.
6. What do you mean by sum of subsets problem? Explain?
Given positive numbers wi, 1 i n, and m, this problem calls for
finding all subsets of the wi whose sums are m. For example, if n =
4, (w1, w2, w3, w4) = (11, 13, 24, 7), and m = 31 then the desired
subsets are (11, 13, 7) and (24, 7) rather than represent the solution
vector by the wi which sum to m, we could represent the solution as
described by the vector (1,2,4) and (3, 4). In general, all solutions
are k-tuples (x1, x2…….. xk), 1 k n, and different solutions may have
different sized tuples. The explicit constraints xj {jlj is an integer
and 1 j n}. The implicit requires that no two be the same and that
the sum of the corresponding will be m. Since we wish to avoid
generating multiple instance of the same subject (e.g (1, 2, 4) and
(1, 4, 2) represent the same subject), another implicit constraints

12. 12
that is imposed is xi<xi+1, I i < k.
In another formulation of the sum of subsets problem, each solution
subject is represented by n-tuple (x1, x2, ….. xn) such that xi {0,1},
1 i n. Then, xi = 0, if wi is not chosen then the above instance are
(1, 1, 0, 1) and (0, 0, 1, 1). The formulation express all solutions
using fixed sized tuple. Thus, we conclude that there may be several
ways to formulate a problem so that all solutions are tuples that
satisfy some constraints. One can verify that for both of the above
formulations, the solutions space consists of 2n distinct tuples.
Backtracking algorithms determine problem solutions by systematically
searching the solution space for the given problem instance. This
search is facilitated by using a tree organization for the solution
space. For a given solution space, many tree organizations may be
possible. The next two examples examine some of the ways to
organize a solution into a tree.
We have two possible formulations of the solutions space for the sum
of subsets problem. Fig 6.2 and Fig 6.3 show a possible tree
organization for each of these formulations for the case n=4. The
tree of Fig 6.2 corresponds to the variable tuple size formulation.
The edges are labeled such that an edge from a level i node to a
level i + 1 node represents a value for xi. At each node the solution
space is partitioned into solution space. The solution space is defined
by all paths from the root node to any node in the tree, since any
such path corresponds to a subset satisfying the explicit constraints.
The possible paths are ( ) (this corresponds to the empty path from
the root to itself), (1), (1,2), (1,2,3), (1,2,3,4), (1,2,4), (1,3,4),
(2), (2,3) and so on. Thus, left most subtree defines all subsets

13. 13
containing wi, the next subtree defines all subsets containing w2 but
not w1, and so on.
The tree of Fig 6.3 corresponds to the fixed tuples size formulation.
Edges from level i nodes to level i + I nodes are labeled with the
value of xi, which is either zero or one. All paths from the root to a
leaf node define the solution space. The left subtree of the root
defines all subsets containing wl, the right subtree defines all subsets
not containing wl and so on. Now there are 24 leaf nodes which
represent 16 possible tuples.
At
this
point, it is useful to develop some terminology regarding tree
organization of solution spaces. Each node in this tree defines a
problem state. All paths from the root to other nodes define the
state space of the problem. Solution states are those problem states
s for which the path from the root to s defines a tuple in the
solution space. In the tree Fig 6.2, all nodes are solutions states
whereas in the tree Fig 6.3, only leaf nodes are solution states.
Answer states are those solution states s for which the path from the

14. 14
root to s defines a tuple that is a member of the set of solutions
(i.e. it satisfies the implicit constraints) of the problem.
The tree organization of the solution space is referred to as the
state space
tree.
Suppose we are given distinct positive number and we desire to find
all
combinations of these numbers whose sums are m. This is called the
sum
of subsets problem. The example in above shows how we could
formulate
this problem using either fixed or variable sized tuples. We consider a
back
tracking solution using the fixed tuple size strategy. In this case, the
element
xi of the solution vector is either one or zero depending on whether
the
weight wi, is included or not.
The children of any node in Fig 6.3 are easily generated. For a node
at level
i, the left child corresponds to xi = 1 and the right to xi = 0.
A simple choice for the bounding functions Bk
(x1, ……. xi) = true if
k n
wx + w > m
i 1 i=k+1 i
The given sum problem can be solved using generating functions.
Consider the number of ways to select out of
given integers such that their sum equals , and define
the generating function