Aoa amortized analysis

Amortized Analysis
Algorithm and Complexity
Salabat Khan | 3820160108
1

Amortized Analysis
What is Amortized Analysis ?
In amortized analysis, the time required to perform
a sequence of operations is averaged over all the
operations performed.
 No involvement of probability
Average performance on a sequence of operations,
even some operation is expensive.
Guarantee average performance of each operation
among the sequence in worst case.
Amortized analysis is not just an analysis tool, it is also a way of
thinking about designing algorithms.
2017/05/15 2Amortized Analysis

Amortized Analysis
Methods of Amortized Analysis
 Accounting Method: we overcharge
some operations early and use them
to as prepaid charge later.
 Aggregate Method: we determine an
upper bound T(n) on the total sequence
of n operations. The cost of each will
then be T(n)/n.
 Potential Method: we maintain credit
as potential energy associated with
the structure as a whole.

Amortized Analysis
1. Aggregate Method
Show that for all n, a sequence of n operations take
worst-case time T(n) in total
In the worst case, the average cost, or amortized cost ,
per operation is T(n)/n.
The amortized cost applies to each operation, even
when there are several types of operations in the
sequence.

Amortized Analysis
3 ops:
Push(S,x) Pop(S)
Multi-
pop(S,k)
Worst-case
cost:
O(1) O(1)
O(min(|S|,k)
= O(n)
Amortized cost: O(1) per operation
Aggregate Analysis: Stack Example

Amortized Analysis
Sequence of n push, pop, Multipop operations
Worst-case cost of Multipop is O(n)
Have n operations
Therefore, worst-case cost of sequence is O(n2
)
Observations
Each object can be popped only once per time that it’s
pushed
Have <= n pushes => <= n pops, including those in
Multipop
Therefore total cost = O(n)
Average over n operations => O(1) per operation on
average
Notice that no probability involved
……. Aggregate Analysis: Stack Example

Amortized Analysis
2. Accounting Method
Charge i th operation a fictitious amortized cost ĉi,
where $1 pays for 1 unit of work (i.e., time).
Assign different charges (amortized cost ) to different
operations
 Some are charged more than actual cost
 Some are charged less
This fee is consumed to perform the operation.
Any amount not immediately consumed is stored in the
bank for use by subsequent operations.
The bank balance (the credit) must not go negative!
We must ensure that
for all n.
∑∑
==
≤
n
i
i
n
i
i cc
11
ˆ
Thus, the total amortized costs provide an upper
bound on the total true costs.

Amortized Analysis
3 ops:
Push(S,x) Pop(S) Multi-pop(S,k)
•Assigned
cost:
2 0 0
•Actual cost: 1 1 min(|S|,k)
Push(S,x) pays for possible later pop of x.
….. Accounting Method: Stack Example

Amortized Analysis
….. Accounting Method: Stack Example
When pushing an object, pay $2
$1 pays for the push
$1 is prepayment for it being popped by either pop
or Multipop
Since each object has $1, which is credit, the
credit can never go negative
Therefore, total amortized cost = O(n), is an upper
bound on total actual cost

Amortized Analysis
….. Accounting Method: Binary Counter
k-bit Binary Counter: A[0..k−1]
INCREMENT(A)
1. i ← 0
2. while i < length[A] and A[i] = 1
3. do A[i] ← 0 ⊳ reset a bit
4. i ← i + 1
5. if i < length[A]
6. then A[i] ← 1 ⊳ set a bit
∑ ⋅= −
=
1
0 2][k
i
i
iAx
Introduction

Amortized Analysis
Consider a sequence of n increments. The
worst-case time to execute one increment is
Θ(k). Therefore, the worst-case time for n
increments is n · Θ(k) = Θ(n⋅ k).
WRONG! In fact, the worst-case cost for n
increments is only Θ(n) ≪ Θ(n⋅ k).
Let’s see why. Note: You’d be correct
if you’d said O(n⋅ k).
But, it’s an overestimate.

Amortized Analysis
Ctr A[4] A[3] A[2] A[1] A[0] Cost
0 0 0 0 0 0 0
1 0 0 0 0 1 1
2 0 0 0 1 0 3
3 0 0 0 1 1 4
4 0 0 1 0 0 7
5 0 0 1 0 1 8
6 0 0 1 1 0 10
7 0 0 1 1 1 11
8 0 1 0 0 0 15
9 0 1 0 0 1 16
10 0 1 0 1 0 18
11 0 1 0 1 1 19
A[0] flipped every op n
A[1] flipped every 2 ops n/2
… … … … …
A[i] flipped every 2i
ops n/2i
Total cost of n operations

Amortized Analysis
 
)(
2
2
1
2
1
lg
1
n
nn
n
i
i
n
i
i
Θ=
=<




=
∑
∑
∞
=
=
Cost of n increments
Thus, the average cost of each increment operation is
Θ(n)/n = Θ(1).

Amortized Analysis
Example:
Charge an amortized cost of $2 every time a bit is set from 0 to 1
• $1 pays for the actual bit setting.
• $1 is stored for later re-setting (from 1 to 0).
At any point, every 1 bit in the counter has $1 on it… that pays for
resetting it. (reset is “free”)
0 0 0 1$1
0 1$1
0
0 0 0 1$1
0 1$1
1$1
0 0 0 1$1
1$1
0 0
Cost = $2
Cost = $2

Amortized Analysis
When Incrementing,
Amortized cost for line 3 = $0
Amortized cost for line 6 = $2
Amortized cost for INCREMENT(A) = $2
Amortized cost for n INCREMENT(A) = $2n =O(n)
INCREMENT(A)
1. i ← 0
2. while i < length[A] and A[i] = 1
3. do A[i] ← 0 ⊳ reset a bit
4. i ← i + 1
5. if i < length[A]
6. then A[i] ← 1 ⊳ set a bit

Amortized Analysis
Accounting Method vs. Aggregate Method
Aggregate Method :
First analyze entire sequence
Then calculate amortized cost per operation
Accounting method:
First assign amortized cost per operation
Check that they are valid .
Then compute cost of entire sequence of
operations

Amortized Analysis
3. Potential Method
IDEA: View the bank account as the
potential energy (as in physics) of the
dynamic set.
FRAMEWORK:
Start with an initial data structure D0.
Operation i transforms Di–1 to Di.
The cost of operation i is ci.
Define a potential function Φ : {Di} → R,
such that Φ(D0 ) = 0 and Φ(Di ) ≥ 0 for all i.
The amortized cost ĉi with respect to Φ is
defined to be ĉi = ci + Φ(Di) – Φ(Di–1).

Amortized Analysis
Like the accounting method, but think of the
credit as potential stored with the entire data
structure.
 Accounting method stores credit with
specific objects while potential method
stores potential in the data structure as a
whole.
 Can release potential to pay for future
operations
Most flexible of the amortized analysis
methods .
….. Potential Method

Amortized Analysis
ĉi = ci + Φ(Di) – Φ(Di–1)
potential difference ∆Φi
 If ∆Φi > 0, then ĉi > ci. Operation i
stores work in the data structure for later
use.
 If ∆Φi < 0, then ĉi < ci. The data
structure delivers up stored work to help
pay for operation i.

Amortized Analysis
The total amortized cost of n operations is
Summing both sides telescopically.
)()( 0
1
DDc n
n
i
i Φ−Φ+= ∑=
( )∑∑ =
−
=
Φ−Φ+=
n
i
iii
n
i
i DDcc
1
1
1
)()(ˆ
∑=
≥
n
i
ic
1
since Φ(Dn) ≥ 0 and Φ(D0 ) = 0.

Amortized Analysis
….. Potential Method: Stack Example
Define: φ(Di) = #items in stack Thus, φ(D0)=0.
Plug in for operations:
Push: ĉi = ci + φ(Di) - φ(Di-1)
= 1 + j - (j-1)
= 2
Pop: ĉi = ci + φ(Di) - φ(Di-1)
= 1 + (j-1) - j
= 0
Multi-pop: ĉi = ci + φ(Di) - φ(Di-1)
= k’ + (j-k’) - j k’=min(|S|,k)
= 0

Amortized Analysis
….. Potential Method: Binary Counter
Define the potential of the counter after the ith
operation
by Φ(Di) = bi, the number of 1’s in the counter after the ith
operation.
Note:
• Φ(D0 ) = 0,
• Φ(Di) ≥ 0 for all i.
Example:
0 0 0 1 0 1 0
0 0 0 1$1
0 1$1
0 Accounting method)(

Amortized Analysis
The amortized cost of the i th INCREMENT is
ĉi = ci + Φ(Di) – Φ(Di–1)
= (ti + 1) + (1 − ti)
= 2
Assume ith INCREMENT resets ti bits (in line 3).
Actual cost ci = (ti + 1)
Number of 1’s after ith operation: bi = bi–1 – ti + 1
Therefore, n INCREMENTs cost Θ(n) in the worst case.

Amortized Analysis
….. Dynamic Tables
Problem: if too many items inserted, table may be too
small
Implementing a table (e.g., hash table) for dynamic
data, want to make it small as possible
Idea: allocate more memory as needed

Amortized Analysis
In some applications:
We don't know how many objects will be stored in a table.
Similarly, if many objects are deleted from the table:
It may be worthwhile to reallocate the table with a smaller size .
This problem is called
Dynamically Expanding and Contracting a table.
….. Dynamic Tables: Why Dynamic Tables?

Amortized Analysis
Using amortized analysis we will show that :
The amortized cost of insertion and deletion is O(1).
Even though the actual cost of an operation is large when it triggers an
expansion or a contraction.
We will also show how to guarantee that:
The unused space in a dynamic table never exceeds a constant fraction of
the total space.
Operations on Dynamic Table
TABLE-INSERT: Inserts into the table an item that occupies a single slot.
TABLE-DELETE: Removes an item from the table & frees its slot.
Load Factor: For empty its 1 and for non empty its given below
….. Dynamic Tables:
tabletheof)(
tablein thestoreditemsofNumber
)(
slotsofnumbersize
T =α

Amortized Analysis
Assumption:
Table is allocated as an array of slots.
A table fills up when:
All slots have been used.
Equivalently, when its load factor becomes 1.
Table-Expansion occurs when:
An item is to be inserted into a full table.
A Common Heuristic:
Allocate a new table that has twice as many slots as the old one.
Hence, insertions are performed if only :
….. Insertion-Only Dynamic Tables:
1 / 2 ≤ α(T) ≤ 1

Amortized Analysis
Table Insert:
….. Insertion-Only Dynamic Tables:
1 / 2 ≤ α(T) ≤ 1
TABLE-INSERT (T, x)
1. if size[T] = 0 then
2. allocate table[T] with 1 slot
3. size[T] ← 1
4. if num[T] = size[T] then
5. allocate new-table with 2.size[T] slots
6. copy all items in table[T] into new-table
7. free table[T]
8. table[T] ← new-table[T]
9. size[T] ← 2.size[T]
10. insert x into table[T]
11. num[T] ← num[T] + 1
12. end
table[T] : pointer to block
of table storage
num[T] : number of
items in the table
size[T] : total number of
slots in the table
Initially, table is empty,
so num[T] = size[T] = 0

● Let ci = cost of ith insert
● ci = i if i-1 is exact power of 2, 1 otherwise
● Example:
■ Operation Table Size Cost
Insert(1) 1 1 1
Amortized Analysis

● Example:
Insert(1) 1 1 1
Insert(2) 2 1 + 1 2
Amortized Analysis

● Example:
Insert(1) 1 1 1
Insert(2) 2 1 + 1 2
Insert(3) 4 1 + 2 3
Amortized Analysis

● Example:
Insert(1) 1 1 1
Insert(2) 2 1 + 1 2
Insert(3) 4 1 + 2 3
Insert(4) 4 1 4
Amortized Analysis

Let ci = cost of ith insert
ci = i if i-1 is exact power of 2, 1 otherwise
Example:
Insert(1) 1 1 1
Insert(2) 2 1 + 1 2
Insert(3) 4 1 + 2 3
Insert(4) 4 1 4
Insert(5) 8 1 + 4 5
Amortized Analysis

● Example:
Insert(1) 1 1 1
Insert(2) 2 1 + 1 2
Insert(3) 4 1 + 2 3
Insert(4) 4 1 4
Insert(5) 8 1 + 4 5
Insert(6) 8 1 6
Amortized Analysis

● Example:
Insert(1) 1 1 1
Insert(2) 2 1 + 1 2
Insert(3) 4 1 + 2 3
Insert(4) 4 1 4
Insert(5) 8 1 + 4 5
Insert(6) 8 1 6
Insert(7) 8 1 7
Amortized Analysis

● Example:
Insert(1) 1 1 1
Insert(2) 2 1 + 1 2
Insert(3) 4 1 + 2 3
Insert(4) 4 1 4
Insert(5) 8 1 + 4 5
Insert(6) 8 1 6
Insert(7) 8 1 7
Insert(8) 8 1 8
Amortized Analysis

● Example:
Insert(1) 1 1 1
Insert(2) 2 1 + 1 2
Insert(3) 4 1 + 2
Insert(4) 4 1
Insert(5) 8 1 + 4
Insert(6) 8 1
Insert(7) 8 1
Insert(8) 8 1
Insert(9) 16 1 + 8
1
2
3
4
5
6
7
8
9
Amortized Analysis

Amortized Analysis
1. Aggregate Method
Table is initially empty.
Observe:
o i-th operation causes an expansion only when i-1
is an exact power of 2.



=
otherwise1
2ofpowerexactanisif ii
ci

Amortized Analysis
…Aggregate Method
Therefore the total cost of n TABLE-INSERT
operations
 The amortized cost of a single operation is 3n/n=3 = O(1)
 
nnnnnc
n
j
j
n
j
j
n
i
i 3222
lg
0
lg
01
=+=+<+= ∑∑∑ ===
i 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 ...
ci
1 2 3 1 5 1 1 1 9 1 1 1 1 1 1 1 17 1 1 1 ...
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 ...
Expansion
cost
1 2 4 8 16

Amortized Analysis
Assign the following amortized costs
– Table-Expansion : 0
– Insertion of a new item : 3
Insertion of a new item
a) 1 (as an actual cost) for inserting itself into the table
b) 1 (as a credit) for moving itself in the next expansion
c) 1 (as a credit) for moving another item (in the next
expansion) that has already moved in the last expansion
… Accounting Method: Dynamic Tables

Amortized Analysis
Size of the table: M
Immediately after an expansion (just before the insertion)
num[T] = M/2 and size[T] = M where M is a power of 2.
Table contains no credits
…Accounting Method : Dynamic Tables

Amortized Analysis
1st insertion
2nd insertion
(a) $1 for
insertion(b)
(c)

Amortized Analysis
M/2th Insertion
Thus, by the time the table contains M items and
is full
– each item in the table has $1 of credit to pay for its
move during the next expansion

Amortized Analysis
Define a potential function Φ that is
– 0 immediately after an expansion
– builds to the table size by the time table becomes full
Next expansion can be paid for by the potential.
One possible Φ is:
Φ(T) = 2*num[T] –size[T]
Immediately after an expansion
size[T] = 2*num[T] ⇒ Φ(T) = 0
Immediately before an expansion
size[T] = num[T] ⇒ Φ(T) = num[T]
The initial value for the potential is 0
……. Potential Method: Dynamic Tables

Amortized Analysis
Definition of Φ:
Since the table is at least half full
(i.e. num[T] ≥ size[T] / 2)
Φ(T) is always nonnegative.
Thus, the sum of the amortized cost of n TABLE-INSERT
operations is an upper bound on the sum of the actual
costs.

Amortized Analysis
Analysis of i-th Table Insert:
numi : num[T] after the i-th operation
sizei : size[T] after the i-th operation
Φi : Potential after the i-th operation
Initially we have numi = sizei = Φi = 0
Note that, numi = numi-1 +1 always hold.

Amortized Analysis
Insertion without Expansion:
sizei = sizei – 1 ; ci = 1
Insertion with Expansion:
sizei = 2.sizei – 1 and sizei-1 =numi – 1 =numi - 1

Amortized Analysis
Adding Delete Operation:
TABLE-DELETE: Remove the specified item from the
table. It is often desirable to contract the table.
In table contraction, we would like to preserve two
properties
 Load factor of dynamic table is bounded below by a
constant
 Amortized cost of a table operation is bounded above
by a constant
We assume that the cost can be measured in terms of
elementary insertions and deletions

Amortized Analysis
A natural strategy for expansion and contraction
• Double the table size when an item is inserted into a
full table
• Halve the size when a deletion would cause
< 1 / 2
This strategy guarantees
Unfortunately, it can cause the amortized cost of an
operation to be quite large
)(Tα
1)(
2
1
≤≤ Tα

Amortized Analysis
Worst-Case for α(T) ≥ ½
Consider the following worst case scenario
– We perform n operations on an empty table
where n is a power of 2
– First n/2 operations are all insertions , cost a
total of Θ(n)
at the end: we have num[T] = size[T] = n/2
– Second n/2 operations repeat the sequence
I D D I
that is I D D I I D D I I D D I ...

Amortized Analysis
Example: n=16
In the second n/2 operations
– The first INSERT cause an expansion
– Two further DELETEs cause contraction
– Two further INSERTs cause expansion ... and so on
Hence there are n/8 expansions and n/8 contractions
The cost of each expansion and contraction is ≈ n/2

Amortized Analysis
Thus the total cost of n operations is Θ(n2
) since
– First n/2 operations : 3n
– Second n/2 operations : (n/4)*(n/2)=n2
/8
The amortized cost of an operation is Θ(n)
The difficulty with this strategy is
– After an expansion, we do not perform enough
deletions to pay for a contraction
– After a contraction, we do not perform enough
insertions to pay for an expansion

Amortized Analysis
Improving Expansion – Contraction
We can improve upon this strategy by allowing α(T)
to drop below ½
We continue to double the table size when an item
is inserted into a full table
But, we halve the table size (perform contraction)
when a deletion causes α(T) < ¼ rather than α(T)
< ½ ,
Therefore, ¼ ≤ α(T) ≤ 1

Amortized Analysis
Improving Expansion – Contraction
Hence after an expansion, α(T) = ½ , thus at least
half of the items in the table must be deleted
before a contraction can occur.
Similarly, after a contraction α(T) = ½ , thus the
number of items in the table must be doubled by
insertions before an expansion can occur.

Amortized Analysis
Define the potential function as follows
• Φ(T) = 0 immediately after an expansion or
contraction
• Recall that, α(T) = ½ immediately after and
expansion or contraction,
therefore the potential should build up to num[T]
as α(T) increases to 1 or decreases to ¼
• So that the next expansion or contraction can be
paid by the potential.

Amortized Analysis
Description of New Φ: One such Φ is
• Φ(T) = 0 when α = ½
• Φ(T) = num[T] when α = ¼
• Φ (T)= 0 for an empty table (num[T] = size[T]=0)
• Φ is always nonnegative





<−
≥−
=Φ
2
1
(T)if][
2
size[T]
2
1
(T)if][][2
)(
α
α
Tnum
TsizeTnum
T

Amortized Analysis
Operations are:
– TABLE-INSERT
– TABLE-DELETE

Amortized Analysis
Table Insert:
 αi-1 ≥ ½
 Analysis is identical to that for table expansion so ĉi = 3.
 αi-1 < ½ and αi < ½
Expansion may not occur (ĉi = 1 , sizei = sizei-1 )

Amortized Analysis
Table Insert:
 αi-1 < ½ but αi ≥ ½
Thus, amortized cost of a TABLE-INSERT operation is at most 3.

Amortized Analysis
Table Delete:
 αi-1 < ½ (No contraction) :sizei = sizei-1
 αi-1 < ½ (Contraction) ĉi =numi + 1 , sizei /2= sizei-1 /4 = numi-
1= numi + 1

 Thus, the amortized cost of a TABLE-DELETE operation is at
most 2
 Since the amortized cost of each operation is bounded above by
a constant
 The actual time for any sequence of n operations on a Dynamic
Table is O(n)
Amortized Analysis
….. Dynamic Tables

Amortized Analysis

Aoa amortized analysis

More Related Content

What's hot

Similar to Aoa amortized analysis

Recently uploaded

Aoa amortized analysis