This document discusses instruction pipelining as a technique to improve computer performance. It explains that pipelining allows multiple instructions to be processed simultaneously by splitting instruction execution into stages like fetch, decode, execute, and write. While pipelining does not reduce the time to complete individual instructions, it improves throughput by allowing new instructions to begin processing before previous instructions have finished. The document outlines some challenges to achieving peak performance from pipelining, such as pipeline stalls from hazards like data dependencies between instructions. It provides examples of how data hazards can occur if the results of one instruction are needed by a subsequent instruction before they are available.

1. Pipelining

2. Overview
 Greater performance can be achieved by
taking advantage of improvements in
technology like:
 Faster Circuitry.
 Processor organization improvements.
 Use of multiple registers instead of single
accumulator.
 Use of Cache memory
 Another important approach is instruction
pipelining

3. Pipelining
 An instruction pipeline is a technique used in the
design of computers to increase their instruction
throughput (the number of instructions that can be
executed in a unit of time).
 Pipelining does not reduce the time to complete an
instruction, but increases instruction throughput by
performing multiple operations in parallel.
 In pipelining new jobs are accepted at one end
before previously accepted inputs appear as
outputs.

4. Pipelining
 Pipelined organization requires sophisticated
compilation techniques.
 Pipelining is widely used in modern
processors.
 Pipelining improves system performance in
terms of throughput.

5. Basic Concepts

6. Traditional Pipeline Concept
Laundry Example
Adil, Bilal, Cathy, David
each have one load of clothes
to wash, dry, and fold
Washer takes 30 minutes
Dryer takes 40 minutes
“Folder” takes 20 minutes
A B C D

7. Traditional Pipeline Concept
 Sequential laundry takes 6
hours for 4 loads
 If they learned pipelining,
how long would laundry
take?
A
B
C
D
30 40 20 30 40 20 30 40 20 30 40 20
6 PM 7 8 9 10 11 Midnight
Time

8. Traditional Pipeline Concept
 Pipelined laundry takes
3.5 hours for 4 loads
A
B
C
D
6 PM 7 8 9 10 11 Midnight
T
a
s
k
O
r
d
e
r
Time
30 40 40 40 40 20

9. Traditional Pipeline Concept
 Pipelining doesn’t help
latency of single task, it
helps throughput of entire
workload
 Multiple tasks operating
simultaneously using
different resources.
 Potential speedup = Number
of pipe stages
 Unbalanced lengths of pipe
stages reduces speedup
 Time to “fill” pipeline and
time to “drain” it reduces
speedup
 Stall for Dependences
A
B
C
D
6 PM 7 8 9
T
a
s
k
O
r
d
e
r
Time
30 40 40 40 40 20

10. Concept of Pipelining in
Computers
 Each instruction is split into a sequence of
dependent stages:
[Fetch,Decode,Execute,Write].
 The first step is always to fetch the instruction
from memory; the final step is usually writing the
results of the instruction to processor registers or
to memory.
 Pipelining seeks to let the processor work on as
many instructions as there are dependent steps.

11. Use the Idea of Pipelining in a
Computer
F
1
E
1
F
2
E
2
F
3
E
3
I1 I2 I3
(a) Sequential execution
Instruction
fetch
unit
Execution
unit
Interstage buffer
B1
(b) Hardware organization
Time
F1 E1
F2 E2
F3 E3
I1
I2
I3
Instruction
(c) Pipelined execution
Figure 8.1. Basic idea of instruction pipelining.
Clock cycle 1 2 3 4
Time
Fetch + Execution

12. Use the Idea of Pipelining in a
Computer
F4I4
F1
F2
F3
I1
I2
I3
D1
D2
D3
D4
E1
E2
E3
E4
W1
W2
W3
W4
Figure 8.2. A 4-stage pipeline.
(a) Instruction execution div ided into f our steps
F : Fetch
instruction
D : Decode
instruction
and f etch
operands
E: Execute
operation
W : Write
results
Interstage buff ers
(b) Hardware organization
B1 B2 B3
Fetch + Decode
+ Execution + Write

13. Role of Cache Memory
 Each pipeline stage is expected to complete in
one clock cycle.
 The clock period should be long enough to let
the slowest pipeline stage to complete.
 Since main memory is very slow compared to
the execution, if each instruction needs to be
fetched from main memory, pipeline is almost
useless.
 Fortunately, we have cache.

14. Pipeline Performance
 The potential increase in performance
resulting from pipelining is proportional to the
number of pipeline stages.
 However, this increase would be achieved
only if all pipeline stages require the same
time to complete, and there is no interruption
throughout program execution.
 Unfortunately, this is not true.

15. Pipeline Performance
F1
F2
F3
I1
I2
I3
E1
E2
E3
D1
D2
D3
W1
W2
W3
Instruction
F4 D4I4
Clock cy cle 1 2 3 4 5 6 7 8 9
Figure 8.3. Effect of an execution operation taking more than one clock cycle.
E4
F5I5 D5
Time
E5
W4

16. Pipeline Performance
 The previous pipeline is said to have been stalled for two
clock cycles.
 Any condition that causes a pipeline to stall is called a
hazard.
 Data hazard – any condition in which either the source or
the destination operands of an instruction are not
available at the time expected in the pipeline. So some
operation has to be delayed, and the pipeline stalls.
 Instruction (control) hazard – a delay in the availability of
an instruction causes the pipeline to stall.
 Structural hazard – the situation when two instructions
require the use of a given hardware resource at the
same time.

17. Pipeline Performance
F1
F2
F3
I1
I2
I3
D1
D2
D3
E1
E2
E3
W1
W2
W3
Instruction
Figure 8.4. Pipeline stall caused by a cache miss in F2.
1 2 3 4 5 6 7 8 9Clock cy cle
(a) Instruction execution steps in successiv e clock cy cles
1 2 3 4 5 6 7 8Clock cy cle
Stage
F: Fetch
D: Decode
E: Execute
W: Write
F1 F2 F3
D1 D2 D3idle idle idle
E1 E2 E3idle idle idle
W1 W2idle idle idle
(b) Function perf ormed by each processor stage in successiv e clock cy cles
9
W3
F2 F2 F2
Time
Time
Idle periods –
stalls (bubbles)
Instruction
hazard

18. Pipeline Performance
 Again, pipelining does not result in individual
instructions being executed faster; rather, it is the
throughput that increases.
 Throughput is measured by the rate at which
instruction execution is completed.
 Pipeline stall causes degradation in pipeline
performance.
 We need to identify all hazards that may cause the
pipeline to stall and to find ways to minimize their
impact.

19. Data Hazards

20. Data Hazards
 We must ensure that the results obtained when instructions are
executed in a pipelined processor are identical to those obtained
when the same instructions are executed sequentially.
 Hazard occurs
A ← 3 + A
B ← 4 × A
 No hazard
A ← 5 × C
B ← 20 + C
 When two operations depend on each other, they must be
executed sequentially in the correct order.
 Another example:
ADD AX,R2
SUB R3,AX