The document discusses how buffer size affects file input/output (I/O) throughput and latency. It explains that there is generally a "sweet spot" buffer size that yields the best throughput for file I/O on magnetic tapes, hard drives, and over networks. For most operating systems and hardware, the optimal buffer size is around 8 KB, though the size also depends on factors like the storage medium, connection speed, and file system cluster size. Properly sizing buffers can improve performance by reducing latency from issues like skipping bad blocks or needing multiple storage tapes.

1. BUFFER SIZE MATTERS, WITHIN LIMITS
FILE AND NETWORK I/O BUFFERING
File C:USERSDAVEDOCUMENTSARTICLES_2016BUFFER_SIZE_MATTERS.DOCX
Page 1 of 3 pages © 2016, David A. Gray. All rights reserved world wide. Current as of 2016-08-27 15:54:00
A recent news article, “The hidden cause of slow Internet and how to fix it,” reminded me of a
closely related subject that I have studied since the early 1980’s, when I made my first study of
the relationship between buffer size and file I/O throughput.i
While the media have changed over the last 40 years, I have found that there was always a
“sweet spot” buffer size that yielded the best throughput.
FILES ON NINE TRACK TAPES
That original study was actually my second encounter with the relationship between buffer size
and throughput, which arose in the late 1970’s when I had the opportunity to work with nine-
track magnetic tapes while I was a graduate student at The University of Alabama. To help pay
my way through school, I took a part-time job at the Center for Business and Economic
Research, an agency of the college of Commerce and Business Administration that gathered
and analyzed economic data on the state of Alabama and provided programming services to the
faculty of the business school. During that time, most of our really big data sets arrived on
magnetic tapes, from which we extracted subsets, which were archived on other magnetic tape
files that we created.
Bad Spots = Unusable Tape
Anybody who was more than about six years old during the 1970’s knows that magnetic tapes
can suffer from a phenomenon called bad spots, portions of the tape that couldn’t be used to
record anything. On audio tapes, such as those onto which your favorite music was recorded,
bad spots manifested as sudden drops in sound volume or decreased sound fidelity. Since
digital data has but two states, ON (1) and OFF (0), the result can be much worse; since an
entire block had to occupy a contiguous section of the tape, which was divided into frames of a
fixed size that was a function of the size of the blocks, a bad spot rendered a length of tape
sufficient to hold an entire block unusable. Worse yet, if a tape had many bad spots, even small
ones, and the buffer size was large, long runs of a reel were rendered unusable. At the very
least, this decreased the overall storage capacity of the reel, and in extreme cases, a large file
that was expected to fit onto a single reel of tape might need two or more.
Long Unusable Runs = I/O Latency
These long runs of unusable tape adversely affected latency in two ways.
1. Skip Delays. Even if everything fit onto a single reel, reading the tape encountered a
slight delay whenever a bad spot was encountered, because the tape drive had to move
the tape past it before it could read the following block This delay was a function of the
length of the bad block and the forward speed at which the transport mechanism
operated. Since the tape speed is constant, it takes the same amount of time to read or
write a block as it does to skip a bad one.
2. Tape Mount Delays. If the tape onto which the file was recorded had too many bad
spots, the file required two or more tapes. This introduced further delays as the tape
was rewound, a prompt appeared on a system console in the computer room, and an
operator performed all of the following tasks.

2. BUFFER SIZE MATTERS, WITHIN LIMITS
FILE AND NETWORK I/O BUFFERING
File C:USERSDAVEDOCUMENTSARTICLES_2016BUFFER_SIZE_MATTERS.DOCX
Page 2 of 3 pages © 2016, David A. Gray. All rights reserved world wide. Current as of 2016-08-27 15:54:00
a. Find a new blank tape.
b. Dismount the tape that just rewound.
c. Remove the write wring from said tape.
d. Insert the write ring that was removed from the tape that just rewound.
e. Mount and thread the tape.ii
Even if you weren’t born in time to remember audio tapes, the video tapes that you used in
your VCR could, and did, suffer from the same factors, as did the DAT (Digital Audio Tape) that
was the stalwart of many mainframe and server backup systems until just a few years ago.
THE PROBLEM PERSISTS WITH HARD DRIVES
Though the impact is usually less, and more subtle, the same phenomenon affects files stored
on even the most modern high speed hard drives. Two factors make it less noticeable to most
users, but the problem persists, and remains measurable.
1. Improved Hard Drive Reliability. A combination of two factors favorably affects the
influence of bad spots.
a. Though by no means immune, modern hard drive platters are less prone to
developing bad spots.
b. Firmware in modern disk controllers manage bad spots transparently, decreasing
their adverse effect on performance, and move data from failing blocks to spare
blocks set aside for this purpose.
2. Improved Hard Drive I/O Latency. Likewise, two design factors significantly
decreases the adverse impact of a bad spot on a hard drive.
a. Higher platter rotation speed decreases the time required to read or skip a block
in exactly the same way that transport forward motion speed does for magnetic
tapes.
b. Closer spacing of tracks decreases the travel distance and time required to seek
to the track on which the desired block of data is stored.
Though I am not as familiar with the mechanical details of how blocks are stored on hard
drives, I know that, at the operating system level, they are divided into logical units called
clusters, which is the smallest unit at which reads and writes happen. In any case, there is
some degree of latency associated with skipping over a bad segment, although the amount of
time is much less than it is for tapes.
THE FINE ART OF BUFFER SIZING
From time to time, I have created and run benchmarks using the I/O facilities available to code
written in a variety of programming languages, including interpreted scripting languages such
as VBScript and Perl, compiled languages such as C and C++, and “in-between” languages,
such as C# and VB.NET.

3. BUFFER SIZE MATTERS, WITHIN LIMITS
FILE AND NETWORK I/O BUFFERING
File C:USERSDAVEDOCUMENTSARTICLES_2016BUFFER_SIZE_MATTERS.DOCX
Page 3 of 3 pages © 2016, David A. Gray. All rights reserved world wide. Current as of 2016-08-27 15:54:00
Each time, the result has been fairly consistent and independent of the language chosen and
the mechanical specifications of the underlying hardware. The sweet spot has always been
around 8 KB (213
= 8,192 bytes) or the cluster size as seen by the operating system, whichever
is greater.
NETWORK I/O IS FILE I/O OVER A LONG WIRE
It is no accident that most operating systems treat network connections and file handles largely
alike, even to the extent of using the same APIs to support them. Fundamentally, network
connections are essentially very fast serial I/O connections, which suffer from the same
engineering limitations as any other serial connection, whether it is composed of TCP packets
traveling over Ethernet at megabit speeds, USB packets traveling at some large fraction of a
megabit or more, or RS-232 packets traveling at 56 kilobits (57,344 bits per second) or less
over an analog telephone circuit.
In many respects, network connections more closely resemble blocks headed to or from a tape
drive than blocks headed to or from a hard drive. For all practical purposes, there is no such
thing as a partial buffer at any junction in the network. Even if the hardware can salvage a
partial buffer to recover one or more small packets contained therein, an individual buffer is a
queue with one server, and packets wait their turn for parsing and extraction at every point
along its route from origin to destination where it is extracted from a buffer or inserted into a
new one.
Although I am not equipped to do so, I suspect that careful benchmarking would reveal that
network connection buffer size has a sweet spot, that it is a lot smaller than many people
expect, and that its value depends on such factors as the average measured speed of the
segment on which it is traveling. This would suggest that it would almost certainly be beneficial
for the originating end point of each major segment to assemble new buffers sized to suit the
capacity of the next segment on their route.
i
“The hidden cause of slow Internet and how to fix it,” Phil Hippensteel, Network World, 23 August 2016,
online at http://www.infoworld.com/article/3110654/internet/the-hidden-cause-of-slow-internet-and-how-
to-fix-it.html?token=%23tk.IFWNLE_nlt_infoworld_daily_2016-08-
26&idg_eid=1471b8de74b15f29e8b842567372c3aa&utm_source=Sailthru&utm_medium=email&utm_ca
mpaign=InfoWorld%20Daily:%20Morning%20Edition%202016-08-
26&utm_term=infoworld_daily#tk.IFW_nlt_infoworld_daily_2016-08-26, retrieved Saturday, 27 August
2016.
ii
From time to time, I had to handle my own tapes, and have seen at first hand that these tasks can take
nearly as long as it does to read or write the entire tape, even if you anticipate the need, and already
have an extra tape ready to go.