Many people in the IT industry need to produce reports and make presentations, sometimes to their technical colleagues and sometimes to more senior people. This paper discusses the best ways to present technical information to higher levels of management. The presenter needs to take account of the intended recipient's knowledge, interest and preconceptions. When it is necessary to present results that have significant consequences for the organisation, it is good psychology to prepare the ground in advance, perhaps by "leaking" the more contentious items so that they become part of the accepted pool of knowledge. Depending on the circumstances, there is much to be gained by regular reporting in a known and accepted format. The paper therefore goes on to discuss the extent to which automation of report publishing is both desirable and practical, and shows the results of some recent work on the automatic interpretation of computer performance data.

1. What do they Really Want to Know?
Presenting technical reports to management
Andy Mardo
Principal Consultant
Metron Technology Ltd
Abstract
Many people in the IT industry need to produce reports and make presentations, sometimes to their technical
colleagues and sometimes to more senior people. This paper discusses the best ways to present technical
information to higher levels of management. The presenter needs to take account of the intended recipient's
knowledge, interest and preconceptions. When it is necessary to present results that have significant
consequences for the organisation, it is good psychology to prepare the ground in advance, perhaps by
"leaking" the more contentious items so that they become part of the accepted pool of knowledge.
Depending on the circumstances, there is much to be gained by regular reporting in a known and accepted
format. The paper therefore goes on to discuss the extent to which automation of report publishing is both
desirable and practical, and shows the results of some recent work on the automatic interpretation of computer
performance data.
_____________________________________________________________________________
1. Introduction In order to make a successful presentation,
especially one with financial implications, you
should learn as much as you can about the target
This paper is motivated by many years' experience audience. You should try to understand:
of delivering technical presentations, and of being
on the receiving end of other people's efforts. As in • The pressures they are under
all fields of endeavour, there have been both • Their priorities
successes and failures. The first objective of this
• Their technical awareness.
paper is to share some of the author's understanding
of what makes for a successful technical
presentation. The second objective is to look at the 2.1 Pressures and Priorities
technology of report production.
Most of the pressures on managers originate from
2. Knowing the Target Audience their own superiors, who are possibly even more
pressed for time and even less technically literate
than they are. In order to keep his superiors happy,
These days, everyone works under pressure. We
your boss probably has to juggle a conflicting set of
all have to face deadlines, handle difficult clients,
priorities. He will probably not react well to you
and cover for sick colleagues. We wrestle with
trying to redefine them. Don't, for example, spend
unfamiliar software, fix bugs to keep the critical
a lot of time explaining how important the
applications running, and risk our lives travelling to
conclusions of your report are. What is deeply
and from the office every day. Sometimes, when
important to you may be of only marginal interest
the stars and the dice are right, we find some time
to him.
to do our real jobs.
The pressures faced by senior management may be
different in kind but are certainly not different in 2.2 Technical Awareness
degree - indeed, they are likely to be fiercer. These
pressures will affect the way in which your boss
reacts to information from "down below". Most of us have what I will refer to as a "domain of
competent understanding". For example, you may
1

2. have detailed knowledge of how a particular • Be precise. Ensure that the conclusions can be
application works, or how to configure a Windows logically derived from the supporting material.
network, or how to tune an Oracle database. Only Phrases like "tend to imply" or "may have a
very rare individuals will have all these skills (or bearing on" might be suitable in a government-
more). Your boss's domain of competent sponsored report on public transport, but have
understanding is unlikely to include your own no place in a technical presentation.
domain as a subset. This means that unless you are
careful, you could overwhelm your target audience
with information that they are not capable of 4. Presenting contentious or unpleasant results
relating to.
The defence reaction of someone who is The nature of a surprise is that people react
overwhelmed by the contents of a report, is usually unpredictably when confronted with one. What
to latch on to some relatively minor ambiguity or happens next depends on whether the surprise is
inconsistency in an effort to discredit the whole. pleasant or unpleasant.
Don't fall into the trap of making this possible.
You may find yourself in the fortunate position of
being the bearer of good tidings. For example:
2.3 Establishing a bond of trust • "Our website has taken far more hits than
expected and the server is performing really
well".
Emotionally, your audience needs to trust what you
are telling them. They will do this much more • "Last week's memory upgrade solved all our
readily if you make your presentation in terms that performance problems and the users are very
they can understand. They need to trust you happy".
because they may subsequently need to pass the • "IBM has halved the price of their disk drives
relevant information up to a higher level, and only and we're going to be able to install twice as
the very brave, or the foolhardy, will stake their many as we budgeted for".
reputation or credibility on anything that they don't
fully understand. This is especially true if the But how often is the converse true?
information that you are trying to convey has • "Our web server is overloaded and our
significant financial implications. customers are deserting us in droves".
Therefore, you need to organise your material in • "The memory upgrade made no difference.
such a way that the audience is psychologically Our clients are threatening to sue".
capable of trusting it.
• "The lead time on hardware delivery has
slipped to six months and we're already nearly
3. Guidelines for a trustworthy out of disk space".
presentation
4.1 Denial
The author recommends the following guidelines
when you are constructing or delivering a technical
No one wants to hear any kind of bad news for the
presentation.
first time during a formal presentation, especially if
• Start at the beginning. Before proceeding to other people are present. If the news is bad
the unknown, take time to describe what is enough, the well-known psychological
already understood. This immediately phenomenon of Denial comes into play. "Denial"
establishes your credibility with the audience - involves saying to oneself "This is so bad that I
in their eyes, you are demonstrating that you don't want to believe it, so I will make up evidence
know what you are talking about. to convince myself that it can't be true". The worse
• Keep focussed. Eliminate any unnecessary the pain, the more compelling the imaginary
details that do not have a direct impact on the counter-evidence becomes. If your information
information being presented. Don't run the risk comes as no surprise, the "denial" reaction is
of "glazed eye" syndrome, especially if you are unlikely to occur. Provided all other attributes of a
presenting to a group. At the end of the trustworthy presentation are in place, your report
presentation, there will probably be a will be, however reluctantly, accepted.
discussion. Peer-group pressure makes it likely
that everyone will feel the need to participate in
this discussion. If anyone lost his way during
the presentation, that person's contribution is
likely to be negative.
2

3. 4.2 Preparing the ground reports are rarely or never printed out as hard
copy.
Recall that a good and trustworthy presentation • Daily reports. Data is typically presented at
starts from what is known, and proceeds towards its relatively high resolution, with data samples
conclusion in logical steps. So, if possible, you every two to five minutes. These reports are
should ensure that the bad situation you are intended to convey detailed information about
reporting on is already part of the common fund of recent events. This type of report, and the
accepted knowledge before you start discussing it. following ones, may be distributed as hard
There is no pre-ordained way of doing this. copy, or (more likely) as an e-mail attachment.
However, you may be able to take advantage of the • Weekly reports. These are presented at a
informal hierarchy that exists in any medium to lower resolution, perhaps with one data point
large organisation. This informal hierarchy exists for each aggregated hour, or possibly with just
alongside, and largely independently of, the formal one aggregated point per day.
reporting structure. By planting a word in the right
• Monthly reports. The graphs in a monthly
ear, you can ensure that everyone from the
report will typically show one aggregated point
managing director to the car park attendant will
per working day. A month's worth of data is
soon know what you want them to know.
usually the least amount that can be used for
trending purposes.
5. Automatic Reporting • Year-to-date reports. The graphs in a year-
to-date report will be at the lowest level of
Not all presentations are face-to-face. The act of e- resolution, certainly with no more than one
point per day and more likely with one point
mailing a report to your boss is still a
"presentation", even though you are not there in per week. Depending on the particular
measurement(s) being reported on, the primary
person when they read it. In many ways the
objective of such a report is to display trend
requirements for clarity, appropriateness etc. of
such a report are much higher than if you present it information.
in person, because the e-mailed report has to speak
for itself in every way - it cannot get any more 5.2 Report Management Techniques
assistance from you, the author.
Automatic reporting is a practical technology that
is appropriate for many routine presentations. Automatic reporting has its dangers as well as its
Further value can be added to automatic reports if benefits. How many reports are produced, pinned
the information on the graphs can be summarised to the wall or posted on the intranet, and never
and interpreted in plain English. Even more benefit looked at again? In order to make reports relevant,
can be obtained if the interpretations are used to interesting and useful, think about the following.
trigger exception events, for example to warn • Top N reporting. Concentrate on the few
automatically that a critical system will become busiest nodes, or devices, or users. Ensure that
overloaded in some number of months' time unless your automatic reporting application (if you
corrective action is taken soon. use one) can identify the Top N instances
itself, even if they are different each time,
without any intervention from you.
5.1 Report types and purposes
• Filtering. Ensure that reports are produced for
periods of time that are important to the
Many performance analysts are tasked with business. If you are in a 24*7 environment,
producing regular reports on the performance of then all times are important. If your
one or several servers (possibly including organisation works a 5-day week and even lets
mainframes) for which they are responsible. These you off for public holidays, then the non-
reports are often, but not necessarily, in the form of working days should be filtered out of the
Word or HTML documents containing annotated reports.
graphs. They fall into a number of categories:
• Correlation. What are the key resource
• Near-real-time reports. Examples of these drivers? Which particular activities have the
kinds of reports include continually updated biggest effect on the total pattern of system
graphs or charts that are posted to an intranet loading? In many cases, correlation analysis
web site, accessible through a standard lets you predict large-scale performance
browser. Each new data sample will cause a changes caused by relatively small changes in
new point to be displayed on the chart. These the nature of the workload or in user
behaviour.
3

4. • Exception reporting. If you have 100 nodes - Print.
in your installation, do you really want to • A common format or house style. Most
report on all of them? Much more likely, you people react well to having the same kind of
only want a report to be produced if some kind information presented in the same way each
of exception condition is detected. time. This applies to:
A good Automatic Reporting and Automatic - The sequencing of the report contents
Advisor regime will incorporate all these facilities.
- The appearance of the graphs and tables
- The means of transmission (via e-mail, as
5.3 Outline of Automatic Reporting hard copy, etc).
technology
If suitable automation is available, it means that in
order to produce a regular report, the performance
In order to produce a report manually, the analyst only needs to carry out the following
performance analyst has to carry out the following actions once:
sequence of actions: • Write an outline of the report. Ideally the
• Write the outline of the report outline should contain mostly "boilerplate" text
• Obtain the relevant data from whatever sources that is not going to change from one issue of
are available the report to the next, though of course the
analyst may want to edit the text to match the
• Create graphs and tables of the data for the graphs that are actually generated on any
required period of time particular occasion.
• Insert these graphs and tables into the report • Create "sample" graphs and tables, from
document existing data, to illustrate the report. The
• Dispatch the finished report to its intended graphs should tell the story in the most
recipient. understandable way. This point is expanded
upon next.
These activities can be time-consuming and
tedious, especially when the only significant • Specify a schedule of when the report is to be
difference between one report and the next is the updated, for what period of time, and who is to
name of the server that it relates to, or the period of receive it by what means of transmission.
time that it covers.
However, notice that all those different kinds of 5.4 Presenting information graphically
reports have a number of common features that
make them ideal candidates for automation. They
are: Over the years, a great deal of work has been
carried out to determine the kinds of graphical
• A regular production date. For example, a
presentation that make data most easily understood.
daily report will be produced at 9 am each day
Here are some practical examples.
to display the previous day's data. A weekly
report will be produced every Monday. A
monthly report will be produced on the first of 5.4.1 Numerical proportions
every month, and so on.
• A consistent analysis period. The analysis
Figure 1 shows three ways of displaying the fact
period is the period of time that the graphs in a
that certain workloads are the largest contributors
given report cover - a day, a week, a month,
to the total loading on a system. The table of
the year to date and so on. A common
numbers, while true and accurate, is difficult to
requirement for an analysis period is to go
assimilate quickly. The horizontal bar chart shows
back some number of days, weeks or months
the relative magnitudes at a glance, but does not
from the date of report production.
convey the additional information that the elements
• A known, stable recipient list. Each report add up to a particular total. The pie chart shows
will be sent to a named individual or team, or the relative magnitudes and also conveys the
is intended for saving in a "well-known information that the elements account for
location", for example a particular folder on a "everything".
Web server. Any of the following techniques
could be used for distributing a report:
- Via e-mail (as an attachment)
- File Save or FTP (for Web-based
reporting)
4

5. Figure 2 - Stacked area chart contrasted with a
Figure 1 - Ways of representing proportions of a
non-stacked bar chart
total
5.4.3 Magnitude or Variability?
5.4.2 Areas and scaling
You may be using the same set of data to
Some variables, for example different categories of
emphasise two (or more) different attributes of the
CPU utilisation, can logically be summed to
measurement in question. For example, you might
present a total value. If plotting two (or more) such
want to display a graph of CPU utilisation for at
variables over time, it is good practice to stack the
least two different reasons:
individual values so that this total value is clearly
displayed. In the example shown, the measured • To show how large (or small) the utilisation is
values are hourly aggregations of what is in fact a on average
continuous variable, namely the CPU utilisation • To show how the utilisation varies over time.
over time.
A good rule of thumb is:
The fact that the variable is continuous is most
clearly brought out by displaying the results as an • To emphasise magnitude, use an area chart
area graph rather than as stacked bars. Use stacked • To emphasise variability, use a line chart.
bars when the values are snapshots made at specific
This is illustrated in Figure 3.
times, for example the number of users logged on
at particular times of the day.
If possible, show the results against a fixed vertical
scale, rather than accepting whatever automatic
default your graphics package determines for you.
There are two reasons for this:
• The viewer can see at a glance how much scope
there is for a potential increase in the value of
whatever is being presented
• If the graph is going to be updated by the use of
Automatic Reporting technology, fixed scaling
contributes to consistency between one version
of a report and the next. This makes it much
easier to compare different versions of the same
report that have been produced at different
times. Figure 3 - Area chart compared with line chart
These points are illustrated in Figure 2.
6. Adding value to reports - Automatic
Trending
6.1 Basic trends
Daily or weekly performance reports are useful
aids to performance management. They let you
5

6. and your colleagues understand what is happening - it will not happen until 20th August. Clearly the
over a relatively short timescale, so that immediate trend has changed.
or recent problems can be addressed and rectified.
As mentioned earlier, monthly (or less frequent)
reports are most useful if they include an element
of trending. This gives the significant additional
benefit of identifying likely dates in the future
before which action will have to be taken in order
to avoid potential problems.
Figure 4 shows two months' worth of data about the
total CPU utilisation of a particular server, with a
trend line applied. As shown, the analysis window
informs us that the trend line will reach a value of
70% on 17th June. There may be a good reason for
wanting to know when the trended daily average
CPU utilisation will reach 70%. This is a popular
value among performance analysts. It is the typical
maximum value beyond which you would not want Figure 5 - Updated graph and trend, with
to run a server processing a critical workload. additional data for March
Although it appears a relatively low value,
remember that this trend is based on a daily
average, so it allows for some normal variation 6.2 Discontinuous trends
during the day. Normally, the best way of
determining whether 70% (or any other particular
value) is an appropriate cut-off point is to carry out Linear trends are extremely useful. However, they
analytical modelling of the system being studied. have a particular shortcoming. Suppose you know
This will show the performance impact of running that conditions are going to change in a few weeks'
at that loading level. or months' time, in such a way as to affect the
direction of the trend in a predictable fashion.
Examples of this kind of situation are when you
know that a particular new project is going to be
rolled out onto the server in question, or that the
number of users is going to increase suddenly, or
that the server is going to be upgraded.
In these circumstances, it is extremely useful to be
able to apply a specific change to the nature of the
trend at a future date. For example, you may want
to specify that the slope of the trend is going to
increase by a known proportion of its value at that
time. This kind of change can be expressed as a
"What-If". In the case where an automatic trend
line has had a What-If change specified, you want
this change to be honoured the next time the chart
and its trend is automatically updated.
Figure 4 - Trended graph with analysis window, Figure 6 shows an example of a What-If change
with underlying data from January and February specified to the trend on a particular chart. It
shows the trend suddenly turning upwards at a
At this point, we are assuming an environment particular date, perhaps because the implementation
where a chart and its associated trend can be of a new project will cause a greater rate of
updated automatically. The previous chart covers increase in the server workload. As a result, the
January and February. Suppose it was generated value of 70% is predicted to occur earlier than
by an automatic reporting mechanism that produces originally thought, in fact on 15th May rather than
a new year-to-date report on the first of every 17th June, as was the case for the trend without the
month. Figure 5 shows the chart that was specified change (figure 4).
generated on April 1st, with the trend automatically
re-calculated to fit the new March data as well as
the previous data. The analysis window now
shows a revised projection for 70% total utilisation
6

7. performance problem or to alleviate an existing
problem.
The following list gives examples of types of
automatic analysis.
• Top N analysis. This analysis can determine
the few busiest or most resource-hungry users,
devices, Oracle sessions or similar. Simply
identifying them is a good start. Better,
however, is to see their pattern of activity over
time.
• Mean value versus thresholds. This is a
simple and straightforward check that the mean
value of a measured data item is not too high or
Figure 6 - Automatic trend with What-If change too low. Failure to stay within threshold
applied bounds can be made to generate an exception
event.
Again, if the chart is automatically updated with • Proportion of time within threshold ranges.
data for another month, the trend is recalculated, Typically the performance analyst will want to
and the existing What-If is automatically applied to set two threshold levels for the value of certain
the revised trend. This gives a new projected date critical data items - a lower, warning threshold
for the 70% utilisation level. and a higher, alarm threshold. It is
straightforward to report automatically on the
Clearly, the ability to create and modify trends proportion of the measurements that fell into
automatically gives significant added value to the each of the three ranges - below the warning
charts, and to the reports that they will be value (and therefore satisfactory), between the
embedded in. warning and the alarm level, and above the
alarm level. This gives valuable information
7. Adding value to reports - Automatic about the relationship between peaks and
Interpretation averages.
• Variability around the mean value. A given
set of measurements will have a mean value,
Probably the best way of adding value to reports is and each individual measurement will typically
to generate automatically an interpretation of the be some amount higher or lower than the mean
data that is being presented. This relieves the value. It is often useful to categorise the
analyst from the task of modifying the report text measured value as "fairly constant", "rather
so that it matches the information in the charts. variable" or "very variable" based on the
The final sections of this paper present the outline proportion of time when the measured values
of an Automatic Advisor system, intended to are close to or far away from the mean value.
facilitate web-based publication of complete Again, if variability is a concern, this analysis
performance reports with minimal user can be made to generate an exception event.
intervention.
• Trended value versus thresholds. A very
useful automatic analysis is to determine the
7.1 Interpretation Techniques date at which the value of a particular metric is
projected to exceed a certain threshold, or to
reach some other predetermined boundary
Given a chart with its underlying data, it is value (e.g. zero, 100% etc.) An exception can
practical to apply a number of analyses be generated on several different attributes of
automatically. In most cases, the analysis can the trend, for example the fact that it will reach
result in the automatic generation of an "exception a boundary value or will cross a threshold value
incident", which will be e-mailed to a responsible on or before a predefined date.
person or team. Additionally, the performance
analyst can specify that reports be generated and • Correlation analysis. Used carefully and with
published only if certain exception conditions in a sensible selection of metrics, Correlation
fact occur. Depending on the circumstances, the Analysis can identify causal as well as
results of an automatic analysis can be turned into statistical relationships between data values.
automatic advice, which gives guidance on actions For example, it is easy to identify UNIX users
that should be taken to avoid a potential or Windows processes whose activity has a
large effect on total CPU utilisation. Similarly,
7

8. the analysis can identify particular I/O devices
that are associated with important warning
metrics such as CPU Wait for I/O Completion.
7.2 Automatic Advice
In order for an Automatic Advisor's reports to be
accepted, they must be:
• Trustworthy - i.e. the conclusions are
recognisably correct and are based on firm Figure 7 - System Summary Status report from an
evidence Automatic Advisor application
• Specific - i.e. the recommendations are specific
enough to be acted on without the need for
further detailed analysis Each of the underlying reports will contain detailed
information about the selected aspect of the
• Understandable - many advice systems in the selected system, including all the interpretation and
past have proved more difficult to understand advice described previously. For any item that is
than reading the relevant technical not shown as "happy", these drill-down reports will
documentation itself. show trustworthy and specific advice for making it
Based on the types of interpretation outlined in so.
section 7.1, it is possible to offer trustworthy, Depending on the size of the installation and the
specific and understandable advice about such number of systems being reported on, this
things as: Summary Status report could be produced at
• CPU upgrades, for example if utilisation regular short intervals, so giving an effectively
thresholds are currently being exceeded, or if continuous summary of the installation's health.
trend analysis shows that they will be exceeded
soon
8. Conclusion
• Memory upgrades, for example if paging and
swapping rates are (or will soon be) high, or if
cache hit rates are low Producing a good report manually takes a lot of
effort. There are a number of psychological factors
• Upgrades or tuning of the I/O subsystem, for to consider, in addition to the purely technical ones:
example if particular devices are becoming
hotspots, or if queuing is becoming a high • What are the needs and interests of the intended
proportion if I/O service time. recipient?
• How can the report be made credible and
trustworthy?
7.3 System-wide Exception Reporting - Traffic
Lights A regime of automatic reports with intelligent
interpretation can add significant value to the work
of a system performance analyst. The reports can
Even if reports are produced and distributed be interesting, credible, trustworthy - and perhaps
automatically, the recipient needs to be directed to most important, timely. The analyst is now free to
any problems, rather than having to read through concentrate on the serious business of maintaining
all the reports in order to find them. and enhancing the performance that is provided to
"Traffic Lights" are a common and convenient way the people who really matter - the organisation's
of showing the health (or otherwise) of a large customers
number of systems at once. Figure 7 shows an
example of a status report covering a number of
systems. By clicking on any of the icons, one can
see the full details of all the underlying reports .
from which the colour of the icon (green, yellow or
red) was generated.
Author
Andy Mardo
Principal Consultant
8

9. Metron Technology Limited
Osborne House
Trull Road
TAUNTON
TA1 4PX
Tel: +44 1823 259231
Fax: +44 1823 334502
Email: andy.mardo@metron.co.uk
www: http://www.metron.co.uk
9