The 7th International Post Keynesian Workshop
Conference held in
Kansas City, Missouri
29 June - 3 July 2002
AN EMPIRICAL ANALYSIS OF STOCK MARKET SENTIMENT
ANDREA TERZI (Franklin College Switzerland and
Catholic University, Milano, Italy)
GIOVANNI VERGA (University of Parma, Italy)
One can view the behavior of stock market prices in one of three possible ways:
a) Market efficiency: Stock prices equal the present value of the rational expectation future cash flows;
b) Irrationality: Stock prices reflect psychological waves of unjustified optimism and pessimism;
c) Conventional consistency: Stock prices reflect a market sentiment regarding future cash flows (i.e., a
general sense of directions of business affairs and of market liquidity resulting from heterogeneous
investors) and a discount factor (a long-term nominal interest rate) at which expected cash flows are
A relationship between stock prices and interest rates is expected, under both a) and c), when we
assume that the influence of all remaining variables is negligible. Shiller (1982), however, finds little
comovement between stocks and bonds, even after correcting for movements in other relevant variables,
and thus supports b). On the contrary, Barsky (1989) supports market efficiency by providing ad hoc
explanations for the major deviations of stock prices from interest rates (especially the 1973-80 period when
low (real) rates went along with low stock prices). Within a portfolio allocation approach, Chan, Norrbin
and Lai (1997) found that stock and bond prices move apart over time, thus justifying a tactical asset
The link between stock prices and interest rates is controversial. A theoretical explanation that
assumes market efficiency predicts such correlation exists between stock prices and real interest rates. On
the other hand, it is fairly obvious that assuming market irrationality one does not expect to find any strong
relationship of such kind. Finally, assuming conventional consistency, one would expect a link between
stock prices and nominal interest rates. Of course, under the latter, the interest rate is not a rational
expectations equilibrium rate (Keynes would rather describe it as a conventional interest rate), and the
expected future cash flows are those implied by the general "market sentiment".
Although the assumption of market efficiency is common (and convenient) in financial studies, the
approach of market analysts in trying to identify periods of overvaluation or undervaluation is alive, and
significant resources are spent in the attempt of estimating market mispricing and of forecasting market
convergence to "fundamental value". To this end, several analytical tools are used, including the
price/earning ratio. The use of this ratio is problematic, in so far as it disregards the interest rate effect on
stock prices. The widespread use of p/e ratios to compare market evaluation at different points in time
provides a poor measure of "over/under-valuation", since it does not consider the discount factor, and
seems to assume either a constant real discount factor, or market irrationality. A better use of this approach
is when analysts care to calculate the differential between the price/earning ratio and an appropriate
interest rate. This, however, may still be unreliable especially when other market conditions (such as
business cyclical conditions or inflation) vary.
One old research study that attracted our interest is the one by Macauley (1938). With less
sophisticated statistical tools than those available today, Macauley found that stock and bond prices do
move apart over time and that their cyclical movements are strongly correlated.
In this paper we study this hypothesis: that although bond and stock prices move apart over time
they are cyclically correlated. If we find a satisfactory statistical result, we can then attempt to remove the
cyclical interest rate effect (i.e., the discount factor effect) from a stock price series and thus measure a
"market sentiment indicator". This indicator captures changes in market sentiment net of discount factor
effects. It may offer itself as a tool to generate warning signals when this market sentiment reaches "unusual
values". We finally explore whether this indicator may be a better warning signal than price/earning ratios
or even the differential of the price/earning ratios and interest rates.
In the empirical analysis that follows we
i. Study the correlation between cyclical movements of stocks and bonds;
ii. Explore the regression residuals to find unusual values;
iii. Construct an indicator of stock market sentiment (after removing the interest rate
iv. Calculate a price/earning ratio that eliminates the discount factor effect and explore
Our data sample in this version is 1980-2002 and reference is made exclusively at US financial
2. Methodology and empirical results.
(i) We analyze the cyclical correlation between stocks and bonds with the following technique:
• We first smooth a stock price index series (the SP500) using a smoothing tool that
generates brief cycles over the period (see Fig.1). This is done using a simple technique adopted by
• We identify brief segmented trends, whose corners correspond to the peaks of the
smoothed stock price series (see Fig.2). Within each trend, we then study the cyclical relationship
between the stock price index and an appropriate interest rate. An investment on stocks implies a
long-term horizon, irrespectively of the expected holding period. Thus, the regression should
include a long-term rate. The most appropriate rate should be the yields on corporate bonds. The
best solution, both from a theoretical and an empirical point of view, is to consider the yield on
bonds of highest quality (AAA) plus an indicator of risk such as the interest rate differential
between corporate bonds of different quality (AAA and BAA). We have tested the additional
relevance of the long-term default-free (government) interest rate. Although when taken alone, it
has good explanatory power, the explanatory power of the government bond yield vanishes when
the regression also includes the AAA yield.
• The "cycle trends" technique is used to estimate the underlying correlation between
interest rates and stock prices over each cycle. Having identified the stock price cycles, we run the
log(P) = α0 + α1log(RAAA) + α2 (RABB-RAAA)/RAAA + Σj=0 βj Tj
where P = stock price index; RABB and RAAA = yields on AAA and BAA corporate bonds,
respectively; T0 is a standard linear trend, and Tj (for j>0) is a linear trend starting in coincidence
with the jth peak of the smoothed stock price series.
• The outcomes of the regression are as follows:
Variables Cost log(RAAA) (RABB-RAAA)/RAAA T0 T1 T2 T3 T4 T5 T6
Coefficients -0.80 -0.76 -1.13 .0310 -.0279 .0046 .0000 -.0068 .0151 -.0156
t-values -0.80 -11.95 -5.88 7.23 -5.85 4.03 .04 -6.51 22.85 -19.13
R2 = .9939
• We then studied the additional explanatory power provided by the short-term interest
rates, assuming these reflect monetary policy action and thus influence investors' expectations. The
3-month Treasury bills yield is significant, and the coefficient for the AAA yield is slightly reduced.
Variables Cost log(RAAA) Log(R3) (RABB-RAAA) T0 T1 T2 T3 T4 T5 T6
Coefficients -1.04 -.64 -.11 -1.41 .032 -.030 .006 .001 -.009 -.018 -.031
t-values -1.06 -9.72 -4.46 -7.22 7.75 -6.54 4.98. .96 -8.02 20.85 -17.72
R2 = .9944
(ii) We explore the regression residuals to find "unusual" values.
• The residuals of the two regressions above are quite similar. They show (see Fig. 3
for equation 2) a few large deviations from zero: 1980, 1982, 1987, 1988, 1998, 2000 and 2001. An
interpretation is that the short-term stock market behavior in these three occasions was clearly out
of line: in some cases reflecting a “Minskyian” deviation (the bubble that precedes the October 87
crash and the New Economy bubble of 2000), and in other cases a “Davidsonian” deviation (the
international liquidity crises of 1982 and 1998). The terrorist attacks crisis of 2001 can also be
(iii) We then construct an indicator of "stock market sentiment" to be interpreted within a conventionally
consistent market hypothesis.
• A sentiment indicator can be generated after removing the interest rate correlation,
using the following:
log(P*) = log(P) – α11 [log(RAAA) –log(RAAA')] - α12 [log(R3) – log(R3')]
where P* is the market sentiment indicator; RAAA' and R3' are the value of interest rates of AAA
corporate bonds and 3-month bills, respectively, at the arbitrarily chosen date of reference. An
obvious choice is the beginning of the period of our sample (January 1980), when RAAA' was 11.09%
and R3' was 12.0% (see Fig. 4).
(iv) We finally calculate a price/earning ratio that eliminates the discount factor effect, and we explore
• As shown above, the estimates of the relation between the cyclical movements of
stocks and bonds provide the basis for eliminating the interest rate effect from the market value of
stocks. What we obtain we call a "market sentiment indicator", corrected for the influence of interest
rates. Such indicator may be used to measure a corrected version of the price / earning ratio (P/E).
We call such corrected version P*/E ratio. As shown in Fig. 5a, the corrected version is more stable.
This should not surprise, since the standard P/E ratio reflects the wide movements of interest rates
in this period. The extraordinary high value of the P/E ratio at the end of our sample period is
largely explained by the reduction of interest rates. When we eliminate the interest rate effect, the
P/E ratio drops from about 45 to about 25!
• In order to improve further the reliability of our indicators it may be wise to
eliminate the transient effects of corporate earnings from our P/E ratios. A common procedure is to
apply the Hodrick-Prescott filter to earnings. Fig. 5b shows the result when the denominators of the
series shown in Fig. 5a are properly filtered. The filtered P*/E ratio is even less volatile: the volatility
caused by transient variations of earnings as well as the one caused by interest rates is eliminated.
When we compare the standard P/E ratio with the P*/E ratio with filtered earnings we notice a
very different behavior in general and in the last 18 months of our sample, in particular.
• One idea that we have stressed above is that a P/E ratio of, say, 40 means very
different things depending on whether the interest rate is 5% or is 10%. One indicator that can be
used to "rescale" the P/E ratio with respect to the current level of interest rates is the differential
between the E/P ratio and an appropriate interest rate. Although this is a better indicator than the
regular P/E, we feel that it is still inferior to the residuals of the equation 2 above, when one wants
to derive indications on "unusual" market movements. Fig 6 shows the differential of E/P and a
long-term interest rate (the 10-year government bond yield) compared with equation 2 residuals.
The latter shows the unusual movements noticed above under (ii): specifically, the unusual highs of
1987 and 2000, as well as the unusual lows of 1980, 1982, 1988, 1998 and 2001. Each of these dates
refers to well-known events of stock market instability. The E/P minus interest rate differential, on
the contrary, shows the biggest anomalies in 1980 and 1987, and other minor anomalies in 1982,
1983, 1988, 1992, 1995, and 2001.
We conducted an empirical analysis of the correlation between cyclical movements of stocks and
bonds using a cycle-trend technique. The link between stocks and corporate yields is statistically strong.
This provided the basis for the construction of an indicator of stock market sentiment that removes the
interest rate effect from the actual stock price series.
We considered alternative ways of indicating market anomalies, including the P/E ratio and its
differential with the interest rate. We also found that the residuals of the equation based on the segmented
trend technique provide useful information about short-run anomalies.
As an application, Fig 7 shows the Market Sentiment Indicator for the last three years on a daily
basis. Our indicator signals a downward trend of market sentiment during year 2000, at a time when the
actual stock index was oscillating around a flat trend. This is because a drop in interest rates had initially
offset a drop in sentiment in 2000, and it may have gone unnoticed without proper recognition of the role of
Fig. 1: SP500: actual and smoothed(in log)—Arrows indicate peaks
Fig. 2: SP500: actual, segmented trend and residuals (in log)
Fig. 3: Residuals of Equation 2
Fig. 4: SP500 and the Market Sentiment Indicator (Base: January 1980)
Fig. 5a:Price / earning and P* / e ratios
Fig. 5b: Price / earning and P* / e ratios (with filtered earnings)
Fig. 6: A comparison between the differential of E/P and interest rates vs. equation 2 residuals
Fig. 7: Daily Market Sentiment Indicator (Base: June 1st, 1999)
Barsky, R.B. (1989) "Why Don't the Prices of Stocks and Bonds Move Together?" American
Economic Review, December, Vol.79, No.5, 1132-45.
Chan, Norrbin and Lai (1997) "Are Stock and Bond Prices Collinear in the Long Run?"
International Review of Economic and Finance, v.6, iss.2, pp193-201.
Faliva, M. (1978) “Optimal Filtering for Seasonal Adjustment of Quarterly Data”, Rivista
Internazionale di Scienze Sociali, gennaio-marzo.
Macauley, F. R. (1938) Some Theoretical Problems Suggested by the Movements of Interest Rates, Bond
Yields and Stock prices in the United States since 1856, New York: National Bureau of Economic Analysis.
Shiller, R. (1982) "Consumption, Asset Markets, and Macroeconomic Fluctuations", Carnegie
Rochester Conference Series on Public Policy, Autumn, 17, 203-38