Firm weights are applied to the qualitative responses of participants to calculate business tendency survey (BTS) results. Sector weights are employed to produce higher levels of aggregation. What impact does weighting have on the accuracy of the BTS results?

1. The impact of weight adjustment on the
accuracy of business tendency surveys
An assessment of the manufacturing survey of South Africa
George Kershoff, BER

2. What are business tendency surveys?
 Use the same questionnaire
 to collect qualitative information
 from the same selection (panel) of firms.

3. Use weights to calculate the survey results
 Sample weights
 Seize weights
 Firm weights
 Sector weights

4. Firm weights
No. of Size class Firm
employees weight
1 – 19 1 1
Small 20 – 49 2 4
50 – 99 3 10
100 – 199 4 19
200 – 299 5 34
300 – 399 6 48
Large
400 – 499 7 62
500 – 999 8 94
1 000+ 9 286

5. Firm weights
No. of Size class Firm
employees weight
1 – 19 1 1
Small 20 – 49 2 4
50 – 99 3 10
100 – 199 4 19
200 – 299 5 34
300 – 399 6 48
Large
400 – 499 7 62
500 – 999 8 94
1 000+ 9 286

6. Sector weights
Sector Weight %
Food 15.1
Beverages 4.7
Textiles 2.4
Clothing 2.3
Wood 2.3
Paper 4.6
Printing 4.0
Chemicals 8.7
Plastic 3.3
Non-metal minerals 3.3
Basic metals 7.6
Metal products 5.4
Machinery 6.0
Electrical machinery 4.5
Transport equipment 18.1
Furniture & other 5.3
Not published 2.2
Total excl petroleum 100.0

7. Calculating the results in practice
Size Firm Sector Total Up Same Down Total
class weight weight weight
A. Small 2 4 0.051 0.204 0.204
B. Small 3 10 0.051 0.510 0.510
M C. Large 6 48 0.051 2.448 2.448
e
a D.
8 94 0.051 4.794 4.794
t Large
Sum 0.204 2.958 4.794 7.956
% 3 37 60 100
V E. Small 3 10 0.106 1.060 1.060
e
h F. Large 8 94 0.106 9.964 9.964
i
c
G. Large 8 94 0.106 9.964 9.964
l
Sum 11.024 0.000 9.964 20.988
e
s % 53 0 47 100
T Sum 11.228 2.958 14.758 28.944
o
t % 39 10 51 100

8. Calculating the results in practice
Size Firm Sector Total Up Same Down Total
class weight weight weight
A. Small 2 4 0.051 0.204 0.204
B. Small 3 10 0.051 0.510 0.510
M C. Large 6 48 0.051 2.448 2.448
e
a D.
8 94 0.051 4.794 4.794
t Large
Sum 0.204 2.958 4.794 7.956
% 3 37 60 100
V E. Small 3 10 0.106 1.060 1.060
e
h F. Large 8 94 0.106 9.964 9.964
i
c
G. Large 8 94 0.106 9.964 9.964
l
Sum 11.024 0.000 9.964 20.988
e
s % 53 0 47 100
T Sum 11.228 2.958 14.758 28.944
o
t % 39 10 51 100

9. Calculating the results in practice
Size Firm Sector Total Up Same Down Total
class weight weight weight
A. Small 2 4 0.051 0.204 0.204
B. Small 3 10 0.051 0.510 0.510
M C. Large 6 48 0.051 2.448 2.448
e
a D.
8 94 0.051 4.794 4.794
t Large
Sum 0.204 2.958 4.794 7.956
% 3 37 60 100
V E. Small 3 10 0.106 1.060 1.060
e
h F. Large 8 94 0.106 9.964 9.964
i
c
G. Large 8 94 0.106 9.964 9.964
l
Sum 11.024 0.000 9.964 20.988
e
s % 53 0 47 100
T Sum 11.228 2.958 14.758 28.944
o
t % 39 10 51 100

10. Calculating the results in practice
Size Firm Sector Total Up Same Down Total
class weight weight weight
A. Small 2 4 0.051 0.204 0.204
B. Small 3 10 0.051 0.510 0.510
M C. Large 6 48 0.051 2.448 2.448
e
a D.
8 94 0.051 4.794 4.794
t Large
Sum 0.204 2.958 4.794 7.956
% 3 37 60 100
V E. Small 3 10 0.106 1.060 1.060
e
h F. Large 8 94 0.106 9.964 9.964
i
c
G. Large 8 94 0.106 9.964 9.964
l
Sum 11.024 0.000 9.964 20.988
e
s % 53 0 47 100
T Sum 11.228 2.958 14.758 28.944
o
t % 39 10 51 100

11. Calculating the results in practice
Size Firm Sector Total Up Same Down Total
class weight weight weight
A. Small 2 4 0.051 0.204 0.204
B. Small 3 10 0.051 0.510 0.510
M C. Large 6 48 0.051 2.448 2.448
e
a D.
8 94 0.051 4.794 4.794
t Large
Sum 0.204 2.958 4.794 7.956
% 3 37 60 100
V E. Small 3 10 0.106 1.060 1.060
e
h F. Large 8 94 0.106 9.964 9.964
i
c
G. Large 8 94 0.106 9.964 9.964
l
Sum 11.024 0.000 9.964 20.988
e
s % 53 0 47 100
T Sum 11.228 2.958 14.758 28.944
o
t % 39 10 51 100

12. Average number of response per quarter
400
350
300
250
200
150
100
50
0
2001 2002 2003 2004 2005 2006 2007 2008 2009
Sector Total - Small (<100) Sector Total - Large

13. Impact of non-responses
Size Firm Sector Total Up Same Down Total
class weight weight weight
A. Small 2 4 0.051 0.204 0.204
B. Small 3 10 0.051 0.510 0.510
M C. Large 6 48 0.051 2.448 2.448
e
a D.
8 94 0.051 4.794 4.794
t Large
Sum 0.204 2.958 4.794 7.956
% 3 37 60 100
V E. Small 3 10 0.106 1.060 1.060
e
h F. Large 8 94 0.106 9.964 9.964
i
c
G. Large 8 94 0.106 9.964 9.964
l
Sum 11.024 0.000 9.964 20.988
e
s % 53 0 47 100
T Sum 11.228 2.958 14.758 28.944
o
t % 39 10 51 100

14. Impact of non-responses
Size Firm Sector Total Up Same Down Total
class weight weight weight
A. Small 2 4 0.051 0.204 0.204
B. Small 3 10 0.051 0.510 0.510
M C. Large 6 48 0.051 2.448 2.448
e
a D.
8 94 0.051 4.794 4.794
t Large
Sum 0.204 2.958 4.794 7.956
% 3 37 60 100
V E. Small 3 10 0.106 1.060 1.060
e
h F. Large 8 94 0.106 9.964 9.964
i
c
G. Large 8 94 0.106 9.964 9.964
l
Sum 11.024 0.000 9.964 20.988
e
s % 53 0 47 100
T Sum 11.228 2.958 14.758 28.944
o
t % 39 10 51 100

15. Impact of non-responses
Size Firm Sector Total Up Same Down Total
class weight weight weight
A. Small 2 4 0.051 0.204 0.204
B. Small 3 10 0.051 0.510 0.510
M C. Large 6 48 0.051 2.448 2.448
e
a D.
8 94 0.051 4.794 4.794 3.162
t Large
Sum 0.204 2.958 4.794 7.956
% 3 37 60 100
V E. Small 3 10 0.106 1.060 1.060
e
h F. Large 8 94 0.106 9.964 9.964
i
c
G. Large 8 94 0.106 9.964 9.964
l
Sum 11.024 0.000 9.964 20.988
e
s % 53 0 47 24.150
100
T Sum 11.228 2.958 14.758 28.944
o
t % 39 10 51 100

16. Composition of the survey results vis-à-vis
the benchmark
2001 2005 2009 Average
Transport equipment
% of total
Benchmark
15.3 18.1 15.3 17.5
(domestic sales)
Survey (sum of
9.9 8.6 4.7 8.1
responses)

17. Sector composition of survey results using
fixed weights
100%
90% Not publ -
Transp equip -
80% Elec machin -
Machinery -
70%
Metal products -
60% Basic metals -
Non-metal min -
50% Chemicals -
Wood -
40%
Clothing -
Textiles -
30%
Beverages -
20% Food -
Plastics -
10% Printing -
Paper -
0%
Furn & other -
02Q3
04Q2
06Q2
08Q1
09Q4
01Q2
01Q3
01Q4
02Q1
02Q2
02Q4
03Q1
03Q2
03Q3
03Q4
04Q1
04Q4
05Q1
05Q2
05Q3
06Q1
06Q3
06Q4
07Q1
07Q2
07Q3
07Q4
08Q3
08Q4
09Q1
09Q2
09Q3
04q3
08q2

18. Weight adjustment
 Adjust individual weights ex post to align the survey’s
sector composition with that of the reference series
 Provide for
 Over and under representation
 Non-responses

19. Weight adjustment in practice
Size Firm Sector Total Up Same Down Total
class weight weight weight
A. Small 2 4 0.051 0.204 0.204
B. Small 3 10 0.051 0.510 0.510
M C. Large 6 48 0.051 2.448 2.448
e
a D.
8 94 0.051 4.794 4.794 3.162
t Large
Sum 0.204 2.958 4.794 7.956
% 3 37 60 100
V E. Small 3 10 0.106 1.060 1.060
e
h F. Large 8 94 0.106 9.964 9.964
i
c
G. Large 8 94 0.106 9.964 9.964
l
Sum 11.024 0.000 9.964 20.988
e
s % 53 0 47 24.150
100
T Sum 11.228 2.958 14.758 28.944
o
t % 39 10 51 100

20. Sector composition after weight adjustment
100%
90% Not publ -
Transp equip -
80% Elec machin -
Machinery -
70%
Metal prod -
60% Basic metals -
Non-metal min -
50% Chemicals -
Wood -
40%
Clothing -
Textiles -
30%
Beverages -
20% Food -
Plastics -
10% Printing -
Paper -
0%
Furn & other -
09Q3
01Q2
01Q3
01Q4
02Q1
02Q2
02Q3
02Q4
03Q1
03Q2
03Q3
03Q4
04Q1
04Q2
04Q4
05Q1
05Q2
05Q3
06Q1
06Q2
06Q3
06Q4
07Q1
07Q2
07Q3
07Q4
08Q1
08Q3
08Q4
09Q1
09Q2
09Q4
04q3
08q2

21. Net % = % up less % down
-80
-60
-40
-20
0
100
20
40
60
80
Mar-01
Sep-01
Mar-02
adjustment
Sep-02
Mar-03
Sep-03
Mar-04
Sep-04
Without weight adjustment (lhs)
Mar-05
Sep-05
Mar-06
Sep-06
Mar-07
Sep-07
Mar-08
Sep-08
With weight adjustment (lhs)
Survey results with and without weight
Mar-09
Sep-09

22. Net % = % up less % down
-80
-40
-20
0
100
-60
20
40
60
80
-100
Mar-01
Sep-01
Mar-02
Sep-02
Mar-03
Sep-03
Mar-04
Sep-04
Mar-05
With weight adjustment (lhs)
Sep-05
Mar-06
Sep-06
Mar-07
Sep-07
Mar-08
Sep-08
Reference series (rhs)
Mar-09
Sep-09
0
10
30
20
-30
-20
-10
Adjusted survey results vs. reference series
Year on year % change

23. Conclusion
 Weight adjustment did not improve accuracy in the
case of the manufacturing sector between 2001 and
2009
 Accuracy = closeness between survey results and
reference series
 Survey results are therefore robust