Satisfaction of end-users about the quality of water and wastewater services - A methodology João Rosa, Álvaro Carvalho e David Alves 9.º Congresso Mundial da Água Lisboa, 21-26 de setembro de 2014

1. Questionnaire
Questions
Structure
Introduction
Satisfaction of end-users about the quality of water and wastewater services
A methodology
J. Rosa*, D. Alves**, Á. Carvalho***
* The Water and Waste Services Regulation Authority, rosa.joaofilipe@gmail.com
**The Water and Waste Services Regulation Authority, david.alves@ersar.pt
*** The Water and Waste Services Regulation Authority, alvaro.carvalho@ersar.pt
Since 2004, The Water and Waste Services Regulation Authority (ERSAR) has been regulating the quality of service through the implementation of a system of performance indicators, by comparing and publicly disclosing the results achieved by the operators, thus producing an objective measure of benchmarking that pushes operators to improve efficiency and the quality of service. But what are the views of the end-users? Do they have the same perception?
Analysis of user satisfaction and perception of the services provided by water and waste utilities is an important tool for an accurate regulatory assessment of these services. ERSAR has the legal obligation of “measuring, assessing and publicly disclosing the satisfaction levels of the users of (…) water and waste services”. Therefore, ERSAR decided to develop a pilot study that, when implemented, should allow for an understanding of the global satisfaction and perception levels of end-users in water and waste services and to bring forth latent problems in this sector.
This paper presents a methodology developed in that pilot study. It discusses the main problems faced in the process and stresses the solutions found for a sampling plan and for a questionnaire to be addressed at the users of water services.
Study’s scope
inspiring change
Conclusions
www.iwahq.org
This pilot study is a thorough methodological starting point for future assessments. It allows for a nationwide analysis of the satisfaction of end-users of water services while avoiding expensive and time consuming analysis of all municipalities in the country. It is also a useful tool for small scope comparative analysis of a few municipalities. In the future it is expectable that the analysis is spread over to more municipalities in order to improve the benchmarking exercise at regional level.
Target population ▪ Over 18 years old; ▪ Drinking water, sewerage and waste management customers; ▪ Being customer for more than 1 year of service.
Collecting responses
▪ Computer Aided Telephone Interview (CAPI).
Territorial’s scope
▪ National approach [Fig. 1, (a)] - 1068 interviews
▪ Local approach [Fig.1, (b)] - 1176
Figure 1 – Sampling plan (national and local approaches)
(a) National approach
(b) Local approach
The sampling plan shows the best method for the collection of a representative sample of the population in the most efficient way. It is necessarily different on nationwide and municipal approaches.
The national analysis is based on a sample of 1068 interviews in total, regionally stratified and thus accounting for the specific weight of the population in each region. The municipal analysis is based on a random sampling of 1176 interviews, for the municipalities of Lisbon, Oporto, Coimbra, Moura and Portimão. This way, the study will be covering a multiple range of operator profiles in what concerns geographical position and dimension, population served and seasonal demographic variations, urban/rural typologies and management models (direct, delegation and concession management models).
Finally, this sampling methodology sets out the baseline for future analysis of water and waste users satisfaction and perception, in which all 278 municipalities of mainland Portugal will be included.
Single measurement scale with 7 evaluation levels
Length 12 minutes
3 blocks of questions
3 blocks for each service, allowing the interviewer to choose randomly the order of the services that start the questionnaires and thus mitigating the possible length effect on the last block if the structure were a fixed one.
Less than 15 minutes, which is the maximum length recommended in this type of studies.
Using a mixed measurement scale could be confusing for interviewees.
7 levels of choice isn’t hard enough to take a clear response and to appraise a positive or negative evaluation (odd scale).
General satisfaction
Perceived price and perceived value
Awareness about the operator
Customer support
Complaints
Billing
Specific topics like satisfaction with drinking water quality, water pressure, cutting of water supply and time spent by the operator solving cuts/breakdowns in water supply are taken into account.
Common topics address by this questionnaire: the perceived price and the perceived value of the services, awareness about the operator, perceived quality of the customer service, handling of complaints and billing.
The questions address general and specific topics and are based on an analysis of other questionnaires, consumer complaints and regulatory information about the quality of service.
The questionnaire ends with the characterization of the interviewees in terms of gender, age, education, working situation and income level.