Informal MENA-OECD Expert Group event on Measuring Public Service Delivery (co-hosted by SIGMA). Presentation given by Saudi Arabia.

1. 2nd Feb 2023
Beneficiary Experience Dept.
(ADAA)

2. Objectives of the Methodology
Drive the measurement of beneficiary experience through the provision of comprehensive guidelines
2
Prepare and publish beneficiary experience report to related stakeholders
3
Establish a standardized and unified methodology in beneficiary experience measurement aligned with leading practices
1
Enable the adoption of innovative practices around beneficiary experience measurement
4 Support the development of adequate beneficiary experience capabilities across the KSA government
5

3. Beneficiary Experience Definition
The cumulative perception and emotions
of a beneficiary generated from single or
multiple interactions with the Public
Entities on the quality of the service
Renew
trade
license
File a
Complaint
Apply for
Student
Scholarship
Apply for
Marriage
certificate
Register
Agency
Pay
Utility
Bills
Apply for
a job
Request for
facilities
provision
Email
Beneficiary
Experience Definition

4. Voice of Beneficiary Measurement Stages
Plan Measure Analyze Report Improve
Determine the scope
of the VOB initiative,
identify the VOB
methods to be used,
and develop an
execution plan
Initiate the fieldwork
for the selected method
(e.g. satisfaction
surveys, mystery
shopping, focus groups)
based on the defined
scope of the VoB
Gather all beneficiary
feedback and conduct
analysis to generate
insights into beneficiary
perception of services
Develop reports
highlighting the
quantitative and
qualitative findings
with visualization and
synthesis of the data
Develop an action plan to
improve service delivery
based on the beneficiary
findings and monitor
progress against plan
2 3 4 5
Voice of Beneficiary Measurement Stages
1
Success of Voice of the Beneficiary measurement hinges on setting a clear plan with defined objectives and focus areas

5. Impact of Expectations on Beneficiary Satisfaction Levels
How Beneficiaries
Perceive the Experience?
What are their expectations? How did they feel?
Very Satisfied / Delighted
Enjoyable
Satisfied
Neither satisfied nor
dissatisfied
Dissatisfied
Very dissatisfied
Easy
Met the Needs
Hard
Annoying, Degrading…
Cultural
Context
Demographics
Service
Promises
Previous
Experiences
Governments and the private sector are moving from the Satisfaction Model to the Happiness Model (Customer Delight), with a high level
of emotional and rational perception of the service

6. Drivers of Satisfaction for Government Services
Procedures Staff
Channel
Timeliness Service Outcome
Evaluates the availability and accuracy of information,
as well as the ease for beneficiaries to obtain the
service
Evaluates satisfaction with the staff know-
how, and their level of courtesy
Assesses the quality of the channels, physical and digital
(website, mobile application), based on appeal and
accessibility
Evaluates the time spent by beneficiaries to
obtain the service including the service
processing time and the waiting time
Assesses the quality of the service outcome
and whether it is in line with the service
promise
Drivers of
Satisfaction

7. Beneficiary Experience Measurement Components
Beneficiary Level
Sub-Journey
Level
… that can live through
different sub-journeys...
... that can be provided
through different
channels…
… by one or more public
entities.
Satisfaction is measured on
the different beneficiary
categories and segments…
1
Service Level
Channel Level
Government
Entity Level
5
4
3
… composed of a series of
services...
Sub-Journey ‘1’
Service 1
Channel 1
Entity 1
Service 2
Service
‘n’
Service 1
Channel 1
Entity 2
Service 2 Service 3
Service
‘n’
Channel ‘n’
Sub-Journey ‘n’
Entity 1 Entity ‘n’
Entity ‘n’
2

8. Target Government Services for BEX Measurement
A service comprises of a series of related procedures that are aimed to achieve a single defined outcome, and that is
provided by a particular government entity to its beneficiaries, across its channels.
Service
Definition
All services mapped for a selected sub-journey will be measured
To select the services for ad-hoc entity reports, entities would conduct a service prioritization exercise
Overall List of Entity Services Selection & Prioritization of Services* Final List of Services
1 2 3
Service 1
Final list of services for
measurement and
reporting of beneficiary
satisfaction
Service n
• Service 1
• Service 2
• Service 3
• Service n
• Represents a core service of the entity
• Has a high transactional volume relative
to the entity’s other services
• Has a high number of beneficiaries
relative to entity’s other services
• Has a high number of beneficiary
complaints
Service 2
Service 3
Service 4
*Selected services to meet one or more of the
criteria mentioned above
4
3
2
1 5

9. Levels of BEX Measurement
Macro-Journey Level
Measurement
• A list of country wide macro-journeys were defined, each of which comprise
of multiple services and government entities. Beneficiary experience
measurement is conducted on the overall macro-journey level with its
entities and services included
• This level of measurement is considered ‘partial measurement’ for the
entities involved within the macro journey, where a report is generated on
the journey level and shared with involved entities
Entity Level
Measurement
• Beneficiary satisfaction is measured on the level of a particular government
entity, and the services provided by this entity
• There are two options to generate entity reports: collection of services
measured as part of macro-journeys or tailored measurement of main
journeys provided by the entity
• This level of measurement is considered ‘full measurement’ for the entity,
where a report is generated and shared with the entity itself
PLANNED
AD-HOC

10. National satisfaction index NSI
Topic #2

11. ‫العامة‬ ‫األجهزة‬ ‫أداء‬ ‫لقياس‬ ‫الوطني‬ ‫المركز‬
©
، ‫محفوظة‬ ‫الحقوق‬ ‫جميع‬
2021
National satisfaction index in nutshell (1/3)
11
Government effectiveness and citizen satisfaction are
embedded within the DNA of all Vision Realization Programs.
NTP 2.0 has relative initiatives focused on citizen satisfaction
such as Build a unified local index of citizen satisfaction, Create
a ranking index of government channels, Establish a dashboard
to monitor the performance of government services.
Despite the cancellation of building local index for citizen
satisfaction , the current activities of the measurement are
consistent with the achievement of this initiative and achieve
greater coverage for beneficiaries of all kind such as (citizens,
residents, visitors, establishments).

12. ‫العامة‬ ‫األجهزة‬ ‫أداء‬ ‫لقياس‬ ‫الوطني‬ ‫المركز‬
©
، ‫محفوظة‬ ‫الحقوق‬ ‫جميع‬
2021
National satisfaction index in nutshell (2/3)
12
ADAA looked into leading practices such as
ACSI, EPSI, and UKCSI are the most widely
accepted and used national satisfaction
index models in the world
… and defined the decision to guide whether
to develop a custom-built NSI to fit KSA’s
requirements, or leverage existing models
ADAA decided to develop a custom-built
model for the NSI, and to consider leveraging
ACSI model later for private sector expansion
Adaa can develop a proprietary National
Satisfaction Index model, tailored to the region’s
characteristics and objectives. In the past, several
countries developed their own models: Norway,
Sweden, Switzerland and Malaysia, etc. And most
of the international leading models where started
at a country level.
Benchmark Analyze Decide

13. ‫العامة‬ ‫األجهزة‬ ‫أداء‬ ‫لقياس‬ ‫الوطني‬ ‫المركز‬
©
، ‫محفوظة‬ ‫الحقوق‬ ‫جميع‬
2021
National satisfaction index in nutshell (3/3)
Note: Each data point can be linked to actual and recent experience that is being verified
Key characteristic of sub indictors.
Base component of the main indicator which is measured by CSAT
Analyses only to identify drivers' overall performance and root causes
The national satisfaction index (NSI) as a single score for satisfaction on the national level that is designed to measure
beneficiary satisfaction of the service delivery ( transactional and relational ) during high-impact country wide macro-journeys
provided and supervised by public entities.
Definition

14. ‫العامة‬ ‫األجهزة‬ ‫أداء‬ ‫لقياس‬ ‫الوطني‬ ‫المركز‬
©
، ‫محفوظة‬ ‫الحقوق‬ ‫جميع‬
2021
Macro-journeys portfolio in a nutshell
1
Definition
Selection
criteria
A macro-journey portfolio refers to the short listed high-impact country wide macro-journey pool that classify macro
journeys to its relevant sectors to be utilized in the planning phase . Out of the 65 macro-journeys identified, we have
prioritized a set of 42 journeys based on 3 key criteria to be used as larger pool to choose from as per measurement
cycle and stakeholder inputs
Executive Summary

15. ‫العامة‬ ‫األجهزة‬ ‫أداء‬ ‫لقياس‬ ‫الوطني‬ ‫المركز‬
©
، ‫محفوظة‬ ‫الحقوق‬ ‫جميع‬
2021
ADAA has activated the measurement of the Macro-journeys starting from 2020 with gradual
increase of coverage.
1
Sector 2020 2021 2022
Commerce & Finance 20% 20% 60%
Communications 0% 100% 50%
Education 40% 40% 60%
Health & Social Development 50% 50% 50%
Law Enforcement 50% 0% 50%
Public Administration 0% 40% 40%
Tourism & Entertainment 0% 60% 60%
Transportation 0% 100% 67%
Urban Development & Housing 29% 14% 43%
Utilities 100% 100% 100%
Grand Total 26% 43% 55%
It becomes clear that ADAA needs to prioritize macro-
journeys to ensure the continuity and consistency of the NSI
Primary Macro-Journeys: high-impact countrywide macro-journeys that
are characterized by high volume transactions and strategic importance.
ADAA has defined 12 macro-journeys that fall into this category. (see
table 1 macro-journeys in bold ).
Secondary Macro-Journeys: countrywide macro-journeys that give
ADAA room to expand its scope and encourage collaboration among
public entities.
Prioritization proposal

16. 16
Example of a Journey Map (Hajj Journey)

17. Thank you