The document outlines the governance assessment methods and applications of governance data in policy-making related to the WHO's Good Governance for Medicines (GGM) programme, which aims to address corruption and unethical practices in the pharmaceutical sector. It details the phases of program assessment, the development of national frameworks, transparency assessments, and recommendations for enhancing accountability within public pharmaceutical sectors. Moreover, it emphasizes the importance of ethical practices, collaboration, and effective communication in implementing good governance strategies.

1. Governance Assessment Methods and,
Applications of Governance Data in Policy-
Making Workshop
Good Governance for Medicines programme
Mohamed Ramzy
WHO Regional Office for the Eastern Mediterranean
1-4 June 2009
Cairo Egypt
Essential Medicines and Pharmaceutical Policies

2. Objectives
1. Briefly introduce WHO's Good Governance for
Medicines (GGM) programme
2. Share experience on WHO's transparency assessment
methodology
3. Share insights and reflections on translating
assessment results into policies
Essential Medicines and Pharmaceutical Policies
May 2009 – Good Governance for Medicines 2

3. Pharmaceutical sectors is a great target to
corruption and unethical practices
R&D
R&D and clinical trials priorities Unlawful
Patent appropriation
royalties
Manufacturing Counterfeit/
substandard Tax evasion
Registration Conflict
of interest
Cartels Pricing
Selection Pressure
Unethical Procurement & import
donations Collusion Over-
Distribution Thefts
invoicing
Inspection
Falsification Bribery
safety/ Prescription
efficacy data
Dispensing
State Capture Pharmacovigilance
Unethical
Promotion
promotion
Essential Medicines and Pharmaceutical Policies
May 2009 – Good Governance for Medicines 3

4. ‫ﺳﻠﺴﻠﺔ اﻟﺪواء‬
‫ﻗﺪ ﺗﻮﺟﺪ ﻣﻤﺎرﺳﺎت ﻏﻴﺮ أﺧﻼﻗﻴﺔ ﻓﻲ أي ﻣﻦ هﺬﻩ اﻟﻤﺮاﺣﻞ‬
‫أوﻟﻮﻳﺎت اﻟﺒﺤﺚ‬
‫اﻹﻋﺘﻤﺎد‬ ‫اﻟﻌﻠﻤﻲ‬ ‫اﻟﺒﺤﺚ اﻟﻌﻠﻤﻲ واﻟﺘﺠﺎرب اﻟﺴﺮﻳﺮﻳﺔ‬
‫اﻟﻐﻴﺮ ﻗﺎﻧﻮﻧﻲ‬ ‫ﺑﺮاءة اﻹﺧﺘﺮاع‬
‫اﻷدوﻳﺔ اﻟﻤﺰﻳﻔﺔ‬ ‫ﻟﻺﺗﺎوات‬
‫واﻟﻤﺘﺪﻧﻴﺔ‬ ‫اﻟﺘﻬﺮب اﻟﻀﺮﻳﺒﻲ‬ ‫اﻟﺘﺼﻨﻴﻊ‬
‫ﺗﻀﺎرب اﻟﻤﺼﺎﻟﺢ‬ ‫اﻟﺘﺴﺠﻴﻞ‬
‫اﻟﺘﺴﻌﻴﺮ‬ ‫اﻹﺣﺘﻜﺎر‬
‫ﻣﻤﺎرﺳﺔ اﻟﻀﻐﻮط‬
‫اﻹﺧﺘﻴﺎر - اﻹﻧﺘﻘﺎء‬
‫اﻟﺸﺮاء واﻹﺳﺘﻴﺮاد‬ ‫اﻟﺘﺒﺮﻋﺎت اﻟﻐﻴﺮ‬
‫زﻳﺎدة ﻗﻴﻤﺔ‬ ‫أﺧﻼﻗﻴﺔ‬ ‫اﻟﺘﻮاﻃﺆ‬
‫اﻟﺴﺮﻗﺔ‬ ‫اﻟﺘﻮزﻳﻊ‬
‫اﻟﻔﻮاﺗﻴﺮ‬
‫اﻟﺘﻔﺘﻴﺶ‬
‫اﻟﺮﺷﻮة‬ ‫اﻟﻮﺻﻒ‬ ‫ﺗﺰوﻳﺮ ﺑﻴﺎﻧﺎت‬
‫ﺳﻴﻄﺮة‬ ‫اﻟﺴﻼﻣﺔ واﻟﻔﻌﺎﻟﻴﺔ‬
‫اﻟﺼﺮف‬ ‫اﻟﻘﻄﺎع اﻟﺨﺎص ﻋﻠﻲ‬
‫رﺻﺪ اﻵﺛﺎر اﻟﺠﺎﻧﺒﻴﺔ‬ ‫ﺻﻨﻊ اﻟﻘﺮار‬
‫اﻟﺘﺮوﻳﺞ‬
‫اﻟﺘﺮوﻳﺞ‬
‫اﻟﻐﻴﺮ أﺧﻼﻗﻲ‬
‫‪Essential Medicines and Pharmaceutical Policies‬‬
‫4 ‪May 2009 – Good Governance for Medicines‬‬

5. Numerous technical guidelines already exist… the
challenge is to balance them with ethical practices
Technical guidelines Ethical practices
Accountability
GMP
Transparency
GCP
Efficiency and
Counterfeits effectiveness
Manual on Marketing Responsiveness
Authorization
Forward vision
WHO model list of EM
Institutional pluralism
Good procurement
practices Participation
Ethical criteria Rule of law
Etc… Etc…
Essential Medicines and Pharmaceutical Policies
May 2009 – Good Governance for Medicines 5

6. Good Governance for Medicines programme:
a model process
Clearance
MOH
PHASE I PHASE II PHASE III
National Development Implementation
assessment national GGM national GGM
framework programme
Assessment GGM framework GGM Strategic
report officially Plan of Action
adopted
Essential Medicines and Pharmaceutical Policies
May 2009 – Good Governance for Medicines 6

7. PHASE I PHASE II PHASE III Assessment of
transparency and accountability
Assesses vulnerability to corruption of
systems in place
Looks at key functions of the
pharmaceutical sector systems
National Regulation: registration, licensing, inspection,
assessment promotion, clinical trials
Supply: selection, procurement, distribution
Elements evaluated:
Assessment Country's regulations and official documents
report
Written procedures and decision-making
processes
Committees, criteria for membership and
conflict of interest policy
Appeals mechanisms and other monitoring
systems
Essential Medicines and Pharmaceutical Policies
May 2009 – Good Governance for Medicines 7

8. Key players in medicines: Who and What do
we assess?
Medicine
Regulators: DRAs,
Policy makers: Parliament,
Executive branch Regulatory
police, court, customs
Authorities
Key players in
medicines
Others: general public, civil Medicines suppliers:
society, media Public
manufacturers,
Decision makers
procurement
importers, procurement
and agencies, distributors,
Implementers agencies
dispensers
Prescribers: doctors, nurses,
Patients/ consumers paramedics
Essential Medicines and Pharmaceutical Policies
May 2009 – Good Governance for Medicines 8

9. PHASE I PHASE II PHASE III
Development of
National GGM Framework
National GGM
framework
Developed through a national
Discipline Values
based based consultation process with all
approach approach
stakeholders
GGM framework
officially
adopted
Essential Medicines and Pharmaceutical Policies
May 2009 – Good Governance for Medicines 9

10. PHASE I PHASE II PHASE III
Components of a national GGM Framework
1. Ethical framework of moral values & 5. Established anti-corruption legislation
ethical principles
6. Whistle-blowing mechanism
Justice/fairness
Truth
7. Sanctions on reprehensible acts
Service to common good 8. Transparent and accountable
trusteeship
regulations and administrative
procedures
2. Code of conduct
9. Collaboration with other GG & AC
3. Socialization programme initiatives
4. Promotion of Moral Leadership 10. Management, coordination and
evaluation of GGM programme
(Steering Committee & task force)
Values based approach Discipline based approach
Essential Medicines and Pharmaceutical Policies
May 2009 – Good Governance for Medicines 10

11. PHASE I PHASE II PHASE III Implementation of
National GGM Programme
Strengthening systems by increasing
transparency and accountability
Promoting awareness (general public &
Implementation health professionals)
national GGM Dissemination of information (newsletter,
programme website, etc.)
Social marketing (radio, TV jingles, posters, etc.)
Critical thinking and discussions (seminars,
Communication conferences)
plan
Building capacity (managers and public
policy makers)
Technical training programme
Leadership training programme
Essential Medicines and Pharmaceutical Policies
May 2009 – Good Governance for Medicines 11

12. Countries efforts focus on moving from phase I
to phase III
Phase I (10 countries)
Phase II (11 countries)
Phase III (5 countries)
Essential Medicines and Pharmaceutical Policies
May 2009 – Good Governance for Medicines 12

13. Objectives of transparency assessment
study
Provide countries picture of level of
transparency and vulnerability to
corruption of 8 functions:
Regulation: registration, licensing,
inspection, promotion, clinical trials
Supply: selection, procurement, distribution
Recommendations to increase
transparency and accountability in
pharmaceutical public sector
Initiate establishment of GGM
programme in countries
Essential Medicines and Pharmaceutical Policies
May 2009 – Good Governance for Medicines 13

14. Assumption
The more transparent any system is, the less vulnerable
to corruption it will be
Presence of publicly available and easily accessible
documents = sign of transparency reduces vulnerability
to corruption (and vice versa)
Awareness about their existence suggest they have been
disseminated and used on practice
Essential Medicines and Pharmaceutical Policies
May 2009 – Good Governance for Medicines 14

15. Questions use 4 different methods
Method 1
(information)
Evaluates knowledge of KI on structures/procedures
1 question with binary answer (yes/no)
Method 2
Method 4 (information)
(perception) Evaluates knowledge KI
Open questions
1 question including sub-questions
Helps for recommendations with binary answer (yes/no)
Method 3
(perception)
Pose statement & ask KI to what extent agree
Allows comparison between evidence &
real practice
Essential Medicines and Pharmaceutical Policies
May 2009 – Good Governance for Medicines 15

16. Rating indicators method 1
1 question with binary answer: Yes or No
Yes (existence of document) = 1 low vulnerability to corruption
No (document does not exists) = 0 high vulnerability to corruption
Transcribe 1 or 0 on consolidation sheet
Use interpretation guidelines in the manual for
each indicator
Transcribe score calculated (0 or 1) on scoring
sheets
Needs to be supported by evidence
Essential Medicines and Pharmaceutical Policies
May 2009 – Good Governance for Medicines 16

17. Rating indicators method 2
Each question has several criteria
Binary answer for each criteria
On questionnaire calculate scoring for each
answer
Disregard all invalid answers (D.K., N.A.)
Score = total "yes"/total valid answers
If more than half are D.K. or N.A. disregard whole
question for this KI
Transcribe score calculated (between 0 and 1) on
score sheet
Essential Medicines and Pharmaceutical Policies
May 2009 – Good Governance for Medicines 17

18. Example method 2
Indicator: Are there clear written criteria for selecting the
members of the essential medicines committee?
No Yes D.K.
1. Written criteria 0 1
2. Criteria publicly available 0 1
3. Specify professional qualification required 0 1
4. Specify the technical skills and work experience related to the area 0 1
5. Require declaration of conflict of interest (e.g. investment in 0 1
pharmaceutical business)
6. Give a timeframe to serve as a committee member 0 1
Total 1 4 1
Total yes 4
Total valid answers 5
Scoring (total yes/total valid answers) 0.8
Essential Medicines and Pharmaceutical Policies
May 2009 – Good Governance for Medicines 18

19. Rating indicators method 3
Asks KI level of agreement with a statement
Remember may be sensitive
Uses Likert scale
Strongly Strongly
Disagree Undecided Agree N.A. D.K.
disagree agree
Essential Medicines and Pharmaceutical Policies
May 2009 – Good Governance for Medicines 19

20. Example method 3
To what extent to you agree with the following
statement: "the members of the registration committee
are systematically and objectively selected based on the
written criteria in force in your country"?
9
8
7
6
No. of answers
5
4
3
1
2
1
0
ed
e
.
.
e
ee
e
.A
.K
re
re
re
id
gr
N
D
ag
ag
Ag
ec
sa
is
ly
nd
di
D
ng
U
ly
ro
ng
St
ro
St
N.A.= not applicable; D.K.= do not know
Essential Medicines and Pharmaceutical Policies
May 2009 – Good Governance for Medicines 20

21. Rating indicators method 4
Open questions mainly for narrative report
Brings additional information useful for
recommendations
Essential Medicines and Pharmaceutical Policies
May 2009 – Good Governance for Medicines 21

22. Score each function to define level
"vulnerability to corruption" (Annex 5)
For each indicator (method 1 & 2)
Transcribe rate for each KI on score sheet
Calculate average rating (possible range between 0
and 1)
For each function
Calculate score "vulnerability to corruption"
10-point scoring system
0.0 ‐2.0 2.1 ‐4.0 4.1‐6.0 6.1‐8.0 8.1 – 10.0
Extremely Very Moderately Marginally Minimally
vulnerable vulnerable vulnerable vulnerable vulnerable
Essential Medicines and Pharmaceutical Policies
May 2009 – Good Governance for Medicines 22

23. Example of scoring registration function
KI 1 KI 2 KI 3 KI 4 KI 5 KI 6 KI 7 KI 8 KI 9 KI 10 Total Average
per
question**
The Total for
Profession* G G G G N P P each line is P
P P
Indicator I.1 1 1 1 1 1 1
given by the 1
1 1 1 10.00 1.00
Indicator I.2 1 1 0.88 1 1 1 0.88 0.88 0.88 0.88 9.38 0.94
Indicator I.3 1 1 1 1 1 1
sum of KI's
1 0.86 0.86 0.86 9.57 0.96
Indicator I.4 0.83 0.50 0.50 0.50 1 0.67 0.33 scores. 0.33
0.50 0.40 5.57 0.56
Indicator I.5 1 1 1 1 1 1 1 0.875 1 1 9.88 0.99
Indicator I.6 1 1 1 1 1 D.K D.K 1 D.K 1 7.00 1.00
Indicator I.7 1 1 1 1 1 1 1 1 1 1 10.00 1.00
Indicator I.8 0.63 0.63 0.63 0.38 0.63 0.60 0.50 0.63 0.85 0.50 5.95 0.60
Indicator I.9 0.75 0.75 0.75 0.63 0.63 0.60 0.71 0.57 0.33 0.50 6.22 0.62
Indicator I.10 0 0 0 0 0 0 0 0 0 0 0.00 0.00
Indicator I.11 ** The average for each
Indicator I.12 0.86 0.57 0.29 question is calculated only 0.29
0.57 0.29 0 0.25 0.60 0 3.71 0.37
Indicator I.13 1 1 1 on1valid responses and all
1 1 1 1 1 1 10 1
Indicator I.14 D.K. and N.A. answers are
Indicator I.15 see text in narrative report
discarded
Indicator I.16 see text in narrative report
Total 9.03
***Final score registration 7.52
Essential Medicines and Pharmaceutical Policies
May 2009 – Good Governance for Medicines 23

24. Example of scoring registration function
KI 1 KI 2 KI 3 KI 4 KI 5 KI 6 KI 7 KI 8 KI 9 KI 10 Total Average
per
question**
Profession* G G G G N P P P P P
Indicator I.1 1 1 1 1 1 1 1 1 1 1 10.00 1.00
Indicator I.2 1 1 0.88 1 1 1 0.88 0.88 0.88 0.88 9.38 0.94
Indicator I.3 1 1 1 1 1 1 1 0.86 0.86 0.86 9.57 0.96
Indicator I.4 0.83 0.50 0.50 0.50 1 0.67 0.33 0.50 0.40 0.33 5.57 0.56
Indicator I.5 1 1 Method 3 question. The KI is 1
1 1 1 1 1 0.875 asked1 9.88 0.99
Indicator I.6 1 1 his/her level of (dis)agreement in
1 1 1 D.K D.K 1 D.K 1 7.00 1.00
Indicator I.7 1 1 1 respect 1 a given statement. 1
1 to 1 1 1 1 10.00 1.00
Indicator I.8 0.63 0.63 0.63 0.38 0.63 0.60 0.50 0.63 0.85 0.50 5.95 0.60
Indicator I.9 0.75 0.75 0.75 0.63 0.63 0.60 0.71 0.57 0.33 0.50 6.22 0.62
Indicator I.10 0 0 0 0 0 0 0 0 0 0 0.00 0.00
Indicator I.11 Method 4 questions:
Indicator I.12 0.86 0.57 0.29 0.57 0.29 question is
0 0.25 0.60 0 0.29 3.71 0.37
An open-ended
Indicator I.13 1 1 1 1 1 1 1 1 1 1 10 1
Indicator I.14
asked to the KI.
Indicator I.15 see text in narrative report
Indicator I.16 see text in narrative report
Total 9.03
***Final score registration 7.52
Essential Medicines and Pharmaceutical Policies
May 2009 – Good Governance for Medicines 24

25. Example of scoring registration function
KI 1 KI 2 KI 3 KI 4 KI 5 KI 6 KI 7 KI 8 KI 9 KI 10 Total Average
per
question**
Profession* G G G G N P P P P P
Indicator I.1 1 1 1 1 1 1 1 1 1 1 10.00 1.00
Indicator I.2 1 1 0.88 1 1 1 0.88 0.88 0.88 0.88 9.38 0.94
Indicator I.3 1 1 1 1 1 1 1 0.86 0.86 0.86 9.57 0.96
Indicator I.4 0.83 0.50 0.50 0.50 1 0.67 0.33 0.50 0.40 0.33 5.57 0.56
Indicator I.5 1 1 1 1 1 1 1 0.875 1 1 9.88 0.99
Indicator I.6 1 1 1 1 1 D.K D.K 1 D.K 1 7.00 1.00
Indicator I.7 1 1 1 1 1 1 1 1 1 1 10.00 1.00
Indicator I.8 0.63 0.63 0.63 0.38 0.63 0.60 0.50 0.63 0.85 0.50 5.95 0.60
*** The final score is given by summing
Indicator I.9 0.75 0.75 0.75 0.63 0.63 0.60 0.71 0.57 0.33 0.50 6.22 0.62
Indicator I.10 0 up all the averages for0each question
0 0 0 0 0 0 0 0 0.00 0.00
Indicator I.11 (orange boxes) and dividing the result
Indicator I.12 0.86 the number of 0.29
by 0.57 0.29 0.57 method 1&2 indicators0.29
0 0.25 0.60 0 3.71 0.37
Indicator I.13 1 (12 1 this example). This number is
in 1 1 1 1 1 1 1 1 10 1
Indicator I.14 then multiplied by 10 to be represented
Indicator I.15 see text0-10 scale.
in a in narrative report
Indicator I.16 see text in narrative report
Total 9.03
***Final score registration 7.52
Essential Medicines and Pharmaceutical Policies
May 2009 – Good Governance for Medicines 25

26. Analysing results and report writing
Annex 4: Annotated content list for final
report
Analytical framework
Quantitative (scoring of vulnerability to
corruption)
Qualitative
Summarise relevant information from notes
taken during interviews
Compare answers with evidence found
Compare method 1&2 with method 3
(perception of KIs see table 1)
Recommendations for actions
Flow naturally from questions scoring low
Analysis of answers to open ended questions
Essential Medicines and Pharmaceutical Policies
May 2009 – Good Governance for Medicines 26

27. Selecting key informants: influences
directly the quality of the findings
Knowledgeable and interest in pharmaceutical sector
Multi-perspective
Mix of senior, middle managerial and junior level personnel
Different institutions and organizations: MOH, MRA,
procurement office, committee members, industry,
associations, academia, CSOs, etc. (see box 2, page 12)
Minimum of 10-15 interviews per function saturation
(no new information is coming from interviews)
KIs can be interviewed for several functions (but
counted only once in total number of KIs)
Essential Medicines and Pharmaceutical Policies
May 2009 – Good Governance for Medicines 27

28. For more information …
Visit
www.who.int/medicines/ggm
Or contact
emp@emro.who.int
ramzym@emro.who.int
Essential Medicines and Pharmaceutical Policies
May 2009 – Good Governance for Medicines 28