1. 1T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
Dr. Naseem Naqvi
FBBA FRCP(Edin) FHEA MAcadMed MSc (Blockchain & Cryptocurrency)
President,The British Blockchain Association
Editor in Chief,The JBBA
Founding Chair, Centre for Evidence-Based Blockchain (CEBB)
Opening Keynote
2nd Blockchain International Scientific Conference
11 March 2020, Edinburgh
www.linkedin.com/in/naseemnaqvi
2. 2T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
3/11/2020
3. 3T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
3/11/2020
4. 4T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
5. 5T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
6. 6T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
7. 7T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
8. 8T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
9. 9T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
10. 10T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
11. 11T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
12. 12T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
13. 13T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
14. 14T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
15. 15T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
16. 16T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
17. 17T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
3/11/2020
18. 18T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
3/11/2020
19. 19T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
3/11/2020
20. 20T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
21. 21T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
EVIDENCE BASED ROUTE
22. 22T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
What is the Evidence for
Evidence Based Decision Making?
• CUT failure rates by 50%
• INCREASE the number of RIGHT business decisions by SIX fold
• Achieves desired RESULTS 90% of the time
• EXCEEDS expectations in over 40% of the time
• 98% of Business executives FAIL to make Evidence-Based decisions
23. 23T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
What is the cost of not practicing Evidence
Based decision making?
24. 24T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
25. 25T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
26. 26T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
27. 27T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
28. 28T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
29. 29T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
30. 30T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
•Context and history of evidence-based
practice
•What is Evidence-based Blockchain (EBB)
•5 steps of EBB
•Summary and road ahead
31. 31T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
32. 32T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
33. 33T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
34. 34T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
35. 35T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
Why should you scientifically
evaluate these opinions?
36. 36T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
37. 37T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
Decision Making
38. 38T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
Evidence Based Blockchain: EBB
“Conscientious, explicit, and judicious decision
making based on professional expertise and the best
available organisational, stakeholder and scientific
evidence.”
39. 39T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
July ‘17
to
Dec ‘17
Sources
of
Evidence
Professional
Expertise
Scientific
Evidence
Stakeholder/
Consortium
Evidence
Organisation
Evidence
40. 40T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
Evidence Based Blockchain (EBB): 5 Steps
1. Formulate a precise question: (ASK)
2. Search of the Evidence, looking for answers to the question (ACQUIRE)
3. Critical appraisal of the evidence (APPRAISE)
4. Apply the results to your practice (APPLY)
5. Monitor any changes and evaluate (ASSESS)
41. 41T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
History timeline of Evidence Based Practice:
1970: Healthcare (SirArchibald Cochrane)
1990: Medical Education (Professors Guyatt & Sackett, McMaster University, Canada)
1998: Probation Services (Professor Peter Raynor, University ofWales)
1999: Social Care (National Institute of Clinical Excellence, NICE, UK)
2000: Nursing (Davies et al, University of St Andrews, UK)
2000: Criminal Justice (Professor David Farrington, University of Cambridge, UK)
2005: Employment and HR (Denise Rousseau, Carnegie Mellon University, USA)
2006: Management (Centre for Evidence Based Management,The Netherlands)
2018: Blockchain and Distributed Ledgers (The British Blockchain Association, UK)
42. 42T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
43. 43T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
44. 44T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
45. 45T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
46. 46T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
47. 47T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
48. 48T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
49. 49T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
50. 50T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
51. 51T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
52. 52T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
53. 53T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
54. 54T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
55. 55T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
56. 56T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
57. 57T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
58. 58T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
59. 59T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
60. 60T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
61. 61T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
Which Problem does Evidence-Based Blockchain solve?
62. 62T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
5 major challenges:
• Not asking the right questions
• Not searching for the right information
• Not evaluating the information
• Not applying the right information
• Not assessing the right outcomes
63. 63T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
Evidence Based Blockchain (EBB): Five Steps
1. Formulate a precise question: (ASK)
2. Search of the Evidence, looking for answers to the question (ACQUIRE)
3. Critical appraisal of the evidence (APPRAISE)
4. Apply the results to your practice (APPLY)
5. Monitor any changes and evaluate (ASSESS)
64. 64T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
65. 65T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
66. 66T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
67. 67T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
68. 68T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
Question: - Ask
Should I use blockchain in my Supply Chain business?
69. 69T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
70. 70T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
71. 71T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
72. 72T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
“P.I.C.O” Question
73. 73T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
Question:
“..Compared to existing traditional database based [C]
infrastructure, does private permissioned blockchain [I] of
Hyperledger Fabric save time, reduce costs, improve food
integrity, and increase consumer satisfaction [O] in the tracking of
seafood [P] via the supply chain, based in India..”?
74. 74T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
Formulate a precise question: (ASK)
75. 75T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
Where to search for Evidence? - Acquire
76. 76T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
77. 77T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
78. 78T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
79. 79T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
Where to Search for High Quality Evidence?
• DOAJ
• Semantic Scholar
• Microsoft Academic
• Scopus
• WorldCat
• Web of Science
• Library Catalogues
80. 80T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
81. 81T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
82. 82T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
83. 83T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
84. 84T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
85. 85T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
86. 86T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
Appraise the Evidence
87. 87T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
88. 88T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
89. 89T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
90. 90T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
91. 91T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
92. 92T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
93. 93T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
94. 94T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
95. 95T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
96. 96T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
97. 97T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
98. 98T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
99. 99T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
100. 100T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
101. 101T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
Assess Results
• Why
• When
• How
• Where
• What
• Input - > Out put - > Outcome - > Impact - > Influence
102. 102T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
103. 103T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
104. 104T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
105. 105T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
106. 106T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
107. 107T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
Why are we not using EBB?
• Own beliefs – Cognitive biases
• Organisational culture
• Power / politics
108. 108T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
Risks of not following EBB:
• Waste of resources
• Poor quality benchmarks and frameworks
• Ineffective Policymaking
• Funding
109. 109T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
How to follow EBB:
• As an individual
• As an organisation
110. 110T H E B R I T I S H B L O C K C H A I N A S S O C I A T I O N w w w . b r i t i s h b l o c k c h a i n a s s o c i a t i o n . o r g |
Editor's Notes
How do we know what works?
How can we make better decisions? We make 35,000 decisions in a day (86,400 seconds a day)
4 people in an ecosystem:
Inventor/ Vendor/ suppliers
User
Investor
Regulator/ Policymaker/ Government
System 1 is primarily located in the limbic area, what we used to call “the reptilian brain”. It is largely automatic, instinctual, and unconscious. This is very important because that is a part of the brain that keeps us alive in dangerous situations and makes sure we pay attention to things like hunger, procreation, and rest. It is also hedonistic and mainly considers whether a possible action will make us feel good or satisfy a need. Oh, and it doesn’t “think” using words! So, forget about trying to convince System 1 to be rational!
System 2 is primarily located in the frontal area of the brain. This is one of the last parts of the brain to develop in the transition from childhood to adulthood. If you wonder why your teenager looks sort of like an adult, but certainly doesn’t act like an adult; that’s why! These areas of the brain use conscious, logical “thought”. This is where our adult ability to reflect, deliberate, and analyze is happening. Unfortunately, it works much slower than System 1. In a race between the two, System 2 will always lose. It takes more energy and resources than System 1. It is also in the part of our skull that is quite sensitive to injury (e.g., concussions, sports injuries) and infection. Because it was the last part of our human brain to evolve, many scientists believe it is more prone to processing errors, too.
So, that part of our brain that makes us most “human” is also the part that is most sensitive to being harmed and having functional errors.
In other words, out of every $100 spent, the government should dedicate $1 to making sure the other $99 actually works.
The final step, which is often overlooked, is to monitor any changes and repeat the process.
Process of carefully and systematically examining research to judge its trustworthiness, its value, and relevance in a particular context.”
IMPACT: “ Recorded or otherwise auditable occasion of influence from research on another individual or organization, demonstrated by references to, citations of or a discussion of the research or the researcher’. (LSE)