Misra, D.C.(2009): E-government Monitoring and Evaluation_MDI-12.2.2009


Published on

A comprehensive presentation on
e-government monitoring and evaluation.

1 Comment
No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Misra, D.C.(2009): E-government Monitoring and Evaluation_MDI-12.2.2009

  1. 1. E-government Evaluation E-government Monitoring and Evaluation: Implementing E-business Plan by Dr D.C.MISRA
  2. 2. Thursday, February 12, 2009 11-15 a.m. to 1-15. p.m <ul><li>3rd Post Graduate Diploma in Public Policy and Management Programmme </li></ul><ul><li>(2007-09) </li></ul><ul><li>School of Public Policy and Governance </li></ul><ul><li>Management Development Institute </li></ul><ul><li>P.B.60, Mehrauli Road, Sukhrali, </li></ul><ul><li>Gurgaon 122 001 </li></ul>
  3. 3. E-government Monitoring and Evaluation: A Presentation <ul><li>by </li></ul><ul><li>Dr D.C.Misra, I.A.S. (Retd.) </li></ul><ul><li>E-government Researcher and Consultant </li></ul><ul><li>New Delhi, India </li></ul><ul><li> Email : dc_misra@hotmail.com </li></ul><ul><li> Web : http://in.geocities.com/drdcmisra </li></ul><ul><li> Think Tank : </li></ul><ul><li>http://tech.groups.yahoo.com/group/cyber_quiz </li></ul><ul><li> Tel : 91-11- 2245 2431 </li></ul><ul><li> Fax : 91-11- 4244 5183 </li></ul>
  4. 4. I CONTENTS E-government Monitoring and Evaluation <ul><li>Part A: Monitoring </li></ul><ul><li>Historical Background of Monitoring </li></ul><ul><li>Ministry of Statistics and Programme Implementation </li></ul><ul><li>Reasons for Delay in Project Implementation </li></ul><ul><li>E-government Project Failures - An Indian Example </li></ul><ul><li>Causes of E-government Project Failures </li></ul><ul><li>What can monitoring do? </li></ul><ul><li>What is not monitoring? </li></ul><ul><li>What is then e-government monitoring? </li></ul>
  5. 5. Part A: Monitoring <ul><li>What is E-government Project? </li></ul><ul><li>E-government Project Life Cycle-Five Models </li></ul><ul><li>Logical Framework Approach (LFA) </li></ul><ul><li>Who will do e-government monitoring? </li></ul><ul><li>The e-government monitoring unit </li></ul><ul><li>E-government monitoring methodology </li></ul><ul><li>Guiding Principles of E-government Monitoring </li></ul>
  6. 6. Contents Part B: Evaluation <ul><li>Historical Background of Evaluation </li></ul><ul><li>Programme Evaluation Organisation (PEO) </li></ul><ul><li>Functions of PEO </li></ul><ul><li>Four Generations of Evaluation </li></ul><ul><li>What is not Evaluation? </li></ul><ul><li>What is then Evaluation? </li></ul><ul><li>Four Senses of Term Evaluation </li></ul><ul><li>What is E-government Evaluation? </li></ul>
  7. 7. Part B: Evaluation <ul><li>Types of Evaluation </li></ul><ul><li>Evaluation Timing and Diffusion-Adoption of Curves </li></ul><ul><li>What is to be evaluated? </li></ul><ul><li>How E-government Domains are evaluated? </li></ul><ul><li>Approaches to Evaluation </li></ul><ul><li>Who will evaluate? </li></ul><ul><li>Which Type of Evaluation is Suited Most to E-government? </li></ul>
  8. 8. Part C: E-government M&E Framework <ul><li>E-government Monitoring and Evaluation (M&E) </li></ul><ul><li>E-government Monitoring and Evaluation (M&E) Unit </li></ul><ul><li>Relative Weights to Monitoring and EvaluationI </li></ul><ul><li>E-government M&E Framework: Components </li></ul>
  9. 9. E-government M&E Framework <ul><li>(a) E-government Management </li></ul><ul><li>(b) E-government M&E Unit </li></ul><ul><li>(c) Information Needs Matrix </li></ul><ul><li>(d) E-government M&E Cycle </li></ul><ul><li>(e) Citizens </li></ul><ul><li>A Framework for E-government Monitoring and Evaluation </li></ul>
  10. 10. Part D and Part E <ul><li>Part D : </li></ul><ul><li>My Questions </li></ul><ul><li>End of Presentation </li></ul><ul><li>Thank you </li></ul><ul><li>Part E : </li></ul><ul><li>Your Questions Now </li></ul>
  11. 11. Part A: Monitoring <ul><li>I. Historical Background of Monitoring </li></ul><ul><li> Monitoring in loose form has always been part of </li></ul><ul><li>Indian administration </li></ul><ul><li> Centralised monitoring is a recent phenomenon </li></ul><ul><li> It came into existence at the Centre in 1985 when </li></ul><ul><li>Ministry of Programme Implementation (MOPI) </li></ul><ul><li>was formed </li></ul><ul><li> MOPI is now part of Ministry of Statistics and </li></ul><ul><li>Programme Implementation (MOSPI) </li></ul>
  12. 12. II. Ministry of Statistics and Programme Implementation <ul><li> MOSPI tracks implementation of </li></ul><ul><li>Central Sector Projects > Rs 20 Crore </li></ul><ul><li> Report for January-March, 2007 covers 882 </li></ul><ul><li>projects: </li></ul><ul><li>-- Mega (Greater than Rs 1000 Crore) 69 </li></ul><ul><li>-- Major (Between Rs 100 & 1000 Crore) 432 </li></ul><ul><li>-- Medium (Between Rs 20 Crore & Rs 100 Crore) 381 </li></ul><ul><li> Cost Over-run (33%) 287 </li></ul><ul><li> Time Over-run (1 to 196 months)(35%) 301 </li></ul>
  13. 13. III. Reasons for Delay in Project Implementation <ul><li>Fund Constraints </li></ul><ul><li>Land Acquisition </li></ul><ul><li>Environment Clearance </li></ul><ul><li>Slow Progress </li></ul><ul><li>Delay in Supply of Equipments </li></ul><ul><li>Law and Order </li></ul><ul><li>Others (Technology selection and agreement, Award of contract, Delay in civil works and government clearance, Geomining, Court case, Inadequate Infrastructure and bad weather ) </li></ul><ul><li>(Source: MOSPI) </li></ul>
  14. 14. IV E-government Project Failure: An Indian Example <ul><li>Reinventing EPF (Employees’ Provident Fund) India </li></ul><ul><li> The largest reform project in terms of complexity </li></ul><ul><li> Touches 40 million citizens </li></ul><ul><li> Rs 250 million already spent </li></ul><ul><li> Time over-run: 2001-06: 66 months: Target: 22 </li></ul><ul><li>months </li></ul><ul><li> Scrapped: January 2008: To be started all over </li></ul><ul><li>again </li></ul><ul><li> Contractor: Siemens Information Systems Limited (SISL) </li></ul><ul><li>Source: Dhoot (2008) </li></ul>
  15. 15. V Causes of E-government Project Failure <ul><li>Complexity </li></ul><ul><li>Commitment Failure </li></ul><ul><li>Planning Failure </li></ul><ul><li>Vision Failure </li></ul>
  16. 16. Causes of E-government Project Failure <ul><li>Inappropriate Methods </li></ul><ul><li>Short Time Horizon </li></ul><ul><li>Turbulent Environments </li></ul><ul><li>Failure to Support End Users </li></ul><ul><li>Source: Chiles (2001) </li></ul>
  17. 17. VI What can Monitoring do? <ul><li>It can prevent E-government Project Failures </li></ul><ul><li>It can prevent cost and time over-runs of E-government Projects </li></ul><ul><li>It can keep a track of progress of E-government Project implementation </li></ul><ul><li>It can ensure that resources are expended as planned </li></ul><ul><li>Above all, it can ensure that the benefits of e-government project reach the target group, that is, citizens and non-citizens </li></ul>
  18. 18. VII What is not Monitoring? <ul><li>Reporting ≠ Monitoring </li></ul><ul><li>Inspection ≠ Monitoring </li></ul><ul><li>Supervision ≠ Monitoring </li></ul><ul><li>Audit* ≠ Monitoring </li></ul><ul><li>Surveillance ≠ Monitoring </li></ul><ul><li>Review ≠ Monitoring </li></ul><ul><li>* Audit: 1. Financial, 2. Performance, 3. Development, 4. Social, 5. Citizen (Through RTI Act) </li></ul>
  19. 19. What is not Monitoring? Comprehensive feedback Review 6 Scanning of environment Surveillance 5 Examination of accounts Audit 4 Overseeing implementation Supervision 3 Fault-finding in detail Inspection 2 Routine reporting Report 1 Focus Tool SN
  20. 20. What is not Monitoring? RTI Act ? Citizen / Media Audit 5 Scrutiny from social point Social Audit 4 Quality and durability of assets (IRDP) Development Audit 3 Project experience Performance Audit 2 Conformance to financial rules Financial Audit 1 Focus Tool SN
  21. 21. VIII What is then E-government Monitoring? <ul><li>E-government Monitoring is a specialised , systematic , dynamic , and semi-autonomous management tool to ensure that the E-government Project serves the target group - Citizens and Non-Citizens- in accordance with e-business plan taking into account the interests of various stakeholders and the emerging challenges being faced by E-government </li></ul>
  22. 22. Elements of Definition Emerging Challenges 10 Management Tool 5 Stakeholders 9 Semi-autonomous 4 E-business plan 8 Dynamic 3 Service to Citizens 7 Systematic 2 E-government Project 6 Specialisation 1
  23. 23. IX What is E-government Project? <ul><li>An E-government Project is a development project which aims to transform an inward-looking government to a citizen centric government making best use of information and communication technologies (ICTs) through a carefully designed e-business plan </li></ul>
  24. 24. X. E-government Project Lifecycles <ul><li>Five Models </li></ul><ul><li>Generic Model (Tasmania, Australia) </li></ul><ul><li>Technocratic Model (NIC, India) </li></ul><ul><li>Audit Model (Lea’s Model) </li></ul><ul><li>Systems Model (Heeks’ Model) </li></ul><ul><li>E-government Project Model </li></ul>
  25. 25. 1. Generic Model (Government of Tasmania, Ausralia) Source: Government of Tasmania 2002
  26. 26. 2. E-government Technocratic Model (National Informatics Centre, New Delhi) Source: Mishra 2005 A. Project Initiation and Planning B. Software Development C. ICT Infrastructure Creation D. Service Provision E. System Integration and Testing F. Project Commissioning G. Project Completion and Sign Off H. Maintenance I. Retirement
  27. 27. 3. Lea’s E-government Project Lifecycle (Audit Model) Source: Lea 2003 Initiation Planning and Implementation Monitoring Operations
  28. 28. 4. Heeks’ System Lifecycle Project assessment System Construction Design of the proposed new system Implementation and beyond Analysis of current reality 1 5 4 3 2 Source: Heeks 2006
  29. 29. 5. E-government Project Cycle Modify Modify Modify M&E Unit 4 Evaluate E-business Plan 5 Review E-business Plan 3 Monitor E-business Plan 2 Implement E-business Plan 1 Prepare E-business Plan
  30. 30. XI. Logical Framework Approach (LFA) <ul><li> Developed by Leon J. Rosenberg for USAID in 1969 </li></ul><ul><li> It logically connects project activities to </li></ul><ul><li>results </li></ul><ul><li> The logic is: If x is done (input), y will </li></ul><ul><li>follow (output) under asumptions z (outcome) </li></ul><ul><li> It presents a concise picture of the project in a page or two </li></ul>
  31. 31. Logical Framework Approach (LFA) <ul><li> It is an analytical tool </li></ul><ul><li> LFA (approach) should not be </li></ul><ul><li>confused with logframe (document) </li></ul><ul><li> It is 4x4 Matrix (that is, it is a matrix </li></ul><ul><li>of four rows and four columns ) </li></ul><ul><li>giving rise to 16 cells </li></ul>
  32. 32. Logical Framework Approach (LFA) C44 (Critical Conditions) C43 C42 C41 Inputs (Resources) C34 (Implementation Assumptions) C33 C32 C31 Outputs (Activities) C24 (Project Assumptions) C23 C22 C21 Objectives (Short-term Objectives) C14 (Hypothesis) C13 C12 C11 Goal (Wider objectives) Assumptions and Risks (Principal Methods) Measurement (Means of Verification) Indicators (of Progress) Structure (Type of Information) S.N.
  33. 33. XII. Who will do E-government Monitoring? <ul><li>The E-government Monitoring Unit </li></ul><ul><li> Set up a monitoring unit </li></ul><ul><li> It will be part of organisation but function independently </li></ul><ul><li> It will report directly to Top Management </li></ul><ul><li> It will be a specialised unit </li></ul>
  34. 34. XIII. The E-government Monitoring Unit <ul><li> It will consist of: </li></ul><ul><li>Head of Monitoring Unit </li></ul><ul><li>Database Administrator/System Analyst </li></ul><ul><li>Statistician </li></ul><ul><li>Economist </li></ul><ul><li>Sociologist/Political Scientist </li></ul>
  35. 35. XIV. E-government Monitoring Methodology <ul><li>II. Manual </li></ul><ul><li>Desk Research </li></ul><ul><li>Sample Survey </li></ul><ul><li>Focus Groups </li></ul><ul><li>Case Studies </li></ul><ul><li>Individual/Group Interviews/Discussions </li></ul><ul><li>Participatory Appraisals, etc. </li></ul><ul><li>I. Automated </li></ul><ul><li>1. Online Survey </li></ul><ul><li>(SurveyMonkey) </li></ul><ul><li>Virtual Focus Groups (E-groups) </li></ul><ul><li>E-mail Surveys </li></ul><ul><li>Blog (Comments) </li></ul><ul><li>Wiki (Comments) </li></ul><ul><li>Online Feedback, etc. </li></ul>
  36. 36. XV. Guiding Principles of E-government Monitoring <ul><li>E-government monitoring must be </li></ul><ul><li>Action-oriented </li></ul><ul><li>Web-based </li></ul><ul><li>Top management-oriented </li></ul><ul><li>Specialised </li></ul><ul><li>Citizen-centric </li></ul><ul><li>Simple </li></ul><ul><li>Timely </li></ul><ul><li>Relevant </li></ul><ul><li>Accurate </li></ul><ul><li>Flexible </li></ul>
  37. 37. Part B: Evaluation <ul><li>I. Historical Background of Evaluation </li></ul><ul><li> Systematic Evaluation is older than </li></ul><ul><li>Centralised Monitoring </li></ul><ul><li> Evaluation came into being in 1952 when </li></ul><ul><li>Programme Evaluation Organisation </li></ul><ul><li>(PEO), an independent organisation </li></ul><ul><li> was set up in Planning Commission </li></ul><ul><li> to evaluate India’s Community </li></ul><ul><li>Development (CD) Programme </li></ul><ul><li> PEO survives till date (2008) </li></ul>
  38. 38. Evaluation <ul><li>II Programme Evaluation Organisation (PEO) </li></ul><ul><li> Headed by Adviser (Evaluation ), it has </li></ul><ul><li>3-tier Structure </li></ul><ul><li> 1. Headquarters - Planning Commission </li></ul><ul><li> 2. Regional Evaluation Offices (7) </li></ul><ul><li>(Kolkata, Chandigarh, Chennai, </li></ul><ul><li>Hyderabad,Jaipur, Lucknow and Mumbai) </li></ul><ul><li> 3. Project Evaluation Offices (8) (State </li></ul><ul><li>Capitals- Guwahati, Bhubaneswar, </li></ul><ul><li>Shimla, Bangalore,Bhopal, Patna, </li></ul><ul><li>Thiruvananthapuram and Ahmedabad) </li></ul>
  39. 39. III. Functions of PEO <ul><li> Undertakes evaluation of selected </li></ul><ul><li>programmes/schemes under </li></ul><ul><li>implementation </li></ul><ul><li> Evaluation studies, assess </li></ul><ul><li> the performance , </li></ul><ul><li> process of implementation , </li></ul><ul><li> effectiveness of the delivery systems and </li></ul><ul><li> impact of programmes. </li></ul>
  40. 40. IV Four Generations of Evaluation Source: Guba and Lincoln 1989 Mediator (among conflicting stakeholders) Response (Interaction) & Construction (Methodology ) An Alternative Approach Fourth Generation 4 Judge Reaching Judgements Judgment Third Generation 3 Describer Strengths and Weaknesses Description Second Generation 2 Technical Measuring Instruments Measurement First Generation 1 Role of Evaluator Focus Name Generation SN
  41. 41. V What is not Evaluation? <ul><li>Analysis ≠ Evaluation </li></ul><ul><li>Measurement ≠ Evaluation </li></ul><ul><li>Assessment ≠ Evaluation </li></ul><ul><li>Appraisal ≠ Evaluation </li></ul><ul><li>Audit ≠ Evaluation </li></ul><ul><li>Monitoring ≠ Evaluation </li></ul>
  42. 42. What is not Evaluation? Implementation Monitoring 6 Rules and regulations Audit 5 Investment Appraisal 4 Cost and Benefits Assessment 3 Metrics Measurement 2 Breaking into parts Analysis 1 Focus Tool S.N.
  43. 43. VI. Four Senses of Term Evaluation Calculation of the value of an expression (in Mathematics) Sense 4 4 Work done by professional evaluators Sense 3 3 An autonomous discipline: Study and application of procedures for doing objective and systematic evaluation Sense 2 2 Merit, Value or Worth of Something Sense 1 1 Definition Sense S.N.
  44. 44. VII. What is then Evaluation? <ul><li>“… a robust arena of activity directed at collecting , analyzing , and interpreting i nformation on the need for, implementation of, and effectiveness and efficiency of intervention efforts to better the lot of humankind.” </li></ul><ul><li>---Rossi and Freeman (1989, p-13) </li></ul>
  45. 45. VIII. What is E-government Evaluation <ul><li>E-government evaluation is a systematic , objective , planned and participatory exercise undertaken during the design , implementation and after completion of a project for determining the worth of e-government to citizens against pre-set objectives and criteria for improving e-government services to citizens. </li></ul>
  46. 46. IX. Types of Evaluation <ul><li>Ad hoc evaluation </li></ul><ul><li>Evaluation in vivo </li></ul><ul><li>Pilot project evaluation </li></ul><ul><li>Terminal evaluation </li></ul><ul><li>Formative and summative evaluation </li></ul><ul><li>Meta evaluation (evaluation of evaluations) (Stufflebeam 1981/ Scriven 1991) </li></ul><ul><li>Formal and informal evaluation </li></ul><ul><li>Insider and outsider evaluation </li></ul><ul><li>Compliance , effectiveness , significance and efficiency evaluation </li></ul><ul><li>On-going , process or concurrent evaluation </li></ul><ul><li>Ex ante and ex post evaluation </li></ul>
  47. 47. Types of Evaluation <ul><li>Self-checking evaluation (ibid.) </li></ul><ul><li>Do-it-yourself evaluation (ibid.) </li></ul><ul><li>Systematic evaluation (Rossi and Freeman ) </li></ul><ul><li>Fourth generation evaluation (Guba and Lincoln 1989) </li></ul><ul><li>Scientific evaluation </li></ul><ul><li>Auto or self evaluation (ACCORD 1993) </li></ul><ul><li>Thematic evaluation </li></ul><ul><li>Individual , group and institutional evaluation </li></ul><ul><li>Casual everyday evaluation (Frutchey 1959) </li></ul>
  48. 48. Types of Evaluation 26. Right to information evaluation (being undertaken by Adam Smith International/ Administrative Staff College of India) (2007) 27. Evaluation research 28. Online and offline evaluation 29. E-government evaluation 30. E-government special studies ,etc. <ul><li>21. Adversary </li></ul><ul><li>evaluation </li></ul><ul><li>Empowerment </li></ul><ul><li>evaluation (Fetterman et al.(eds.)(1996) </li></ul><ul><li>23. Utilisation-focused evaluation (Patton 1997) </li></ul><ul><li>24. Transparency </li></ul><ul><li>evaluation </li></ul><ul><li>25. Citizen evaluation </li></ul>
  49. 49. X. Evaluation Timing and Diffusion-Adoption Curves <ul><li>Hypothetical Curves </li></ul>Time (months/years) 1 2 3 4 5 6 0 50 100 (c) (b) (a)
  50. 50. XI. What is to be evaluated? Domains of E-government Evaluation I External Environment III Service Delivery II Organisation
  51. 51. XII. How e-government domains are evaluated? <ul><li>Indicators </li></ul><ul><li>1. Input indicators </li></ul><ul><li>2. Output indicators </li></ul><ul><li>3. Usage/Intensity indicators </li></ul><ul><li>4. Impact/Effect indicators </li></ul><ul><li>5. Environmental/Readiness indicators </li></ul><ul><li>(Source: Jenssen 2005) </li></ul>
  52. 52. Indicators <ul><li>1. Input Indicators </li></ul><ul><li>– Amount of financial resources devoted to eGovernment. Absolute figures, per capita figures. </li></ul><ul><li>– IT/e-Government spending as % of GDP. </li></ul><ul><li>– Amount of resources devoted to Research and Development. </li></ul><ul><li>– Amount of public resources devoted to internet infrastructure. </li></ul>
  53. 53. Indicators <ul><li>2. Output Indicators </li></ul><ul><li>– Number of online services for citizens; </li></ul><ul><li>– Number of online services for businesses; </li></ul><ul><li>– Percentage of government departments that have a website; </li></ul><ul><li>– Percentage of government websites that offer electronic services. </li></ul>
  54. 54. Indicators <ul><li>3. Usage Indicators </li></ul><ul><li>– Number of individuals that have made use of electronic services offered; </li></ul><ul><li>– Number of businesses that have made use of electronic services offered; </li></ul><ul><li>– Percentage of citizens that has visited government websites to search for information; </li></ul><ul><li>– Number of businesses that have made payments online; </li></ul><ul><li>– Percentage of internet traffic that pertains to electronic service delivery. </li></ul>
  55. 55. Indicators <ul><li>4. Impact Indicators </li></ul><ul><li>– reduction of waiting time at government counter x by y %; </li></ul><ul><li>– decrease in case processing time at government organisation x by y %; </li></ul><ul><li>– citizen/business satisfaction levels concerning eGovernment; </li></ul><ul><li>– survey-type questions, e.g.: ‘do you feel more positive to your government, now that you can contact it by email?’ ‘has your government become more efficient, now that you can perform services online?’ </li></ul>
  56. 56. Indicators <ul><li>5. Environment Indicators </li></ul><ul><li>– ICT penetration rates (pc, internet, mobile phone) private households, work, schools; </li></ul><ul><li>– Indicator that measures ‘fear of invasion of privacy’; </li></ul><ul><li>– Online shopping rates as an indicator of trust in online environments; </li></ul><ul><li>– Indicator that measures ‘quality of legislation concerning the information society’; </li></ul><ul><li>– Telephone tariffs, GSM tariffs, Internet access tariffs. </li></ul>
  57. 57. XIII. Approaches to Evaluation Studies <ul><li>Cross-Sectional Studies (Comparison of a group with treatment with another group without treatment ) </li></ul><ul><li>Longitudinal Studies (Comparison of a group before and after treatment) </li></ul><ul><li>Benchmarking Studies (Comparison with best practices) </li></ul><ul><li>“ Value Addition ” Studies (Accenture ) </li></ul>
  58. 58. Approaches to Evaluation Studies <ul><li>5. Citizen Satisfaction Studies (American </li></ul><ul><li>Consumer Satisfaction Index-ACSI)/ </li></ul><ul><li>Citizen’s Report Card -Public Affairs </li></ul><ul><li>Centre (PAC), Bangalore </li></ul><ul><li>6. Department of Information Technology </li></ul><ul><li>(DIT)’s Evaluation Assessment </li></ul><ul><li>Framework (EAF) </li></ul><ul><li>7. U.S. Office of Management and Budget (OMB)’s Program Assessment Rating Tool (PART) </li></ul>
  59. 59. 1. DIT’s Evaluation Assessment Framework (EAF) <ul><li>EAF Version 2.0 (2004) </li></ul><ul><li>Service Orientation </li></ul><ul><li>Technology </li></ul><ul><li>Sustainability </li></ul><ul><li>Cost Effectiveness </li></ul><ul><li>Replicability </li></ul>
  60. 60. Weights for Attributes
  61. 61. Rating of E-government Projects in India 2005-06 Source: DIT 100 37 Total 8 3 Poor (P) 4 3 1 Satisfactory (S) 3 46 17 Good (G) 2 43 16 Extremely Good (EG) 1 Percentage Number of Projects Rating SN
  62. 62. 3. U.S. Program Assessment Rating Tool (PART) <ul><li>Clarity of Purpose and Well-Designed </li></ul><ul><li>Strategic Planning (valid annual and long-term goals) </li></ul><ul><li>Management (program, financial oversight and program improvement efforts) </li></ul><ul><li>Results (accuracy, consistency) </li></ul><ul><li>(Source: ExpectMore.gov) </li></ul>
  63. 63. Assessmet of U.S.Federal Programs by “PART” (2008) DISTRIBUTION OF PROGRAM RATINGS Source: ExpectMore.gov 1004 18% 31% 29% 3% 19% Number of Programs Assessed Effective Moderately Effective Adequate Ineffective Results Not Demonstrated
  64. 64. XIV. Who will evaluate? <ul><li>First decide whether evaluation will be done in-house or by an outside agency </li></ul><ul><li>In-house evaluation is preferable as it builds evaluation capability in-house </li></ul><ul><li>No separate in-house evaluation unit is required </li></ul><ul><li>Entrust the evaluation function to the in-house monitoring unit , suggested earlier </li></ul><ul><li>And call it monitoring and evaluation (M&E) unit </li></ul><ul><li>Let outside agencies also undertake evaluation and special studies after a gap of 3/5 years </li></ul>
  65. 65. XV Which Type of Evaluation is Suited Most to E-government? <ul><li> The type of evaluation will depend upon the specific requirements of an e-government project . </li></ul><ul><li> There are two key stakeholders in e-government: 1. E-government Management , and 2. Citizens </li></ul><ul><li> There are four standards for evaluation: 1. Utility 2. Feasibility 3. Propriety 4. Accuracy </li></ul><ul><li> E-government evaluation must meet the following two criteria: (a) Utility (to stakeholders-Management and Citizens) and (b) Actual Use, both geared to serve citizens </li></ul>
  66. 66. Which Type of Evaluation is Suited Most to E-government? <ul><li> Utilisation-focused Evaluation (Patton 1997) meets our criteria of selection of type of evaluation </li></ul><ul><li> Mere provision of government services online is not e-government </li></ul><ul><li> The online services must be utilised by the target group- the citizens </li></ul><ul><li> The issue of impact of e-government will arise only when the following is satisfied: </li></ul><ul><li>PROVISION UTILISATION IMPACT </li></ul><ul><li>(of e-gov (by citizens) (on general well- </li></ul><ul><li>services) being of citizens) </li></ul>
  67. 67. Part C: E-government M&E <ul><li>I. E-government Monitoring and Evaluation (M&E) </li></ul><ul><li> M&E is an under-developed aspect of E-government </li></ul><ul><li> It has so far not found any systematic application in e-government project implementation </li></ul><ul><li> M&E findings, where available, are not widely diffused </li></ul><ul><li> Its neglect hampers e-government development </li></ul><ul><li> M&E is a tool for development of E-government </li></ul>
  68. 68. II.E-government Monitoring and Evaluation (M&E) Unit <ul><li>E-government M&E unit will have the same staff as the monitoring unit </li></ul><ul><li>A part of the organisation, it will report directly to top management </li></ul><ul><li>It will undertake regular monthly monitoring of e-government project and evaluation six-monthly or annually </li></ul><ul><li>It will give 80% weightage to monitoring and 20% to evaluation </li></ul>
  69. 69. III. Relative Weights to Monitoring and Evaluation <ul><li>Relative Importance of M&E </li></ul>M e E m Initial Stage After 4/5 years
  70. 70. IV. E-government M&E Framework: Components <ul><li>The E-government M&E Framework consists of five components: </li></ul><ul><li>(a) E-government Management </li></ul><ul><li>(b) E-government M&E Unit </li></ul><ul><li>(c) Information Needs Matrix </li></ul><ul><li>(d) E-government M&E Cycle </li></ul><ul><li>(e) Citizens </li></ul>
  71. 71. IV(a). E-government Management <ul><li>Supporting Staff </li></ul>Top Management Middle Management Information Technology (IT) Department
  72. 72. IV(b). E-government M&E Unit Top Management Head of M&E Unit Database Administrator/ System Analyst Statistician Economist Political Scientist/ Sociologist
  73. 73. IV(c). Information Needs Matrix <ul><li>Information Needs of Management </li></ul><ul><li> Information needs differ in three levels of management hierarchy: 1. Top Management 2. Middle Management 3. Supporting Staff </li></ul><ul><li> Undertake information needs analysis of various levels of management hierarchy </li></ul><ul><li> Prepare Information Needs Matrix </li></ul>
  74. 74. Information Needs Matrix Supporting Staff Middle Management Top Management Review Information Utilisation Information Citizen Access Information ICT Infrastructure Information Organisatinal Needs Information Citizen Needs Information Information Needs IT Department
  75. 75. IV(d). E-government M&E Cycle M&E Cycle 4. Evaluate it 2. Implement it 3. Monitor it 5. Review it 1. Prepare E-business Plan
  76. 76. IV(e) Citizens <ul><li>Citizens interact with government in following four ways: As </li></ul><ul><li>(a) Information </li></ul><ul><li>Seekers </li></ul><ul><li>(of government </li></ul><ul><li>activities) </li></ul><ul><li>(b) Service Users (of </li></ul><ul><li>public services) </li></ul>We want service
  77. 77. Citizens <ul><li>(c) Beneficiaries </li></ul><ul><li>(of public programmes like NREGP) </li></ul><ul><li>(d) Compliers </li></ul><ul><li>(with laws, rules and regulations </li></ul><ul><li>like payment of taxes) </li></ul><ul><li>(e) Stakeholders </li></ul><ul><li>(in public policies and programmes) </li></ul>Their needs in these capacities have to be identified and met by E-government M&E
  78. 78. V. A Framework for E-government Monitoring and Evaluation <ul><li> We are now in position to link these components and present </li></ul><ul><li> A Framework for E-government Monitoring and Evaluation </li></ul><ul><li> The Framework conceptualises the complex reality of e-government and provides a Roadmap for E-government M&E Unit </li></ul><ul><li> Here then is the Framework. </li></ul>
  79. 79. A Framework for E-government Monitoring and Evaluation Formal Sources of Information Informal Sources of Information ICT Indicators Forecasting Monitoring & Evaluation Census & Surveys Audit Discussion Group Blog Wiki Social Sites Legislature RTI Act M&E Unit IT Deptt E-business Plan Implementation Sustainability Citizens M&E Unit Information Matrix M&E Cycle (d) (a) (c) (b) Supporting Staff Top Management Middle Management M&E Cycle Citizens We want service Supporting Staff Middle Mgt Top Mgt Review Information Utilisation Information Citizen Access Information ICT Infrastructure Information Organisatinal Needs Information Citizen Needs Information Information Needs IT Department Evaluate Review Prepare E-business Plan Implement Monitor
  80. 80. Contribution of M&E to Good Governance <ul><li>Monitoring information and evaluation findings can contribute to sound governance in a number of ways: </li></ul><ul><li>Evidence-based policy making (including budget decision making), </li></ul><ul><li>Policy development , management , and accountability . </li></ul><ul><li>Many governments around the world have realized much of this potential, including most OECD countries and a small but growing number of developing countries. (Source: Mackay 2007) </li></ul>
  81. 81. Contribution of M&E to Good Governance <ul><li>Examples of well-functioning government M&E system : </li></ul><ul><li>(Source: Mackay 2007) </li></ul><ul><li>1. Australia (by 1994, almost 80 percent of new spending proposals relied on evaluation findings) </li></ul><ul><li>2. Colombia (which has about 500 performance indicators) </li></ul><ul><li>3. United Kingdom (Public Sector Agreements between the Treasury and each of the 18 main departments) </li></ul><ul><li>4. U.S.A. (PART (Program Assessment Rating Tool), created in 2002, rates all 1,000 government programmes), and </li></ul><ul><li>5. Chile (whose Finance Ministry collects 1,550 performance indicators). </li></ul>
  82. 82. Part D: My Questions <ul><li>With this I end my presentation  but ask the following questions  </li></ul><ul><li>What is monitoring? How does it differ from other sources of management information? </li></ul><ul><li>What is evaluation? Describe different types of evaluation. Which type of evaluation is most suited to e-government? Give reasons for your answer. </li></ul>
  83. 83. My Questions <ul><li>What role can an e-government monitoring and evaluation (m&e) unit play in successful implementation of an e-government project? </li></ul><ul><li>Do you agree that development of a framework for e-government monitoring and evaluation (m&e) can provide a useful roadmap for m&e unit? Give reasons for your answer. </li></ul>
  84. 84. Your questions now! <ul><li>Thank you for your attention. </li></ul>Have a nice day . --Dr D.C.Misra