Case studies of performance audits - B5

4,123 views

Published on

Approach of the Slovenian Court of Audit to Perfomance Audit of public bodies. Case examples given.

2 Comments
5 Likes
Statistics
Notes
  • Thank you for constructive feedback Paul. You are quite right. In our defense I can only say that the “watch dog” attitude is something that the public and Parliament demand from us. It is not the kind of approach that we use towards the audited public fund users (at least not the initial one). With them we always try to establish the kind of approach that you have described (and in vast majority of cases - I’d say more than 90% - we even succeed in doing that).
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • I like your presentation. Very comprehensive review of your performance audit process. I am cautious of your guard dog/ watch dog analogy. Performance Audits, from my experience, have the most impact when the client agency embraces the audit process as an improvement and QA assistant/partnership. That is, when the Audit is seen to be looking to assist them by making recommendations to improve the process. Therefore to set up the construct/relationship with a watch dog mentality, it gives the client an unnecessary negative and defensive strategy.
    Having said that, I also think an important part of performance audit is to demystify the process, and you do a good job of that here. well done.
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total views
4,123
On SlideShare
0
From Embeds
0
Number of Embeds
7
Actions
Shares
0
Downloads
197
Comments
2
Likes
5
Embeds 0
No embeds

No notes for slide
  • Opombe za predavatelja
  • Case studies of performance audits - B5

    1. 1. Case-Studies of Performance Audits Slovenian Court of Audit Approach Vienna, Austria, 7. 5. 2010 dr. Miroslav Kranjc, Supreme State Auditor Court of Audit of the Republic of Slovenia
    2. 2. About the Court of Audit of the Republic of Slovenia 1/2 <ul><li>Established in 1995 </li></ul><ul><li>133 staff </li></ul><ul><li>69,9% are auditors, 22,6% support </li></ul><ul><li>67,7% of all staff has university degree and 24,1% M.Sc. or Dr.Sc. </li></ul><ul><li>6.493.038€ budget </li></ul><ul><li>1.282 published reports </li></ul><ul><li>Currently 70 reports per year </li></ul>
    3. 3. About the Court of Audit of the Republic of Slovenia 2/2 <ul><li>3 Cabinets (President and two Deputies) </li></ul><ul><li>6 Audit departments (Sup. State Audit.) </li></ul><ul><ul><li>State budget, jurisdictional system and political parties </li></ul></ul><ul><ul><li>Municipalities and Local government </li></ul></ul><ul><ul><li>Non-commercial public services </li></ul></ul><ul><ul><li>Public utilities, privatisation funds, agencies </li></ul></ul><ul><ul><li>Performance and IT Audit </li></ul></ul><ul><ul><li>European and international Funds </li></ul></ul><ul><li>1 Shared Services Department </li></ul>
    4. 4. About me <ul><li>15 years of professional experience in public and private sector </li></ul><ul><li>5 years as supreme state auditor (SSA) </li></ul><ul><li>leading the Department for Performance Audit and IT Audit since it’s establishment </li></ul><ul><li>B.Sc., M.Sc., Dr.Sc. from automatic control of industrial processes </li></ul>
    5. 5. Lecture overview <ul><li>Legal grounds for Performance Audit (PA) </li></ul><ul><li>PA at the Court of Audit of the Republic of Slovenia (CoA) </li></ul><ul><li>What is PA and what it ought to be </li></ul><ul><li>PA questions and challenges </li></ul><ul><li>PA lifecycle at the CoA </li></ul><ul><li>Case-Studies with Q&A </li></ul><ul><li>Conclusion and additional sources </li></ul>
    6. 6. Legal Grounds for Performance Audit
    7. 7. INTOSAI Auditing Standards 1/2 <ul><li>1.0. 38 The full scope of government auditing includes regularity and performance audit . </li></ul><ul><li>1.0.39 Regularity audit embraces: </li></ul><ul><li>(a) ... ; </li></ul><ul><li>(b) ...; </li></ul><ul><li>(c) ...; </li></ul><ul><li>: </li></ul><ul><li>(f) ... . </li></ul><ul><li>1.0.41 In practice there can be an overlap between regularity and performance auditing, and in such cases classification of a particular audit will depend on the primary purpose of that audit. </li></ul>Regularity audit Performance audit
    8. 8. INTOSAI Auditing Standards 2/2 <ul><li>1.0.40 Performance audit is concerned with the audit of economy, efficiency and effectiveness and embraces: </li></ul><ul><li>(a) audit of the economy of administrative activities in accordance with sound administrative principles and practices, and management policies; </li></ul><ul><li>(b) audit of the efficiency of utilisation of human, financial and other resources, including examination of information systems , performance measures and monitoring arrangements, and procedures followed by audited entities for remedying identified deficiencies; and </li></ul><ul><li>(c) audit of the effectiveness of performance in relation to the achievement of the objectives of the audited entity, and audit of the actual impact of activities compared with the intended impact. </li></ul>
    9. 9. INTOSAI Implementation Guidelines for PA <ul><li>Performance auditing is an independent examination of the e fficiency and e ffectiveness of government undertakings, programs or organizations, with due regard to e conomy, and the aim of leading to improvements . </li></ul>Independence from the subject of the audit Ultimate goal
    10. 10. The Constitution of the Republic of Slovenia <ul><li>Article 150 </li></ul><ul><li>( Court of Audit ) </li></ul><ul><li>(1) The Court of Audit is the highest body for supervising state budged and all public spending. </li></ul><ul><li>(2) The organisation and powers of the Court of Audit are provided by law . </li></ul><ul><li>(3) The Court of Audit is independent in performance of its duties and bound by the Constitution and the laws. </li></ul>
    11. 11. Court of Audit Act 1/3 <ul><li>Article 20 </li></ul><ul><li>(Auditing Powers) </li></ul><ul><li>(1) The Court of Audit shall audit the business operation of users of public funds : </li></ul><ul><li>1. it may carry out regularity and performance audits ; </li></ul><ul><li>2. it may audit any act on past operations as well as any act on planned business operation of any user of public funds. </li></ul>Financial statements, internal controls, financial plans ... Regularity audit Performance audit
    12. 12. <ul><li>Article 20 </li></ul><ul><li>(Auditing Powers) </li></ul><ul><li>(3) Auditing of business operation under this Act shall be the obtaining of relevant and sufficient data to express an opinion on the business operation of the auditee: </li></ul><ul><li>1. Regularity audits provide relevant and sufficient data to enable an opinion to be expressed on compliance of operation with regulations and guidelines that any user of public funds is required to observe in the conduct of business operation; </li></ul><ul><li>2. Performance audits provide relevant and sufficient data to enable an opinion to be expressed on economy , efficiency and effectiveness of business operation. </li></ul>Court of Audit Act 2 /3
    13. 13. <ul><li>Article 20 </li></ul><ul><li>(Auditing Powers) </li></ul><ul><li>(5) The user of public funds under this Act shall be: </li></ul><ul><li>1. any legal entity of public law or a unit thereof; </li></ul><ul><li>2. any legal entity of private law provided that one of the following applies: </li></ul><ul><li>- it has received financial support from the budget of the European Union, state budget or local community budget; </li></ul><ul><li>- it performs public services or provides public goods on concession basis; </li></ul><ul><li>- it is a commercial company, bank or insurance company in which the state or a local community holds the majority share; </li></ul><ul><li>3. any physical person provided that one of the following applies: </li></ul><ul><li>- he/she has received financial support from the budget of the European Union, state budget or local community budget; </li></ul><ul><li>- he/she performs public services or provides public goods on concession basis. </li></ul>Court of Audit Act 3 /3
    14. 14. INTOSAI definitions of 3E in CoA Act  Court of Audit Act INTOSAI Uspešnost E ffectiveness Učinkovitost E fficiency Gospodarnost E conomy Performance audits provide relevant and sufficient data to enable an opinion to be expressed on economy, efficiency and effectiveness of business operation. Performance audit is concerned with the audit of economy, efficiency and effectiveness.
    15. 15. Key attributes of performance audit at the CoA <ul><li>Independence from the audited entity </li></ul><ul><ul><li>CoA possess firm legal ground and other necessary conditions </li></ul></ul><ul><ul><li>Audited entity is ( in rising number of cases ) only object of examination </li></ul></ul><ul><ul><li>CoA provide s information, advice and assurance to the National Assembly to better scrutinize activities and efforts to deliver value for money </li></ul></ul><ul><li>Add value – path to improvement </li></ul><ul><ul><li>Recommendations are mandatory part of the performance audit report </li></ul></ul><ul><ul><li>CoA provide s advice and assistance to auditees to help them improve </li></ul></ul>
    16. 16. <ul><li>We are not in the business of criticizing government! </li></ul>Our role <ul><li>We have dual role of: </li></ul><ul><ul><li>Guard dog (we guard public funds) </li></ul></ul><ul><ul><li>Watch dog (we guide and give recommendations) </li></ul></ul>
    17. 17. Performance Audit at the Court of Audit of the Republic of Slovenia
    18. 18. Performance audit evolution at the CoA <ul><li>Adoption of new Court of Audit Act and Rules of Procedure (2001) </li></ul><ul><li>3 Twinning projects with NAO (2001-07) </li></ul><ul><li>Establishment of PA department (2005) </li></ul><ul><li>Performance Audit manual (2007) </li></ul><ul><li>Close cooperation with: </li></ul><ul><ul><li>INTOSAI </li></ul></ul><ul><ul><li>EUROSAI </li></ul></ul><ul><ul><li>GAO </li></ul></ul><ul><ul><li>ECA </li></ul></ul><ul><ul><li>Good relations and experience exchange with number of European and world SAI’s </li></ul></ul><ul><li>Respecting the wishes of National Assembly </li></ul>
    19. 19. Percentage of Performance Audits at CoA Sources:  Annual reports 01-08,  CoA website (on 4 . 5 . 2010) and  Goal from Slovene Court of Audit Strategy 2007-2013
    20. 20. Which types of PA does CoA performs? <ul><li>We do: </li></ul><ul><ul><li>E fficiency, E ffectiveness, E conomy </li></ul></ul><ul><ul><li>E nvironmental </li></ul></ul><ul><ul><li>IT </li></ul></ul><ul><ul><li>RBB (Results based budgeting) </li></ul></ul><ul><ul><li>E quity </li></ul></ul><ul><ul><li>Quality of Service </li></ul></ul><ul><li>But not yet all of VFM types: </li></ul><ul><ul><li>Sustainability </li></ul></ul><ul><ul><li>Performance measurement </li></ul></ul><ul><ul><li>Consumer interests </li></ul></ul> 
    21. 21. What is Performance Audit and what it ought to be
    22. 22. Performance Audits are ... 1/2 <ul><li>PA are: </li></ul><ul><ul><li>disclosing and describing the reasons for found deficiencies </li></ul></ul><ul><ul><li>researching important new initiatives </li></ul></ul><ul><ul><li>responding to new challenges </li></ul></ul><ul><ul><li>following preceding audits </li></ul></ul><ul><ul><li>evaluating outcomes </li></ul></ul>
    23. 23. Performance Audits are ... 2/2 <ul><li>But: </li></ul><ul><ul><li>have more judgment in interpretation </li></ul></ul><ul><ul><li>have more selective coverage </li></ul></ul><ul><ul><li>use criteria for evaluation that are less pre-defined </li></ul></ul><ul><ul><li>their evidence tend to be more persuasive than conclusive - no standardized opinions </li></ul></ul><ul><ul><li>reports contain more discussion and reasoned argument </li></ul></ul><ul><ul><li>are more flexible in regard to goals, methods used, forming the opinion </li></ul></ul><ul><ul><li>are unique (non repetitive and non-standardized) </li></ul></ul><ul><ul><li>use wider scope and use more diverse range of methods </li></ul></ul><ul><ul><li>are not successors of commercial audits in private sector </li></ul></ul>
    24. 24. Performance Audits ought to be... <ul><li>Economic and effective </li></ul><ul><li>Achieve sustainable improvement </li></ul><ul><li>Useful (achieve financial impact) </li></ul><ul><li>Improve quality of service (time, method of delivery, more for less) </li></ul><ul><li>Constructive, fair and objective </li></ul><ul><li>On time </li></ul><ul><li>Add value! </li></ul>
    25. 25. Special required skills <ul><li>PA Auditors should possess skills in the field of: </li></ul><ul><ul><li>Problem formulation </li></ul></ul><ul><ul><li>Data collection </li></ul></ul><ul><ul><li>Data analysis </li></ul></ul><ul><ul><li>Presentation </li></ul></ul><ul><ul><li>Marketing </li></ul></ul>
    26. 26. However! <ul><li>PA auditors are not über auditors </li></ul><ul><li>PA are not better audits </li></ul><ul><li>PA requires different auditors </li></ul>
    27. 27. Performance Audit questions and challenges
    28. 28. What questions are we trying to answer 1/4 <ul><li>Descriptive questions </li></ul><ul><ul><li>Who, where, when, what ? </li></ul></ul><ul><ul><li>Bring relevant information to light </li></ul></ul><ul><ul><li>Without interpretation of facts </li></ul></ul><ul><ul><li>Straightforward methods </li></ul></ul><ul><li>Example: </li></ul><ul><ul><li>:/ </li></ul></ul>
    29. 29. <ul><li>Explanatory questions </li></ul><ul><ul><li>Why, how ? </li></ul></ul><ul><ul><li>Seek to explain </li></ul></ul><ul><ul><li>Requires research and investigative techniques </li></ul></ul><ul><li>Examples: </li></ul><ul><ul><li>How is Slovenia tackling cancer? </li></ul></ul><ul><ul><li>How the Republic of Slovenia provides Free Legal Aid? </li></ul></ul>What questions are we trying to answer 2 /4
    30. 30. <ul><li>Criteria-based questions </li></ul><ul><ul><li>Were , is ? (also called normative questions) </li></ul></ul><ul><ul><li>Comparison with what “should be” </li></ul></ul><ul><ul><li>Criteria must be robust and defensible </li></ul></ul><ul><li>Examples: </li></ul><ul><ul><li>Is the accessibility of preschool care in Slovenia satisfactory? </li></ul></ul><ul><ul><li>Is VAT collection in Slovenia effective? </li></ul></ul>What questions are we trying to answer 3 /4
    31. 31. <ul><li>Evaluative questions </li></ul><ul><ul><li>Is, what ? </li></ul></ul><ul><ul><li>Consider costs, outcomes and wider impacts </li></ul></ul><ul><ul><li>Measure against “benchmark” or alternative model </li></ul></ul><ul><ul><li>Different evaluative perspectives, variant analysis </li></ul></ul><ul><li>Example: </li></ul><ul><ul><li>What are the consequences of non-establishment of the Unified Public Procurement Portal? </li></ul></ul>What questions are we trying to answer 4 /4
    32. 32. Typical questions are <ul><li>Does audited entity acquires, maintains and use resources that are entrusted upon it economically, efficiently and/or effectively? </li></ul><ul><li>Which are the causes for disclosed deficiencies? </li></ul><ul><li>Which are good/bad practices? </li></ul><ul><li>Does audited entity complies with the regulations in a sense of good governance/economy/efficiency/ effectiveness? </li></ul>
    33. 33. Or wider... <ul><li>Suitable use of resources </li></ul><ul><li>Use of right resources a t the lowest cost </li></ul><ul><li>Minimisation of resources usage for achievement of intended goals </li></ul><ul><li>Avoidance of inappropriate or wrong initiatives </li></ul>Are things being done in the right way? Are the right things being done? Implementation Guidelines for Performance Auditing Chapter 1.4
    34. 34. Performance Audit lifecycle at the Court of Audit
    35. 35. Strategic Planning Annual Audit Programme Agreed Detailed Plan for specific PA Completion of Analysis of Evidence Production of the Report Making an Impact Deciding what goes in the Annual Audit Programme Proposal. Scoping the Audit. Defining Audit Criteria. Choosing Methodologies, project management and producing a communications plan. Conducting the Audit Fieldwork and gathering the evidence. Dinner Party Technique. Contents of the Report, Clearing the Report, Amending Communications Plan. Implementing the Communications Plan. Auditee’s Response. Assessing the Impact. Feedback Our lifecycle
    36. 36. Our strategy <ul><li>CoA Strategy for period from 2007 until 2013 contains: </li></ul><ul><ul><li>mission statement </li></ul></ul><ul><ul><li>vision </li></ul></ul><ul><ul><li>values </li></ul></ul><ul><ul><li>factors affecting our work (with focus areas) </li></ul></ul><ul><ul><li>7 strategic objectives </li></ul></ul><ul><ul><li>critical success factors </li></ul></ul><ul><ul><li>strategies for achievement of goals </li></ul></ul>
    37. 37. Annual programme 1/3 <ul><li>AP is a basis: </li></ul><ul><ul><li>for allocation of resources </li></ul></ul><ul><ul><li>issuing demands to audited entities </li></ul></ul><ul><li>In Q3 of previous year SSA propose at least 30% more studies than we can execute </li></ul><ul><li>President of CoA selects which ones will be executed </li></ul><ul><li>Criteria for selection: </li></ul><ul><ul><li>importance </li></ul></ul><ul><ul><li>public visibility </li></ul></ul><ul><ul><li>feasibility </li></ul></ul><ul><ul><li>clear case... </li></ul></ul>
    38. 38. Annual programme 2/3 Goals and Strategies Research Ideas for Audits Short-term influences (e.g., public and political interests) Develop Criteria for Assessing Audit Proposals Assess Research ideas against criteria If criteria are met then submit idea for inclusion in the Audit Programme Obviously the criteria for assessing audit proposals will influence the research conducted
    39. 39. Annual programme 3/3 <ul><li>Each individual draft plan consists of: </li></ul><ul><ul><li>basic information about the audit </li></ul></ul><ul><ul><ul><li>details of the audited entity </li></ul></ul></ul><ul><ul><ul><li>subject and scope of the audit </li></ul></ul></ul><ul><ul><ul><li>type of audit </li></ul></ul></ul><ul><ul><li>reasons for the audit </li></ul></ul><ul><ul><li>definition (scope) of the audit </li></ul></ul><ul><ul><ul><li>segments of the audit </li></ul></ul></ul><ul><ul><ul><li>timetable </li></ul></ul></ul><ul><ul><ul><li>the manner in which the work is to be done </li></ul></ul></ul><ul><ul><ul><li>audit days allocated to auditors </li></ul></ul></ul><ul><ul><ul><li>cost of an external assistance </li></ul></ul></ul><ul><ul><ul><li>authorised SSA </li></ul></ul></ul><ul><ul><ul><li>authorised Deputy President </li></ul></ul></ul>
    40. 40. Steps in PA <ul><li>Planning </li></ul><ul><li>Examination </li></ul><ul><li>Drafting </li></ul><ul><li>Clearance </li></ul><ul><li>Publication </li></ul><ul><li>Follow-up reporting </li></ul><ul><li>Parliament procedure </li></ul><ul><li>They are the same as in regularity audit but...  </li></ul>
    41. 41. Case-Studies
    42. 42. Case studies <ul><li>Based on actual (published) reports </li></ul><ul><li>Show different types of PA </li></ul><ul><li>On each case we will show: </li></ul><ul><ul><li>Reasons for audit </li></ul></ul><ul><ul><li>Audit approach </li></ul></ul><ul><ul><li>Results </li></ul></ul><ul><ul><li>Follow up activities </li></ul></ul>
    43. 43. Some audits of PA department <ul><li>PA of Government economy policy measures </li></ul><ul><li>P A of the Project of the Elimination of Slovenian Court Backlogs </li></ul><ul><li>P A of Government Office for Local Self-Gov . and Reg . Policy IT System </li></ul><ul><li>RBB audit of the Slovenian Transport Inspectorate </li></ul><ul><li>Exceptional pensions system in Slovenia </li></ul><ul><li>P A of Young Researchers Programme </li></ul><ul><li>Effectiveness of Slovenian Rural development policy </li></ul><ul><li>P A of the Natural Disasters Aid System in Slovenia </li></ul><ul><li>How is Slovenia tackling cancer? </li></ul><ul><li>P A of governmental co-financing of the media </li></ul><ul><li>Prescribing drugs system in Slovenia </li></ul><ul><li>Efficiency of VAT collection in the Republic of Slovenia </li></ul><ul><li>How the Republic of Slovenia takes care of those in need? </li></ul><ul><li>RBB audit of the Slovenian Labour Inspectorate </li></ul><ul><li>Are public funds for medical equipment well managed? </li></ul><ul><li>P A of draught prevention and damage reduction in agriculture </li></ul><ul><li>Does Slovenia perform Regulatory Impact Assessment? </li></ul><ul><li>Efficiency of IT system of the Tax Admin . of the Republic of Slovenia </li></ul><ul><li>Who and how executes Public interest in the field of Sport? </li></ul><ul><li>P A of the Free Legal Aid system </li></ul><ul><li>Efficiency and Economy of use of public funds in the project of Public Procurement Portal ... </li></ul>
    44. 44. Case Studies <ul><li>Construction of new National and Public Library (CS 1) </li></ul><ul><li>Information System of Public Health Institute (CS 2) </li></ul><ul><li>Preschool care availability and users satisfaction (CS 3) </li></ul>
    45. 45. CS 1 – Construction of new National and Public Library – Reasons for audit 1/2 <ul><li>Law on University Library construction from 1994 assumes building of it until 1998 </li></ul><ul><li>Risks related to: </li></ul><ul><ul><li>Documentation </li></ul></ul><ul><ul><li>Staff and organisational issues </li></ul></ul><ul><ul><li>Unrealistic timeframe </li></ul></ul><ul><ul><li>Lack of financing </li></ul></ul><ul><ul><li>Problematic location </li></ul></ul>
    46. 46. CS 1 – Construction of new National and Public Library – Reasons for audit 2/2 <ul><li>Project of national importance for which we have been waiting for almost 20 years (covers 4 libraries) </li></ul><ul><li>Construction has not begun (but it should be finished by 1998) </li></ul><ul><li>Considerable public funds have been spent </li></ul><ul><li>Limitations: </li></ul><ul><ul><li>Project timeframe (1989 – 2007) </li></ul></ul><ul><ul><li>Large scale project with extensive changes </li></ul></ul><ul><ul><li>Lack of documentation, unreliable sources </li></ul></ul><ul><ul><li>Changes in project lead and project staff </li></ul></ul>
    47. 49. CS 1 – Construction of new National and Public Library – Audit approach <ul><li>Main question: </li></ul><ul><li>Does Ministry of Higher Education, Science and Technology provides necessary condition for effective and efficient realization of the project? </li></ul><ul><li>Method: </li></ul><ul><ul><li>main question is further divided into several sub questions that are easier to answer and explain </li></ul></ul><ul><ul><li>analysis of planning and execution of project activities, documentation, resource allocation and funding </li></ul></ul>
    48. 50. CS 1 – Construction of new National and Public Library – Results 1/9 <ul><li>Does Ministry has at its disposal full documentation on a project? </li></ul><ul><li>Findings: </li></ul><ul><ul><li>documentation is not completely collected or kept in a way that guarantees transparent view on a time frame and content of the project </li></ul></ul><ul><ul><li>all past activities cannot be fully chronologically and materially described </li></ul></ul><ul><li>Risks: project documentation will never be complete </li></ul>
    49. 51. CS 1 – Construction of new National and Public Library – Results 2/9 <ul><li>Is the state of human resources and organisational structure enabling quality execution of a project? </li></ul><ul><li>Findings: </li></ul><ul><ul><li>existing HR and organisational structure is insufficient and unsuitable (only 1 person) </li></ul></ul><ul><ul><li>unclear roles and responsibilities of consulting and supervision bodies </li></ul></ul><ul><li>Risks: ineffective leadership may threaten (last) deadline by end of 2011 </li></ul>
    50. 52. CS 1 – Construction of new National and Public Library – Results 3/9 <ul><li>How is Ministry planning the project? </li></ul><ul><li>Findings: </li></ul><ul><ul><li>during the audit there were no plan on finishing the Phase 1 of a project </li></ul></ul><ul><ul><li>there were no plan for Phase 2 and 3 </li></ul></ul><ul><ul><li>there are no sufficient conditions for effective and efficient realization of the project </li></ul></ul><ul><ul><li>risks are not addressed or evaluated </li></ul></ul><ul><li>Risks: project is without control </li></ul>
    51. 53. CS 1 – Construction of new National and Public Library – Results 4/9 <ul><li>How is Ministry planning the funding of the project? </li></ul><ul><li>Findings: </li></ul><ul><ul><li>there were no detailed financial plan </li></ul></ul><ul><ul><li>financing is obscure and unrealistic </li></ul></ul><ul><ul><li>at least 29,3 million € were already spent </li></ul></ul><ul><ul><li>future spending is unrealistic </li></ul></ul><ul><li>Risk: funding is insufficient </li></ul>
    52. 54. CS 1 – Construction of new National and Public Library – Results 5/9 <ul><li>Financing of the project (Ministry only) </li></ul>
    53. 55. CS 1 – Construction of new National and Public Library – Results 6/9 <ul><li>How was Ministry planning the funding for the project? </li></ul>
    54. 56. CS 1 – Construction of new National and Public Library – Results 7/9 <ul><li>How is Ministry providing for the location of the project? </li></ul><ul><li>Findings: </li></ul><ul><ul><li>Ministry does not poses all necessary ground for the building </li></ul></ul><ul><ul><li>serious risks related to getting permissions to build </li></ul></ul><ul><ul><li>real-estate is dangerous </li></ul></ul><ul><li>Risks: additional delays, inability to build </li></ul>
    55. 57. CS 1 – Construction of new National and Public Library – Results 8/9
    56. 58. CS 1 – Construction of new National and Public Library – Results 9/9 <ul><li>28.000m 2 initially </li></ul><ul><li>35.000m 2 in 2002 </li></ul><ul><li>21.000m 2 currently </li></ul>
    57. 59. CS 1 – Construction of new National and Public Library – Follow up activities <ul><li>CoA has demanded: </li></ul><ul><ul><li>preparation of risk assessment </li></ul></ul><ul><ul><li>detailed plan for Phase 1 </li></ul></ul><ul><ul><li>time plan for Phase 2 an 3 </li></ul></ul><ul><li>Ministry has: </li></ul><ul><ul><li>responded within 90 days (post-audit report published in July 2008) </li></ul></ul><ul><ul><li>has made initial promises which were evaluated as “good enough” but with BIG disclaimer from CoA </li></ul></ul><ul><ul><li>CoA has made afterwards two additional pre-audit enquiries regarding the state of the project </li></ul></ul><ul><li>In 2009 Ministry has resigned from the intended project location </li></ul>
    58. 60. Questions about CS 1
    59. 61. CS 2 – IT of Public Health Institute – Reasons for audit 1/2 <ul><li>Law on databases in the field of health puts PHI into the centre of public health records </li></ul><ul><li>Risks related to: </li></ul><ul><ul><li>Quality of records kept (at least 3 previous audits found inefficiency) </li></ul></ul><ul><ul><li>Regularity (private information are not kept in accordance with regulations) </li></ul></ul>
    60. 62. CS 2 – IT of Public Health Institute – Reasons for audit 2/2 <ul><li>Known issues: </li></ul><ul><ul><li>57 databases on public health (not all required by Law are maintained) </li></ul></ul><ul><ul><li>34 of those should contain private information </li></ul></ul><ul><ul><li>there is at least one database which is kept in two different versions </li></ul></ul><ul><ul><li>management and maintenance is not efficient enough to provide for goal achievement of PHI and institutions dependant on it’s information </li></ul></ul>
    61. 63. CS 2 – IT of Public Health Institute – Audit approach 1/2 <ul><li>For efficiency: </li></ul><ul><ul><li>CobiT 4.1 framework </li></ul></ul><ul><ul><li>Assessment of all 34 processes in 4 domains </li></ul></ul><ul><ul><li>Good governance principles </li></ul></ul><ul><li>For regularity part: </li></ul><ul><ul><li>Compliance with Law on databases in the field of health </li></ul></ul><ul><ul><li>Compliance with Law on private information </li></ul></ul>
    62. 64. CS 2 – IT of Public Health Institute – Audit approach 2/2 <ul><li>CobiT 4.1 Maturity model </li></ul>
    63. 65. CS 2 – IT of Public Health Institute – Results 1/6 <ul><li>Plan and organise (PO) </li></ul>
    64. 66. CS 2 – IT of Public Health Institute – Results 2/6 <ul><li>Acquire and implement (AI) </li></ul>
    65. 67. CS 2 – IT of Public Health Institute – Results 3/6 <ul><li>Deliver and support (DS) </li></ul>
    66. 68. CS 2 – IT of Public Health Institute – Results 4/6 <ul><li>Monitor and evaluate (ME) </li></ul>
    67. 69. CS 2 – IT of Public Health Institute – Results 5/6 <ul><li>PHI is not efficient in governing it’s IT system (average grade 1,7, set target for efficiency was 3,0) </li></ul><ul><li>For each deficient area risks were assessed and recommendations given </li></ul>
    68. 70. CS 2 – IT of Public Health Institute – Results 6/6 <ul><li>On compliance with Law on databases in the field of health: </li></ul><ul><ul><li>Only 40 out of 57 databases are kept </li></ul></ul><ul><ul><li>For 8 databases data is not collected in accordance with the law </li></ul></ul><ul><li>On compliance with Law on private information </li></ul><ul><ul><li>Only one database is kept in accordance with Private information act (it is kept in paper form) </li></ul></ul><ul><ul><li>Databases are not properly reported to PI Officer </li></ul></ul>
    69. 71. CS 2 – IT of Public Health Institute – Follow up activities <ul><li>PHI had to: </li></ul><ul><ul><li>Provide plan of activities (with responsible persons and dates) for: </li></ul></ul><ul><ul><ul><li>compliance with the Law or documentation of measures taken for amendments of the Law </li></ul></ul></ul><ul><ul><ul><li>Fulfilment of requirements related to how private information are kept </li></ul></ul></ul><ul><ul><li>Guarantee proper reporting of necessary data about PI to PI Officer </li></ul></ul><ul><li>PHI did all of the above and started to implement serious changes in it’s operation </li></ul>
    70. 72. Questions about CS 2
    71. 73. CS 3 – Preschool care availability and users satisfaction – Reasons for audit 1/2 <ul><li>Half empty or half full? </li></ul><ul><li>Children and their parents ARE satisfied with preschool child care (PCC) </li></ul><ul><li>NOT everything is wrong </li></ul><ul><li>But... </li></ul>
    72. 74. CS 3 – Preschool care availability and users satisfaction – Reasons for audit 2/2 <ul><li>In school year 2005/06 57.137 of all preschool children were included in public or private PCC institutions </li></ul><ul><li>32.699 (36,4%) was not included in any of publicly recognised forms of PCC </li></ul><ul><li>Ministry has no information where this children are </li></ul><ul><li>Risk: second group of children is in position that is not comparable (regarding the QofS) to the first one </li></ul>
    73. 75. CS 3 – Preschool care availability and users satisfaction – Audit approach 1/2 <ul><li>Main question s: </li></ul><ul><li>Is the accessibility of PCC in Slovenia satisfactory? </li></ul><ul><li>Are children and their parents satisfied with the PCC services? </li></ul><ul><li>Methods: </li></ul><ul><ul><li>Evaluation of the existing system </li></ul></ul><ul><ul><li>Surveying </li></ul></ul>
    74. 76. <ul><li>Satisfaction depends at least on: </li></ul><ul><ul><li>Price for </li></ul></ul><ul><ul><ul><li>Parents </li></ul></ul></ul><ul><ul><ul><li>Local municipalities </li></ul></ul></ul><ul><ul><li>Accessibility regarding: </li></ul></ul><ul><ul><ul><li>Time (work hours) </li></ul></ul></ul><ul><ul><ul><li>Geographic location </li></ul></ul></ul><ul><ul><ul><li>Waiting time </li></ul></ul></ul><ul><ul><li>Quality of service in regard to: </li></ul></ul><ul><ul><ul><li>Expertise of work force </li></ul></ul></ul><ul><ul><ul><li>Educational programme </li></ul></ul></ul><ul><ul><ul><li>Extra services and activities </li></ul></ul></ul>CS 3 – Preschool care availability and users satisfaction – Audit approach 2/2
    75. 77. CS 3 – Preschool care availability and users satisfaction – Results 1/9 <ul><li>Law on PCC recognises 12 different forms of PCC. There were: </li></ul><ul><ul><li>108 public kindergartens </li></ul></ul><ul><ul><li>184 kindergartens at primary schools </li></ul></ul><ul><ul><li>18 private kindergartens (with or without concession) </li></ul></ul><ul><ul><li>5 forms of PCC that did not exist at all </li></ul></ul><ul><ul><li>4 forms of PCC that the ministry did not know if they exist or not </li></ul></ul><ul><li>Risk: Possibilities under the law are not fully used (reasons?!) </li></ul>
    76. 78. CS 3 – Preschool care availability and users satisfaction – Results 2/9 <ul><li>Survey shows that among the parents whose children are not included in PCC (roughly): </li></ul><ul><ul><li>1/3 takes care of them on their own </li></ul></ul><ul><ul><li>1/3 with relatives and friend s </li></ul></ul><ul><ul><li>1/3 in grey economy (with nannies that are not registered and accredited) </li></ul></ul>
    77. 79. CS 3 – Preschool care availability and users satisfaction – Results 3/9 <ul><li>Cost of PCC in 2003-2005 ≈ 748 m€ </li></ul><ul><ul><li>≈ 67% programme - local municipality </li></ul></ul><ul><ul><li>≈ 28% programme - parents </li></ul></ul><ul><ul><li>≈ 5% investments (partially funded by the state other local municipalities) </li></ul></ul><ul><li>Risk: PCC present considerable burden for local municipalities </li></ul>
    78. 80. CS 3 – Preschool care availability and users satisfaction – Results 4/9 <ul><li>There are considerable variations in: </li></ul><ul><ul><li>Prices of the “same” programmes (more than 100%) </li></ul></ul><ul><ul><li>Around 80% of costs are related to cost of labour </li></ul></ul><ul><li>Risk: Parents are not treated equally (equity) </li></ul>
    79. 81. CS 3 – Preschool care availability and users satisfaction – Results 5/9 <ul><li>Odd distribution: </li></ul><ul><ul><li>65,7% of parents are in class 0-3 </li></ul></ul><ul><ul><li>they contribute only 37,3% </li></ul></ul><ul><li>Risk: even with a salary only a little above average one should be in the highest classes  W ho has children in institutional PCC? </li></ul>
    80. 82. CS 3 – Preschool care availability and users satisfaction – Results 6/9 <ul><li>Republic of Slovenia has one of the highest progression rates in parent contributions </li></ul><ul><ul><li>Parents with income average or a bit higher income pays one of the highest bills among OECD countries </li></ul></ul><ul><li>Risk: Who has children in institutional PCC? </li></ul>
    81. 83. CS 3 – Preschool care availability and users satisfaction – Results 7/9 <ul><li>Does current legislation enables justly arrangements into paying classes in regard to income and property? </li></ul><ul><li>Are high differences in the cost of equal programmes too high or acceptable? </li></ul>
    82. 84. CS 3 – Preschool care availability and users satisfaction – Results 8/9 <ul><li>Parents and children that are included in PCC are in large majority very satisfied with QofS </li></ul>
    83. 85. CS 3 – Preschool care availability and users satisfaction – Results 9/9 <ul><li>Additional findings </li></ul><ul><ul><li>System is not well integrated into local self-government </li></ul></ul><ul><ul><li>Planning of a program is not adequate </li></ul></ul><ul><ul><li>Strategic goals are unclear </li></ul></ul><ul><ul><li>Information system support is deficient </li></ul></ul>
    84. 86. CS 3 – Preschool care availability and users satisfaction – Follow up activities 1/3 <ul><li>CoA has recommended to the Ministry that it should: </li></ul><ul><ul><li>Provide for variety of forms of PCC </li></ul></ul><ul><ul><li>Investigate t h e reasons why children are not included in formal PCC </li></ul></ul><ul><ul><li>Lower the differences between same programs </li></ul></ul><ul><ul><li>Provide for more equal burden of parents co - financing </li></ul></ul><ul><ul><li>Better monitor of the area and faster response to changes... </li></ul></ul>
    85. 87. CS 3 – Preschool care availability and users satisfaction – Follow up activities 2/3 <ul><li>CoA has demanded a response report which was evaluated as satisfactory (with promise) </li></ul><ul><li>Ministry has changed the law on PCC </li></ul><ul><ul><li>Most of CoA recommendations were followed </li></ul></ul><ul><ul><li>BUT one new “idea” was introduced (free PCC for second child) </li></ul></ul>
    86. 88. CS 3 – Preschool care availability and users satisfaction – Follow up activities 3/3 <ul><li>Since 2008/09 kindergartens are full </li></ul><ul><ul><li>Big issue on local level </li></ul></ul><ul><ul><li>Parents are unhappy! </li></ul></ul><ul><ul><li>Costs are rising </li></ul></ul><ul><li>In Q4 2009 CoA introduced two new audits in this field: </li></ul><ul><ul><li>Follow-up audit of effectiveness and efficiency of activities and measures for PCC availability </li></ul></ul><ul><ul><li>PA of PCC availability records IT System </li></ul></ul>
    87. 89. Questions about CS 3
    88. 90. Conclusion and additional sources
    89. 91. Performance Audit Challenges <ul><li>Interdisciplinary and expert knowledge required </li></ul><ul><li>Costs and time for maintaining desired quality standards </li></ul><ul><li>Target audiences expectations </li></ul><ul><li>Our own ambitions  </li></ul><ul><li>And sometimes things just happen... </li></ul>
    90. 92. Concluding remarks <ul><li>Where to go with PA: </li></ul><ul><ul><li>More detailed and consistent measurement of achievements </li></ul></ul><ul><ul><li>Better cooperation with audited bodies </li></ul></ul><ul><ul><li>Education and inclusion of internal auditors within audited bodies </li></ul></ul><ul><ul><li>Management inclusion (at audited bodies) </li></ul></ul><ul><ul><li>Potential growth (human and other resources) </li></ul></ul>
    91. 93. Resources <ul><li>Code of Ethics and Auditing Standards , INTOSAI, 2001 </li></ul><ul><li>Implementation Guidelines for Performance Auditing , Standards and guidelines for performance auditing based on INTOSAI's Auditing Standards and practical experience, INTOSAI, 2004 </li></ul><ul><li>Court of Audit Act , 2001 </li></ul><ul><li>Slovene Court of Audit Strategy 2007 – 2013 , CoA, 2007 </li></ul><ul><li>Priročnik za revizije smotrnosti poslovanja (Performance Audit Manual) , CoA, 2006 (in Slovene language only) </li></ul><ul><li>Performance Audit Manual , European Court of Auditors </li></ul><ul><li>National Audit Office website </li></ul><ul><li>Government Accountability Office website </li></ul>
    92. 94. Thank you! <ul><li>More info: </li></ul><ul><li>http://www.rs-rs.si </li></ul><ul><li>or </li></ul><ul><li>[email_address] </li></ul>

    ×