For a relatively young profession - and 70 years is youthful as professions go - the changes have been numerous and substantial.
Performance audits provide objective analysis so that management and those charged with governance and oversight can use the information to improve program performance and operations, reduce costs, facilitate decision making by parties with responsibility to oversee or initiate corrective action, and contribute to public accountability.
As stated in the Auditing Standards, performance auditing is not overly subject to specific requirements and expectations. While financial auditing tends to apply relatively fixed standards, performance auditing is more flexible in its choice of subjects, audit objects, methods, and opinions. Performance auditing is not a regular audit with formalized opinions, and it does not have its roots in private auditing. It is an independent examination made on a non-recurring basis. It is by nature wide-ranging and open to judgments and interpretations. It must have at its disposal a wide selection of investigative and evaluative methods and operate from a quite different knowledge base to that of traditional auditing. It is not a checklist-based form of auditing. The special feature of performance auditing is due to the variety and complexity of questions relating to its work. Within its legal mandate, performance auditing must be free to examine all government activities from different perspectives.The character of performance auditing must not, of course, be taken as an argument for undermining collaboration between the two types of auditing.
The point of this brainteaser is THINK OUTSIDE THE BOX.
Management is our customer. We can give what they want the post without understand their business. Most internal auditors whish to be perceived as experts in control. They tell the auditee that they may have no experience in the technical aspects of an activity or unit, but they have both experience and expertise in control. This may well be true. But it doesn’t go far enough. Many auditees regard control as a harsh term, a constricting concept, the function of the nay-sayer. We must be more than experts in control if we are to meet high-minded goals.
Performance auditors can be faced with considerable variety and ambiguity in their work. They require skills in analyzing activities andmanagement practices. They can be faced with the need to become familiar with a wide range of organizational contexts and subjectmatters.
Seeing through management eyes - we will understand the issues management is facing and how to be partners with them to improve the performance. Internal auditors call themselves “control expert”. After all, control is but one of the four functions of management. And if we are to counsel managers we must be experts in all four functions – planning, organizing, and directing, as well as controlling. And we should be educated and prepared with management and business processes. This may be new frontier for some performance auditors. This is where the need is greatest. The supply of skilled, corporate managers is severely limited. Managerial performance is often adversely affected by poor managerial techniques or by the violation of accepted management principles. And that is where the management-oriented performance auditor can make a significant contribution. Being conversant with the principle of good management – not only with control – is the first step toward assuming the role of management counselor rather than of management critic.
Example: Milestones completed, Customer Satisfaction Rating, Number of projects requiring reworkProduction cost overrun Cost of maintenance projectsNumber of Maintenance projectsRemember SMART in your organization’s objective setting
The original 1947 Statement of Responsibilities of the Internal Auditor was not much better. It gave a grudging nod to the internal auditor’s involvement with other than financial activities when it said that internal auditing “deals primarily with accounting matters but may properly deal with matters of an operating nature.”The revised 1957 Statement, somewhat more expansive, defined internal auditing as providing “for the review of accounting, financial, and other operations.”But the 1971 version of the Statement cut the umbilical cord to the books of account from which internal auditors first drew their life support by describing internal auditing quite simply as “the review of operations as a service to management.” Even this definition is too narrow. The Standards for the Professional Practice of Internal Auditing of 1978 expand “service to management” to read “service to the organization.” Thus, it encompasses both management and the board of directors.
Issued in 1968 to promote the ethical culture among the internal audit profession worldwide.What will happen if an internal auditor violate the Code?- Revoke membership in IIARemove certificationBar from taking CIA exam
Examples of External Factors affecting an organization include:Technological developments which can affect the nature and timing of service start-ups, or lead to changes in hiring.Changing legislation or expectations that can affect regulations or operating procedures or customer service.OthersExamples of Internal Factors include:A disruption in information system processing which can adversely affect the organization’s ability to function.The quality of personnel hired and methods of training and motivation that can influence the level of control consciousness within the organization.Others
• Open-ended - good for both hard and soft controls. There are no restrictions as to the type of controls or actions being reviewed.• Disciplined - helps to ensure that all major risks identified are addressed during the review as well as providing the opportunity to identify improvement actions. These forms help ensure that the documentation of work is completed at the time the work is performed and that the client and auditor have reviewed the results and taken the time to identify corrective actions which should be taken.• Risk-based - improves audit effectiveness and efficiency. By focusing on the risks management has expressed concern about, the entire audit process is enhanced since it is doing a review, which will have an impact on the operations and add value to the management team.• Inclusive - documents complete survey thought process. Again, the entire process that management and the auditor followed is documented on a few forms and can be used as a benchmark or as a tool to identify the opportunities
Performance auditing may contribute to strengthening these values by producing public and reliable information on the economy, efficiency, and effectiveness of government programs.
The approach is not to ignore the importance of risk-based approach. The question is what the organization and the management needs more. Canadian Government, GAO and other government auditors have been the leaders of this type of performance audit. They have recognized that the lake of the 3E’s can be a huge risk for an government entity to achieve its objectives.However, in UK, this type of performance audit approach is not recognized. They expect that internal auditors will audit the controls over efficiency, economy and effectiveness, but not make evaluations of performance. A possible exception is where auditors are asked to validate performance reports but we would not treat this normal auditing but a consultancy assignment. They generally address the three Es, Value for money and performance from a risk management perspective so that we would be looking at the effectiveness of the measures put in place to mitigate risks.
Performance audit adding value
Performance AuditAdding ValueICGFM Conference May 19, 2011<br />Lily Bi, CIA, CGEIT, CISA<br />Director, Standards and Guidance<br />Institute of Internal Auditors<br />
Program Objectives<br /><ul><li>Understand the Landscape –
Increase your ability to work with management in a positive and constructive partnership
The International Standards for Professional Practice of Internal Auditing</li></ul>Analyze risks and develop a risk-based performance audit<br />Learn a value-for-money approach for performance audit<br />Final Thoughts – Trend of Internal Audit Profession <br />
Program Topics<br />Unit 1 - Understand the Landscape<br />Unit 2 - Management Functions and Performance Measures<br />Unit 3 - International Standards For Performance Audit<br />Unit 4 - Risk-Based Approach (Case Study)<br />Unit 5 - Value-for-Money Approach (Case Study)<br />Unit 6 – Final Thoughts<br />
Working Agreement<br />P = Participation<br />O = Openness<br />S = Sense of fun<br />E = Enthusiasm<br />
Unit 1<br />Understand the Landscape<br /><ul><li>The road map of internal audit profession
Benefit of performance audit</li></li></ul><li>Road Map of Internal Audit Profession<br />
Road Map of Internal Audit<br />Modern Internal Audit<br />1941 - Internal Audit, <br />a separate and distinctive discipline.<br />
About the IIA<br />Established in 1941, global headquarters in Altamonte Springs, Florida, USA<br />Nonprofit professional association<br />170,000 members worldwide<br />103 national institutes worldwide<br />Key focus:<br />Standards-setting body for internal auditors<br />Professional certifications<br />Global research center<br />Principal educator <br />Global voice for the profession<br />
Images of Internal Auditors<br />Which metaphor do you like?<br />Magnifying glass<br />Telescope<br />Compass<br />Hunting dogs<br />Watch dogs<br />Policemen<br />Consultants<br />Eyes and ears of the Audit Committee<br />
Definition of Internal Auditing <br />Internal auditing is an independent, objectiveassurance and consulting activity designed to add value and improve an organization’s operations. It helps an organization accomplish objectives by bringing a systematic, disciplined approach to evaluate and improve the effectiveness of risk management, control, and governance processes.<br />Source: International Professional Practices Framework (IPPF)<br /> The Institute of Internal Auditors<br />
Definitions of PA<br />INTOSAI: Performance auditing is an independent examination of the efficiency and effectiveness of government undertakings, programs, or organizations, with due regard to economy, and the aim of leading to improvements.<br />US Government Auditing Standards:Performance audits are defined as engagements that provide assurance or conclusions based on an evaluation of sufficient, appropriate evidence against stated criteria, such as specific requirements, measures, or defined business practices. Performance audits provide objective analysis so that management and those charged with governance and oversight can use the information to improve program performance and operations, reduce costs, facilitate decision making by parties with responsibility to oversee or initiate corrective action, and contribute to public accountability. <br />
Working Definition of PA<br /> Performance Audit is an independent and objective examination of a program, function, operation or the management systems of a governmental entity to:<br />assure the entity’s objectives are carried out in an economic, efficient and effective way, and<br />identify opportunity for improvement<br />
Financial vs. Compliance vs. Performance Auditing<br />
What Makes this Performance Audit?<br />An Example:<br />“…to determine whether laws, contracts, policies and procedures have been properly observed and whether all business transactions were conducted in accordance with established policies and with success. In this connection, the auditors are to make suggestions for the improvement of existing facilities and procedures, criticisms of contracts with suggestions for improvement, etc.”<br />
Benefit of PA – Adding Value<br />Relevant<br />Focus on the key initiatives<br />Flexible <br />Define the scope of the audit based on risk<br />Improving organizational performance<br />Strengthen the governance<br />Fraud prevention and detection<br />Gaining public trust <br />
Management Issues and Concerns<br />Cost Containment<br />Human Resources <br />Values and Vision Initiatives <br />Empowered Environments vs. Traditional Structures <br /><ul><li>Technological Changes and Innovations
Performance Auditor’s Roles<br />Evaluate the management processes and identify the heart of the problem<br />Alert to actual and potential changes<br />Identify the opportunity for improvement<br />All units, programs, systems and activities are subject to internal auditor’s evaluations<br />
See though the Eyes of Management<br />Almost every deviation or deficiency results from the violation of some principle of management or good administration.<br />See the organization and its activities through the eyes of management<br />
Three Simple Questions to Ask Management<br />What can go wrong?<br />How do you it won’t go wrong?<br />So what?<br />
Types of Management Performance Measures<br />INPUTS - Measures of service efforts, e.g., number of hours, amount of materials.<br />OUTPUTS - Measures of service level, e.g., number of residences served, amount of service provided.<br />OUTCOMES - Measures of service accomplishments, e.g., measures related to program goals, including effectiveness of quality.<br />EFFICIENCY - Measures that relate service efforts to service accomplishments, e.g., output/unit of input, productivity indexes.<br />
Principles<br />Measure only what are important to the organization<br />Use of output-oriented measures<br />Identify the total costs of service delivery<br />Focus on continuous process improvement<br />Performance measures should interconnect throughout the organization<br />
One Example – Five Performance Categories:<br />Effectiveness – the degree to which process output conforms to requirements<br />Efficiency – the degree to which the process produces the output at a minimum cost of resources<br />Quality – the degree to which the product or service meets customer expectations<br />Timeliness – the degree to which a unit of work was done correctly and on time<br />Safety – the measure of health and the working environment of the organization<br />
Unit 3<br />International Standards <br />For Performance Audit<br />International Professional Practices Framework - IPPF from the IIA<br />
Why the Standards Matter<br />TheStandards<br />Lead<br />Represent<br />Advancement of theProfession<br />
Code of Ethics<br />Integrity<br />The integrity of internal auditors establishes trust and thus provides the basis for reliance on their judgment.<br />Objectivity<br />Internal auditors exhibit the highest level of professional objectivity in gathering, evaluating, and communicating information about the activity or process being examined. Internal auditors make a balanced assessment of all the relevant circumstances and are not unduly influenced by their own interests or by others in forming judgments.<br />Confidentiality<br />Internal auditors respect the value and ownership of information they receive and do not disclose information without appropriate authority unless there is a legal or professional obligation to do so.<br />Competency<br />Internal auditors apply the knowledge, skills, and experience needed in the performance of internal auditing services.<br />
International Standards for Professional Practice of Internal Auditing<br />
Importance of the Standards<br /><ul><li>They define the profession.
They set the bar that every auditor should comply with.
They give you a reference guide for how to conduct yourself.
They lay the ground work, but are not the ultimate goal.
They give our customers peace of mind and confidence they’re getting a quality product.</li></li></ul><li>The International Standards<br />Mandatory requirements consisting of:<br />Statements of basic requirements for professional practice of internal auditing <br />Interpretations which clarify terms or concepts within the Statements.<br />Glossary<br />26 changes effective Jan 2011<br />
Overview of the IIA Standards<br />Attribute Standards:<br /><ul><li>Purpose, Authority and Responsibility……………………1000
Independence and Objectivity………………………………..1100
Proficiency and Due Professional Care……………….….1200
Quality Assurance and Improvement Program……..…1300</li></ul>Performance Standards:<br /><ul><li>Managing the Internal Auditing Activity……………………2000
Referenced on the mandated legislation or regulation in countries or territories, such as
Belgium, Bosnia & Herzegovina, Canada, Chinese Taiwan, Estonia, Poland, Romania, South Africa, Sweden, Thailand, Tunisia, Unites States, United Kingdom, Zimbabwe, and …</li></li></ul><li>IPPF Strongly Recommended Guidance <br /><ul><li>Practice Advisories (56)</li></ul>Address approach, methodology and considerations, but NOT detailed processes and procedures. Concise and timely guidance to assist internal auditors in applying Code of Ethics and Standards and promoting good practices. <br /><ul><li>Position Papers (2)</li></ul>IIA statement to assist a wide range of interested parties, including those not in internal auditing profession, in understanding significant governance, risk or control issues and delineating related roles and responsibilities of internal auditing.<br /><ul><li>Practice Guides (26)</li></ul>Detailed guidance for conducting internal audit activities. Includes detailed processes and procedures, such as tools and techniques, programs, and step-by-step approaches, including examples of deliverables.<br />www.theiia.org/guidance<br />
Unit 4<br />Risk-Based Performance Audit<br /><ul><li>Performance audit process
The importance of clearly defined business objectives and associated performance measures (goals) to a performance audit
Risk assessment using a Risk/Control Matrix methodology
Case Study </li></li></ul><li>Performance Audit Process<br />Planning <br />Examining and Evaluating Information<br />Communicating Results<br />Following Up<br />
IIA Standards Related to Performance Audit Process<br />
Plan Performance Audit<br />The most important part of an audit is the planning phase. <br />Standard 2010 – Planning: The chief audit executive must establish risk-based plans to determine the priorities of the internal audit activity, consistent with the organization’s goals.<br />
Plan Performance Audit<br />Standard 2201 – Planning Considerations: In planning the engagement, internal auditors must consider:<br />The objectives of the activity being reviewed and the means by which the activity controls its performance;<br />The significant risks to the activity, its objectives, resources, and operations and the means by which the potential impact of risk is kept to an acceptable level;<br />The adequacy and effectiveness of the activity’s risk management and control processes compared to a relevant control framework or model; and<br />The opportunities for making significant improvements to the activity’s risk management and control processes.<br />
Risk-based Performance Audit<br />Start with an organization’s objectives and associated performance measures.<br />Focus on an evaluation of performance risks and controls related to those objectives.<br />Help the organization achieve the desirable goals and protect it from bad or undesirable things happening.<br />Help reduce the chance of missed opportunities.<br />Provide suggestions for improvement in controls designed to mitigate the risks associated with meeting performance objectives.<br />
What is Risk<br />Risks are things that could prevent an organization from meeting its objectives.<br />IIA definition - Risk is the possibility of an event occurring that will have an impact on the achievement of objectives. Risk is measured in terms of impact and likelihood.<br />
Business Risk Examples<br />Erroneous records and/or information<br />Business interruption (Government shutdown)<br />Public criticism or legal action<br />High costs<br />Loss or destruction of assets<br />Customer dissatisfaction due to ineffective program/service design<br />Fraud or conflict of interest<br />Inappropriate mgmt. policy and/or decision making process<br />
Focusing on the “Real Risks”<br />Operational 20%<br />Strategic & Business 60%<br />Financial 15%<br />Compliance 5%<br />
Mitigation</li></li></ul><li>Risk Response Strategy<br /><ul><li>Management identifies available risk response options
Considers their effect on event likelihood and impact, in relation to risk appetite and cost versus benefit
Effective enterprise risk management does not dictate which response management should chose, but that the chosen response brings the expected likelihood and impact within the desired risk tolerances</li></li></ul><li>Risk Assessment - Two perspectives<br /> Inherent<br /> Risk<br /><ul><li>Inherent (Gross) - BEFORE RISK RESPONSE
Exercise: Rain and Umbrella<br />When it rains, where are Inherent and Residual Risk (IR and RR)?<br />
When it rains, where are IR and RR?<br />IR<br />IR<br />IR<br />IR<br />IR<br />IR<br />IR<br />RR<br />CR<br />RR<br />RR<br />RR<br />RR<br />IR = All the raindrops<br />RR = The raindrops outside the umbrella<br />CR = Control Risk, possibility the umbrella leaks<br />Risk Appetite = How big the umbrella is<br />
What is Control<br />Controls are things that help meet an organization's objectives.<br />IIA Definition Control - any action taken by management, the board, and other parties to manage risk and increase the likelihood that established objectives and goals will be achieved. Management plans, organizes, and directs the performance of sufficient actions to provide reasonable assurance that objectives and goals will be achieved.<br />
Control to Mitigate These Risks<br />Erroneous records and/or information<br />Business interruption<br />Public criticism or legal action<br />High costs<br />Loss or destruction of assets<br />Customer dissatisfaction due to ineffective program/service design<br />Fraud or conflict of interest<br />Inappropriate mgmt. policy and/or decision making process<br />
Risk Management and Control<br />Two sides of the same coin: <br />Risk is managed by having in place the right controls to safeguard against its occurrence;<br />Internal control exists only in relation to what they do to mitigate risk. <br />Risk management and internal control are integrated parts of an entity’s overall governance and management system.<br />
Control - Who Is Responsible<br /><ul><li>Management is responsible to design, implement and monitor controls
Internal auditors is responsible to assess the adequacy and effectiveness of controls</li></li></ul><li>Risk Control Matrix<br />Use RCM to <br /><ul><li>Plan an audit
Document an audit </li></li></ul><li>Benefits of Risk Control Matrix<br />Open-ended<br />Disciplined<br />Risk-based<br />Inclusive<br /> Most organizations modify, delete, and add columns on the Risk/Control Matrix to fit their own environment.<br />
Difference between Risk-Based and Value-for-Money approaches
Twelve Attributes for Evaluating Effectiveness
Case Study</li></li></ul><li>Needs for Performance Audit<br />To evaluate a unit or program and answer questions like:<br />Do we get value for money?<br />Is it possible to spend the money better or more wisely?<br />Are the right things been done?<br />If so, are things been done in the right way?<br />If not, what are the causes? <br />
Value-for-Money<br />Definition: VFM is utility derived from every purchase or every sum of money spent. VFM is based not only on the minimum purchase price (economy) but also on the maximum efficiency and effectiveness of the purchase.<br />Looks at how well an organization provides value for money.<br />Focuses on economy, efficiency, and effectiveness<br />Based on the Twelve Attributes for Evaluating Effectiveness<br />
Audit Performance Measures – 3E’s<br />The principle of ECONOMY is keeping costs low. It requires that the resources used by the audited entity for its activities shall be made available in due time, in appropriate quantity and quality and at the best price. <br />The principle of EFFICIENCY is getting the most from available resources. It is concerned with the best relationship between resources employed, conditions given and results achieved.<br />The principle of EFFECTIVENESS is meeting the objectives set. It is concerned with attaining the specific aims or objectives set and/or achieving the intended results. <br />
12 Attributes For Evaluating Effectiveness<br />Costs and Productivity<br />Responsiveness <br />Financial Results<br />Working Environment<br />Protection of Assets<br />Monitoring and Reporting<br />Management Direction<br />Relevance<br />Appropriateness<br />Achievement of Intended Results<br />Acceptance<br />Secondary Impacts<br />
Conducting Performance Audit- Planning<br />Gather background information on the audit area.<br />Understand the organization’s business, objectives, mission, etc.<br />Interview management and staff.<br />Use the twelve attributes to scope the audit by looking at each attribute to choose which are most applicable.<br />For the selected attributes, form questions to be answered during the next phase.<br />
Conducting Performance Audit- Examining and Evaluating<br />The questions are answered through:<br />- Interviews with management, employees and others<br />- Industry research<br />- Performance measures (criteria)<br />- Benchmarking (criteria)<br />- Other management and audit reports.<br />- Site visits.<br />
Conducting Performance Audit- Reporting and Following Up<br />Communicating Results Phase<br />Issues should be communicated to client throughout the audit.<br />The report is written and presented to the client. <br />Following Up<br />Management implements action items from the report. Audit assists as required.<br />
Case Study<br />State Department of <br />Fruit and Vegetable<br />
Unit 6<br />Final Thoughts<br /><ul><li>Summary of What We Discussed
Internal Audit - Today and Tomorrow</li></li></ul><li>Summary<br />Understanding of internal audit and performance audit<br />Performance measures<br />IIA’s International Professional Practices Framework (IPPF)<br />Management functions<br />Risk-based performance audit<br />Value-for-money performance audit<br />
Modern Internal Auditing<br /><ul><li>Client-focused, value-added service to management and oversight bodies
Guided by international standards and enhanced emphasis on quality
Started to be part of governance structure</li></li></ul><li>Top 5 Internal Audit Activities Today<br />Operational auditing (89% of respondents).<br />Audits of compliance with regulatory code (including privacy) requirements (75% of respondents).<br />Auditing of financial risks (72% of respondents).<br />Investigations of fraud and irregularities (71% of respondents).<br />Evaluating the effectiveness of control frameworks (i.e., using COSO and COBIT) (69 percent of respondents).<br />2010 IIA Global Internal Audit Study <br />
What Is Next? Top Five Imperatives <br /><ul><li>Assess and align with key stakeholder expectations