Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

El-Paso SOX TestingTraining- June 2007

749 views

Published on

  • Be the first to comment

  • Be the first to like this

El-Paso SOX TestingTraining- June 2007

  1. 1. El Paso Corporation Introduction to Sarbanes – Oxley Section 404 Project June, 2007
  2. 2. 2 What You Will Learn from This Material • Understand the purpose of the SOX 404 compliance project. • Understand what the project team has accomplished and what is left to do. • Understand the roles and responsibilities of the testing participants. • Understand the detailed steps of completing management’s testing.
  3. 3. 3 Success Factors • Know and use available resources – training materials, templates, people. • Understand your assigned business processes – “what can go wrong?”, “what is in place to prevent those failures?” • DON’T BE AFRAID TO ASK QUESTIONS!! • Build a plan and stick to it! • It’s Management’s Risks, Management’s Controls, Management’s Plan & Management’s Conclusions. • Call it like you see it! • Communicate, communicate, communicate. • Don’t go it alone!
  4. 4. 4 What You Will Learn from This Material • Understand the purpose of the SOX 404 compliance project. – Sarbanes-Oxley Act Overview, – Basics of internal control, – COSO Framework, • Understand what the project team has accomplished and what is left to do. • Understand the roles and responsibilities of the testing participants. • Understand the detailed steps of completing management’s testing.
  5. 5. 5 Sarbanes–Oxley Act Overview The Sarbanes – Oxley Act of 2002 (SOX) requires company management to assess and report on the company’s internal control. The Public Company Accounting Oversight Board (PCAOB) was established to set the standards that companies must follow to comply with SOX.
  6. 6. 6 Key Sections Requires an annual report by management regarding internal controls and procedures for financial reporting, and an attestation as to the accuracy of that report by the company’s auditors. The annual report states management’s responsibility for establishing and maintaining an adequate internal control structure as well as management’s assessment of the effectiveness of that internal control structure. Section 404 In general, the new certification requirements require companies to formalize control structures, improve controls and establish monitoring programs to enable CEOs and CFOs to make their evaluations and report their conclusions. Section 302 Requires quarterly certification by the CEO and CFO of all companies filing periodic reports under section 13 (a) or 15 (d) of the Securities Exchange Act of 1934 regarding the completeness and accuracy of such reports as well as the nature and effectiveness of internal controls supporting the quality of information included in such reports. Sarbanes-Oxley Act Overview
  7. 7. 7 “What is internal control?” “Internal control is broadly defined as a process, effected by an entity’s board of directors, management and other personnel, designed to provide reasonable assurance regarding the achievement of objectives in the following categories: • Effectiveness and efficiency of operations. • Reliability of financial reporting. • Compliance with applicable laws and regulations.” (Source: Internal Control – Integrated Framework, COSO) Sarbanes-Oxley Act Overview
  8. 8. 8 What is internal control over financial reporting? • Internal control over financial reporting- • Includes those policies and procedures that pertain to the ability to initiate, authorize, record, process, and report financial data consistent with the assertions embodied in either annual or interim financial statements, or both, prepared in accordance with generally accepted accounting principles. • Specifically includes (a) maintenance of records that in reasonable detail accurately and fairly reflect the transactions and dispositions of the assets of the entity, and (b) policies and procedures that provide reasonable assurance that (1) transactions are recorded as necessary to permit preparation of financial statements in accordance with generally accepted accounting principles, and (2) receipts and expenditures of the entity are being made only in accordance with authorizations of management and directors of the entity. • Certain controls over financial reporting may be in information systems that are primarily designed to achieve objectives other than financial reporting objectives. Basics of internal control
  9. 9. 9 The SEC ruled that management’s evaluation of internal controls must be based on a “suitable framework” and that the COSO Internal Control – Integrated Framework met the criteria of “suitable.” COSO, the Committee of Sponsoring Organizations, was formed in 1985 to improve the quality of financial reporting through business ethics, effective internal controls, and corporate governance. Based on these principles, they developed the COSO framework as a foundation for establishing internal control systems and determining their effectiveness. COSO Framework
  10. 10. 10 The Five Components Control Activities  Policies/procedures that ensure management directives are carried out.  Range of activities including approvals, authorizations, verifications, recommendations, performance reviews, asset security and segregation of duties. Monitoring  Assessment of a control system’s performance over time.  Combination of ongoing and separate evaluation.  Management and supervisory activities.  Internal audit activities. Control Environment  Sets tone of organization- influencing control consciousness of its people.  Factors include integrity, ethical values, competence, authority, responsibility.  Foundation for all other components of control. Information and Communication  Pertinent information identified, captured and communicated in a timely manner.  Access to internal and externally generated information.  Flow of information that allows for successful control actions from instructions on responsibilities to summary of findings for management action. Risk Assessment  Risk assessment is the identification and analysis of relevant risks to achieving the entity’s objectives-forming the basis for determining control activities. All five components must be in place for a control to be effective. COSO Framework
  11. 11. 11 What You Will Learn from This Material • Understand the purpose of the SOX 404 compliance project. • Understand what the project team has accomplished and what is left to do. – SOX Project Plan – Project History ° Set Foundation ° Document Processes, Risks and Controls ° Conduct CFO Walkthroughs ° Develop Management’s Testing Approach – Project Overview • Understand the roles and responsibilities of the testing participants. • Understand the detailed steps of completing management’s testing.
  12. 12. 12 SOX Project Plan Insert Testing timeline here
  13. 13. 13 Launched Teams at Each Business Unit El Paso Corporation Pipelines Production Field Services Merchant Energy Corporate GTM • ANR • TGP • CIG • EPNG • SNG Create new Org chart to include names of leaders
  14. 14. 14 Sub-Cycles Processes are classified by cycles to help ensure efficient testing. Processes leading to a common business objective or processes involving a common set of procedures or common employees, or are otherwise related to one another are grouped within the same cycle. This is a subjective process to some extent, but it helps divide the hundreds of in-scope processes into manageable groupings. Revenue and A/R Expenditure Disbursements Payroll Treasury Inventory Property Tax Financial Reporting Information Technology
  15. 15. 15 Critical & Significant Controls In order to efficiently assess the control environment, management prioritized the controls to be evaluated. An overwhelming number of “key” controls were identified during the documentation initiative. Therefore, those controls were further segregated as critical, significant or non-key. Controls Key Controls Critical Controls Significant Controls The test program writers for 2007 reviewed all controls and re-evaluated the criticality for testing purposes. Some non-key controls were selected for testing while some critical and significant controls were moved to controls not being tested.
  16. 16. 16 Objectives of Management’s Testing • Provide evidence of the operating effectiveness of controls over financial reporting: – Confirm controls operate as designed, and – Verify authorized and competent people execute controls. • Identify exceptions in documentation or operation of controls. • Provide Management with sufficient evidence to conclude on the internal control environment. Finding and documenting what does work right is important. Finding what is NOT working right is even more important because we have to fix those things.
  17. 17. 17 • Testing of controls is a form of validating controls operation. – Periodic testing of controls also evaluates the quality of self-assessment and monitoring processes. • Testers use tests of controls to determine whether selected internal control policies and procedures were operating effectively during a period of time or as of a point in time. • Testers must also assess whether the individuals responsible for executing the controls have the necessary authority as well as the appropriate competencies to do the job. • Tests of controls should be performed at the process level (in addition to the entity level, although this level is not the focus of this phase of the project) -- tests at the process level include tests of: – Manual processing controls , – Automated programmed controls. Validation Methods – Tests of Controls Develop Management’s Testing Approach
  18. 18. 18 “Where do I fit in?” Management’s Responsibilities: • Accept responsibility for the effectiveness of the company’s internal control over financial reporting. • Evaluate the effectiveness of the company’s internal control over financial reporting using suitable control criteria. • Support its evaluation with sufficient evidence, including documentation. • Present a written assessment about the effectiveness of the company’s internal control over financial reporting as of the end of the company’s most recent fiscal year. PwC Audit Responsibilities: • Express an opinion on the financial statements. • Express an opinion on internal control over financial reporting, which includes: ‫־‬ Evaluating management’s assessment of the effectiveness of internal control over financial reporting. ‫־‬ Testing the effectiveness of internal control over financial reporting directly. Project Team Responsibilities: • Assist Management in meeting its responsibilities by performing Management’s Testing. In order to comply with Sarbanes-Oxley Act, Section 404, Management must meets its obligations for PwC to give El Paso a clean opinion under its responsibilities outlined below.
  19. 19. 19 What You Will Learn from This Material • Understand the purpose of the SOX 404 compliance project. • Understand what the project team has accomplished and what is left to do. • Understand the roles and responsibilities of the testing participants. – Testing Team Structure, – Roles and Responsibilities, – Project Team Structure. • Understand the detailed steps of completing management’s testing.
  20. 20. 20 El Paso ManagementRoles & Responsibilities • Accepts responsibility for the effectiveness of the entity’s internal control over financial reporting. • Manages overall direction of the testing activities, making all critical decisions. • Mobilizes resources. • Reviews and approves testing plan. • Assumes sole responsibility for the sufficiency of the accumulation, gathering and summarization processes of the testing activities. • Evaluates and concludes on testing results. • Presents a written assessment about the effectiveness of the company’s internal control over financial reporting as of the end of the company’s 2007 fiscal year.
  21. 21. 21 PMO • Fosters clear communication and synchronizes activities among multiple testing teams. • Coordinates resources and skill sets. • Provides guidance to testing teams on standard methods and procedures, including documentation standards. • Collaborates with Testing Team Managers on the sequencing of the testing schedule for each Business Unit. • Coordinates testing of entity-level controls. • Measures and reports progress of testing teams. • Monitors processes requiring remediation to track completion of Remediation Action Plans and coordinate re-testing. • Anticipates project efficiency issues; takes action to keep project on-time/on-budget. • Develops and communicates Management Reporting requirements for the project. • Performs Quality Assurance review of testing. • Coordinates management’s review of testing activities. • Coordinates communications with external auditor. • Coordinates review of testing with external auditor. • Manages and populates the Sarbox Portal. • Coordinates planning of refresh testing. • Supports The Self-Assessor team in building self-assessment program. Roles & Responsibilities
  22. 22. 22 Testing Team Manager • Fosters clear communications and synchronizes activities within testing team. • Collaborates with the PMO on the sequencing of the testing schedule for each Business Unit. • Collaborates with the PMO on resourcing requirements. • Provides guidance to testers in every step of testing. • Coordinates testing activities with process owners. • Manages contact with process owners to minimize interruptions. • Understands process and relevant controls within cycle to manage testing. • Selects controls for testing. • Coordinates testing of entity-level or cross-cycle controls. • Reviews and approves the test plan and any subsequent changes. • Evaluates exceptions to document retention standards. • Evaluates results of tests. • Communicates internal control gaps to process owners. • Alerts PMO when processes require remediation and further testing. • Reviews Remediation Action Plans to ensure that plans adequately address identified gaps. • Tracks timing and progress of testing activities and routinely reports status to PMO. • Reviews testing efforts to ensure timely accomplishment of objectives. • Reviews quality of testing activities periodically and before submission of documentation to PMO. • Schedules PMO Reviews. • Plans refresh testing. Roles & Responsibilities
  23. 23. 23 Testing Team Lead Tester • Assists Testing Team Manager in overseeing day-to-day operations. • Also performs duties of tester. IT Specialist • Provides guidance on testing process level IT controls, which are generally application controls. • Also performs duties of tester or Testing Team Manager, as assigned. Tester • Understands specific processes and relevant controls to anticipate testing issues. • Writes the detailed instructions for performing the tests of controls. • Communicates testing requirements and document requests to process owner. • Coordinates with process owner to gather evidence required for test execution. Follows-up routinely. • Notifies Testing Team Manager of any changes to the testing plan. • Performs testing. • Routinely updates Testing Team Manager on timing and progress of testing activities. • Informs Testing Team Manager of any issues/concerns arising during the testing activities. • Validates testing results with process owner. • Communicates with process owner to understand the root cause of exceptions. • Employs standardized templates for documenting test plans and results. • Understands documentation standards & maintains neat, orderly workpapers. • Maintains testing documentation on El Paso shared drive. Roles & Responsibilities
  24. 24. 24 Process OwnerRoles & Responsibilities • Accommodates multiple requests for data and information related to testing activities. • Makes available key process personnel on a timely basis for inquiries, observations, etc. • Gathers and organizes all requested documentation to be used in testing activities. • Coordinates data requests directly with IT personnel, with assistance of tester. • Communicates any issues causing delays relating to personnel and documentation availability. • Confirms or resolves deficiencies identified during testing activities. • Prepares Remediation Action Plans for controls deemed to be operating ineffectively. • Corrects any processes and controls requiring remediation.
  25. 25. 25 Project Team Structure The project team is organized in a hybrid structure according to the Business Units and the Cycles that were established during earlier phases of the project. Business Unit Focus: Revenue, Expenditure, Inventory & Property Cycles Merchant Energy Production Pipelines (Revenue) Pipelines (Expenditure, Inventory & Property) Cycle Focus: Merchant, Production, Pipelines, Corporate Tax Financial Reporting Information Technology Payroll, Treasury, Control Environment, Corp Expenditure & Corp Property Field Services is not included in this chart. Their interaction with the overall project structure is dependent on the Gulf Terra merger & is still under consideration.
  26. 26. 26 What You Will Learn from This Material • Understand the purpose of the SOX 404 compliance project. • Understand what the project team has accomplished and what is left to do. • Understand the roles and responsibilities of the testing participants. • Understand the detailed steps of completing management’s testing. – Test Plan Creation – Evidence Compilation – Test Plan Execution – Quality Assurance
  27. 27. 27 “How do we test controls?” The steps of completing detailed testing can be organized into four major activities, as illustrated below. I. Test Plan Creation II. Evidence Compilation III. Test Plan Execution IV. Quality Assurance 1. Conduct initial planning 2. Select controls 3. Draft test plan 4. Approve test plan Test Step 5. Determine evidence accessibility 6. Select sample 7. Coordinate evidence collection 8. Manage testing support Test Step 9. Execute test plan 10. Evaluate test results 11. Validate conclusions 12. Coordinate Remediation Action Plans 13. Report results of testing 14. Anticipate refresh testing Test Step 15. Adhere to documentation standards 16. Review progress and quality of work 17. Schedule PMO reviews Test Step
  28. 28. 28 “Where do we record our work?” Each process should have a Controls Test Matrix like the one below.
  29. 29. 29 “How do we test controls?” The following graphic demonstrates the relationship of the Controls Test Matrix to the testing steps. The Controls Test Matrix fields are beside the testing step to which they relate. I. Test Plan Creation Controls Test Matrix Field ------ ------ A-P Q II. Evidence Compilation Controls Test Matrix Field R-T U-V ------ ------ III. Test Plan Execution Controls Test Matrix Field W-Y Z AA-AB, AE AC-AD ------ AF IV. Quality Assurance 1. Conduct initial planning 2. Select controls 3. Draft test plan 4. Approve test plan Test Step 5. Determine evidence accessibility 6. Select sample 7. Coordinate evidence collection 8. Manage testing support Test Step 9. Execute test plan 10. Evaluate test results 11. Validate conclusions 12. Coordinate Remediation Action Plans 13. Report results of testing 14. Anticipate refresh testing Test Step 15. Adhere to documentation standards 16. Review progress and quality of work 17. Schedule PMO reviews Test Step Controls Test Matrix Field ------ ------ ------
  30. 30. 30 Steps 1 – 4: Test Plan Creation I. Test Plan Creation 1. Conduct initial planning 2. Select controls 3. Draft test plan 4. Approve test plan Test Step Controls Test Matrix Field ------ ------ A-P Q II. Evidence Compilation III. Test Plan Execution IV. Quality Assurance
  31. 31. 31 What has been done?I. Test Plan Creation • The teams have substantially completed the initial planning which included: understanding the processes, classifying the processes into “mini-cycles”, scheduling testing with process owners and setting roles and expectations of team members. What needs to be done: Start by gaining an overall understanding of the processes in which the controls to be tested reside as well as your specific role on your team. Assist with outstanding activities such as coordinating schedules with process owners. • The teams have reviewed the classification of controls as critical and significant and have determined which controls they will be testing. Controls selected will cover all relevant financial statement assertions. • The teams are currently drafting test plans for all controls selected for testing. The plans will be documented in the Controls Test Matrix (CTM) and will include relevant data from the RCM as well as details regarding the test of controls to be performed. Fields A-Q of the CTM will be completed at the end of I. Test Plan Creation. • All test plans must be approved by the Test Team Manager before execution. This approval will be recorded in Field Q of the CTM. If at any point during the testing, the test plan changes, the Test Team Manager must re-approve the plan. 2. Select controls 1. Conduct initial planning 3. Draft test plan 4. Approve test plan Ongoing
  32. 32. 32 3. Draft test plan Who does this? Testing Team What needs to be done? The test plan consists of the detailed instructions for verifying the operating effectiveness of controls. The plan will include information such as testing approaches, scopes and sample sizes. The teams will use the Controls Test Matrix to document the test plan, execution, and results. A Controls Test Matrix should exist for each process or subprocess (each Bold red or light red process on the PCF), and ALL critical and significant controls should be recorded on that Controls Test Matrix, even though only selected significant controls will be tested. I. Test Plan Creation
  33. 33. 33 3. Draft test plan (cont’d) “What tools are available to assist in drafting the test plan?” The test team should leverage off the suggested test plan in the Risk Control Matrix when completing the test plan. However, the test team should constructively evaluate the suggested test plan to ensure that it conforms with the guidance provided here relating to the requirements set forth by management for performance of Management’s Testing. In addition, the teams should review the materials gathered during the CFO Walkthroughs as evidence of control performance. These materials will provide additional information regarding the evidence that will be available for performing Management’s Testing. I. Test Plan Creation
  34. 34. 34 3. Draft test plan - CTMI. Test Plan Creation A. Header - The Header is the information gathered from the Risk Control Matrix: the Organization, Process, Process Owner, COSO Objective, Financial Reporting Element and Business Cycle. B. Test Number - This is a unique, sequential number, one for each control test. Each process will start over with one. C. Control Number - This is the number of the control being tested, as identified on the Risk Control Matrix and Process Maps. Control Description - This is the detailed description of the control being tested. It should agree word-for-word to the Control Description in the Sarbox Portal. No changes, additions, clarifications, etc. may be made to the wording of fields C/D. C. Control Number/Description (editable) - This is the editable version of the control number and description in fields C and D. E. Clarifying Remarks - This field should be used to provide additional detail regarding the control description or the process activities to which the control applies. This could include specific activities performed by specific individuals to accomplish the control’s overall objective; references to specific documents, reports or fields; more detailed linkage to process activities; etc. – any clarifying language needed to create a clear test. The Testing Team may need to talk to the process owner to obtain this clarifying information. F. (C)ritical or (S)ignificant - This is the designation of the control as critical or significant, as indicated in the Sarbox Portal. G. Control Type - This field will identify whether the control is “preventive” or “detective”. H. Control Method - This field will identify whether the control is “automated” or “manual”. I. Control Role - The control role refers to the function of the control within the cycle. The options are: Data Capture, Data Transfer, Monitoring, Fraud, Non-routine or Other. A particular control may perform a combination of these roles and the teams may select more than one option. C/D.2 * * * * * * * These fields are populated by downloading information from the Sarbox Portal. C/D.*
  35. 35. 35 3. Draft test plan – CTM (cont’d) J. Assertion Type - The Assertion Type refers to the El Paso financial statement assertion that is achieved by the control. The El Paso assertions are: Authorization, Completeness & Accuracy, Evaluation of Balances, Access to Assets, Substantiation of Balances, Proper Classification, Presentation & Disclosure. I. Test Plan Creation * * These fields are populated by downloading information from the Sarbox Portal. J. COSO Assertion Type – The COSO Assertion Type refers to the COSO financial statement assertion that is achieved by the control. The COSO assertions are: Existence, Occurrence, Completeness, Rights & Obligations, Valuation or Allocation, Presentation & Disclosure. 2
  36. 36. 36 “Before I get lost. . .” 23DRAFT – For Discussion A. Header What goes in this section? The Header is the information gathered from the Risk Control Matrix: the Organization, Process, Process Owner, COSO Objective, Financial Reporting Element and Business Cycle. Organization Identify the specific Business Unit(s) for which the test plan will apply. Process List the PCF numbers and Process/Subprocess names that will be included in the particular control test. Process Owner Identify the process owner who is responsible and will serve as primary contact for the process being evaluated. COSO Objective List the COSO objective that is being achieved by the process. Financial Reporting Element Identify the financial statement accounts or disclosures that originate from or are impacted by the process. Business Cycle List the business cycle that is most directly impacted by the process. Example: Header on RCM 3. Draft test plan3. Draft test plan LETTER refers to a field on the Controls Test Matrix Subsequent slides will be labeled with a LETTER (e.g., “A” or “K”). Each of these letters and these slides refer directly to a field that must be filled out on the Controls Test Matrix. The fields are discussed within the specific test step in which they apply. As we discuss these fields, take a look at the example Controls Test Matrix so that you know exactly where the field is. NUMBER refers to the test step to which the field applies
  37. 37. 37 Initial Data Capture Data Transfers Monitoring Data capture controls help ensure that all information that should be in the financial reporting makes its way into the control environment in the first place. Data transfer controls help ensure that data moves through the system completely and accurately and that reported results have integrity. Monitoring controls (e.g., analytical review such as variation analysis and reasonableness tests performed by El Paso’s management) play a key role in ensuring that significant errors and omissions do not make their way into reported information. I. Control Role2. Select controls What goes in this field? The Control Role refers to the function of the control within the cycle. The options are: Data Capture, Data Transfer, Monitoring, Fraud, Non-routine or Other. A particular control may perform a combination of these roles and the teams may select more than one option. What issues and concerns must be taken into account? “What do the various control roles mean?”
  38. 38. 38 I. Control Role – Fraud2. Select controls One additional consideration in choosing which significant controls should be tested is the role that a control plays in preventing or detecting fraud. Fraud includes fraudulent financial reporting (e.g. earnings management), misappropriation of assets (e.g. payroll fraud), expenditure and liabilities for improper purposes (e.g. bribery), fraudulently obtained revenue and assets and/or cost and expenses avoided (e.g. tax fraud). Many of the controls surrounding fraud are entity-level controls, as opposed to process-level controls. These include controls surrounding the code of conduct, ethics hotline, audit committee and human resources policies. However, some fraud controls may be embedded within the processes. For example, approval, authorizations, reconciliations and segregation of duties may all be relevant for preventing and detecting fraud. “What if there are no controls in my area specifically related to fraud prevention and detection?” Processes that have a risk of fraud that could have a material effect on the financial statements should have fraud related controls and those controls should be evaluated. If a particular process does not have any of these type of controls identified, then the Testing Team Manager should contact the PMO to coordinate further documentation.
  39. 39. 39 I. Control Role – Non-routine Transactions2. Select controls “Why are non-routine transactions important?” The testing team may consider emphasizing testing of controls that involve the processing of non-routine transactions. Non-routine transactions may involve significant judgments and estimates (e.g., allowance for doubtful accounts), or they may involve management overrides of other controls (e.g., the booking of “topside” entries that undo or lessen the impact of other process-based entries). Processing of Routine Transactions Control 1 Control 2 Control 3 Control 4 Control 5 Control 6 Control 7 Control 8 $5,000,000 “Topside Entry” $(4,000,000) $1,000,000 All these controls may operate effectively 100% of the time. . . . . .but one non- routine transaction. . . . . .could undo all those controls. NET EFFECT
  40. 40. 40 J. (El Paso) Assertion Type What goes in this field? The Assertion Type refers to the El Paso financial statement assertion that is achieved by the control. The El Paso assertions are: Authorization, Completeness & Accuracy, Evaluation of Balances, Access to Assets, Substantiation of Balances, Proper Classification, Presentation & Disclosure. What issues and concerns must be taken into account? “Do all assertions apply to all processes?” All relevant assertions must be addressed within each cycle. Consider the following financial statement assertions which were used during the Documentation initiative. Authorization - management has defined and communicated criteria for authorizing transactions. Completeness and Accuracy - all transactions have been recorded or considered in the proper period. Evaluation of Balances - all balances are recorded at appropriate amounts in accordance with standards. Access to Assets - physical safeguards restrict access to assets in accordance with management authorization. Substantiation of Balances - reports and database contents should be periodically substantiated. Proper Classification (All Transactions) - items in the financial statements are properly described and classified. Presentation and Disclosure - all items are fairly presented in conformity with generally accepted accounting principles. If all relevant assertions are not addressed by critical controls, then additional controls must be designated as critical. In other words, if authorization is not covered by any of the critical controls within a particular cycle, then a significant or other control must be changed to be a critical control. I. Test Plan Creation
  41. 41. 41 J2. COSO Assertion Type What goes in this field? The COSO Assertion Type refers to the COSO financial statement assertion that is achieved by the control. The COSO assertions are: Existence, Occurrence, Completeness, Right & Obligations, Valuation or Allocation, Presentation & Disclosure. What issues and concerns must be taken into account? “Why are there two set of assertions?” The PCOAB has indicated that companies should use the COSO framework for assessing internal controls over financial reporting. These standards were clarified after El Paso had started its documentation phase. During that documentation phase, El Paso chose an alternative set of assertions. Now, to ensure compliance with the PCAOB standards, El Paso is also recording the COSO assertion for controls that are being tested. Therefore, in addition to the El Paso assertions, the teams must also identify the relevant COSO assertions for each control included on the Controls Test Matrix.
  42. 42. 42 J2. COSO Assertion Type (cont’d) “How do El Paso’s assertions relate to the COSO assertions?” The following table presents the linkage between the COSO assertions (adopted by the PCAOB) and El Paso’s assertions. For illustration, consider a balance sheet, the assets of which include only one line item: Desks. . . $100. See the appendix for further guidance. COSO Assertion (adopted by PCAOB) Definition Example Related El Paso SOX Assertions Existence Assets, liabilities, and ownership interests exist as of a point in time. Desks exist as of the reporting date. • Substantiation of Balances • Access to Assets Occurrence Recorded transactions represent economic events that actually occurred during the specified time period. (If a purchase of desks had been made during the period, the cash flow statement might include “Purchases of desks. . . $25.” The purchase occurred.) • Completeness and Accuracy Completeness All transactions, events and circumstances that occurred during a specific period that should have been recognized in that period, have, in fact, been recorded and disclosed. All the desks I own are represented here. • Completeness and Accuracy Rights and Obligations All assets on the balance sheet are property rights legitimately owned, and all liabilities are true obligations as of the date specified. The desks are mine. I own them. • Authorization • Substantiation of Balances Valuation or Allocation All transactions, events, and circumstances represented in the financial statements have been recorded at appropriate amounts according to relevant accounting principles. The desks are truly worth $100. I paid $100 for them [assuming no depreciation]. • Completeness and Accuracy • Evaluation of Balances • Substantiation of Balances Presentation and Disclosure Items in the financial statements are adequately described, properly classified, and fairly presented. What I am representing here is desks (as opposed to, say, chairs) and we can agree on what a generally accepted definition of chair is. The geography on the financial statements is correct. • Presentation and Disclosure • Proper Classification
  43. 43. 43 K. Test Name3. Draft test plan What goes in this field? This is a short description of the test being performed that should also distinguish the test from other tests. A more detailed test description will be documented later. What issues and concerns must be taken into account? “Why do we need to create a “name” when we will be creating a detailed test description?” The Sarbox Portal (as currently configured) will use this name field to distinguish one test from another. Therefore, attempt to strike a balance between brevity and clarity. Since the Portal will eventually be used to list many tests, a user trying to differentiate multiple tests named, say, “Account Reconciliation” would have to look within each test individually until she finds the one she wants. Examples: REP-MgrReviewOfInv-GLCodingMgrSignature; INQ-AP,Pol&Pro; OBS-AP_SegOfDuties; RETEST5-REP-MgrReviewOfInv-GLCodingMgrSignature “What specific guidelines can we use to create test names?” 1. The Test Name field in the Portal is limited to 50 characters, so the name may not be more than 50 characters long. 2. The first three letters of the name should be abbreviations of the test type: Inquiry = INQ, inspection = INS, observation = OBS and reperformance = REP (an exception will be for a retest which should begin with RETEST# then the abbreviation of the testing method.) 3. The rest of the name should be as descriptive as possible in revealing the nature of the test, including relevant distinguishing control and attribute features. 4. Use the list of abbreviations provided on the following slide (or others) wherever possible.
  44. 44. 44 K. Test Name (cont’d)3. Draft test plan
  45. 45. 45 L. Test Type What goes in this field? This field documents whether the test is: Inquiry, Inspection, Observation or Reperformance. No other test types may be recorded here, and each test should be only one of these test types. A test could not be BOTH inquiry and reperformance. If a control is tested using both approaches, then the control would be associated with two tests - one inquiry test and one reperformance test. Example: Test – Discuss with warehouse personnel the criteria used when accepting goods for receipt into the warehouse. Observe warehouse personnel receiving goods, verifying goods match open purchase order and goods are not damaged. The discussion would be an inquiry test while the observation would be a separate observation test. What issues and concerns must be taken into account? “What distinguishes inquiry, inspection, observation and reperformance?” The following exhibit details examples of each type of test: 3. Draft test plan Shadowing Viewing Performance Metrics Analytical Reviews Inquiry Observation Inspection Reperformance Scanning Spot Checks Document Reviews Examinations Witnessing Interviews Facilitated Sessions Surveys Confirmations Representations Reconciliations Attributes Testing
  46. 46. 46 L. Test Type (cont’d) • Definition: “Asking the client about controls” • Provides relevant information but not sufficient when used alone – Does not provide adequate basis for management assessment • Should include discussions on non- routine transactions and management override 3. Draft test plan Inquiry Obser- vation Inspec- tion Re- perfor- mance • Definition: “Seeing the control in action” • Useful when when physical controls are present (example: blank checks are safeguarded) • Risky to solely rely on observation as a testing technique • Definition: “Reviewing relevant control documentation” • Used when evidence exists of control operation (e.g. check marks, written explanations, etc.) • Potential to conclude control is operating effectively when process owners are only “going through the motions” • Should combine with limited inquiry • Definition: “Redoing the operation of the control using selected transactions” • More extensive than inquiry, observation, and examination • Use when serious consequence of misstatement could occur if control is not operating effectively • Potential to conclude control is operating effectively when process owners are only “going through the motions” • Should combine with limited inquiry
  47. 47. 47 L. Test Type (cont’d)3. Draft test plan “How do I know which test type to choose?” Performing a combination of two or more of the four methods may be necessary to test each control. The conclusion regarding operating effectiveness is more reliable when corroborative evidence is obtained from a combination of testing methods. When choosing a test type consider: • Test Objective – What are you trying to determine in this test? – What level of assurance do you want from the test? • Testing Methodology – How are you going to test this control? – What evidence is necessary to show that the control is operating effectively? • Sample – What data are you going to need to use in the test? – What data elements are you going to need to perform the test? – How is the data’s completeness ensured? – Is there any items that need to be excluded from the test? – How accessible is this data? – Where is this data located (e.g. system)?
  48. 48. 48 “How do I know which test type to choose?” (cont’d) We want an efficient test plan but we do not want to compromise effectiveness for the sake of efficiency. Consider what evidence is available to test in order to choose the most effective combination of testing types. 3. Draft test plan L. Test Type (cont’d) • Is there any documentation available? – Can the steps in the process to generate the documentation be reperformed? In this case, it would make sense to combine inspection of documentation with reperformance. – Does the documentation indicate who performed each step? If so, it makes sense to perform inquiry as well. • Does the control lend itself to observation? – Manual controls are usually directly observable within the process. – Some system controls may not be directly observable but outputs may be obtained and reviewed. – It is important to take into account the timing of the control execution when considering this testing approach. Controls that are exercised infrequently may not lend themselves to observation. • Who can you talk to about the operation of the control? – The control owner and/or others executing the control can provide valuable insight as to how consistently and effectively the control is operating. This test type is most effective when combined with other types of tests to corroborate the information provided by the control owner and/or others executing the control. • How can you reperform the execution of the control? – There are a number of controls that can be reperformed by the test team as part of the test plan. We would suggest reperforming the controls within the test if it is feasible. • `
  49. 49. 49 L. Test Type (cont’d)3. Draft test plan This attributes test is a REPERFORMANCE-type test, since it requires the tester to essentially reperform the type of analysis that the approving manager would have gone through to provide his approval. The risk of this test is that the signed initials only imply the working of the control, since the tester cannot know whether the manager went through the analysis before signing – or whether the manager routinely ignores the control aspect of this procedure and signs everything that hits his desk without analyzing it. Note: Although the tester is evaluating 4 attributes, this is one test. The following discussion will help provide a context for evaluating the appropriateness of test types chosen for the testing. Controls testing to support Sarbanes-Oxley compliance will be somewhat different from the testing traditionally performed in external and internal audits. Example: Control - Each operational manager reviews all invoices in his area for correct G/L coding, as well as accuracy and appropriateness of charges (i.e., it is an expense that correctly “belongs” to one of their projects). If all is correct, he manually signs and dates, indicating approval to pay. Incorrect or questionable invoices are investigated and resolved. Test - Choose a sample of 30 invoices. •Review for correct G/L coding. •Review for appropriateness of business expenditure. •Recompute charges to verify mathematical accuracy. •Review for appropriate manager signature.
  50. 50. 50 L. Test Type (cont’d)3. Draft test plan Sarbanes-Oxley Compliance Approach – REPERFORMANCE + INQUIRY In order to mitigate the risk that a reperformance test is not sufficient by itself to gain comfort that the control is working the team should supplement testing with the following: For a sample of invoices reviewed, inquire of the approving manager the following: What analysis, specifically, do you perform on the invoices that you approve? What procedures do you perform with respect to any invoices that you find to be questionable? Please provide the most recent one or two examples of invoices that you questioned and walk me through the process by which you determined that it should be approved or rejected for payment. Conclude as to the effectiveness of this control based both on the reperformance and the subsequent inquiries. This approach provides a much more thorough understanding of the control’s operating effectiveness than reperformance alone. However, 30 individual follow-up inquiries would require a significantly greater level of resources to complete the testing than reperformance alone. The Testing Team must choose carefully the level of assurance needed and the means for getting there. Note that the inquiry is a separate test from the reperformance.
  51. 51. 51 L. Test Type (cont’d)3. Draft test plan “Why is an attributes test classified as a reperformance?” When performing an attributes test, like the one describe on slide earlier, the tester should be actually reperforming the steps that led to the particular attribute originally occurring. Note that “inspecting” for the appropriate manager signature is only one step of reperforming this approval control. The objective is to reperform the analysis that the manager undertakes to determine whether approval to pay is appropriate. When a single test such as this one involves two test types (reperforming the review, reperforming the computation, inspecting the signature) the more stringent test type should be chosen as the test type.
  52. 52. 52 M. Test Description What goes in this field? Document in detail the specific steps necessary to complete the control test from beginning to end. These steps should include adequate information so that a third-party could read the test procedures and understand the overall purpose of the test and could execute the test. Be clear and concise in writing the test procedures. The test description should also indicate the proposed sampling unit, or item to be tested. The sampling unit constitutes one item in the population, such as a document, an entry, or a line item. Example: If the testing objective is to determine whether disbursements have been authorized and the prescribed control activity requires a duly authorized voucher before processing the disbursement, the sampling unit might be defined as the voucher. On the other hand, if one voucher pays several invoices and the prescribed control activity requires each invoice to be authorized individually, the line item on the voucher representing the invoice might be defined as the sampling unit. If a significant control is not selected for testing, this is the area where the testing team manager should document the considerations leading to this conclusion. 3. Draft test plan
  53. 53. 53 M. Test Description (cont’d) What issues and concerns must be taken into account? “Can I test the financial statement account for which transactions are being controlled?” No. The purpose of the SOX 404 testing is to verify internal controls. Tests should be designed to test the operation of those controls. Although the controls may be designed to provide comfort around account balances or disclosures, testing the affected account balance or disclosure does not provide the type of information needed to assess the operating effectiveness of the control. Misstatements detected by the substantive procedures might alter the external auditor’s judgment about the effectiveness of controls. However, substantive tests of details do not provide the evidence needed to support an assertion on the effectiveness of internal control. The absence of misstatements detected by substantive procedures does not provide evidence that controls being tested are effective. Tests of controls in operation are required. (Source: PCAOB Release No. 2003-017; Paragraphs 143 & 144) Example: Consider the controls related to accounts receivable. Test of Balances A test-of-balances approach used by an external auditor would include confirming specific receivables with third parties. This approach provides information about the balance, but it does not reveal much about the operation of the controls. Thus, the approach is not useful for controls evaluation. Test of Controls A test-of-controls approach would more likely involve an inquiry of the Accounts Receivable Manager regarding the specific criteria she uses to judge the collectibility of accounts during her review of the weekly A/R Aging report, along with an inspection of specific examples of accounts that she determined needed to be written off according to the specified criteria. The tester should strive to understand the manager’s process for dealing with both routine and non-routine events related to the collectibility of accounts. 3. Draft test plan
  54. 54. 54 M. Test Description (cont’d) “Are there any special considerations for monitoring controls?” When writing a test for a monitoring control, the tester should consider a review of the integrity of metrics, information and reports used during the monitoring process as well as understanding the actions taken by the process owner when any exceptions are identified, including identification of the root causes of the exceptions and ensuring appropriate process improvements or other necessary actions are taken to avoid the occurrence of future exceptions. The test teams should remember to include inquiries regarding non-routine transactions and management overrides. 3. Draft test plan
  55. 55. 55 N. Failure Conditions What goes in this field? The test plan developer must make a precise statement of what constitutes a failure so that the individuals performing the testing procedures will have specific guidelines for identifying deviations. What issues and concerns must be taken into account? “How do I know what a failure is?” A failure in tests of controls is a departure from adequate performance of the prescribed control activity. In other words, a failure is “what can go wrong” with respect to the operation of the control. Assertions provide a good starting point for identifying potential failure conditions. Example: Suppose a prescribed control requires every disbursement to include an invoice, a voucher, a receiving report, and a purchase order, all stamped “paid.” If the existence of an invoice, voucher, receiving report and purchase order stamped “paid” are necessary to indicate adequate performance of the control, then a failure may be defined as “a disbursement not supported by an invoice, voucher receiving report or purchase order that have been stamped “paid.” All four documents must be present. Note: Lack of sufficient documentation of control performance is a failure for all tests of controls. 3. Draft test plan
  56. 56. 56 O. Assumptions3. Draft test plan What goes in this field? The test plan developer will likely make certain assumptions about the data available to test. Assumptions may include but should not be limited to: 1. Information/circumstances that need to exist prior to the control test beginning, and/or 2. Information/circumstances that will provide additional assistance in executing the test plan. What issues and concerns must be taken into account? “What is an example of an assumption?” Assumptions may be made relating to the availability of certain reports, for example. Since the teams will be using the documentation from the CFO Walkthroughs as a starting point for drafting the test plans, they should document any assumptions that are being made so that they may confirm those assumptions with the process owners.
  57. 57. 57 P. Control Frequency3. Draft test plan What goes in this field? This is a description of how often a control is performed. Examples: Daily, Monthly, Quarterly The purpose of this field is to aid in selecting an appropriate sample for testing. What issues and concerns must be taken into account? “What do I include for controls that do not have any specific frequency?” Certain controls may not have a frequency. Example 1: Accounts Payable personnel maintain formal policies and procedures depicting the activities involved in the Accounts Payable process. Example 2: Segregation of duties exist between the employees who enter invoices and employees who issue checks to vendors. In these instances, place “Ongoing” in the field. No sampling is necessary. The controls testing will involve inspection of the policies, interviews with relevant individuals, etc. to ensure that the controls are operating as intended.
  58. 58. 58 Steps 5 – 8: Evidence Compilation I. Test Plan Creation II. Evidence Compilation 5. Determine evidence accessibility 6. Select sample 7. Coordinate evidence collection 8. Manage testing support Test Step Controls Test Matrix Field R-T U-V ------ ------ III. Test Plan Execution IV. Quality Assurance
  59. 59. 59 5. Determine evidence accessibility Who does this? Testing team and Testing Team Manager What needs to be done? In order to execute the testing plan, the team must first select the items to be tested. This may include selecting people to interview and/or observe as well as documents to inspect and/or be used in reperformance. This will likely involve a discussion with the process owner to determine which people can answer questions and what documents are available. The team should also consult the CFO Walkthrough files to determine what documents are available. At this point, the team will likely evaluate whether the test requires transactional or non-transactional support. Inquiries and observations are generally non-transactional in nature while inspections and reperformances are transactional. Transactional tests generally require the teams to obtain a population. A population consists of all of the items constituting the class of transactions subject to testing, as defined by the control. Example 1: If the tester tests a control designed to ensure that all shipments are billed, is the appropriate population shipped items or billed items? Answer: Shipped items. Example 2: Consider the following control: “Invoices from alliance vendors are deemed approved when received by Accounts Payable because only approved purchasers are allowed to make purchases from alliance vendors.” In this instance the population is invoices from alliance vendors. II. Evidence Compilation
  60. 60. 60 5. Determine evidence accessibility (cont’d) Note: In some instances the process owner may direct the test team to IT personnel to assist with gathering population information. The inclusion of IT is encouraged, however, the team should not work directly with IT without the inclusion of the process owner. The process owner, who has knowledge of the process and the system fields, should coordinate all requests to IT to ensure that the appropriate information is being gathered. Ultimately, it is the process owner who is responsible for ensuring the achievement of the control objective. IT’s role, though important, is one of “helper,” not “owner” in achieving that objective, even if the control is completely automated. The testers should remain an active participant in this process. “What if the information needed to execute the test is not available?” The testing team and Testing Team Manager need to determine why information is not available. If the issue is simply related to the way the test is worded and other testable evidence can be obtained from the process owner, the test team should rewrite the test as appropriate. If, however, no alternative evidence is available for performing the test, the control can be considered to be not operating effectively. Based on the proposed PCAOB standards and PwC’s guidance, inadequate documentation is a deficiency. “What do I do when process owner or IT does not provide data timely?” The teams should set deadlines when requesting information and they should work with the process owners to ensure those deadlines are met. If the process owners are not responding in a timely manner, the testers should notify the Testing Team Manager. If the issue persists, the Testing Team Manager should contact the PMO for assistance. II. Evidence Compilation
  61. 61. 61 R. Population What goes in this field? Clearly document the characteristics of the population, including all supporting information requested and the source from which the population was extracted. This could be a system file, binder, customer file, etc. Also, identify the employee (process owner) within the Business Unit who is responsible for maintaining/monitoring the information contained in the population. The Business Unit will be responsible for providing the testers with the information (sampled items) to test. For non-transactional tests, such as inquiries and observations, the team should include information related to the “population” of people being interviewed or observed. Example 1: From the previous example, the population characteristics are invoices from alliance vendors. The supporting information would be the invoice number, vendor number, vendor name, the invoice date, the payment date, amount, and G/L coding. The population source is the PeopleSoft A/P module. The responsible person is John Doe. Example 2: Consider the following control: “Account reconciliations are performed monthly comparing the data transmitted from system A to system B. Any discrepancies are researched and resolved.” In this instance, the population consists of the reconciliations for the months within the testing period. Example 3: 20 Revenue Accountants 5. Determine evidence accessibility
  62. 62. 62 R. Population (cont’d) What issues and concerns must be taken into account? “How do I know that I have the entire population?” It is not sufficient for a process owner to provide a file or list and say, “This is everything.” Testers must assume a posture of “professional skepticism” and obtain objective assurance that the population presented is both complete and accurate. If a process owner were trying to disguise an ineffectively operating control, the items he would be most likely to “leave out” of the population are the items we most likely want to see in our testing. It is the responsibility of testers to use sampling techniques that ensure they have a reasonable chance of choosing items indicating ineffective operation. Therefore, the tester must use professional skepticism to ensure the population is complete. Note: One way to address this issue is to be involved with the process owners’ requests for data and having IT send population information directly to the tester. However, the process owner should be involved in discussions requesting the data, as indicated on slide 82. 5. Determine evidence accessibility
  63. 63. 63 S. Testing Period Start Date What goes in this field? The testing period start date should be January 1, 2004, unless the control was not in operation at that time. In that case, the start date should be the date the control was placed into operation. For quarterly and yearly controls, the start date should be in 2003 to include Q4 ’03 for quarterly controls and the last time the control was performed for yearly controls. What issues and concerns must be taken into account? How long should a control be in operation before testing? In general testing of controls for which gaps have been identified and remediation plans have been implemented should be delayed until the controls are known to be operating. Controls should be operating for a time period sufficient to measure via testing. If, for example, a control was remediated one week ago and that control has, therefore, been operating effectively for only one week, it is unlikely that it makes sense to test this control immediately. The control should be in operation for a “reasonable” period (i.e. at least long enough to select a sample according to the chart on slide 88.) For further guidance, see slide 106. 5. Determine evidence accessibility
  64. 64. 64 T. Testing Period End Date5. Determine evidence accessibility What goes in this field? The testing period end date will be the date the population is requested. What issues and concerns must be taken into account? “What if the testing period isn’t the entire year?” Tests of controls may be applied to transactions executed throughout the year or during the period from the beginning of the year to an interim date. When the testing period does not extend to the end of the year, “refresh” testing may need to be performed. See slide 120 for further information. “Should we use a “clean” cut-off date?” Where appropriate the team may choose to use a “clean” cut-off dates, such as month-end. However, the team should strive to to use as long a period as possible to have as large a population as possible.
  65. 65. 65 6. Select Sample Who does this? Testing Team What needs to be done? In order to execute testing, the team must first select a sample to test. This involves determining which employees to select for interviews/observations as well as what documents to select for inspections/reperformances. Before selecting the sample, however, the team must determine the sample size and selection method. The following slides provides additional information regarding sample sizes and methods. The team should also consider using data mining techniques as a way of executing a thorough (100%) test when performing such a test would take about the same amount of time it takes to test a sample of items. The Testing Team Manager should consult the PMO for further guidance if your team identifies an area where data mining can be used. II. Evidence Compilation
  66. 66. 66 U. Sample Size What goes in this field? The team should enter the number of items that will be selected for performing the tests of controls. What issues and concerns must be taken into account? “How many items should I test?” For manual controls, the minimum sample size should be determined based on the following table. Critical Minimum Sample Size Significant Minimum Sample Size Multiple-Times-Per-Day Control 30 5 Daily Control 20 3 Weekly Control 10 1 Monthly Control 3 1 Quarterly Control 3 1 Yearly Control 2 1 6. Select sample These sample sizes take into account minimum requirements specified by PwC. The Testing Team should consider the following when determining the actual number of items to test: • Extent of manual oversight or involvement, • Complexity of control and • Planned reliance on control (critical versus significant). Based on the results of the testing, additional items may need to be evaluated. (See Step 10. Evaluate Test Results.)
  67. 67. 67 U. Sample Size (cont’d) The sample sizes listed in the table on the previous slide are not absolute, though teams are discouraged from using smaller samples. Larger samples may be used for particularly significant controls or whenever a greater level of comfort is warranted. “How do I test 3 quarterly and 2 yearly controls when we are starting testing in Q1?” Measuring the effectiveness of a once-a-year-only or quarterly control is of added risk because the external auditor will have to test the 2004 control directly, and it is unlikely there will be an opportunity to remediate the control if it fails. The team should initially test Q4’03 and Q1’04 for quarterly controls and’03 for yearly controls to identify any gaps. The teams must also test Q2’04 for quarterly controls and ’04 for yearly controls once those controls have been performed and are available for testing. “If I have an automated control, do I have to test more than one?” Limiting the testing of automated controls to one test should only be done once IT general controls have been tested and found to be effective. The IT general controls are being tested simultaneously with the process controls (as a separate cycle); therefore, the teams should carefully consider limiting the sample sizes for automated controls. If the teams decide to assume the IT general controls are operating effectively and therefore conclude to limit testing of automated controls, those teams may have to do additional testing at a later date (prior to July 16) to ensure operating effectiveness of the process-level system controls. Keep in mind that “testing one” means testing one of each condition as defined by the business rules, not just one transaction. Example 1: The system requires invoices less than $500 to have one approval; over $500 have two. A tester must test two transactions: one less than $500 and one greater than $500. 6. Select sample
  68. 68. 68 V. Sample Selection Method What goes in this field? The team should document the process that was used for selecting the sample items. What issues and concerns must be taken into account? “What selection methods are available?” 6. Select sample Unrestricted random numbers Intervals Stratifications Haphazard Selection Methods • Each item in population has an equal chance of being selected • Most common method • There is a uniform interval between each item selected after a random start • The population is segregated into two or more classes which are sampled separately • Sample selected without any conscious bias, that is, without any special reason for including or omitting items from the sample; not careless Explanation • When items in population are numbered or are listed in a complete and accurate record • Random sampling burdensome • No pattern in population that will bias the sample • Items missing in population can be identified • Considerable variation in population • More reliability from breaking the population down into groups of comparable items • No systematic way of selecting items Population Characteristics MORE Desirable LESS Desirable
  69. 69. 69 V. Sample Selection Method (cont’d) “What role does materiality play in sampling in SOX compliance controls testing?” None. Materiality (i.e., of financial statement accounts) was indeed one factor considered in the Sarbanes-Oxley work several months ago during the selection of significant processes. But materiality plays no further role in controls testing. Materiality of balances or transactions should NOT influence samples, unless the control explicitly states that the control applies ONLY to transactions of specified unit size or dollar value. If the control does not explicitly state such a condition, then that control applies to all transactions irrespective of their materiality or “significance” to particular financial statement accounts. In the absence of such a condition, sampling should NOT take into account transaction sizes or values (e.g., using stratified populations or dollar unit-type methods). Example 1: A company has two bank accounts, one with a high level of activity and a high dollar balance, and one with a low level of activity and a low dollar balance. A control chosen for testing reads: Each company bank account is reconciled each month to its corresponding general ledger account. All differences between book and bank are investigated and resolved via the reconciliation. The general ledger cash account is promptly adjusted to reflect any unresolved differences. If the testing team were required to test only one reconciliation, it would NOT be appropriate to choose the higher-activity, higher- balance account on the basis of its higher activity and higher balance. The sampling method should strive to give equal weight to EACH instance of the control’s operation, in this case, EACH reconciliation. Each bank account should have an equal chance of being chosen for testing. 6. Select sample
  70. 70. 70 V. Sample Selection Method (cont’d) “What role does materiality play in sampling in SOX compliance controls testing?” (cont’d) Example 2: A control governing payment processing that operates regardless of the dollar value of the payments or the underlying invoices should also be tested on a sample that is chosen without respect to payment or invoice values. In contrast, consider a control specifically written to address the value of expenditure transactions: All individual invoices totaling $50,000 or greater must be authorized for payment by the appropriate operational manager and at least one other Director-level employee in the department or project having budgetary authority for the expenditure. If such a control were chosen for testing, the testing team would, of course, request a population that would isolate expenditures of $50,000 or greater – an appropriate use of population stratification. Example 3: A testing team plans to test control around inventory cycle counts. The control states that cycle counts should be performed monthly, with all inventory items being included at least once per quarter, with the exception of items individually valued at less than $25. The tester requests documentation of the last three cycle counts as well as a comprehensive listing of all inventory items individually valued at $25 or greater. Upon inspecting the documentation, the tester determines that cycle counts are routinely performed for only the top dollar value accounts. No cycle counts were performed on the other items in inventory during any of the last three months. The inventory accounting manager responds, “But why don’t you review the top-ten dollar value accounts like our external audit firm does when they perform cycle counts?” The tester responds, appropriately, “The external auditor is likely using their test to help evaluate the inventory balance in the general ledger. By contrast, we are testing the operational effectiveness of controls. Since the controls apply to each and every inventory item worth $25 or more, it would not be appropriate to exclude any items worth $25 or more from our review.” It is important to remember that we are not conducting a financial statement audit when we perform tests of controls. If the testing team has any questions about how this affects the design or execution of their testing plans, they should consult the PMO. 6. Select sample
  71. 71. 71 7. Coordinate evidence collection Who does this? Testing Team What needs to be done? The testing team needs to submit document requests and schedule inquiries and observations with the process owner. The team should clearly set deadlines for receiving requested documentation. The following form will facilitate the team’s interaction with the process owners. II. Evidence Compilation This is an example format for a document request list. It is not a required workpaper, however, the volume of data requests will likely make the use of such a form necessary. It is recommended that the Testing Team Managers specify a suitable format for each of their teams, if this one does not meet their needs.
  72. 72. 72 7. Coordinate evidence collection The Testing Team Manager should encourage coordination within the Test Team to limit the number of disturbances to process owners. When appropriate, multiple tests should use the same samples. The testers should consult previous data requests to see if the test could use earlier requested samples. A team calendar is also recommended to document scheduled inquiries and observations to allow for coordination among testers. As noted in the planning section, the team may want to include process owner “black out” dates on their calendar. II. Evidence Compilation
  73. 73. 73 8. Manage testing support Who does this? Testing Team What needs to be done? The team will need to maintain testing documentation in a neat and orderly manner to facilitate review by the Testing Team Manager and PMO. Documentation will include: • High-level planning documentation by Business Unit - Cycle. • Controls Test Matrix by subprocess. • Requested documents list. • Master calendar of inquiry and observation meetings. • Workpaper files – Spreadsheets used for attributes tests – Memos documenting inquiries and observations – Copies of all documentary evidence obtained from process owners A standard workpaper referencing scheme has been devised to aid in the organization and review of the workpapers.The teams should adhere rigorously to these documentation standards. Any deviations must be approved by the PMO. Note: The only items that will be submitted to the PMO will be the Controls Test Matrix and the Workpaper files. The other items are suggested only for managing your work. II. Evidence Compilation
  74. 74. 74 Standard Templates These templates should be used for documenting detailed test results for inclusion in the Workpaper files. 8. Manage testing support
  75. 75. 75 Steps 9 – 14: Test Plan Execution I. Test Plan Creation II. Evidence Compilation III. Test Plan Execution 9. Execute test plan 10. Evaluate test results 11. Validate conclusions 12. Coordinate Remediation Action Plans 13. Report results of testing 14. Anticipate refresh testing Testing Program Step Template Field W-Y Z AA-AB, AE AC-AD ------ AF IV. Quality Assurance
  76. 76. 76 9. Execute test plan Who does this? Testing Team What needs to be done? The testing team should perform procedures described in the test description. The performance of the test procedures should be documented as described in 8. Manage testing support in accordance with the standards set forth in Step 15. Adhere to documentation standards. “What if the control is not actually performed as described in the RCM?” The tester must review the control and the related risk to determine if changing the control description to reflect the actual performance of the control is acceptable or if changing the control would result in an unmitigated risk. If the control cannot be modified, then the control is deemed to be not operating effectively the team should consult slides 104 and 105 to determine how to proceed. If, however, the control as performed does mitigate the risk, then the test should be performed using the updated control description. Note, even changes in frequency (quarterly instead of monthly, for example) should be evaluated to determine if the control is mitigating the associated risk. Example: Consider the following control: “Account reconciliations are performed monthly comparing the data transmitted from system A to system B. Any discrepancies are researched and resolved.” If during the testing, the tester discovers that the reconciliations are only performed quarterly, the tester and Testing Team Manager should evaluate whether quarterly reconciliations are sufficient for mitigating the risk. The sample size and selected sample would be adjusted accordingly. Any changes in the control description should be reviewed by the Testing Team Manager, as it may affect testing. A change control process is being developed for instances when the control description is incorrect and can be changed. III. Test Plan Execution
  77. 77. 77 W. Test Status9. Execute test plan What goes in this field? Documentation status provides the status of the testing. The field should include one of the following: Complete, Not Started, Not Applicable, In Progress. What issues and concerns must be taken into account? “Why do I need to mark the status?” Ultimately, documentation will be entered into the Sarbox Portal which includes this status field. However, in the interim, this field allows the Testing Team Manager to evaluate the progress of the Testers and to promote efficient review of each test.
  78. 78. 78 X. Tester9. Execute test plan What goes in this field? List the person who actually performs the controls testing. This person takes ownership of the testing and will assume responsibility for the timeliness and quality of the execution of the test and the accomplishment of the goal of determining whether a control is operating effectively. What issues and concerns must be taken into account? “What if multiple people perform a test?” This field will be used to populate the Sarbox Portal which can only accommodate one name. Therefore, the team should appoint one person who is primarily responsible for each test and who can answer any questions related to the test.
  79. 79. 79 Y. Workpaper Reference What goes in this field? This field is used to record references for all workpapers that contain testwork related to the test. Example 1: X-CE-50 Example 2: P-IT-1, P-IT-2, P-IT-18 Example 3: M-AR-300 through M-AR-379 What issues and concerns must be taken into account? “What if multiple workpapers are used for one test?” There should be no workpapers related to a particular test that are not included in this field for that test. Completeness and accuracy are as important in this exercise as they are anywhere else in the testing, as reviewers, over-testers, and any others placing reliance on the testing will look to this field to determine where the testwork for any given test is documented. Please see further guidance in Step 15. Adhere to Documentation Standards. 9. Execute test plan
  80. 80. 80 10. Evaluate test results Who does this? Tester, with thorough input from Testing Team Manager What needs to be done? The goal of testing is to determine whether a control is operating effectively. In many cases, the answer will be clear. Example 1: In a test of five “attributes” in 30 sample items, there was not a single exception. In follow-up inquiries with the six employees who performed the control, it was clear that each was aware of his or her control responsibility; each indicated that he or she consistently performed all steps of the control; each was aware of exception-handling routines and provided at least one example of how he or she handled an exception; and each was able to provide the tester documentation supporting the handling of the exception. The tester would conclude that the control was operating effectively. In other instances, the answer may not be as clear. Example 2: In an inspection of two months’ variation analyses performed by the Director of the group, neither month’s package of supporting documents tied to the explanations indicated on the analysis. Upon inquiring about these apparent differences, the Director responded that the variations that she was questioning were resolved to her satisfaction, but that in neither month did she have time to include the appropriate supporting documents and actual variance explanations in the analysis package. The tester requested the evidence that the Director used to resolve her questions, discussed this evidence with her and understood how the evidence did indeed support the resolution in her analysis. It is likely the tester would conclude that the control is operating ineffectively. Lack of sufficient documentation is generally not going to produce an acceptable control outcome. III. Test Plan Execution
  81. 81. 81 10. Evaluate test results (cont’d) “What do I do when a control fails?” The actions to take when a control fails depends on the importance of the control (critical versus significant), the frequency of the control operation and various other factors within those categories. The next two slides provide more detailed information. Using the initial sample sizes, one failure should lead the team to consult the guidance on these slides. In other words, the team DOES NOT need to finish testing the original sample. After having considered alternative controls and expansion of testing, when the tester observes a “non- negligible deviation” when testing control performance and there is not an adequate explanation, it should generally be concluded that the control is “ineffective”. • We generally should never agree a control is operating effectively when there is a “greater than insignificant” error rate. • Our expectation is that the external auditor is likely to rarely, if ever, concur with a conclusion on effectiveness in situations where there are a significant number of errors. III. Test Plan Execution
  82. 82. 82 10. Evaluate test results (cont’d) Multiple times per day Discuss failure with process owner to determine if additional failures are likely. Remediate Retest using original sample size based on frequency. Critical Control III. Test Plan Execution Daily Weekly Monthly Quarterly Yearly Frequency End Effective Ineffective Additional failures likely Expand sample* using statistical sampling: • 95% Confidence Level • 2% Upper Error Limit • 1% Expected Error Rate Absolutely no failures anticipated Ineffective** End Effective “What do I do when a control fails?” (cont’d) *Results in sample size of 97. **May have 2 exceptions before test fails.
  83. 83. 83 10. Evaluate test results (cont’d) Multiple times per day Discuss failure with process owner to determine if additional failures are likely. Significant Control III. Test Plan Execution Daily Weekly Monthly Quarterly Yearly Frequency Additional failures likely Expand sample using statistical sampling: • 95% Confidence Level • 2% Upper Error Limit • 1% Expected Error Rate Absolutely no failures anticipated Ineffective End Effective “What do I do when a control fails?” (cont’d) Determine if an alternate control mitigates the same risk(s) Test alternate control using initial sample sizes*** Ineffective Remediate original control End Remediate both controls Effective Retest original control using original sample size based on frequency. Alternate control available Ineffective** ***The assessment of operating effectiveness of the original control will still be ineffective. However, the comments should reflect that the risk is mitigated by an alternate control which was deemed effective. Alternate control NOT available Remediate original control Effective *Results in sample size of 97. **May have 2 exceptions before test fails.
  84. 84. 84 Once remediation has been completed, Testing Teams will need to “retest” controls to ensure operating effectiveness. “How long should we wait before retesting?” The controls should be operational for an adequate time period to enable management and the auditor to obtain sufficient evidence of the controls’ effective operation. Based on the suggested sample sizes (slide 68) we recommend the controls be in operation for the following time period before testing: Note, however, that all testing must be completed by July 16, 2004. Therefore, a smaller sample size may need to be tested at this interim date to ensure the control is in fact operational as intended and a larger sample may be tested as part of the refresh testing to confirm operating effectiveness. Minimum Time for Control Operation Multiple-Times-Per-Day Control 1 Month Daily Control 1 Month Weekly Control 10 Weeks Monthly Control 3 Months Quarterly Control 2 Quarters III. Test Plan Execution 10. Evaluate test results (cont’d)
  85. 85. 85 10. Evaluate test results (cont’d) “What if the results of the testing aren’t clear?” Much as we all would like pure black-and-white outcomes 100% of the time, there will be testing situations where the results do not produce black-and-white answers. Judgment is necessary. Example considerations include: Do inquiries result in situations where people performing controls pay “lip service” to a control’s correct operation, but other evidence indicates they don’t follow through consistently? Is the control one of a series? If it failed, how likely would it be that the failure would be detected and corrected by a process-level or monitoring “backup” control? Are observation-based tests taking into account the likelihood that “watched” performers will perform controls correctly? What comfort do you have that the same performers behave similarly when they are not being observed? The testing of the “other evidence” likely needs to be expanded. Results will need to fall within tolerable error limits for statistical samples. Observation-based tests should be supplemented with additional testing. The “backup” control should be tested and the results of that testing evaluated. Consideration Possible Implication III. Test Plan Execution When in doubt, the Test Team Manager should contact the PMO.
  86. 86. 86 Z. Summary of Test Results What goes in this field? Identify the test results for each attribute included in the control test. Document the number of exceptions identified through the controls testing in total and percentage. Example: Attribute 1 failed 0 times (0.0%); Attribute 2 failed 1 time (6.7%); Attribute 3 failed 0 times (0.0%). This indicates the control failed 1 time (6.7%). What issues and concerns must be taken into account? “How can I record all the information about the test results in one field?” This is not the place for in-depth, detailed explanations of exception conditions or circumstances. This is a summary only. Detailed explanations should be documented in the supporting workpapers such as a memo, or a matrix that records the results of an attributes test, including explanations of any exceptions. The testing team, with the assistance of the process owner, should document the resolution of each exception and whether an internal control deficiency exists. Once the tester has worked with the process owner to resolve all deficiencies possible, remaining deficiencies must be recorded on the Remediation Action Plan. (See Step 12. Coordinate Remediation Action Plan.) 10. Evaluate test results
  87. 87. 87 11. Validate conclusions Who does this? Testers. What needs to be done? Testers should inform process owners of identified exceptions and set firm deadlines for addressing these items. Testers – and Testing Team Managers if necessary – should emphasize to process owners that addressing such items is a high priority because exception items generally indicate control gaps. Before finally concluding that a tested item represents an exception, testers should notify process owners that such a conclusion is pending. This allows the process owner a “last chance” to seek and provide additional evidence that might result in a different conclusion. Process owners should be given a reasonable amount of time to address these items, but test teams should also balance this need with the need to wrap up testing in a timely manner. It is generally not acceptable for process owners to ask for several days to address items, because this will result in a lengthy delay in completing testing. “What form should the notification take?” Testing teams are encouraged to attach the standard cover page (see exhibit on the following slide) to the list of exception items of which the process owners are being notified. III. Test Plan Execution
  88. 88. 88 11. Validate conclusions (cont’d) This is an example of a cover page that could be attached to the list of exceptions provided to process owners. It is intentionally designed to direct the reader’s attention to the serious consequences that might result from unresolved exception items. Testers are encouraged to discuss a proposed deadline with the process owners and fill in the blank during the discussion so as to ensure that the process owner acknowledges the deadline. The use of this form is intended to streamline the notification to the process owners. Other than a quick follow-up e-mail (at the tester’s discretion), it is not expected that testers should have to contact process owners again to obtain the information. It is the process owners’ responsibility to provide the information once requested. III. Test Plan Execution
  89. 89. 89 AA. Assessment of Operating Effectiveness What goes in this field? Effective, Ineffective or Not Tested. What issues and concerns must be taken into account? “Why would I use ‘not tested’?” “Not tested” should be used in the instances of significant controls that were not selected for testing. “Is this the final assessment of operating effectiveness?” This is the team’s recommendation regarding the operating effectiveness. The final conclusion of operating effectiveness is the responsibility of management. Results from both the interim and refresh testing phases will be considered when finalizing the determination of operating effectiveness. If testing results differ between the interim testing and refresh testing phases, consideration should be given to the nature and timing of errors and whether a control was or should be remediated and the related impact on other controls. “How do I record results for expanded tests, tests of alternate controls, or retests?” All results of testing must be maintained. Expanded tests should be recorded on the same line as the original test and the assessment should be based on the results of the expanded sample. For tests of alternate controls or retests, the original test should be marked ineffective and a separate line should be created on the Control Test Matrix for the test of the alternate control or retest with the additional assessment based on that new test. In the comments field for the original test, the Test Team should simply indicate the control was retested and deemed effective. Example: This control was remediated and deemed effective by RETEST1-REP-MgrReviewOfInv- GLCodingMgrSig. 11. Validate Conclusions
  90. 90. 90 AB. Severity Level of Deficiency What goes in this field? Deficiency, Significant Deficiency or Material Weakness. What issues and concerns must be taken into account? “How do I choose the severity level?” The Test Team Manager should evaluate the significance of a deficiency in internal control over financial reporting initially by determining the following: • The likelihood that a deficiency, or a combination of deficiencies, could result in a misstatement of an account balance or disclosure, and • The magnitude of the potential misstatement resulting from the deficiency or deficiencies. The tester then should conclude whether the ineffective control is likely to result in a deficiency, significant deficiency or material weakness. • Deficiency – an internal control deficiency exists when the design or operation of a control does not allow management or employees, in the normal course of performing their assigned functions, to prevent or detect misstatements on a timely basis. • Significant deficiency – a deficiency (or combination of deficiencies) that adversely affects the company’s ability to initiate, record, process or report external financial data reliably in accordance with GAAP. • Material weakness – significant deficiency (or combination of significant deficiencies) that results in more than a remote likelihood that a material misstatement of the annual or interim financial statements will not be prevented or detected. Any significant deficiency or material weakness should be reported to the PMO immediately. 11. Validate Conclusions
  91. 91. 91 AB. Severity Level of Deficiency “Is there any guidance on what should be included in the various levels?” Per guidance from PWC, deficiencies in the following areas are at least considered significant deficiencies: • Controls over the selection and application of accounting policies that are in conformity with GAAP. • Antifraud programs and controls. • Controls over non-routine and nonsystematic transactions. • Controls over the period-end financial reporting process. Note that PwC’s guidance continues to evolve as the PCAOB’s rules are released, clarified and interpreted. 11. Validate Conclusions
  92. 92. 92 AE. Comments What goes in this field? Any additional information that would be useful to management in their overall evaluation of the test results for purposes of issuing a written assessment about the effectiveness of the company’s internal control over financial reporting. What issues and concerns must be taken into account? “Is there anything that should be in the comments field?” When a team has done additional testing, the comments field is the place to note that subsequent testing. Since the original test must maintain the ineffective assessment, the comments field should indicate the control was retested and deemed effective. 11. Validate Conclusions
  93. 93. 93 12. Coordinate Remediation Action Plans Who does this? Testers, and, if necessary, Testing Team Managers. What needs to be done? Once control gaps are identified in the testing and the results validated by the process owners, testing teams should assist process owners in preparing Remediation Action Plans to address the gaps. This may require meeting with the process owners to discuss potential resolutions. When the decision tree leads to remediation, the team should consider the nature of the exception. Is the Nature of the exception due to: • A poorly designed control (a design deficiency not detected during the earlier evaluation of design effectiveness? • A properly designed control not operating as designed? • The person performing the control not possessing the necessary authority or qualifications to perform the control effectively? Depending on the nature of the gap, it might require recharacterizing a control, i.e., changing it to state more specifically what steps are actually performed to mitigate the associated risk. For all ineffective controls, an action plan should be developed to remediate the weakness as soon as practical. The remediation plan should allow sufficient time for validation by management AND the external auditor prior to year-end. Once the Remediation Action Plan is prepared by the process owner, the Testing Team Manager should review the plan to ensure that it effectively addresses the gap and that the resulting control, if changed, is designed to appropriately mitigate the risk. Once the plan is deemed to sufficiently address the gap identified by testing, the Testing Team Manager should forward the plan to the PMO. III. Test Plan Execution
  94. 94. 94 12. Coordinate Remediation Action Plans Internal Control Gap Additional Comments 5 Third party consultants (Mercer) prepare and coordinate with the Benefits department the assumptions for pension liability. The assumptions to be used in the Actuarial data are reviewed for reasonableness and approved by the El Paso Benefits Committee. El Paso Benefits Committee doesn't currently review the assumptions for reasonableness. S The Committee Chair will place review of assumptions on agenda for next committee meeting; include monthly. Linda Camarillo 1/15/2004 Responsible Party Target Completion Date Process Map Linkage Key Control Description (C) ritical or (S)ignificant Control Action to be Taken The Testing Team should use the following form to document the action plan for resolving any gaps noted during the Management’s Testing. When completing the Remediation Action Plan, do not consider controls in isolation. Include discussion on processes, controls, roles and responsibilities, authority, reporting relationships, and training. III. Test Plan Execution Tester Process Owner
  95. 95. 95 AC. Internal Control Gaps What goes in this field? This field is used to describe the specific gaps in the operating effectiveness of the control, including the root cause of the gap. In some cases the entire control may fail. In others, a portion of the control may fail. Example: A control might be the performance of analytical review procedures (e.g., variation analysis) as well as the complete and accurate documentation of that analytical review. Testing might discover that the analytical review is performed as stated, but the documentation is unacceptable. The gap should specify the “piece” of the control that fails, in this case the failure of the documentation step. What issues and concerns must be taken into account? “Isn’t this already on the Remediation Action Plan?” The Remediation Action Plan is a separate document used to develop resolutions to internal control gaps. These gaps, which are discovered during the testing, must also be documented on the Controls Test Matrix. 12. Validate Conclusions
  96. 96. 96 AD. Overall Recommendation12. Remediation Action Plans What goes in this field? This field contains the testing team’s recommendations regarding any gaps identified. Recommendations may include specific suggestions for remediation, changes to the control, or anything else that might benefit the control’s operation. What issues and concerns must be taken into account? “Isn’t the process owner supposed to decide what action to take?” It is the process owner’s responsibility to prepare any necessary Remediation Action Plan steps related to the identified control gaps. The testing team should strive to provide meaningful assistance in this regard, but the recommendation field is not intended to place the burden of writing the Remediation Action Plan steps on the testing teams.
  97. 97. 97 13. Report results of testing Who does this? Testing Team Manager. What needs to be done? Testing teams should furnish the test results and associated Remediation Action Plans to the PMO. In addition, the Testing Team Manager should communicate the results of testing to the Business Unit’s respective CFO. The CFOs will want to know what is working well, but they will also be responsible for ensuring that whatever remediation efforts are needed are undertaken and completed. The CFOs will ultimately be responsible for resolving all gaps in the controls documentation, the design effectiveness and operation. Remember, any significant deficiencies or material weaknesses should be reported to the PMO immediately. III. Test Plan Execution
  98. 98. 98 14. Anticipate refresh testing Who does this? Testing Team What needs to be done? When the test covers an interim period, the tester must determine what additional evidence needs to be obtained for the remaining period until year end (“refresh” testing). Considerations include: -The significance of the assertion involved. -The specific controls that were tested during the interim period. -Any changes in controls from the interim period. -The results of the tests of controls performed during the interim period. -The length of the remaining period. Assuming that controls have not changed from the interim period, testing teams should provide their recommendation for updating testing at year end. III. Test Plan Execution
  99. 99. 99 “What is refresh testing?” Refresh testing relates to situations where management has tested operating effectiveness at an interim date and must obtain additional evidence to support assertions as of the report date. When testing is executed through an interim date, management must update testing through year-end. Testing controls at multiple points during the year helps to verify that evidence exists that a control is operating effectively throughout the year. However, it is necessary to perform sufficient testing through year-end to support the year-end assertion. Further, some controls may function only at year-end, thus it may only be feasible to test at year-end. The Self-Assessor (TSA) will assist with performance of refresh testing. In addition, management will consider areas for additional review as suggested by the testing teams. III. Test Plan Execution 14. Anticipate refresh testing (cont’d)

×