1. The document discusses the objectives, methodologies, and phases of performing an information systems audit. 2. Key methodologies discussed include the top-down and bottom-up approaches, with the top-down being business and risk focused and the bottom-up focusing on control objectives. 3. The phases of an audit include pre-engagement work, data collection through testing, interviews and documentation, data analysis to identify findings and risks, developing recommendations, and reporting results.

1. IS Audit Preparing and Correctly Deploying an Audit Marco Raposo, CISSP-ISSMP, QSAp, ABCP October, 2007

2. Agenda 1. Audit Drivers and Objectives 2. Assessment in SDLC 3. Methodologies 4. Audit Phases 5. “Take Away’s”

3. Daniel E. Geer, Sc.D., “Risk Management Is Where the Money Is” (1998) Andrew Jaquith, “ Risk Management Is Where The Confusion Is “ (2007) Why audit? Auditing is part of a quality control and risk management process Auditors must be independent External Auditors hired to present an independent vision and evaluation Internal Auditors integrating a separate line of reporting in order to preserve independence Security Audits are performed to ascertain the validity and reliability of existing controls and countermeasures Common Audits Risk Assessment Compliance Assessment Technical Assessment External Assessment (pen testing) Performance audit

4. System Development Life Cycle and Security Initiation - Security Categorization - Preliminary Risk Assessment Implementation - Inspection and Acceptance - Security Control Integration - Security Certification - Security Accreditation Operations / Maintenance - Configuration Management and Control - Continuous Monitoring Disposition - Information Preservation - Media Sanitization - Hardware and Software Disposal Security Assessment in SDLC Acquisition / Development - Risk Assessment - Security Functional Requirements Analysis - Security Assurance Requirements Analysis - Cost Considerations and Reporting - Security Planning - Security Control Development - Developmental Security Test and Evaluation - Other Planning Components

5. Top-Down Approach Business and Risk focus Early Stages of SDLC Business Analysis Qualifies Risk Evaluates Controls efficiency Outputs residual risk Risk Business Mitigation Controls Bottom-up Approach Control Objectives focused Later Stages of SDLC Qualifies residual risk Evaluates Controls efficiency Outputs new control objectives Assessment Top-Bottom approaches

6. Objectives Security Standards and best practices… Use Industry standards and best-practices as control objectives Risk Management IS best-practices Business Continuity Management Federal and government Payment Industry IS Operation Network Security Application Security Physical Security Business 27002 27001 20000 BS 25999-2 VISA PCI 18028-2 27005 27003 27004 OSSTMM OWASP COBIT Information Security Standard NIST SP 800-53 Technology Processes

7. Audit Phases Audit Workflow Vulnerability/ impact report Security Scorecard Prioritized recommendations Supplied Information Testing Interviews/Observation Analysis Recommendations Results Discussion Consolidation Results Presentation

8. Obtain an understanding of the organization and its processes Define Audit Scope for assessment Define an Audit Plan that includes the auditee and business objectives Create a “Term of reference”, a document that confirms the client’s and the IS auditor’s acceptance of a review assignment Plan should be endorsed by the audit management Define an Audit Program Describe the audit steps planned Identify management and personnel resources Identify any limitations in the audit and the program Establish Contingency Actions for sensible tests Pre-engagement work includes taking responsibilities, setting boundaries and planning. Phase 1 - Pre-Engagement Work Setting the rules Supplied Information Testing Interviews/ Observation Analysis Recommendations Results Discussion Consolidation Results Presentation

9. Interviews Document all interviews Hold one to one Interviews Ensure confidentiality Ask subjective questions Give the interviewee time to develop Take notes Ask questions outside interviewee scope Direct observation Observe procedures during normal operation Don’t assume an active role Phase 2 - Data Collection Data Sources Testing Use trustworthy tools Operate in read only mode Protect audit tools Don’t affect system operation integrity Evidence collection Collect logs Take pictures Take screenshots Identify when/how/who regarding all the evidence Supplied Information Testing Interviews/ Observation Analysis Recommendations Results Discussion Consolidation Results Presentation

10. Technical reports Check for false positives Establish a “Proof of Concept” Seek for compensatory controls Mark verified problems as “findings” Phase 3 – Data analysis “Findings” Data Flows Check unnecessary data flows Check inbound-outbound data flows Identify data flows between distinct security levels Mark all above as “findings” Dependencies Check operative dependencies Identify permission dependencies Mark abnormal issues as “findings” Supplied Information Testing Interviews/ Observation Analysis Recommendations Results Discussion Consolidation Results Presentation

11. Workflow Terminate all processes Search for unfinished endpoints Search for missing links Identify non existing connections Mark above items as “Findings” Phase 3 – Data analysis “Findings” Compliance Radar Mark controls as “Compliant” or “Non Compliant” Seek for 85% compliance with benchmarking Identify low areas of compliance Mark non compliant controls as “Findings”

12. Phase 3 – Data Analysis Measuring Risk RISK

13. Recommendations should directly derivate from findings One finding should link to a recommendation Recommendations should be considered for immediate mitigation or strategic Best practices are “nice to have”’s that don’t present current risk Observation is accepted and documented risk Phase 4 – Recommendations The Responsive Actions Finding 2 Strategic Finding 1 Immediate Finding 3 Best Practice Observation … Finding N High Risk Low Effort High Effort Major Changes Holistic Low Effort Recommended Low risk Risk documentation Finding Recommendation Supplied Information Testing Interviews/ Observation Analysis Recommendations Results Discussion Consolidation Results Presentation

14. Recommendations must be categorized and a priority assigned. Recommendations may be subject to risk management Compensatory controls should be used Risk acceptance is a viable outcome Phase 4 – Recommendations Establishing Priorities Auditor task is to document and support decisions, not to take decisions

15. Phase 4 – Recommendations Establishing Priorities 4 III I F1 F2 F4 Technical Reports Data Flow Dependencies Workflows Compliance Radars Inputs Outputs Output Input 1 2 Implementation Effort Change factor Exposure Impact 3 II Findings Priority Risk Management Decision F3 Mitigate Compensatory controls Best Practices Risk Acceptance Priority = Function (Risk, Cost, Change Impact)

16. Intended Audience Version Control Executive Summary Objectives Actions Results & Findings Recommendations Data report Discuss Accept Present Phase 5 - Reporting & Results Presentation Closing the Audit Supplied Information Testing Interviews/ Observation Analysis Recommendations Results Discussion Consolidation Results Presentation

17. “ Do’s” Present a non technical executive summary Identify top 5 problems Be factual Be flexible Be descriptive Be open minded Collect all evidence Document all actions Be practical and result oriented Use graphics, workflows and radars “ Don'ts” Be biased towards a specific person, technology or solution Be influenced State something that you can’t sustain Exclude specific constraints Push decisions regarding risk management Focus on a particular problem Phase 5 - Reporting & Results Presentation Best Practices and Common Mistakes

18. Audit is a useful tool in the SDLC Audit is a science. However, should be performed as an art Follow a specific methodology and rules Use experienced auditors Both technical and human skills are important Explicit control objectives Consolidate information from distinct planes Findings and symptoms of existing problems, not the problem itself Try get the parachute view Analyze the issues from a risk perspective Present synthetic information Present graphical information Don’t take decisions, provide information to management decision “ Take Away’s”