Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upcoming SlideShare
Rutgers Research Center
Rutgers Research Center
Loading in …3
×
1 of 53

The Coming Age of Continuous Auditing

4

Share

Related Books

Free with a 30 day trial from Scribd

See all

Related Audiobooks

Free with a 30 day trial from Scribd

See all

The Coming Age of Continuous Auditing

  1. 1. The coming age of continuous monitoring and auditing IAAIA Workshop on Continuous Monitoring Dubai, February 2011 Miklos A. Vasarhelyi KPMG Professor of AIS, Rutgers Business School Technology consultant, AT&T Labs
  2. 2. Outline • The Real Time Economy – Electronic measurement and reporting (XBRL) – Monitoring and control • Continuous Assurance – Continuous data assurance – Continuous control monitoring – Continuous risk management and assessment • Evolving towards the future 2
  3. 3. The Real Time Economy 3
  4. 4. Latencies Time it takes for a decision to Time it lead to an takes to Time it outcome perform a takes to process pass It may take informatio time to reach n between a decision processes Business Business Process Process Decision 1 2 OutcomeOutcomes latency Intra-process latency Decision latency Inter-process latency 4
  5. 5. Electronic measurement and reporting (XBRL) • XBRL although a very positive step on the route towards automation perpetuates some of the weaknesses of the “paper oriented” reporting model – Audits to improve their social agency function should be of corporate measurement and databases not of financial reports – XBRL is a rigid model not fit for representing the interlinked fuzzy boundary organizations of today – As most substantive regulatory based changes XBRL presents a series of unintended consequences including • Pressure toward standardization of reporting • Facilitation of more frequent reporting • Evolutionary force towards the standardization of the semantics of accounting reporting • A poor conduit to represent corporate transactions (XBRL/FR) • XBRL/FR will eventually lead to XBRL/GL –great airline 5 effects
  6. 6. Continuous Audit (CA)vs Continuous Monitoring (CM) Continuous Auditing Continuous Monitoring Performed by Internal Audit Responsibility of Management • Gain audit evidence more • Improve governance – effectively and efficiently aligning business/compliance • React more timely to risk to internal controls and business risks remediation • Leverage technology to • Improve transparency and perform more efficient react more timely to make internal audits better day-to-day decisions • Focus audits more specifically • Strive to reduce cost of • Help monitor compliance controls and cost of with policies, procedures, testing/monitoring and regulations • Leverage technology to create efficiencies and opportunities for performance improvements From CA/CM as Preventive Care against Fraud by James R. Littley and Andrew M. Costello, KPMG 6
  7. 7. CA/CM roadmap * • 1. Develop the business case • 2. Develop a strategy for adoption • 3. Plan Design and Implementation • 4. Build and Implement the CM or CA system • 5. Monitor Performance and Progress, and refine as needed * Deloitte: Continuous monitoring and continuous auditing: from idea to implementation 7
  8. 8. Five levels of RTE processes • Business Process • Measurement of the processes • Relationship models • KPI monitoring • Continuous monitoring and assurance 8
  9. 9. Monitoring and Control: 5 levels of activity Continuous Reporting Continuous Assurance Transaction assurance, Estimate assurance, Compliance assurance, Judgment evaluation Analytic monitoring level KPIs: Marketing/Sales Ratio •Drill Down Inventory turnover •History Intra-company transfers •Distribution Relationship level Sales change = Incremental Marketing cost * 2.7 +- 12% •Drill Down E-Care queries = number of sales * 4.1 •History Delay relationships •Distribution Data level •Product •Collection •Investment detail •Aging of •Inventory •Drill Down •Regions •Regions receivables •Distribution •History •Clients •Clients •Clients •Ownership •Distribution •Dynamics •Dynamics •Dynamics •Dynamics Structural level Cash Inventory Marketing Sales E-Care 9 A/R Bad Debts Provisioning 5
  10. 10. Continuous Assurance 10
  11. 11. An evolving audit framework Report level Process level Data level Assurance Assurance Assurance Assurance of Assurance of Assurance of Reports Key Processes Data elements •Process reviews a la •Compliance reports Systrust •XML/ XBRL datum becoming commonplace •Internal or outsourced •Generated and modified •Traditional audit is an •Third party processes by different processes instance of RLA are to become the norm •Balkanization of data •Generated and modified •Intra and Inter process •Control / Assurance tags by different processes controls an issue 11
  12. 12. An evolving continuous audit framework Continuous Risk Monitoring and •Automation Assessment •Sensoring Continuous Continuous Data Control •ERP Audit Monitoring •E-Commerce Continuous Audit 12
  13. 13. Continuous data assurance 13
  14. 14. Continuous Process Auditing at AT&T (1986-1991)
  15. 15. CPAS concepts • metrics • analytics • standards: – of operation – of variance – others • alarms • measurement vs monitoring 15
  16. 16. CPAS definition • The Continuous Process Audit System (CPAS) approach can be defined as a philosophy of auditing that aims to monitor key corporate processes on a continuous basis, in order to achieve audit by exception. 16
  17. 17. Alerts, Causal linkage, Confirmatory Follow the life- CPAS effort cycle of the extranets, Advanced CRMA application, analytics linked to • This methodology will change different timing for processes, different cycles data rich, new the nature of evidence, Audit by exception methods timing, procedures and effort involved in audit work. 17
  18. 18. Continuity Equations / Long Distance Billing reservations flights Billing colections R e c e iv in g C a ll C re a tin g d a ta s e ts S p littin g c a ll P o s tin g f ro m o n e R a tin g e a c h d e ta il d a ta fro m o n e -to -o n e d e ta il in to b ille r file to a c c o u n ts B illa b le in d e p e n d e n t m a n y -to -m a n y file s to b e in s e v e ra l b illin g C u s to m e r te le p h o n e o n e -to -m a n y ' p o s te d to c y c le s c o m p a n ie s in d iffe re n t m a g . ta p e s b ille rs 1 2 3 4 5 18
  19. 19. CPAS OVERVIEW System System Operational Reports Workstation DF-level 2 Operational Operation Report al Report DF-level 1 DF-level 1 DF-level 1 Operational Report Filter Alarm DF-level 0 Data Flow Diagrams Database Reports Analytics Metrics
  20. 20. CPAS effort (II) • The auditor will place an increased level of reliance on the evaluation of flow data (while accounting operations are being performed) instead of evidence from related activities (e.g. preparedness audits). • Audit work would be focused on audit by exception with the system gathering knowledge exceptions on a continuous basis. 20
  21. 21. FlowFront - Interactive Flow Diagram Viewer - AT&T Bell Laboratories - Murray Hill, NJ fer Date: 04/01/89 Set Date Recalculate Metrics Recalculating With Check. Help Text Quit! FlowFront Hierarchy Billing System - Customer Billing Module Trans Customer Database Bill Upda Billing AmtDue Extract Calculate Update Billing Info Customer Amount Accounts 1000 Due 1000 Pay Overview Journal Files Format Bill Print Bill 998 988 Inquiry Accounts 2 Missing: Journal Files 10 Table Process Errors 0 Errors
  22. 22. FlowFront - Interactive Flow Diagram Viewer - AT&T Bell Laboratories - Murray Hill, NJ fernsu fer Date: 04/01/89 Set Date Recalculate Metrics Plot Request graph.level 1 RPC: Silver Springs PE: 60 Help Text Quit! FlowFront Hierarchy Billing System - Overview S Graphics Percent Of Accounts Successfully Billed 99 100 99 100 99 Tra 98 98 97 98 95 Bill Upda 80 85 Billing AmtDue 67 Percent Billed 60 Pay 40 Overview Trans 23 Data 20 Inquiry 100 0 3/16 3/17 3/18 3/21 3/22 3/23 3/24 3/25 3/28 3/29 3/30 3/31 4/1 Pro 4/1/89 Mean: 89.076923076923 StdDev: 21.872591442494 Errors
  23. 23. FlowFront - Interactive Flow Diagram Viewer - AT&T Bell Laboratories - Murray Hill, NJ fe Date: 11/27/89 Set Date Recalculate Metrics Starting S analysis server, please wait... RPC: Silver Springs PE: 60 Help Text Quit! Units: Records prod/svc. 4.3 LDS Billing Subfunctions fulfillment request Order Cus Input Volumes to Message Validation PE: 60 RPC: Silver Springs minutes (x100k)messages (x100k) 8 7 * 6 * 5 4 3 * * * * 2 * * * * * 1 0 * * * * 60 nals CTJ 50 * 40 * 30 * * 20 * * 10 * * * * * 0 * * * * Hierarchy 8.5 * * min / msg 8.0 * 7.5 * * * * * 7.0 * * * * * 6.5 * * 6.0 paymt arr 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 1 2 3 4 5 6 7 8 9 10 11 12 Oct Oct Oct Nov 1989 1989 1989 1989
  24. 24. FlowFront - Interactive Flow Diagram Viewer - AT&T Bell Laboratories - Murray Hill, NJ fer Date: 11/27/89 Set Date Recalculate Metrics Text Request for MPS.cam RPC: Silver Springs PE: 60 Help MPS.CAM Text Quit! FlowFront Hierarchy NEXT PREVIOUS FIRST LAST SAVE QUIT Page 1 of 5 Order Processing Update GOR TOPAS MPFB Flowchart: Message Processing S Functional Level: UPE, RPE Hierarchical Level: 1 Parent Process: CAM MPS. ca Primary Source of Information: Business LDS 4.3 AT&T 55626 Functional and System Process Flows: AT&T End User Profile usag e usage MsgTr 1.0 PURPOSE • To edit messages entering system LEC • To guide and rate messages 677951 • To reject messages that are in error to the Message rej Investigation Unit (MIU) drop • To send messages that should be billed in another PE or by BNA.r the LEC there via the Returns and Transfer function Toll.m 2.0 MAJOR INPUTS • Usage tapes from the Local Exchange Companies Journal EHDB • Independent Telephone Companies (ICOs) • Recorded Information Collection System (RICS) • Correct Messages from MIU CAM MIU.ca UCase • Messages from other parts of the Billing system • Control Accounts (formerly Auditors Prefix) NError 3.0 MAJOR OUTPUTS CCase • Guided and rated messages passed to APE billers for bill lub and transfer msgs preparation CError • Messages sent to other parts of the billing system or returned to the LEC return, lub, transfer • Dropped messages that should be billed in another billing system 248773 • Errors to MIU PayRP 4.0 PROCESSES
  25. 25. Itau-Unibanco projects • Branch monitoring through KPIs and transaction monitoring Sale Monitoring / Agent • Transitory Accounts Transitory accounts • Product Sales Project incentives • Implementation considerations – Hiring a systems integrator – Effect on downstream systems – Behavioral changes 25
  26. 26. Branch Monitoring • Heuristics for 17 monitoring procedures that monitor about 1400 branches are being re-calculated • Have retained IBM as the “systems integrator” for hardware expansion and systems implementation of continuous audit analytics • Is focusing on transitory accounts – About 10,000 general ledger accounts – Unclear how many are transitory – Range a large number of business units 26
  27. 27. Unibanco – Some CA Program Features • Automated monitoring of over 5 million customer accounts on a daily basis using 25 automated procedures to: – Detect errors – Deter inappropriate events & behaviors – Reduce or avoid financial losses – Help assure compliance with existing laws, policies, norms and procedures • Examples of “low hanging fruit:” – Customer advances – Excess over credit limit – Returned checks – Federal tax payment cancellations – TED emissions (should this be omissions?) 27
  28. 28. Unibanco – Advances to Clients Monitoring 28
  29. 29. Transitory Accounts • Level 1 – Analytic review of all accounts • Level 2 – Monitoring of risky accounts at the mainframe level • Level 3 – Daily analytics on transactions and generic characteristics of high risk accounts – Generic filter to analyze daily transactions of particular accounts flagged in daily level 2 monitoring • Level 4 (future) – Continuity equations and relationships 29
  30. 30. Overall Quality of each account Skewed N (skewness > 1) Y Peaked N Peaked (Kurtosis > 1) (Kurtosis > 1) Y N Y Lowest Bound Low Bound Normal Bound (e.g., P90) (e.g., P95) (e.g., P99) or 30
  31. 31. Continuous Data Assurance (CDA) at HCA • HSP is a large national provider of healthcare services, composed of locally managed facilities that include numerous hospitals and outpatient surgery centers. • IT internal audit provided access to unfiltered extracts from their transactional databases, comprising all procurement cycle daily transactions from October 1st, 2003 through June 30th, 2004: Over 500,000 data points. • Dataset mimics what a CDA system has to deal with: highly disaggregate data flowing through CA system in real time. • Audit procedures have to be developed for this environment. 31
  32. 32. Lessons from HCA project • Intricate processes can and must be monitored • This may be done at the transaction levels, in addition to more aggregate levels • Models are necessary that are adaptive and can react to current circumstances • Errors may be automatically corrected • A tool may be derived from the performed work that could be superior to existing tools 32
  33. 33. Metlife • Data stream of over 200K wire transfers • Data only currently available for the wires and the records possess little information • Little context knowledge of the major feeding streams • No fraud training data available • Worked during the audit supplementing the audit team work • Developed a series of data filters relating to specific conditions and trends • Working on an aggregate weighting model • Need in the field verification of picked data 33
  34. 34. Metlife (Project 2-3) • Usage of clustering techniques to extract aberrations in data in parallel to the above discussed effort • Usage of clustering techniques in the evaluation of exceptions in life insurance claims 34
  35. 35. Visualizing combination of attributes, we will be able to see similarity and differences among claims
  36. 36. 60.00% Cluster by Insured information 50.00% POPULATION 40.00% 663 14000 30.00% 15500 31000 20.00% 106103 1023000 10.00% 0.00% cluster0 cluster1 cluster2 cluster3 cluster5 cluster6 cluster7 cluster8 Population 22.60% 47.05% 7.21% 17.19% 3.43% 0.64% 1.55% 0.33% 663 28.89% 44.02% 3.84% 20.54% 1.35% 0.45% 0.68% 0.23% 14000 19.02% 50.92% 1.23% 26.99% 1.23% 0.00% 0.00% 0.61% 15500 14.40% 56.27% 2.11% 23.12% 2.38% 0.13% 0.53% 1.06% 31000 56.22% 19.40% 3.98% 16.92% 1.99% 0.00% 1.00% 0.50% 106103 43.27% 35.67% 0.00% 20.47% 0.58% 0.00% 0.00% 0.00% 1023000 4.49% 52.65% 0.82% 39.59% 2.45% 0.00% 0.00% 0.00% Cluster 0 Cluster 1 Cluster2 Cluster3 Cluster5 Cluster6 Cluster7 Cluster8 We can cluster claims using different group of attributes and flag the claims from specific groups in specific clusters. Several clustering of different groups of attributes can make up the score.
  37. 37. P&G (work with the audit innovation team) • KPI projects • Automating order to cash • Vendor files / duplicate payments • Risk dashboard 37
  38. 38. KPI project • Company has facilities in over 160 countries • Some facilities are manufacturing, some are pure distribution and sales • Content is local and world sourced • Substantive part of the work is building models for inventory and sales flow and trying to understand / model the level and flow variables • The objective is to detect out of the normal events both of business and exception nature (errors and fraud) • There are 4 large ERPs feeding the data / data is extracted in ACL and modeled in SAS • 16 different models have been developed and are being tested 38
  39. 39. Order to cash project -> selective automation • This project aims to selectively automate parts of the audit using order to cash as the context – Audit action sheets – Taxonomization of protocols – Change of nature of evidence – Classification of automation level • Manual • Deterministic • Table comparison • Historical / stochastic – Architecture of the Structure – Prototyping of selected models 39
  40. 40. Continuous control monitoring 40
  41. 41. Siemens projects • Focused on audit automation – First project looked at automating CCM in SAP – Second project focused on a wider scope of automation – A third project would think about reengineering the audit action sheets – The fourth project aims at formalizing SOD, activities, and control structures 41
  42. 42. The Siemens project learnings • ERPs are very opaque • Ratings schema are used and desirable • 20-40% of the controls may be deterministically monitored • Maybe other 20-40% may be convertible to be monitorable • New form of alarm evidence that we do not know how to deal • Continuous risk management and assessment needed for weighting evidence and choice of procedures 42
  43. 43. Continuous Risk Monitoring and Assessment Assurance on Risk Management 43
  44. 44. Increasing emphasis on risk assessment  In compliance with SOX, management must monitor internal controls to ensure that risks are being assessed and handled well.  With ERM companies should identify and manage all risks to achieving its objectives.  In compliance with Basel, banks are required to assess their overall capital adequacy in relation to their risk profile.  Regulatory authorities have encouraged financial institutions to validate their risk-related models to increase the reliability of their risk assessment. 44
  45. 45. • CRMA is a real time based integrated risk assessment approach, aggregating data across different functional tasks in organization to assess risk exposure, providing reasonable assurance on firm’s risk assessment. • CRMA continuously computes key risk indicators (KRIs) with firm’s cross-functional data from its key business processes, as well as from external sources and validates them against whether they are linked to the firm’s risk exposure. 45
  46. 46. • KRI provides early warning systems to track the level of risk in the organization • KRI can be identified through analysis of key business activities – 6 steps for KRI identification (Scandizzo, 2005) • Well identified and computed KRIs provide a reliable basis for computing the riskiness of firm for specific risk, such operational risk, liquidity risk, as well as the overall riskiness of firm. – f(KRI(i),KRI(ii),…KRI(n))= Risk exposure – External risk factors may be mapped manually into the computation of KRIs and risk exposure. 46
  47. 47. • Within CRMA, key business processes and risk factors are identified to compute KRIs and risk exposures, and they are continuously monitored. • CCM monitors violation against business controls and provides assurance on whether controls are followed. This mitigates control KRIs. CRMA continuously monitors changes in control KRIs with CCM, assuring CCM is working. • CDA validates transaction data and evaluates quality of transactions. This mitigates data management KRIs. CRMA assures CDA is effective by keep monitoring. CDA also helps feeding CRMA with validated transaction data. 47
  48. 48. CRMA KRI CDA CMM f(KRIs) = Risk exposure KRI Enterprise System Accounting Process 1 R&D HR Process 2 Marketing Purchasin Process 3 Warehouse g 48
  49. 49. Evolving towards the future 49
  50. 50. Opportunities for research • Creating Control system measurement and monitoring schemata • Creating standards for Business Process Monitoring and Alarming • Automatic Confirmation Tools • Development of a variety of modular Audit bots (agents) to be incorporated into programs of audit automation • Creation of alternative real-time audit reports for different compliance masters 50
  51. 51. Complementary research needs • Expansion of assurance to non-financial processes to relate to these through continuity equations (Kogan et al, 2010) • Standards are needed for CA (CICA/ AICPA, 1999; IIA, 2003; ISACA 2010) • Research on the development of complementary assurance products 51
  52. 52. Reconsideration of concepts and standards • Independence needs to be re-defined • The external audit billing model has to be restructured to bill on function not hours • Audit firms must put improved knowledge collection and management processes to feed their audit analytic toolkit • Audit firms have to engage in auditor automation and pro- actively promote corporate data collection during-the-process • Value added must be justified in terms of data quality • Materiality needs to be redefined 52
  53. 53. • http://raw.rutgers.edu – A wide range of presentations / videos and papers from the multiple CA and CR conferences promoted by Rutgers • http://raw.rutgers.edu/dubai 53

×