It32015 slides

4/27/2015
1
Copyright © 2014 AuditNet® and Richard Cascarino & Associates
AuditNet® Training without Travel™
Audit Use of CAATs May 5 2015
Guest Presenter:
Richard Cascarino,
MBA, CIA, CISM, CFE
Richard Cascarino &
Associates
Jim Kaplan CIA CFE
• President and Founder of
AuditNet®, the global resource
for auditors (now available on
Apple and Android and Windows
devices)
• Auditor, Web Site Guru,
• Internet for Auditors Pioneer
• Recipient of the IIA’s 2007
Bradford Cadmus Memorial
Award.
• Author of “The Auditor’s Guide
to Internet Resources” 2nd
Edition

4/27/2015
2
Richard Cascarino MBA CIA CISM CFE
• Principal of Richard Cascarino &
Associates based in Colorado USA
• Over 30 years experience in IT
audit training and consultancy
• Past President of the Institute of
Internal Auditors in South Africa
• Member of ISACA
• Member of Association of Certified
Fraud Examiners
• Author of Auditor's Guide to IT
Auditing
Webinar Housekeeping
• This webinar and its material are the property of AuditNet® and Richard Cascarino
and Associates. Unauthorized usage or recording of this webinar or any of its material
is strictly forbidden. We are recording the webinar and you will be provided with a link
access to that recording as detailed below. Downloading or otherwise duplicating the
webinar recording is expressly prohibited.
• Webinar recording link will be sent via email within 5-7 business days.
• NASBA rules require us to ask polling questions during the Webinar and CPE
certificates will be sent via email to those who answer ALL the polling questions
• The CPE certificates and link to the recording will be sent to the email address you
registered with in GTW. We are not responsible for delivery problems due to spam
filters, attachment restrictions or other controls in place for your email client.
• Submit questions via the chat box on your screen and we will answer them either
during or at the conclusion.
• After the Webinar is over you will have an opportunity to provide feedback. Please
complete the feedback questionnaire to help us continuously improve our Webinars
• If GTW stops working you may need to close and restart. You can always dial in and
listen and follow along with the handout.

4/27/2015
3
Disclaimers
• The views expressed by the presenters do not necessarily represent the
views, positions, or opinions of AuditNet® or the presenters’ respective
organizations. These materials, and the oral presentation accompanying
them, are for educational purposes only and do not constitute accounting
or legal advice or create an accountant-client relationship.
• While AuditNet® makes every effort to ensure information is accurate and
complete, AuditNet® makes no representations, guarantees, or warranties
as to the accuracy or completeness of the informationprovided via this
presentation. AuditNet® specificallydisclaims all liability for any claims or
damages that may result from the informationcontained in this
presentation, including any websites maintained by third parties and
linked to the AuditNet® website
• Any mention of commercialproducts is for information only; it does not
imply recommendationor endorsement by AuditNet®
Today’s Agenda
• System testing techniques
• Computerized application systems
• Non-computerized systems
• CAAT types
• Source code review
• Use of Test Data
• Parallel Simulation
• Integrated Test Facilities
• Snapshot Techniques
• SCARF
• Retrieval Software
• Generalized Audit Software
• Specialized Audit Software
• Utility Software
• ACL
• IDEA

4/27/2015
4
Testing of Computerized
Systems
7
What is a "System"
–Manual - pre-computer
–Computer Application
–Computer Environment
–Manual - post-computer
–Integrated Systems
All subject to control
Manual – Pre-Computer
8
Business Control Objectives
Control normally exercised by:
–Supervision
–Authorization
–Authentication
–Procedures

4/27/2015
5
Computer Applications
9
–Control objectives have not changed
–Control points may vary
–Controls themselves may be:
Computerized
Manual
–Effective / Efficient trade-off
Application Controls
10
–Prime Areas
Recording, Classifying and Summarizing
Authorized Transactions
Updating Files
Reporting the results of processing
–Can data be relied upon? - Is it :
Complete
Accurate
Valid

4/27/2015
6
Computer Environment
11
–Operating Environment
Operating System
Networking Software
Database Management Systems
–Control Environment
Operation Controls
Change Control
Operations Controls
12
–Custodial Controls
Physical Site Controls
Operations Standards and Procedures
Library and File Controls
Backup / Restart Controls
Disaster Recovery Planning

4/27/2015
7
Supervisory Controls
13
–Run Schedules
–Checklists
–Exception Reports
–Reconciliation Procedures
–Log Books
–Computer Logs
Administrative Controls Cover
14
–Reliability of Information
–Timeliness
–Nature and type of Information
–Speed of Error Detection / Correction
–Appropriateness of Management
Decisions

4/27/2015
8
Integrity Controls Include
15
–Implementation Controls
–Program Security Controls
–Computer Operation Controls
–Data File Security Controls
–System Software Controls
–Change Control
When Changes are made is Risk Controlled or Introduced?
Are Changes Authorized?
Are Authorized Changes Carried Out
Are Changes Controlled or Recorded?
Who Does the Changes?
Polling Question 1

4/27/2015
9
Selecting Controls for Testing
17
–Establish "prime" Controls for an Area
–Identify Controls covering several Areas
–Identify Stand-alone Controls
–Controls which provide Evidence
–Do NOT try to prove a Negative
Primary Areas of Concern
18
–Complex Systems cannot be re-created manually
–Many computer records are intelligible only to
computers
–Most systems allow multiple access
–"Computers can be trusted"
–Disasters really mean Disaster

4/27/2015
10
Control Concepts and IT
19
–Extent of manual controls reduced
–Sources of data have shifted
–Transaction trails may be discontinuous
–Control points have migrated
–Opportunities for human judgment are less
–Documentation becomes critical
Lack of hard-copy audit trails
Continuity Control
Maintenance Control
–Data Custody Shifted
Logical vs. Technical Controls
20
Logical Controls are :
–Business controls
–Functional in nature
–Either people or computer enforced
Technical controls are concerned with
technical complexities (e.g. parity
controls)

4/27/2015
11
Automated Tools (CAATs)
21
Test Data Generators
Flowcharting Packages
Specialized Audit Software
Generalized Audit Software
Utility Programs
Specialized Audit Software
22
Can accomplish any audit task but
–High development and maintenance cost
–Require specific I.S. skills
–Must be "verified" if not written by the auditor
–High degree of obsolescence

4/27/2015
12
Generalized Audit Software
23
"Prefabricated" audit tests
Each use is a one-off
Auditor has direct control
Lower development cost
Fast to implement
Application of GAS
24
Detective examination of files
Verification of processing controls
File interrogations
Management inquiries

4/27/2015
13
Polling Question 2
Types of Audit Software
26
Program generators
Macro languages
Audit-specific tools
Data downloaders
Micro-based software

4/27/2015
14
Hardware / Software
Compatibility (Desirable)
27
–Across manufacturers
–Across operating environments
–Across machine size
–Mainframe / mini / micro
There are some about
Audit Software Functions
28
File access
Arithmetic operations
Logic operations
Record handling
Update
Output
Statistical Sampling
File comparison
Graphics

4/27/2015
15
Determining the Appropriate
CAAT
29
Depends on the Audit Objective and
selected technique
Application Audit Techniques
Purposes
–1 To verify processing operation
–2 To verify the results of processing
Areas of Control in IT Systems
30
–Application controls - unique to individual
user systems
–Systems development controls - assuring
systems are likely to fulfill objectives
–Physical controls - controlling operating
environment
–System integrity controls - securing the
logical environment
–A balance must be struck

4/27/2015
16
Polling Question 3
CAAT Types and Their Usage
32
–Application audit tools are not always CAATs
–"Any tangible aid that assists an auditor"
Tools to obtain information
Tools to evaluate controls
Tools to verify controls
Automated tools

4/27/2015
17
Obtaining Information
33
–Interviews
–Questionnaires
–Analytical audit flowcharts
–Flowcharting software
–Documentation Review
Control Evaluation
34
Application control matrix
–Components
–Concerns
Adequate
Inadequate

4/27/2015
18
Cascarino Cube
35
Control Verification
36
Audit around
Test data
Re-performance of key functions
Reprocess selected items

4/27/2015
19
Source code review
–Requires programming skill
–Slow
–Expensive
–Boring
–Proves little
–May be useful for specialized review
Confirmation of Results
38
e.g. Debtors certification
–Slow
–Uncertain
–Only shows up errors in your favor
–Very labor intensive

4/27/2015
20
Test Data
39
–Selected to test both correct data and errors
–Require little technical background
but Lacks Objectivity
–Influenced by what is expected
–Assumes program tested is "LIVE" program
Integrated Test Facility (ITF)
40
–Establishes a "dummy" entity
–Process data together with live data
–Excluded from live results
–Under the auditor's control but
–May result in system catastrophe

4/27/2015
21
Advantages of an ITF
41
–Little technical training required
–Low processing cost
–Tests system as it routinely operates
–Understood by all involved
–Tests manual function as well as computer
Disadvantages of an ITF
42
–ITF transactions must be removed before
they interfere with live totals
–High cost if live systems require
modification to implement
–Test data affects live files - danger of
destruction
–Difficult to identify all exception
conditions
–Quantity of test data will be limited

4/27/2015
22
Snapshot Technique
43
–A form of transaction trail
–Identifiable inputs "tagged"
–Trail produced for all processing logic
–Useful in high-volume systems
–Used extensively by I.S. staff in testing
systems
Sampling
44
–"Liars, Damned Liars and Statistics"
–A tool for audit quality control
–May be the only tool possible in a high-volume
system
–Not well understood by auditors
–At computer speeds 100% sampling may be
practicable
May not be desirable

4/27/2015
23
Types of Stat Sampling e.g.
45
–Attributes Sampling
–Variables Sampling
–Systematic selection
–Random selection
–Stratified random selection
–Discovery sampling
–Stop-go sampling
Parallel Simulation
46
Uses same input data
Uses same files
Uses different programs
From a different source
To produce the same results?

4/27/2015
24
Polling Question 4
Common CAAT Problems
48
–Getting the wrong files
–Getting the wrong layout
–Documentation is out of date
–Prejudging results
Never believe what the first printout tells you

4/27/2015
25
In any Application System
49
–Try to identify the controls the user relies on
–Documentation is often misleading
–Not everything needs to be audited
–Program logic mirrors business logic
–You can always ask for help
Industry-Related Software
50
–Audit procedures commonly available for:
Accounts receivable
Payroll
General ledger
Inventory
–May be customizable
–Industry-related audit software available for:
Insurance
Health care
Financial services

4/27/2015
26
Industry- Related Drawbacks
51
–Requires
Conversion of input to standard
package layouts
Selection of appropriate parameters
A degree of IS skill for conversion
–Software itself normally
Cost-effective
Efficient
Customized Audit Software
52
–To run in unique circumstances
–To perform unique audit tests
–To produce output in unique formats
–Expensive to develop
–Normally require a high level of IS skills
–May not tell you what you think they do
–May be the only viable solution

4/27/2015
27
Information Retrieval Software
–Report writers and Query Languages
–Not specifically written for auditors
–Can perform many common audit routines
–Includes
Report writers
Program generators
4th generation languages
54
–Designed specifically for auditors
–Potential uses
Examine records
Test calculations and make computations
Compare data on separate files
Select and print audit samples
Summarize or re-sequence data
Perform analyses
Compare audit data to other sources

4/27/2015
28
Benefits
55
Handling of volumes
Output can be used for further computer
processing
Time to audit can be reduced
Auditor freed to spend time interpreting
results
Limited programming skills required
Audit reliance on IS staff reduced
Limitations
56
Hardware and software environments may
be restrictive
Number of files handle able may be
restrictive
Types of record structures may not be
comprehensive
Number of computations may be limited
Number of reports per "pass" may be
restrictive

4/27/2015
29
Polling Question 5
Excel as a CAAT
58

4/27/2015
30
ACL as a CAAT
59
Idea as a CAAT
7

4/27/2015
31
Importing the Data
61
 Bring a copy to the audit machine
Copies can be reanalyzed later if need be
Live data moves on
You cannot corrupt live data working on a copy
 Bringing it into the audit software
Depends on the software
Most modern systems can import from a variety of data types
 What’s where in the data
Data layout is critical
May automatically extract the data layout from metadata (data
about the data)
ODBC databases
Excel layouts etc.
If the structure is flat you will need the file layout from IT (Make
sure it’s up-to-date)
Acquiring the Data
62
If all you can get is the hard copy
 Can they print it to a file instead
Comma Delimited if possible
Fred Smith, Internal Audit,3/13/2011,
Individual data fields separated by commas
Easy for the software to identify individual fields
 If it’s a printout scan it
1 field of 120 characters for example
The audit software will allow you to define fields within the 120
characters
You can even define different layouts for different rows

4/27/2015
32
Acquiring the Data
63
You’ve got the data – now what?
 Make sure it’s what you asked for
Timeliness – does it reflect the right period?
Accuracy – is it the live data?
Completeness – is it all the data?
 It’s embarrassing to come to an adverse conclusion only to find you were
given the “wrong” file / layout etc.
 Its even worse if you came to a non-adverse conclusion
 Check against known
 Control totals
 Dates
 Transactions
 Never believe what the first printout tells you
From your Manual Audit
64
It seems to be the right data – now what?
 You know what you wanted to find
 You knew where the data resided
 Now you’ve got it
 Go ahead with the analysis you planned
 You have the answer
 NOW CHECK IT
 Remember – Never Believe What The First Printout
Tells You
Particularly if its what you want to believe

4/27/2015
33
Acquiring the Data
65
Remember…
 Modern Audit Software can handle almost any data structure
Variable length block can still cause problems
 We can take it in any format (even pdf) IT has it in and
we’ll handle it from there
 Even if it’s on tape we can handle it with an appropriate
tape drive (on loan?)
 Once you’ve got the data you still have to
Run your tests
Interpret the results
Form your conclusions
Convince someone to do something (perhaps(
If it’s fraud, maintain the chain of custody
Provide expert testimony
 Ensure you have Strength in Depth
Polling Question 6

4/27/2015
34
Questions?
• Any Questions?
Don’t be Shy!
Coming Up Next
IT AUDIT BASIC
4. Auditing Contingency Planning May 7
5. IT Fraud and Countermeasures May 12
IT AUDIT ADVANCED
1. Advanced IT Audit Risk Analysis for Auditors May 14
2. Advanced IT Audit Securing the Internet May 19
3. Advanced IT Audit IT Security Reviews May 21
4. Advanced IT Audit PerformanceAuditing of the IT
Function May 26
5. Advanced IT Audit Managing the IT Audit Function May
28

4/27/2015
35
Thank You!
Richard Cascarino, MBA, CIA, CISM, CFE
Richard Cascarino & Associates
970-291-1497
rcasc@rcascarino.com
Jim Kaplan
AuditNet LLC®
800-385-1625
www.auditnet.org
webinars@auditnet.org

It32015 slides

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Similar to It32015 slides

Similar to It32015 slides (20)

More from Jim Kaplan CIA CFE

More from Jim Kaplan CIA CFE (20)

Recently uploaded

Recently uploaded (20)

It32015 slides