A Retrospective in Analytic Auditing and What’s Ahead Description The speaker will outline salient best practices in establishing an analytic program based on lessons learned looking back on the past two and a half decades. Specific learning objectives include: o Review key dates in the last two decade’s timing that led to the advancement of audit data analytic programs. o Highlight lessons learned over the years through case study examples. o Outline the effective culture around the analytics program to serve as its foundation. o Learn to apply analytics across the entire lifecycle from risk assessment, to planning, fieldwork, and reporting. o Present analytic best practices being deployed by top performing organizations.

1. A Retrospective in Analytic
Auditing and What’s Ahead
About Jim Kaplan, CIA, CFE
 President and Founder of AuditNet®,
the global resource for auditors
(available on iOS, Android and
Windows devices)
 Auditor, Web Site Guru,
 Internet for Auditors Pioneer
 IIA Bradford Cadmus Memorial
Award Recipient
 Local Government Auditor’s Lifetime
Award
 Author of “The Auditor’s Guide to
Internet Resources” 2nd Edition
Slide 1

2. About AuditNet® LLC
• AuditNet®, the global resource for auditors, serves the global audit
community as the primary resource for Web-based auditing content. As the first online
audit portal, AuditNet® has been at the forefront of websites dedicated to promoting the
use of audit technology.
• Available on the Web, iPad, iPhone, Windows and Android devices and
features:
• Over 2,800 Reusable Templates, Audit Programs, Questionnaires, and
Control Matrices
• Webinars focusing on fraud, data analytics, IT audit, and internal audit
with free CPE for subscribers and site license users.
• Audit guides, manuals, and books on audit basics and using audit
technology
• LinkedIn Networking Groups
• Monthly Newsletters with Expert Guest Columnists
• Surveys on timely topics for internal auditors
Introductions
Slide 2
HOUSEKEEPING
This webinar and its material are the property of AuditNet® and its Webinar partners.
Unauthorized usage or recording of this webinar or any of its material is strictly forbidden.
If you logged in with another individual’s confirmation email you will not receive CPE as the
confirmation login is linked to a specific individual
This Webinar is not eligible for viewing in a group setting. You must be logged in with your
unique join link.
We are recording the webinar and you will be provided access to that recording after the
webinar. Downloading or otherwise duplicating the webinar recording is expressly prohibited.
If you have indicated you would like CPE you must attend the entire Webinar to receive CPE
(no partial CPE will be awarded).
If you meet the criteria for earning CPE you will receive a link via email to download your
certificate. The official email for CPE will be issued via NoReply@gensend.io and it is
important to white list this address. It is from this email that your CPE credit will be sent.
There is a processing fee to have your CPE credit regenerated post event.
Submit questions via the chat box on your screen and we will answer them either during or at
the conclusion.
You must answer the survey questions after the Webinar or before downloading your
certificate.
Slide 3

3. IMPORTANT INFORMATION
REGARDING CPE!
SUBSCRIBERS/SITE LICENSE USERS - If you attend the entire Webinar you will receive
an email with the link to download your CPE certificate. The official email for CPE will be
issued via NoReply@gensend.io and it is important to white list this address. It is from
this email that your CPE credit will be sent. There is a processing fee to have your CPE
credit regenerated post event.
NON-SUBSCRIBERS/NON-SITE LICENSE USERS - If you attend the entire Webinar and
requested CPE you must pay a fee to receive your CPE. No exceptions!
We cannot manually generate a CPE certificate as these are handled by our 3rd party
provider. We highly recommend that you work with your IT department to identify and
correct any email delivery issues prior to attending the Webinar. Issues would include
blocks or spam filters in your email system or a firewall that will redirect or not allow
delivery of this email from Gensend.io
Anyone may register, attend and view the Webinar without fees if they opted out of
receiving CPE.
We are not responsible for any connection, audio or other computer related issues. You
must have pop-ups enabled on you computer otherwise you will not be able to answer the
polling questions which occur approximately every 20 minutes. We suggest that if you
have any pressing issues to see to that you do so immediately after a polling question.
Slide 4
The views expressed by the presenters do not necessarily represent the views,
positions, or opinions of AuditNet® LLC. These materials, and the oral
presentation accompanying them, are for educational purposes only and do not
constitute accounting or legal advice or create an accountant-client relationship.
While AuditNet® makes every effort to ensure information is accurate and
complete, AuditNet® makes no representations, guarantees, or warranties as to
the accuracy or completeness of the information provided via this presentation.
AuditNet® specifically disclaims all liability for any claims or damages that may
result from the information contained in this presentation, including any websites
maintained by third parties and linked to the AuditNet® website.
Any mention of commercial products is for information only; it does not imply
recommendation or endorsement by AuditNet® LLC
Slide 5

4. AuditNet® and cRisk Academy
If you would like forever access
to this webinar recording
If you are watching the
recording, and would like to
obtain CPE credit for this
webinar
Previous AuditNet® webinars
are also available on-demand for
CPE credit
http://criskacademy.com
http://ondemand.criskacademy.com
Use coupon code: 50OFF for a
discount on this webinar for one week
Slide 6
Richard B. Lanza, CPA, CFE, CGMA
• Director of Audit Data Analytics for Grant Thornton, LLP
• Over 25 years of ACL, Excel and other software usage
• Received the outstanding achievement in business award by the
Association of Certified Fraud Examiners for developing the publication
Proactively Detecting Fraud Using Computer Audit Reports as a
research project for the IIA
• Recently was a contributing author of:
• Detecting Corruption with Analytics: A Roadmap – The
International Institute for Analytics
• Global Technology Audit Guide (GTAG #13) Fraud In An
Automated World – Institute Of Internal Auditors.
• Cost Recovery – Turning Your Accounts Payable Department
Into A Profit Center – Wiley And Sons.
• Data Analytics: A Roadmap for Expanding Capabilities
(published 2018 in partnership with the IIA's Internal Audit
Foundation)
• In 2015, discovered a new textual analytic technique using letters
called the Lanza Approach to Letter Analytics (LALA)TM
Slide 7
The views expressed by the
presenters do not necessarily
represent the views, positions, or
opinions of Grant Thornton, LLP.
These materials, and the oral
presentation accompanyingthem,
are for educational purposes only
and do not constitute accounting
or legal advice or create an
accountant-client relationship.
rich.lanza@us.gt.com

5. Today’s Agenda
Review key dates in the last two decade’s timing that led to
the advancement of audit data analytic programs
Highlight lessons learned over the years through case
study examples
Outline the effective culture around the analytics program
to serve as its foundation
Learn to apply analytics across the entire lifecycle from risk
assessment, to planning, fieldwork, and reporting
Present analytic best practices being deployed by top
performing organizations
Slide 8
Rich Lanza
30 Year Career in a Slide
Slide 9
Start – 1980s
Fast Food Math
TRS 80
Lotus 123/Excel
Accounting Intern
Current
Grant Thornton, LLC
Audit Data Analytics
Controls and Risk Analytics
1990s
KPMG – MUS Sample
ACL and IDEA
Centralized Analytics
Writing / Speaking
AuditWatch
Early 2000s
AICPA / CPA2Biz
Toys “R” Us
Application & Research
Buyers Guide
Cost Recovery
2004 - 2016
Cash Recovery Partners, LLC
Audit Software Partners LLC
AuditNet ®
AuditSoftwareVideos.com

6. Why Are We Still Talking About Implementing Analytics?
Distrust of Technology
Audit report issued by Marwick, Mitchell & Co. based on their audit of Northern
Shoe Company, Duluth, Minnesota - February 5, 1908
The adding machine in its riper place is an excellent institution but when it is resorted
to on every occasion for the simple addition of two and two it becomes not only
useless but dangerous…….Apart altogether from the time wasted in this way, the use
of the machine which at first may be due to laziness on the part of the employee
becomes at length a necessity for it is a well-known fact that the constant use of any
mechanical calculator leads inevitably to trophy of the calculating faculty. A man who
cannot calculate nimbly and accurately is of very little use as a bookkeeper. We
would strongly advise that the adding machine be removed from the office or its use
very much restricted.
Slide 11
https://caats.ca/

7. Slide 12
News Flash: Projects Fail
Nothing New – KPMG Study
 56% of firms have had to write off at least one IT project in the
last year as a failure.
 The average loss incurred – approximately $13 Million
 The single biggest write-off - almost $210m
 Among the reasons cited for failure:
 inadequate planning
 poor scope management
 poor communication between the IT function and the business
“IT project failure is rampant – KPMG” The Register
Published 26th November 2002
http://www.theregister.co.uk/2002/11/26/it_project_failure_is_rampant/
What Did the I.I. Analytics Find?
Slide 13

8. AuditNet® Survey - 2012 Data
Analysis Audit Software
• Profile of Over 550 Responses:
• 70% 10 or less auditors
• 30% had not purchased data analysis software
• Cost was the top reason for not buying the software
followed by training
• 57% felt that training was the key reason analytics is
not required on audits
• 59% of auditors do not require analytic steps in audits
Slide 14
AuditNet®’s 2017 State of Technology
Use By Auditors – 408 Responses
Slide 15

9. Reasons For Not Using
Data Analytic Software
2015 AuditNet® Audit Data Analysis Software Survey
Slide 16
Top Five Reasons For Not Implementing Audit Technology
The software costs too much(budget constraints)
We do not have a budget for audit software training
The training costs too much
The annual support costs too much
Audit management does not mandate it's use
Slide 17
AuditNet®’s 2017 State of Technology
Use By Auditors – 408 Responses

10. How Can We Increase Analytic Usage?
Take My Manual Audit, circa 1998
Make data analytics a priority on every audit
 Data exists for every process
 Try to use analytics in every audit and explain if you do not
Replace manual tests with automated ones
 You need to replace to provide the time for analytics
 Do one less audit and spend the time “thinking” this year
 Focus on I T testing – user and segregation of duties testing
Slide 19

11. Analytic Benefit in Sum:
Doing More With Less
Page 20
2015 AuditNet® Audit Data Analysis Software Survey
Surveillance is the quickest and
lowers fraud impacts
2016 Report to the Nation – Association of Certified Fraud Examiners
Slide 21
2/3 less costly
3x detection time

12. My Top Audit Savings Ever
http://bit.ly/2Fb5oOd
Over $100MM identified, $40MM recovered
Led to people, process and technology improvements
 It focused on turning the “F” word into the “R” word
Was based on a simple aging report
 Positive values were aged separate of negative values
Slide 22
Crossing the Chasm
by Geoffrey Moore
Slide 23

13. Learning from Other Professions
Engineers, AutoCAD and the 1990s
Attorneys and E-Discovery efficiency
Medical profession
 Collecting, collating and categorizing physicians’ experiences
 Correlations between DNA and disease
 Robotic and virtual guided surgery
 Predictive modeling of mental state from phone messages
...some people will be replaced, the remaining will be augmented
Slide 24
Follow the external auditors
 Identify top areas of manual effort or deficiency
Collaborate with I.T. (They have budget)
 Build testing modulus to automate security and data privacy concerns
 Make their life easier / Reduce risk in their areas of domain
 Play to their ego / They can use best practice automated control
Collaborate with BPOs (They have more budget)
 Make their life easier / Reduce manual testing & questions
 Reduce risk in their areas of domain
 Focus less on risk testing and more on business value opportunities
Slide 25
People
Collaborate With the “Purse Strings”

14. Train everyone (Opportunists and Champions)
 Different training for each group
 Both are experts in their own right
 Communicate frequently
Train staff with competency based training
 Allow team members to gain internal certifications
 Just in time training of key skills seen as most desirable
Have an internal or external coach
 Get the experience you need when you need it – private lesson style
 Provide a safe place to talk about what worked….and what didn’t
Slide 26
People
Training Approaches
Have a backup
 There can not only be one!
Code their knowledge
 Less than 45% have ever scripted their work
Document, save, and videotape your work
 Audit logs in software, flowcharts, and documents take minutes
to develop to the hours to later remember what was done
 Video editing (Camtasia) can be used to show how to run
applications for future auditors
Slide 27
People
Closing the “Lone Ranger” Revolving Door

15. Flipping the Model of Analytics
Slide 28
Data Management
Extraction, Transformation and Enrichment

16. Overcoming Data Challenges
Normalizing data is 80% of the time (in the beginning)
 “By most accounts, 80 percent of the development effort in a big data project
goes into data integration and only 20 percent goes toward data analysis.” —
Intel Corporation
Data is in every process
 It may not be ERP / It may be in your “Big Data”
 90% of data is text
Audit (Internal & External) is the best partner to get the data
 They are independent / Not proving the data is a scope limitation
 Tend to establish the most secure data warehouses
Page 30
Automated Data Normalization
Store procedures for data cleanup once
Create a normalized set of data fields named by YOU
Ensure data quality tests are run prior to analysis
Automate these routine tasks to increase analyst’s time
Enrich the data by organizing it by type codes
Page 31

17. Sample Data Validation
Accounts Payable Other Questions
Make a checklist (to make sure you follow it):
Statistical analysis should also be completed as part of the validation
analysis
Agreement to batch totals, sample data and hardcopies is critical
Page 32
Codify Journal Entry Data
Type of account – Revenue, expense, etc.
Balance Sheet impact and strata category
Income Statement impact and strata category
If the entry has a revenue posting in it
If the entry is associated with a key words table of
unusual words
Manual vs. automated entry
The rank, average value, and std deviation for that
account, if relevant
Slide 33

18. Asking Questions of Data
Lessons from WorldCom/ MCI
Slide 35
In 2002, a small team of internal auditors at
WorldCom worked together, often at night and
secretly, to investigate and reveal $3.8 billion
worth of fraud….
The fraud was accomplished primarily in two
ways:
1.Booking "line costs" (interconnection
expenses with other telecommunication
companies) as capital expenditures on the
balance sheet instead of expenses.
2.Inflating revenues with bogus accounting
entries from "corporate unallocated revenue
accounts".

19. 101 ACL Applications
Over 100 reports including all scripts, audit
objectives, comments to scripts and suggested
audit steps to take with the results
Page 36
Reducing False Positives
http://bit.ly/2H5lYM5
37
Blog article explains
specifically explains
how to:
Remove noise
Filter top
predictors
Duplicate hit rates
increased to 50%
vs. less than 1%

20. Focusing Our Lens
With Our Eyes and Statistical Models
Query Viewpoints
Slide 39

21. Getting Around The Pivot
 Right click on the amount fields (data part) to
see field list and edit various Pivot Table options
 Select from the Top Ribbon Menu the Options
and Design menus for Pivot Tables (Excel
creates them when a Pivot Table is created) –
best and easiest way to make changes
Slide 40
Benford’s Law: The Basis of
Digital Analysis
41

22. Scatter Chart
Slide 42
Scatter Chart Explanation
Slide 43
1 – high dollar change and low count (outliers)
2 – charges that make sense
3 – changes that don’t make sense
4 – inefficiency that is developing

23. Key Control Reports & Scoring
Of the top scored
reports for “fraud”,
find the highest
percentage vendors,
enterers, approvers
etc.
Look for patterns in
the score and the
people related to the
transactions.
Slide 40
Transactional Score
Benefit Patterns Example
Slide 45

24. Focusing on the Edges of
the Normalized Curve
Slide 46
Expanding Our Data Perspective
Moving past structured data

25. The Population of Data Type
Slide 48
Structured Data
 Accounting records
 Sub ledger details
 Monthly performance
measures
Unstructured Data
 Documents (Excel, PDF,
Word)
 Emails
 Network Logs
External Data
 Geomap Service
 OFAC, SAM.Gov Watch Lists
 IRS Tax ID Match
Top Areas To Start Your Own
(Big) Data Analytics
General Ledger
Revenue Cycle
Accounts Payable
Procurement
Payroll Expenses
T&E Spend
Customer and Vendor Feedback
Employee Feedback / Whistleblower Hotline
Process Feedback (Defects, Notes)
Slide 49

26. 80% of Data is Text Based When
Did You Last Audit Text?
It works fast to quickly gain a perspective of the business process data:
 Look for deviations over a 3-year moving average of phrase usage
 Help with new accounting standards relying on contract analysis
If digital analysis/Benford’s Law is latitude, letter analytics is longitude
 Text is far richer in business value and providing a picture than simple
digit theory
 The unique word chart provides a more normalized view of activity
while total word occurrences by letter provides a more dynamic chart
 The trends can be seen quickly to ask relevant questions and to
highlight fraud
Why not use another approach, such as Benford’s Law, to look at ALL data?
Slide 50
Behavioral Analysis
Key Words and the Fraud Triangle
Slide 51
Opportunity
CHANNEL STUFFING
COOKIE JAR RESERVES
FRAUDULENT SHAM
QUID PRO QUO
SECRET ACCOUNT
Rationalization
DON’T KNOW WHAT ELSE TO
DO
PIECE OF THE ACTION
TREAT ME THIS WAY
WORRY ABOUT IT LATER
THEYLL BE SORRY
Pressure
FROM PETER TO PAY PAUL
MANAGE EARNING
TICKING TIME BOMB
WALL STREET
EXPECTATIONS
Per AuditNet® Key Words
http://bit.ly/2oCgEJO

27. American Language’s First
Letters - COCA
Slide 52
The Benford’s Law of Words
Journal Entry Analysis
Word usage in ledger
• Same words tend to occur year over year
• Changes may indicate some change in
the client that could affect risk
assessment
Slide 53
http://bit.ly/1TGwvPS and http://bit.ly/21mEbsU - ACFE Fraud Magazine articles on
“The Benford’s Law of Words – Parts 1 and 2”

28. What’s Ahead?
Strengthen the Three
Lines of Defense
The Three Lines of Defense
Common Constraints
• Slow, manual
• Lack of coordination
• Human subjectivity
• Magnitude of data with limited data resources
The Opportunity
• Insights to identify, monitor, control and
mitigate risks across 3 lines of defense
• Analytic processes directly aligned with
COSO framework
• An opportunity to create efficiencies and
improve effectiveness of the 3 lines of
defense
The Result
• Active data monitoring reduces fraud
losses and fraud detection time by 50%
• Create an opportunity shape the
business' direction and create value
across the organization
Slide 55

29. Audit Analytics Panel
February 22, 2018 (90 minute video)
http://bit.ly/2CXAcMs
Key Notes
Slide 56
80% of the time getting the data / 20% of the time doing analysis
Random samples are defensible / 100% analysis is insightful
The definition of a test is transforming using data
Embedding KPI analytics into the ERP rollout plan
We are at the cusp of continuous auditing
It is becoming the expectation from companies
The receptivity has never been greater
Data Analytics
IIA Research Guides
Other Thought Leadership
• Internal Audit Analytic Surveys – Grant Thornton partnered with the Internal Audit Foundation >>
https://www.grantthornton.com/library/articles/advisory/2017/internal-audit-new-value-data-analytics.aspx
• White Paper – Driving Enterprise Value through Data Analytics >>
https://www.grantthornton.com/library/articles/advisory/2017/enterprise-value-through-data-analytics.aspx
• Data Analytics: A Roadmap for
Expanding Capabilities (published 2018
in partnership with the IIA's Internal Audit
Foundation)
• Data Analytics: Elevating Internal Audit's
Value (published 2016 in partnership with
the IIA's Internal Audit Foundation)
Books
Slide 57

30. Lumen by Grant Thornton LLP
http://gt-us.co/2I2EK8f
Slide 58
Questions?
Any Questions?
Don’t be Shy!
Slide 59

31. More Courses by Rich Lanza
In the Queue for 2018
Many additional training opportunities are available on
www.auditnet.org
June 6
Super-Data-Charging Your Corruption Reviews With Integrated Analytics
September 12
The Future of Auditing and Fraud Detection
December 12
When Data Visualizations and Data Imports Just Don’t Work
Slide 60
Learn More: Go to AuditNet®
AuditNet® and cRisk Academy
If you would like forever access
to this webinar recording
If you are watching the
recording, and would like to
obtain CPE credit for this
webinar
Previous AuditNet® webinars
are also available on-demand for
CPE credit
http://criskacademy.com
http://ondemand.criskacademy.com
Use coupon code: 50OFF for a
discount on this webinar for one week
Slide 61

32. AuditSoftwareVideos.com
Now Free (But Not for Long!)
70+ Hours of videos accessible for FREE subscriptions
Repeat video and text instruction as much as you need
Sample files, scripts, and macros in ACL™, Excel™, etc.
available for purchase
Bite-size video format (3 to 10 minutes)
>> Professionally
produced videos by
instructors with over 20
years experience in
ACL™, Excel™ , and
more
Slide 62
Thank You!
Jim Kaplan
AuditNet® LLC
1-800-385-1625
Email: webinars@auditnet.org
www.auditnet.org
Richard B. Lanza, CPA, CFE, CGMA
Contact Information
D: +1 732 516 5527
M: +1 732 331 3494
Email: rich.lanza@us.gt.com
Slide 63