AGA 2015 Conference - Data Analytics - Amtrak OIG v3
1.
2. PROACTIVE USE OF DATA
ANALYTICS
AMTRAK OFFICE OF INSPECTOR
GENERAL
3. WHO Background and Introduction
WHAT Data Analytics work at Amtrak OIG
HOW Data Analytics strategy at Amtrak OIG
Examples
Reasons to use Data Analytics
Challenges of Data Analytics Work
How to leverage Data Analytics for your work
Agenda
4. Amtrak
OIG: - 50 Auditors, 30 Investigators,
17 Support Staff
- Data Analytics Team - 8 (5
employees, 3 contractors)
DA Team: - Performs own audits
- Provides support to other audits
- Supports Investigations group
Background and Introduction
5. Data Analytics work at Amtrak OIG
Access to 15+ systems (80% of Amtrak’s
financial data)
Audit in 10 different business areas including-
Accounts Payable
Procurement
Payroll
Human Capital
Health care
Operations
150+ tests
18 Reports
6. Opportunities to reduce cost
Opportunities to increase revenue
Opportunities to improve control effectiveness
or increase program efficiency
Recalculations/Compliance testing
Identify potential fraud
Data Analytics for different Audit Objectives
7. Support – OIG and Amtrak leadership
Strategy – Shared services model
Sourcing – Hired technical expertise
Environment – Centralized
Value – Pilot quick win
Tools – Limit to ACL and Excel
Security – Encrypt all data
Data Analytics program at Amtrak OIG
9. From October 2005 - June 2013, $14.1 billion paid to vendors
Data Analysis identified potential duplicate invoices paid of
about $7.5 million
Finance staff recovered about $3.5 million
Four major causes:
Clerks processed known duplicate payments despite system
warnings
Duplicate vendors not detected by the automated controls
Clerks did not ensure correct invoice numbers are entered
Same Invoices received by different departments were paid
Background: Duplicate Payments
10. Duplicate Payments
Vendor Name
Keyed
Invoice No
Invoice
Date
Invoice
Amount
ERICO PRODUCTS INC 130587 06/07/11 8,062.32
VOSSLOH TRACK MATERIAL INC 0000130587 06/07/11 8,062.32
ADT SECURITY SERVICES INC 75277679 07/07/12 5,224.02
DO NOT USE 75277679 07/07/12 5,224.02
W FRANKLIN LP 51017 10/21/11 3,600.00
Lorraine K Koc 51017A 10/21/11 3,600.00
FEDEX 790264830 05/28/12 1,901.04
FEDEX EXPRESS 790264830 05/28/12 1,901.04
11. Background: Material Price Variance
Company buys materials form different vendors for
different plants across the country
Analyzed $35 million worth of material POs
If lowest price vendor was selected for all materials
bought in CY 2013, company could have saved $3.4
million
Causes:
Weaknesses in material requirement forecasting
Limited number of approved vendors
12. Material Price Variance
Material Number
Vendor
Name
Nbr Of
POs PO Amount PO Qty
Avg Unit
Price
%
Variance
000000003710500004 EXXONMOBIL 6 $29,921 5,376 $6 16.52
000000003710500004 VALDES ENTERPRISES INC 1 $14,953 2,304 $6 16.52
000000000299900382 GE TRANSPORTATION SYSTEMS 11 $36,895 235 $157 12.74
000000000299900382 GE TRANSPORTATION SYSTEMS 1 $885 5 $177 12.74
000000000104500004 KOPPERS INDUSTRIES 11 $723,013 364 $1,986 11.34
000000000104500004 LB FOSTER 17 $705,459 319 $2,211 11.34
Material Number PO Amount PO Quantity
Lowest Avg
Unit Price
Amt At
Lowest Price
Variance To
Lowest
000000000104500004 $1,428,472 683.00 $1,986 $1,356,643 $71,829
000000003628500557 $161,299 4,940.28 $21 $104,092 $57,208
000000003733300001 $2,480,196 371,450.80 $7 $2,425,574 $54,622
000000000256404084 $66,486 60.00 $952 $57,120 $9,366
13. Background: Profile of Timesheet Data
Amtrak’s major expenses is labor – $1.2
billion paid to union employees in CY 2014
Amtrak has 14 unions and 23 bargaining
agreements representing different crafts
6 timekeeping systems
Data revealed trends and patterns that
raise questions about whether overtime
and regular time is appropriately reported
16. Summary of Consecutive Days Worked
Top Occurrences – Consecutive Days Worked
SAP ID Job Title Union Start Date End Date
Days
Worked
MAINTAINER SD BRS-SW 12/19/2013 5/14/2014 147
AGENT TICKET
CLK FC
TCU-OFF 4/30/2014 9/8/2014 132
MAINTAINER SD BRS-SW 12/26/2013 5/1/2014 127
MAINTAINER SD BRS-SW 2/20/2014 6/18/2014 119
COACH CLEANER JCC 3/2/2014 6/17/2014 108
C XXXXXXXXX XXXXXX XXXXXX XXXXXX XXX
TICKET/ACCTNG
CLERK
TCU-OFF 6/4/2014 9/15/2014 104
ENG WORK EQUIP
SD B
BMWE-NEC 6/24/2014 10/1/2014 100
17. Identify risk areas with high degree of
assurance in finding results
Mine through 100% of transactions
Advantage over business
Read data from any system, no size limitations
Bring disparate sets of data in one view – hard
for business to do
Helps break down complex business processes
Reasons to use Data Analytics
18. Individual Level
New skill to acquire – lack of commitment to learn
Lack of vision and support from management
Overwhelmed with the data - not knowing where to
start
Unclear objectives – a fishing exercise
Understanding the data and the business process
Uncooperative auditee makes the process difficult to
get meaningful results
Challenges of Data Analytics work
19. Organization/Agency Level
Obtaining access to the data
Storing and securing sensitive data
Recruiting, training, and retaining
Building sustainable processes and
infrastructure
Challenges of Data Analytics work
20. Build the right team
Pick right projects - low hanging fruits first
Identify the need for data analysis at the
beginning of your project
Understand the data and the business
process
Validate your results
How to Leverage DA for Audits
21. Sample Tests
Duplicate Payments
Compare two transactions with same Invoice Number, Invoice Date,
Invoice Amount
Check for duplicate entries in vendor master – same name, tax ID/SSN,
bank account, address, phone number
Identify vendors who repeatedly submit duplicate invoices
Procurement
Compare contract price (PO or BPO) against invoice price to verify if
vendor is honoring agreed upon pricing terms
Check if vendor is honoring discount terms – compare PO vs Invoice
Check if Accounts Payable is losing early discounts because of late
payments – compare discount allowed per PO/Invoice vs discount taken
Check if there is opportunity to negotiate longer payment terms with the
vendors – most companies are asking for 45 to 60 days
22. Material Price variance (use following steps)
1. Aggregate PO quantity and amount by material no and vendor no.
2. Calculate average price per material unit per vendor (Sum PO Amt /
Sum PO Qty).
3. Identify the lowest price per material unit per vendor and segregate
vendors who charged 10% more than the lowest priced vendor.
4. Calculate the higher amount paid for each material by multiplying the
average lowest price paid for that material with the total quantity
bought from all vendors.
Timekeeping
Filter timecards with more than 24 regular and overtime hrs in a day.
Identify employees who submit excessive regular hours in one day to
hide overtime (16 hrs or more).
Filter employees who reported regular and overtime hrs on 30 or more
consecutive calendar days.
Filter employees whose weekly overtime hrs were at least as many or
more than their regular hrs.
Sample Tests
23. Health Care Fraud Indicators
Practitioners with high average payments
Practitioners charging 3 times more than the average amount
per Procedure Code
Practitioners using a Procedure code 3 times more frequently
than other practitioners with similar patient volume
Practitioners with 3 times more than average units per
Procedure Code
Practitioners charging 6 times more than the average amount
per Diagnosis Code
Practitioners with high number of new patients
Practitioners with high transaction volume
Practitioners serving patients with high medical visits
Practitioners not charging copay in high number of visits
Sample Tests
Amtrak OIG is small in size for the volume of business it oversees. Amtrak has only about $4B in revenue, but it is a labor intensive business. Amtrak employs roughly about 20k employees. Comparatively Office of Inspector General may be about the right size or little small.
In the Audit shop, we have Assistant Inspector General for Audits and there are six Senior Directors under him, including me. Each senior director typically focuses on a business area such as train operations audits, financial audits, procurement and contract audits, IT audits. My focus is Data Analytics audits.
If I have to tell you top 5 factors for successful use of DA in our audits, access to all of Amtrak’s data will be one of them. We have our own servers that have a direct pipes built to Amtrak’s key systems such as SAP and some key applications related to our business operations.
Since 2012, we have developed more than 150 different tests and issued 18 audit reports so far. Some of these 18 are from my group where the report findings are data driven, while other reports have used data analytic tests for developing some of their many findings.
We are all about finding economy and efficiency for the agency we are auditing. Generally, your audit objectives will fall into one of these categories. At Amtrak OIG, we have done audits that fell into each of these categories. We have done audits to reduce cost, increase revenue, increase program efficiency, recalculations, and identify potential fraud. The first 4 of these 5 generally focus on process or technology – your recommendations are to improve processes or make changes to technology. The last one focuses on people – potential fraudsters.
The point is that if you have not yet explored the option of using certain data analytic technique because you say that my audit objectives don’t lend themselves to it, my counter argument is that we have plenty to show how you can use data analytics on most of your jobs no matter what your audit objective is.
So, HOW are we making this happen?
The first and foremost is SUPPORT. Without support from the top, it is very difficult to get any momentum. Our IG has been a big proponent of data analytics, and he and our senior leaders from our office worked closely with Amtrak’s leadership to obtain their buy-in. In all, our reports are well-regarded and it also makes it easier for us to obtain access to the data we need to do our jobs.
Most of you have never used data analytics. Most of you want to do performance audits – the way you have always done it in the past, don’t want to learn something new, and do not want to do anything that has to do with technology - makes you nervous – you want to run away from it
But we all have one thing in common - we all do lot of critical thinking for our work. We all have questioning mind and we all are trained to connect dots. Or, at least most of us. We all want to give our best performance and have a desire to find something meaningful in our audits and investigations.
I understand if you don’t want to use data analytics. Except for people in our team, no one in our office wants to learn ACL and do this work. But yet our office is successful in using data analytics in many of our audits and investigations because they all know who to go to for analyzing data. They all are trained to think about data first for their audits. They work hand-in-hand with my technical experts to analyze data. They act as a liaison between my team and the auditee. They understand the audit objective, talk with the auditee to understand the business processes, learn about what data is available and how it is used and translate that to my team member who will then help them answer their audit objective with the use of data analytics. What I am saying is that you don’t have to be a Data Analyst. Find someone in your organization and work with them for your audit needs. Use Excel to the fullest. Get Excel training if you have to for data analysis. Find someone who knows SQL.
Have Courage
Be Persistent
Have Faith
Be Creative
Be Persuasive – show quick wins
Data Reliability
Tie it to you financial system
Sometimes it is the best data source available – qualify your report
Assess how much of this data is unreliable – measure the error rate (in $ or in number of transactions) if possible – acceptable, if less than 5% - say that in your report but move forward with your analysis