Fall 2013 Process Monitoring project tracking Audit cycle times for a company before and after a process change.

1. Audit Report Cycle
Time Duration Study
Process Monitoring - Pharmaceuticals 5629 (101) – Fall 2013
Neil Norton, Chuck Kemmerer, Karen Otrupchak

2. Objectives
• To identify potential root cause for
Audit to Final Audit Report cycle
time to approach the 42-day limit
• To identify sources of variation
• To determine potential actions to
create fixes - reduce cycle time

3. Process Flow
Open audit Template
Document audit from notes and observations
Route report to management/audit team for
review and comment
Edit report adding comment and corrections
requested
Print, sign, and scan report
Post report and observations into Audit tracking
Issue report to supplier

4. Inputs and Outputs
Inputs (X’s)
• Responsibility: Auditing Group/Department
• Audit Date: Date the audit started
• Lead Auditor: Code letters given for each lead auditor
• Audit Team Size: Number of team members
• Observations: Numbers of Critical, Major, and Minor
observations found
• Supplier Type: Type of facility being audited (API, Commodity,
Device, lab, etc.)
• Change Control: Routine vs. non-Routine
• Audit Process: “Stage” of the Process. (process
changed/streamlined for Routine Audits April2013
• Audit Duration: Number of days spent at a site
• Report Issue Date: Date final report was issued
Output (Y)
• Report Cycle Time: Time from Audit start date to Report Issued
(Limits - be less than 42 days)

5. Data Collected*
Audit
Responsibility
Lead
Auditor
Audit
Team
Size
Audit
Date
Audit
Duratio
n
(days)
Report
Issue
Date
Report
Cycle
Time
(Days)
Critical
Obs
Major
Obs
Minor
Obs
Supplier
TYPE
* Data compiled from 105 audits over a duration of 12 months
Change
Control
History
Global Audit EU B 1 09/05/12 2 09/21/12 16 0 0 3 API Routine
Global Audit
09/11/12
EXC
Americas E 1
1 10/08/12 27 0 2 4
Routine
Global Audit
Americas E 2
09/12/12
2 10/03/12 21 0 0 0
Lab
Routine
Global Audit EU B 3 09/18/12 1 09/27/12 9 0 6 8 TPM Routine
Global Audit EU D 7 09/19/12 2 10/19/12 30 1 4 4 Device Routine
Global Audit EU D 1 09/20/12 1 09/27/12 7 0 1 0 EXC Routine
Global Audit
09/20/12
Device
Americas A 1
1 10/25/12 35 0 0 0
Routine
Global Audit
Americas E 2
09/26/12
2 10/19/12 23 0 1 0
TPM
Routine
Global Audit
Americas G 3
09/28/12
2 10/30/12 32 0 4 4
TPM
Routine
…
…
…

6. I-Chart - Report Cycle Times
JMP
Note: Special Test (6) - 4
out of 5 points more than
1 standard
deviation from center line
(on one side of CL) at
points: 25, 27

7. Dot Plot – Report Cycle Times
Minitab

8. Lead Auditor
Pareto Plot
110
100
90
80
70
60
50
40
30
20
10
0
Count
E A B C D F G H J K L
Lead Audi tor
100
90
80
70
60
50
40
30
20
10
0
Cum Percent
Plots
30
25
20
15
10
5
0
Number of Audits
20
17
16
10
26
6 6
1 1 1 1
A B C D E F G H J K L
Lead Audi tor
Bar Chart
JMP
Note: Auditor E conducted
the most number of audits

9. Lead Auditor (ANOVA)
JMP
Oneway Analysis of Report Cycle Time (Days) By Lead Auditor
45
40
35
30
25
20
15
10
5
0
Report Cycle
Time (Days)
A B C D E F GHJKL
Lead Audi tor
Oneway Anova
Rsquare
Adj Rsquare
Root Mean Square Erro r
Mean of Response
Observations (o r Sum Wgts)
0.372156
0.305364
8.098001
21.19048
105
Summary of Fit
Source
Lead Audi tor
Error
C. Total
DF
10
94
104
Sum of
Squares
3653.8941
6164.2964
9818.1905
Mean Square
365.389
65.578
F Ratio
5.5719
Prob > F
<.0001*
Analysis of Variance
Means for Oneway Anova
Note: Auditor E had the
shortest overall mean turn
around time for report

10. Bar Chart: Counts of total
observations
Minitab

11. Total Observations (ANOVA)
Minitab
One-way ANOVA: Report Cycle Time (Days) versus Total Observations
Source DF SS MS F P
Total Observations 14 1246.7 89.1 0.94 0.525
Error 90 8571.5 95.2
Total 104 9818.2
S = 9.759 R-Sq = 12.70% R-Sq(adj) = 0.00%

12. Change in Process
Open audit Template
Open audit Template
Routine
Audit?
Document audit from notes and observations
Routine audit report
template
Non-Routine audit
report template
Route report to management/audit team for
review and comment
Document audit from notes and observations

13. Change in Process
Minitab

14. 2-Sample t-test by audit stage
Two-Sample T-Test and CI: Report Cycle Time (Days), Audit Process
Two-sample T for Report Cycle Time (Days)
Audit
Process N Mean StDev SE Mean
1 35 23.5 10.7 1.8
2 40 20.15 9.05 1.4
Difference = mu (1) - mu (2)
Estimate for difference: 3.34
95% CI for difference: (-1.26, 7.93)
T-Test of difference = 0 (vs not =): T-Value = 1.45
P-Value = 0.152 DF = 67
Based on this data, the change in routine audits had no
statistically significant effect

15. Initial IVP: Cycle time vs. Audit
Responsibility

16. Audit Responsibility- After Process
Change
Minitab
Note: one “outlier”
removed from
Global Audit Americas

17. Routine Audit Report Cycle Time
Comparison of Data- Audit Responsibility after change
Audit
Responsibilit
y
# of Audits Mean
Cycle Time
(days)
Std. Dev SE Mean
Americas 21 15.71 7.85 1.7
PAA 5 19.20 7.33 2.0
EU 14 27.19 6.34 2.8
T-Test Results
Comparison DF P-Value Stat. Sig.
Diff
Americas
vs. EU
29 0.000 YES
Americas
vs. PAA
7 0.328 NO
EU vs. PAA 8 0.050 YES

18. Conclusions
 Auditor E has conducted the most audits and has
the lowest average turn-around time
 There was no factor that could be linked to high
cycle time
 Initial analysis of the change in audit process
didn’t appear to show any statistically significant
change in cycle time
 Further analysis showed that certain groups
weren’t following the process change.
 There was a statistically significant improvement
in cycle time for those groups using the new form
versus those that weren’t