Zaborowski Sixsigma Nov07

XBT Refunds
Six Sigma Project
Final Presentation

Mary Ann Zaborowski
November 15, 2007

Problem Statement:
CBT appointment cancellations have resulted in 56,842 partial
refunds being issued totaling $3.7 million and representing 3% of
total registrations from Jan ’05 – April ’06.

Goal:
Reduce partial refunds as percent of total registrations by 50% to
an average 1-2% per program

2

Confidential and Proprietary. Copyright ©2006 Educational Testing Service.

Y metric # Cancellations B/4 Refundable Date
# Tests Taken + # NS + # Customer Cancellations

CBT Partial Refunds as % of Total Appointments
Appointments = Test Taken + Partial Refunds
Percent
Note: Includes TOEFL, GRE and Praxis CBT
0.056
Refunds / Total Appointments

n = 3,140,655 1
1

0.044
1 1
1
1
1
0.032 UCL=0.03120
_
X=0.02188
0.020

LCL=0.01256
1 1 1
Jan-04 Apr-04 Jul-04 Oct-04 Jan-05 Apr-05 Jul-05 Oct-05 Jan-06 Apr-06
Date

1
Refunds / Total Appointments

0.012
UCL=0.01145

0.008

__
0.004
MR=0.00350

0.000 LCL=0

Jan-04 Apr-04 Jul-04 Oct-04 Jan-05 Apr-05 Jul-05 Oct-05 Jan-06 Apr-06
Date

Although refunds have averaged 2% (Jan 04-Apr 06),
3
From Sept 05-Apr 06 the average increased to 4%

Financial Impact
Hypothesis

Scenarios and Assumptions

• Scenarios evaluated
– Best Case: 100% of customer cancels in 3-X days forfeit
their registration
– Worst Case: 100% of customers take test as originally
planned
– Most Likely: Alternate candidate action taken as follows

GRE TOEFL Praxis
Probabilities
Reschedule 20% 20% 20%
Forfeit entire fee 25% 25% 25%
Cancel earlier than 7 days prior to admin 45% 45% 45%
Take test anyway as originally scheduled 5% 5% 10%
Not register for the test in the first place 0% 0% 0%
Register later and pay the late fee 5% 5% 0%
100% 100% 100%

5


Most Likely Analysis (TOEFL / GRE)
12-Month Impact at Current Volumes

GRE TOEFL Total TOEFL - Low Total - Low
5-DAY SCENARIO
Base Scenario (3 Day Policy)
ETS Net $ 175,560 $ 421,920 $ 597,480 $ 36,000 $ 211,560
5 Day Policy Scenario
ETS Net $ 240,065 $ 561,681 $ 801,746 $ 47,925 $ 287,990
Change in Net $ 64,505 $ 139,761 $ 204,266 $ 11,925 $ 76,430

7-DAY SCENARIO
ETS Net $ 331,980 $ 772,200 $ 1,104,180 $ 61,200 $ 393,180
ETS Net $ 452,700 $ 1,029,600 $ 1,482,300 $ 81,600 $ 534,300
Change in Net $ 120,720 $ 257,400 $ 378,120 $ 20,400 $ 141,120

10-DAY SCENARIO
Preferred ETS Net $ 486,420 $ 1,022,400 $ 1,508,820 $ 158,400 $ 644,820
Option ETS Net $ 663,300 $ 1,363,200 $ 2,026,500 $ 211,200 $ 874,500
Change in Net $ 176,880 $ 340,800 $ 517,680 $ 52,800 $ 229,680

TOEFL volume differences range from 5,000 – 12,000
6

cancellations between 5-day to 10-day scenarios

Approved Plan
• Change refund policy for GRE and TOEFL effective
with the new academic year (July 2007)

However, due to volume trends and competitive pressure from IELTS,
7
Global Division postponed changing the TOEFL refund policy as planned

So what happened with
GRE Program?

Since the GRE Refund Policy changed (7/07) the percent of
registrations with a partial refunds have declined
GRE CBT Partial Refunds as % of Total Appointments
Before After
12.00% n=697,202 n=217,201
Individual V alue

10.00%

8.00% U C L=8.28%

_
6.00% X=5.63%

4.00%
LC L=2.97%
Jan-06 M ar-06 M ay -06 Jul-06 S ep-06 N ov -06 Jan-07 M ar-07 M ay -07 Jul-07 S ep-07
Date

Before After
4.00%
M oving Range

U C L=3.27%
3.00%

2.00%
__
1.00% M R=1.00%

0.00% LC L=0.00%
Date

9
Before: X=7.59% After:
X=5.63%

Correspondingly the number of cancellations with
no-refund as a percent of registrations has increased
GRE Cancel No Refunds as % of Total Appointments
Before A fter
n=1,376 U C L=1.09%
1.00% n=1,161
Individual V alue

0.75% _
X=0.65%
0.50%

0.25% LC L=0.21%

0.00%

Date

Before A fter
0.60%
U C L=0.54%
0.45%
M oving Range

0.30%
__
0.15% M R=0.17%

0.00% LC L=0.00%
Date

10
Before: X=0.21% After:
X=0.65%

GRE Cancellations by Status
Before/After Policy Change
Cancel Status - Before Cancel Status - After
Jul 07-Oct 07
Jan 06-Jun 07
12000 100 100
2500

# Appointment Cancellations
# Appointment Cancellations

10000
80 2000 80

8000
60 60

Percent

Percent
1500
6000
40 1000 40
4000

20 500 20
2000

0 0 0 0
Cancel Status Partial Refund No Refund Cancel Status Partial Refund No Refund
Before 10459 1311 After 1413 1160
Percent 88.9 11.1 Percent 54.9 45.1
Cum % 88.9 100.0 Cum % 54.9 100.0

The percentage of appointment that are cancelled with
11

no-refund has increased from 11% to 45%

GRE Refund Policy Before
and After Process Change
BEFORE AFTER CHANGE
Time Period Jan06–Jun07 Jul07+ ----

# Refunds Avoided/Mth 86 290 204

Net Savings/Month $ 4,816 $ 16,240 $ 11,424

Annual Savings $57,792 $194,880 $137,088

Since policy was changed the net change has been $34K
12

Annual net change is projected at ~$140K

So what happened with
TOEFL Program?

TOEFL Program Refund
Policy Change
• Although TOEFL program did commit to
change their refund policy consistent with
GRE this decision was later reversed
– A result of the “TOEFL Go Anywhere from Here”
campaign for market share
– 3-day cancellation with no-refund policy remained
in effect 07-08 academic year
• This policy change was deferred for
implementation until Jul 08

14


GRE Refunds Control Plan
• Monitor appointment cancellations and partial refunds
issued through data warehouse query for next six
months
– Closely review candidate behavior by days before test day
• If new signal emerges GRE Program staff will identify
the root causes and any corrective action steps that
will be taken

15


Lessons Learned
• Data on money flow processes such as registration,
after-market, and cancellations should be routinely
monitored to provide early warning of potential
margin loss and revenue erosion
• There is strong resistance to adjusting policies
among most program staff
– Data is a powerful tool to address tribal knowledge
• We need to balance our policies with competitors

16


Zaborowski Sixsigma Nov07

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (6)

Recently uploaded

Recently uploaded (20)

Zaborowski Sixsigma Nov07