Avoiding the material weakness: Case studies in developing effective controls. Originally presented by Melanie Dunn and Mark Spong at the 2018 Valuation Actuary Symposium.

1. Session 22PD:
Avoiding the material weakness:
Case studies in developing effective
controls
MELANIE DUNN, FSA, MAAA
MARK SPONG, FSA, CERA, MAAA
August 27, 2018

2. What do we mean by deficiencies and weaknesses?
A material weakness is a deficiency, or a combination of deficiencies, in
internal control over financial reporting, such that there is a reasonable
possibility that a material misstatement of the company's annual or interim
financial statements will not be prevented or detected on a timely basis
A significant deficiency is a single weakness or a combination of
weaknesses in the internal controls associated with financial reporting, that is
less severe than a material control weakness and yet is sufficient to merit the
scrutiny of those responsible for administering an entity's financial reporting
*Source: Auditing Standard No. 5, Public Company Accounting Oversight Board

3. Material weaknesses may be more common than you think
Internet Retail Company Reports Material Weakness:
Second Control Deficiency in Three Years
Audit Analytics
Global Retailer finds
‘material weakness’ in
controls over
accounting leases
Reuters
Insurance Company
Announces GAAP
Restatement
Business Insider
Leading Life Insurer
Discloses Its Second
Material Weakness
This Year
Bloomberg
Large Insurer shares
fall 10% on ‘material
weakness’ warning
Financial Times
School Districts get
financial
accountability
grades
Business Insider
A Red Flag
on Auto
Company
New York Times
Aerospace company says
numbers are unreliable
due to control weakness
MarketWatch
Insurer
Stock
Tanking
Today After
Finding
‘Material
Weakness’
The Street

4. Annual assumption review
Historical examples of material weakness triggers
Discovery of a material financial misstatement
often indicates a weakness in underlying
controls on financial reporting.
Material misstatements
For example, in 2015, an insurer reported a
material restatement to 2013 financials due to
an error in the 2013 annual assumption review.
In March of 2018, another insurer disclosed a
material weakness after reserves on a VA block
were determined to be too high and released.
Releasing excess reserves
In 2015, a third insurer disclosed insufficient
controls on implementation of methodology and
assumption changes for LTC claim reserves.
Controls on methodology changes

5. Market Reputation Financial
Remediation
effort
Stock price
drop
Lack of trust Costs of
remediation
Strategic priorities
must be shifted
How would you, your team, and your department be affected by a
material weakness?
Consequences of a material weakness
Morale
Positive outlook
eroded

6. Material weakness
describes the control
environment, not the
accuracy of financial
statements

7. Common Pitfalls – Case Studies & Discussion

8. Spreadsheet SNAFU
“This isn’t a model, it just organizes the results. The
governance standards for models would be overkill!”
• The valuation model works flawlessly with top notch
controls
• Results are dumped into Excel, transformed into usable
form, and aggregated with other products via links and
macros
• But business day 8 comes along:
– A last minute update to the process is not flowing
through correctly
– New products are not captured
– Balances are transposed
During the normal course of quarter close, management is
not able to prevent misstatements on a timely basis.
1
WHAT • What are the control standards at your
company for End User Computing
applications, such as Excel?
2
• Models are usually defined as having input,
processing and output components. Why
does End User Computing tend to fall under
the radar?
WHY
3
• How can actuaries structure and design
controls to address the underlying issue?
HOW
End User Computing Hand-offsAssumptions
Model StakeholdersData Emergency Protocol

9. Assumption Malfunction
“We update lapse and mortality every year and have a clear
oversight process. Otherassumptionsstillseemappropriate.”
• The assumption inventory for a critical high risk model
appears to be complete and annually reviewed
• The model is complex and certain assumptions
associated with mean reversion are not well understood
by the assumption review committee
• As a result, there is a lower degree of scrutiny on those
assumptions plus lack of scrutiny on implicit
assumptions
• The result is economic simulations that are not reflective
of the prolonged low interest rate environment
The assumption review process was not designed to place
sufficient scrutiny on technical aspects of modeling design.
1
2
3
• Why might controls around assumption
management be challenging to keep up?
• How can actuaries structure and design
controls to address the underlying issue?
WHAT • What do the assumption review and update
process look like at your company?
WHY
HOW
End User Computing Hand-offsAssumptions
Model StakeholdersData Emergency Protocol

10. Hand-off Hardships
“We are the model owners and they are the model users”
• A domestic actuary is the “model owner” for a model and
operation is outsourced to a “model user” in another country
• Hand-offs of model updates and review occur over email, since
the model owner and model user don’t work the same hours
• While attributing impacts between quarters, the model owner
discovers an error from incorrectly mapping new issues during
routine updates in Q2
• The model user hadn’t been educated on model governance
requirements, and didn’t know what level of review was
required for the mapping updates
• The model owner didn’t know that the mapping updates had
been made, and didn’t review them in Q2
Hand-offs have increased risk, and effective controls execution
requires clear standards for communication when processes and
data get handed off.
1
WHAT • What does a hand-off look like during
quarter end at your company?
2
• Why is just emailing someone a model
with quarter close updates a problem?
WHY
3
• How can actuaries structure and design
controls to address the underlying
issue?
HOW
End User Computing Hand-offsAssumptions
Model StakeholdersData Emergency Protocol

11. Data Disaster
“Of course we do reasonability checks on the inforce data,
but we don’t have time to completely audit it”
• The admin system is dropping a small number of
policies each quarter in the inforce data feed
• Data controls are focused on quarter over quarter
changes of counts and face amounts, so nothing stands
out
• After 18-24 months reserve balances are significantly
off
The existing control was operating as designed but still did
not meet the objective and reserves are misstated.
1
WHAT • What data quality checks would you
realistically expect to routinely run on
inforce files?
2
• Big changes from period to period are
noticeable. Why might small changes like
this still constitute a significant deficiency?
WHY
3
• How can actuaries structure and design
controls to address the underlying issue?
HOW
End User Computing Hand-offsAssumptions
Model StakeholdersData Emergency Protocol

12. Modeling Mishap
“I’m just relying on what the pricing team provided”
• After pricing a new product, the pricing team hands off
the pricing model to valuation
• Valuation independently defines business requirements
for the inforce model and determines whether any
features not modeled for pricing need to be modeled
• Risk and modeling teams are not involved
• The risk team just checks the results at the end
• Since pricing decisions are made before valuation,
modeling, and risk become involved, those stakeholders
do not have input into the decisions
This may result in modeling and risk teams that do not have
the practical authority to design and perform effective
controls.
1
WHEN • How early do the risk, modeling, and
valuation teams get involved in the pricing
process at your company?
2
• Why might the pricing team have more
influence within the organization?
WHY
3
• How can actuaries coordinate between
teams to address the underlying issue?
HOW
End User Computing Hand-offsAssumptions
Model StakeholdersData Emergency Protocol

13. Error Emergency
“We continuously keep track of modeling issues and address
them as soon as possible”
• An analyst finds a potential coding error in a production
model on business day 3, two days before results are booked
• There is limited time to thoroughly investigate to confirm the
error or to assess its materiality
• Multiple team members work late and the root cause
appears to be identified and reasonable to address
• A fix is implemented just in time and the impact on financials
is attributed to a methodology enhancement
• After quarter close, it was discovered that the change had
unintended consequences for related products
No emergency protocol was in place to guide action when an
issue was found during quarter close.
1
WHAT • What is the emergency procedure at your
company if an issue is found during the
quarter close process? Is it a formalized or
informal procedure?
2
• Why is it a problem to rely on a judgment
call from management when an issue pops
up during quarter close?
WHY
3
• How can actuaries structure and design
controls to address the underlying issue?
HOW
End User Computing Hand-offsAssumptions
Model StakeholdersData Emergency Protocol

14. End User Computing1
Assumptions2
Recap
Model Stakeholders5
Emergency protocol6
Hand-offs3
Data4