1. Milliman White Paper
February 2016
As the actuarial world moves toward a more strategic, value-added approach to
modeling—rather than the ad-hoc approaches of the past—technology becomes a
critical enabler both in standardizing the inner workings of models and in automating
the repeatable processes necessary to establish the modeling discipline of the future.
However, balance across all three elements of the modeling environment—technology,
process, and people—is necessary for maximum effectiveness.
As the industry changes, the way people and processes interact with
governance changes as well. In the past, individual responsibility
for models was the norm. A single actuary might be in charge of
processes from gathering data, developing assumptions, and loading
data to designing the model, configuring the model, producing
output, and analyzing results. Today, these responsibilities are
becoming the realms of specialists in sync with larger processes.
New, different people are becoming more involved and have more
interest in the modeling environment; processes transcend functional
units and ripple across more areas of the organization.
Developing and implementing a model governance framework in this
new era requires special attention to the softer aspects of people,
collaboration, and organizational change. Other papers have been
written on the topic of model governance: what should be governed,
how it should be governed, etc. This paper focuses instead on the
human aspect of building and implementing a new model governance
framework and how these softer aspects can be critical determinants
of model governance success.
THE POWER OF BUY-IN
As more people take interest in the success of the overall modeling
effort, it is essential that stakeholders come along for the governance
ride. Those who do not have the opportunity to provide input or be
a part of designing a new model governance framework may reject
the new paradigm as too complex and too time consuming. It is
important to give them formal opportunities to contribute ideas and
be included in designing, developing, and implementing the solution.
This creates a sense of “buy-in.” Because they have had the
opportunity to shape its creation, they are more willing to embrace
the solution—not least because, ideally, their input has been
incorporated in such a way as to create the most harmony with
established ways of working. Instead of being told what to do,
people will grow to embrace model governance because it helps
them do their work more effectively.
Buy-in from the beginning: Defining the change
The first task—before any actual change happens—is to form
consensus around the definition and fundamentals of model
governance, which fall into two main areas: policies and procedures,
and the operating model itself. Some aspects will be based on
industry standards, while others will be unique to the organization.
But the importance of establishing and using a common language as
soon as possible is difficult to overstate. Policies and procedures to
be considered at this stage may include:
§§ Standard model-development process
§§ Change-management procedures
§§ Coding, testing, and documentation standards
§§ Architectural standards
§§ Appropriate use guidelines
§§ Standards for reviewing and peer reviewing
Within the operating model, one might include:
§§ Organizational design
§§ Roles and responsibilities
§§ Approval processes
§§ Frameworks for making and executing decisions
§§ Communication channels
This is the time to become closely connected to key constituents in
functional units, teams, or areas. Understanding current processes
and practices—including what is working well and what is not—is
essential because it enables a focus on commonalities rather than
conflict and provides a starting point for developing solutions that will
dovetail naturally with what already exists wherever possible.
Developing and implementing model governance:
Collaboration and buy-in as drivers of successful change
Brian Fomby, FSA, MAAA
2. Milliman White Paper
February 2016Developing and implementing model governance:
Collaboration and buy-in as drivers of successful change
2
When introducing or exposing a new model governance framework,
it is important to provide only the level of detail necessary for the
given audience, which can be difficult for actuaries leading the
process who may be accustomed to high levels of granularity. Many
participants will be intimidated or turned off by too much detail at the
beginning, and this gives them less room for input. It can be better to
begin with the “bones” of the plan and build consensus as the details
are fleshed out. Easy-to-consume communications that focus on the
visual—such as high-level diagrams of processes—can be valuable in
reaching non-specialist audiences. The details can come later.
Communication should also be established early between those
responsible for model governance at the line of business level and
those responsible for enterprise risk management. In many cases,
the two teams will be pursuing model governance simultaneously,
and if they do not communicate early and often, duplicate or even
conflicting efforts can result. Model risk management and model
governance overlap significantly and can be undertaken more
efficiently as part of a coordinated effort.
Away we go: Flexible implementation and the
effectiveness of an Agile process
Once it is time to begin implementing the protocols and procedures
that actually comprise model governance, it can be tempting to put
the emphasis on control rather than collaboration, on perfection rather
than continuous learning. While this may provide the appearance
of reducing risk, it can actually increase risk by reducing valuable
experimentation. Rapid prototyping, on the other hand, can enable
participants to see and feel how a revised approach is different and
how it can add value—and enables them to shape the new process or
procedure. Accepting and learning from small failures in implementing
model governance along the way can prevent larger failures that have
higher cost—and that are more likely when a big change is rolled out
to constituents without their input along the way.
When implementing new model governance and during the actual
modeling activities themselves, it can be helpful to use a flexible
approach that has more in common with contemporary software
development than heavy-handed organization-change procedures
of the past. Known formally as the Agile methodology, this approach
can be an effective way to simplify participation with stakeholders.
Features of Agile that can support improved collaboration include:
§§ Regular peer reviews in real-time, short, practical “sprints” rather
than long, infrequent, information-packed meetings.
§§ Spreading work among more people, which increases the
number of eyes on a given element to reduce risk and moves the
process forward instead of getting stuck in bottlenecks caused by
individuals’ changing workloads, schedules, or priorities.
§§ More emphasis on delivering value over endless planning. This
often results in experimentation and rapid prototyping rather than
feeling a need for perfection in the first iteration. This type of “fail
fast” mindset will naturally surface potential issues sooner rather
than later.
§§ Breaking documentation (critical in light of ever-changing
regulations and modeling environments) into manageable
deliverables that are completed along the way rather than all
at once at the end. This not only makes documentation more
manageable but also improves quality by capturing more details to
provide greater value to future users and auditors.
§§ The identification and execution of quick wins to engender support
and build enthusiasm. By celebrating these successes, positive
momentum will be built.
SIMPLIFY, SIMPLIFY, SIMPLIFY
In order to create a positive user experience, every element of the
model governance framework should be made as simple as possible
while still fulfilling its purpose. Technology is a critical enabler of
simplicity, making governance easier for people to manage and
easier to understand and comply. With automation of modeling
processes and workflows, participants need not worry as much
about remembering or enforcing each step, tracking every document,
or ensuring that signoffs are properly obtained. Technology can help
enforce segregation of duties with role-based access to specific
functionality. If it is properly designed, it can reduce the amount of
time required to train people and help them understand how they are
part of the greater whole. They can simply get on with their jobs.
Simplicity also requires restraint on the part of those in charge of
creating the model governance framework. Once things get rolling,
it can be tempting to throw every control and procedure imaginable
into the mix to reduce risk to an absolute minimum. However, if the
model governance framework becomes onerous, it will not be used
effectively, which can actually create more problems than it solves.
Emphasizing collaboration and communication helps to right-size the
process to fit the need.
TAKING STOCK: MEASURING SUCCESS IN SYNC
As a new model governance framework is implemented, those with
oversight will want to evaluate or measure success. Measuring the
benefits of model governance can be a challenge. The benefits
can be ambiguous and difficult to quantify. One place to look for
benchmarks is the pre-governance environment, but this does
require some forethought in the sense that statistics and data
must be captured before the new framework is in place. These
measurements could include:
§§ How many mistakes, errors, or misstatements had to be corrected
in the prior year?
§§ How long did model changes and enhancements take?
§§ How many peer reviews were executed for new models or model
changes?
§§ How much time was spent troubleshooting, maintaining, and
executing models versus analyzing results and deciphering the
story around the numbers?
