How to link SOX and CMMI

1. Cutter The Journal of
IT Journal Information Technology Management
Vol. 20, No. 1
January 2007
“The final area of debate is
this: given the cost and the
Sarbanes-Oxley:
complexity of implementing
SOX, at the end of the day, is
What Have Companies Learned
it worth it?” En Route to Compliance?
— Robert N. Charette,
Guest Editor SOX Stinks SOX Is Super
SOX is an ongoing nightmare — a cost SOX is the small investor’s best friend. It
creator and value destroyer. It needs radical may be costly to implement, but the cost
change, if not outright repeal. is worth it to ensure trustworthy and
transparent financial reporting.
Opening Statement
by Robert N. Charette . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Cut the SOX Clutter with IT Best Practices
by Niel Nickolaisen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
The Voice of Experience: What One IT Executive Has Learned
About SOX Compliance
by Scott Stribrny . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Surfing the SOX Wave Thanks to CMMI
by Laurent Janssens and Peter Leeson . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Complying with Sarbanes-Oxley: Addressing the IT Issues and Risks
by Mahesh Raisinghani and Bhuvan Unhelkar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

2. LIFE OF PI
Surfing the SOX Wave Thanks to CMMI
by Laurent Janssens and Peter Leeson
In the wake of the Enron and WorldCom scandals, embracing the concept of process improvement (i.e.,
a new era of corporate governance has begun, focus- CMMI), while others were dragging behind and did not
ing on accountability, responsibility, transparency, see why they needed to do this, for a variety of reasons.
and ethical behavior. In the US, Congress passed the
Then along came Sarbanes-Oxley. The requirement to
Sarbanes-Oxley Act, the objective of which is to certify
implement SOX meant that all areas were required to per-
that corporate financial statements are reliable by plac-
form a number of control and quality activities. This facil-
ing increased personal responsibility on senior manage-
itated the further implementation of CMMI-compliant
ment and ensuring that their behavior matches the
activities within the company and helped people under-
responsibilities they have accepted. As IT supports
stand the reasons behind some of the quality-related
the business processes, IT is once again a major player
activities they had been asked to implement.
in the survival of the organization.1 IT management
processes, through the IT general controls (ITGCs), In this article, we will explain first, at a high level, how
must provide reasonable assurance that “undesired Sarbanes-Oxley was satisfied at Company X. Then we
events will be prevented or detected.” will describe how its CMMI-based PI program facili-
tated the SOX project and reduced its cost. We will also
look at how the SOX project had positive impacts on
COMPANY X the company’s PI program.
The experiences we share in this article are based on As in any real-world case, not everything is running
the implementation of CMMI, then SOX, within the as expected. So, later in the article, we will present some
Belgian subsidiary of one of the largest European finan- suggestions for improvement.
cial institutions. We will call this financial institution
“Company X.” In addition to the usual challenges of
process improvement (PI) and SOX initiatives, Company THE SOX STEPS
X had to overcome the fact that the company was split The IT development department of Company X used a
into several sites. Its people — many of whom had five-step approach for its SOX compliance project (see
joined the company through a succession of mergers — Figure 1).
spoke different languages, had different priorities, and
worked in different businesses (including banking and 1. Risk Identification
insurance). The difference in language and cultures
throughout the company meant that some areas A risk is anything that can happen in a project to prevent
were more willing to change than others. Some were you from reaching your objectives. In this context, we
Process
Risk
documentation Evaluation Reporting External audit
identification
and maintenance
Figure 1 — SOX steps.
1
IT has found itself in a similar position previously, with the e-commerce revolution, the introduction of the Euro, Y2K, and other such
incidents.
1 CUTTER IT JOURNAL January 2007 ©2007 Cutter Information LLC

3. were dealing with IT risks, which we identified using of the SOX documentation. The company must there-
version 3.2 of COBIT, the international framework pub- fore keep track of the various versions of the control
lished by the IT Governance Institute. In practice, we description — including their validity dates — because
took all the control objectives of COBIT and documented this is important information for the external auditors.
them with regard to the practices used at Company X to
Consider the following example. Say you have CTRL-A,
check for specific risks. We called this our “matrix of con-
which is described as “obtain the vice president’s
trol activities.” For any gaps we detected, management
approval of all project plans.” Then imagine that on
defined an action plan and tracked it on a regular basis.
1 June a new policy is enacted in which the department
Before official tests or walkthroughs, we submitted the head now approves the project plan if the project in
matrix results to our external auditor, one of the “big question is less than 500 person-days in duration.
four” companies, to ensure that what we delivered at CTRL-A must now be changed, deployed, and com-
the end of the SOX exercise would be near our external municated. When you do the SOX test in August, you
auditor’s expectations. This resulted in a list of 23 will take a sample of projects started from 1 January to
key SOX controls to deploy to the entire organization 31 July. If you have not managed the validity date, your
(around 450 people). The point here is that it is vital to tests will fail, as you will find that the VP has approved
keep your external auditors informed as early as possi- plans for projects with fewer than 500 person-days. This
ble; otherwise, when they come for the walkthrough in small detail could be important.
June or July, it may be too late!
2. Process Documentation and Maintenance HOW CMMI HELPS SOX
Based on this control matrix, we identified a number The IT development department of Company X is run-
of controls as key. The key controls were clustered ning a CMMI-based PI program in order to improve the
in management processes, and a process owner was quality of its development. In November 2005, it was
assigned accordingly. The process owner then provided appraised at CMMI Maturity Level 2. This means that
a complete description of the control (who performs project management and control processes are system-
the control, how frequently it is performed, etc.). atically implemented and respected throughout the
IT development organization.2 The CMMI-compliant
3. Evaluation processes also allowed a stabilization of customer
requirements throughout the development lifecycle.
After those key controls are communicated to all the
Altran CIS (www.altran.com), an innovation consulting
operational teams, quality assurance (QA) assesses their
firm located in Brussels, participated in both programs.
efficiency through walkthroughs or compliance testing.
Q:PIT Ltd (www.qpit.ltd.uk), a UK-based SEI partner,
facilitated the change process.
4. Reporting
The SOX coordinator communicates the results of the Organization Structure
tests to senior management. This individual identifies
One of the first steps in Company X’s PI program was to
gaps and defines a remediation plan for the QA coordi-
define a structure with three independent departments,
nators to follow, according to the priorities set by
reporting directly to the CIO (see Figure 2):
management.
1. The Software Engineering Process Group (SEPG) is
5. External Audit responsible for the coherence of processes and their
alignment with business goals and stakeholders’
External auditors assess the effectiveness of those
needs. The SEPG participates in the definition of
controls.
pragmatic processes based on field experience.
An Iterative Cycle 2. Operational teams apply the processes and the
controls; they highlight improvement opportunities
This five-step approach is iterative. The gaps identified
based on field experiences.
during the evaluation process could lead to a redesign
2
An interesting by-product of the PI program was the reduced learning curve enjoyed by project managers who were completing their
PMP certifications.
Get The Cutter Edge free: www.cutter.com Vol. 20, No. 1 CUTTER IT JOURNAL

4. 3. QA provides to management, operational teams, By clearly establishing the support of management
and the SEPG an independent insight into the effec- and stakeholders in the PI program and considering
tiveness and efficiency of the processes being used. Sarbanes-Oxley as a change request — albeit an impor-
tant one — for the ongoing program, the cultural
The CIO sets the objectives in the policy. The SEPG, like
changes required on a short delay by SOX were imple-
the other project teams, implements that policy through
mented quite smoothly. And those required cultural
the products it develops in order to respond to the cus-
changes were considerable. The first audit (walk-
tomers’ expressed needs. QA ensures that the policies,
through) was planned at the end of June; our external
strategies, processes, standards, and so on are being
auditor had reviewed the 23 key controls at the end of
correctly respected and implemented. Any deviations
January. We had exactly six months to inform the proj-
are reported and resolved as needed.
ect teams, train the QA coordinators, organize the tests,
Through this structure, roles and responsibilities are and perform the tests successfully. There were two phe-
clearly defined. The SOX controls are embedded in the nomena that made the cultural changes most difficult:
IT development management processes, and their coher-
1. Business partner intransigence. Business partners
ence is ensured in all supporting documents (process
asked us, “Why do I need to sign this document?
description, trainings, etc.) as part of the quality checks
This has to do with your internal controls; it’s not
of the PI initiative. Moreover, SOX testing is done in a
my problem.” When we began to implement CMMI,
professional way by QA team members as part of their
the business partners often considered it “overhead.”
“business-as-usual” activities. As a result, the cost of
Company X learned to live with this situation, but
deploying and maintaining the SOX controls is largely
when SOX arrived, there was no longer any choice.
part of the PI program, and those controls are mainly
If a test plan was not signed, the control would fail.
institutionalized without significant additional effort.
So the nature of the relationship between IT and the
business partners changed: after all, we were now in
IT Senior Management Commitment
the same boat!
Thanks to the generic practices required by CMMI for
2. Lack of discipline in evidence gathering. In the first
a Maturity Level 2 rating, the major stakeholders of the
weeks, we would say, “Show me the approval of the
organization perceived the added value of the improve-
test plan,” and we would hear, “Uh, I think it was
ments. IT senior management established policies to
approved in March. I’ll have a look at it.” Later on
provide direction, state the need for improvement and
they would say, “It got lost in the mail.” Afterwards,
control, and define how the processes are to support
once the SOX controls were well institutionalized,
the business objectives. The necessary corresponding
this became “Please find the test plan attached;
embedded controls and reviews to be applied were
also identified.
CIO
QA
Execute Define
Operational Test Software
teams Engineering
Process Group
(SEPG)
Figure 2 — The organization structure.
3 CUTTER IT JOURNAL January 2007 ©2007 Cutter Information LLC

5. the approvals were done by e-mail, and they are are referenced in the control activities matrix and serve
included.” as an important basis for the exhaustive SOX control
documentation.
Changing the culture for SOX requires, first of all, that
people take responsibility for their actions and deci- The IT management processes are fully documented
sions. They must also have the wherewithal to back and available for the whole development community.
them up and to demonstrate that they know what The SEPG writes the processes, and for each one, a
they are doing. Previously, the culture at Company X developer can refer to precise procedures, templates,
focused more on blaming problems on other people or and so on. On the other hand, the SOX documentation
circumstances beyond one’s control. But as in dynamics, requirement is just a part of this complete process. It
it is easier to move a body that is in movement than one is crucial that the detailed process documentation and
that is at rest, because higher energy levels are needed the SOX documentation be completely aligned. The
to compensate for inertia. That energy had already been process owners themselves perform the verification of
used to get PI started. this alignment.
It is also interesting to note that by defining the organi- Auditors (internal and external) have other require-
zational structure and obtaining the commitment of IT ments for the process documentation, mainly for high-
senior management, the control environment required lighting the key controls. When we were in the SOX
by SOX is already partly present. Most of the pervasive control definition stage (with regular refinements and
controls — controls designed to manage and monitor minor updates), management decided to maintain two
the IT environment — are operating efficiently thanks sets of documentation. Now that the SOX documenta-
to CMMI. tion is stabilized, the SEPG has integrated the two sets
in the IT management processes according to a defined
Control Definition and Execution roadmap approved by our external auditor and follow-
ing a well-defined process of deployment.
COBIT defines 34 high-level control objectives, which are
divided into four domains: The main objective of the “development and mainte-
nance” ITGC at Company X is to ensure that every item
1. Plan and organize
put in production is under control. Based on the risk
2. Acquire and implement assessment, the IT development department defined
3. Deliver and support 23 key controls in five processes, as shown in Figure 3.
These 23 controls are common sense; there is no added
4. Monitor and evaluate complexity, just enforced management processes.
For Company X’s IT department, the most important Each of these SOX controls is linked to a phase of the
COBIT control objectives applied were identified as software development lifecycle. They are embedded in
those in the “plan and organize” and “acquire and the milestone review checklists, and (as with any other
implement” domains. These are largely covered by major issue in the project or application) if the expected
the CMMI practice areas. As IT management processes result of a control is not achieved, the next phase of
based on CMMI practice areas are established, they the project may not be started. The CMMI states that
Project management
lifecycle
IT governance Release
process management
Application
management lifecycle
Test process
Figure 3 — Five ITGC processes.
Get The Cutter Edge free: www.cutter.com Vol. 20, No. 1 CUTTER IT JOURNAL

6. conducting milestone reviews is part of monitoring from the field were “refreshed” on the roles everyone
activities and is the responsibility of the project man- plays with regard to SOX. And we did it in a funny
ager. Company X has institutionalized this milestone manner. Since the theme of the forum was around
review process, demonstrating yet another way CMMI “SOX,” the invitation asked, “Which sox will you
and SOX are interrelated. wear?” There was also a quiz in which, for each cor-
rect answer, attendees received a sock with chocolate
SOX Testing (Belgian, of course) inside. Through these communica-
tion efforts, we rectified some bad perceived messages.
Sarbanes-Oxley Section 404 states that the organization
must report on the assessment of controls over financial
reporting. This implies that the controls should be eval- POSITIVE IMPACTS OF SOX:
uated in an independent way. Company X largely ful- SOME BENEFICIAL SIDE EFFECTS
filled this requirement through ongoing and continuous
“objective evaluation” of adherence to the processes, Implementing a CMMI-based process improvement
plans, and standards by an authority that is identified effort is a serious challenge, even when you have the
within the CMMI as responsible for “process and prod- full support of top management. First of all, it is not
uct quality assurance.” As shown in the organization easy to instill the discipline required to transform a cul-
structure (see Figure 2), QA team members are ideally ture in which project team members do what appears to
placed to assess the SOX controls with regard to effi- be correct based on today’s pressures and priorities into
cient operation. Once the training of QA has been a culture in which they understand, plan, document,
adapted to include formal gathering of evidence and the and monitor their activities, even under pressure. The
job assignments have been modified, QA performs SOX need to comply with Sarbanes-Oxley is a great motiva-
testing in a business-as-usual mode. The overhead cost tor to respect the process, even in case of emergency.
of SOX testing is thus kept low. Secondly, in order to stabilize project requirements,
The more challenging point is to avoid a conflict situa- CMMI says that it is important to obtain stakeholders’
tion in the QA role. On one side, QA is a PI change commitment to the plans. For cultural/historical rea-
facilitator, helping the teams reach the expected matu- sons, this was sometimes overlooked at Company X,
rity in the defined processes. On the other side, as SOX with negative consequences as the projects progressed.
tester, QA has a mandate to escalate SOX issues to Now, however, eight of the 23 SOX-defined ITGCs are
senior IT management for direct action, which may lead related to obtaining formal approvals from stakeholders
to stopping the project until issues are addressed. The during the different phases of a project. Thanks to SOX,
change agent and the “SOX cop” roles are somewhat we were able to make business partners aware that
difficult to combine. a formal sign-off is more than just bureaucracy and
that SOX control failure in these cases is not only the
Consider the history of QA at Company X. The QA responsibility of IT.
function was created in 2004, and once the team was
staffed (in 2005), the QA coordinator conducted Finally, three of the 23 SOX key controls are linked to
two QA audits for training purposes. The project teams governance practices, enforcing management awareness
perceived the QA staff as “police” who were always of the importance of maintaining the business case of
saying, “You’re not doing this right,” “You are not OK a project.
for this part of the process,” and the like. QA’s main
role, however, was not to play the “enforcer” but to
ONGOING CHALLENGES
provide support for reaching CMMI Level 2 (and later,
CMMI Level 3). QA staff were there to act as change Of course, there are limitations in the way things have
facilitators, to inform the project teams of the good prac- been implemented in Company X. In this section, we
tices used in other teams, to help in the deployment of would like to identify some of the challenges that are
new processes, and so on. Eventually the project teams in the process of being managed or resolved.
came to trust QA and to see the added value of this
change facilitator role, which is an important success To Standardize, or Not to Standardize?
factor in future PI steps. A CMMI Maturity Level 2 organization focuses on
Now, with the advent of SOX, QA is again perceived project management activities and the stabilization
as the auditor — as the “bad cop.” How did we tackle of project requirements and other practices. The idea
this? By organizing voluntary IT forums where people behind Maturity Level 2 is to encourage projects and
5 CUTTER IT JOURNAL January 2007 ©2007 Cutter Information LLC

7. teams to deliver, in their own way, according to their
own approaches, the results needed to deliver the
products (e.g., actuals) and to measure and report
those results (e.g., How do you measure the actuals?
How do you report them?). Projects are encouraged
to try different approaches within the context of the
detailed organizational policy (laying out manage-
ment’s needs and expectations) and the required quality
and reliability controls.
Naturally, the QA team, which needs to test the con-
trols, would have an easier job finding the appropriate
artifacts and evidence required if everyone did things
the same way. And as the organization progresses, best
practices can be identified in a “bottom-up” way and
the knowledge shared and standardized across the
board. The point of this approach to standardization is
to ensure that the organization does not blindly adopt
an “ideal” approach invented by some theoretician in
a university that does not correspond to the culture Figure 4 — COBIT IT governance focus areas.
and needs of the customers or the management of the
company. The sharing and standardization of best prac-
tices is the focus of the CMMI Maturity Level 3, which
to support ongoing continuous improvement, not to
Company X hopes to achieve by 2008.3 This should fur-
guarantee levels of quality. SOX, on the other hand,
ther reduce the cost of the SOX compliance.
requires that audits be performed on a yearly basis. The
level of quality achieved is a continuous requirement, to
PI vs. SOX
be respected at all times, even the week after the audit,
SOX testing seeks to ensure that controls are operating even during the holidays. You must stay SOX-compliant
efficiently. For example, SOX guarantees that the right from the first of January until the end of December!
business representative signs off on the test plan, but it
does not guarantee the quality of this test plan (in terms PI and SOX: Toward a Peaceful Coexistence
of effectiveness, completeness, and so on). SOX is there
Company X has defined a roadmap for its improvement
to limit the risks but not to improve the quality of the
program, laying out in time the different initiatives by
process or that of the product. That is the main differ-
focusing on the benefits to be achieved. This roadmap
ence between SOX and a PI program. In the latter, qual-
includes a number of improvements related to the busi-
ity should be embedded not only in a way of working,
ness needs and priorities, focusing first on known areas
but also as a kind of philosophy. In a PI program, you
of “lesser strength”; then on overall consistency in the
do not produce quality because you must, but because
processes, collaboration, and communication between
you “think” quality. In the case of SOX, you do it
teams (internal and external); then on known weak-
because you must be compliant! Quality and continuous
nesses and continuous improvement.
improvement are a mindset, while the SOX principles
are external audit-like requirements. Focusing on quality Areas of lesser strength are usually easier to correct.
will ensure that audits are easier to pass, as the needs, These are typically things that are implemented and
products, and controls are well defined to start with. understood, but not done systematically, or not done
completely. By starting with correcting some of the
CMMI does not offer certification. While a CMMI
easier items (i.e., “picking the low-hanging fruit”), an
appraisal’s “validity” is limited to three years, there is
organization can make rapid and visible improvements.
no requirement to perform a new appraisal or to main-
This will encourage and motivate the participants, as
tain the results achieved previously. The model is there
3
The time Company X needs to move up another level is longer than for most organizations mainly because of the size of its IT depart-
ment and the variance in the staffing. This is a company that has grown largely through acquisitions and mergers, combining a number
of different cultures, products, legacy systems, and locations, as well as working on a daily basis in three languages!
Get The Cutter Edge free: www.cutter.com Vol. 20, No. 1 CUTTER IT JOURNAL

8. well as quickly free up some time and/or resources to While SOX is a legal requirement today, there are a
focus on the more difficult areas. number of issues that remain open. Implementing the
approach we have outlined above can assist companies
Unfortunately, these successive improvements and
with a number of problems, but it is not the solution to
changes can seriously impact the results of SOX
the problems that were the impetus for the law. The
compliance efforts and the corresponding controls. As
Enron scandal, to name one such problem, was largely
a consequence, each new or improved process needs
related to an illegal collaboration in masking data
to go through a double SOX control. When the PI staff
between the top management of the company and its
wishes to start up a PI project, the SOX compliance spe-
auditors. Since the implementation of SOX, this can only
cialist performs an initial review. This specialist reviews
now be done through an illegal collaboration in mask-
the PI project’s summary and determines whether there
ing data between top management and auditors! SOX
is a known impact and whether it is large or small, or
claims to place the responsibility for any fraud with the
whether there is a potential impact. He or she might
management; however, Enron’s management were rec-
decide:
ognized as responsible for the malfeasance without the
This probably has no impact; go ahead with the law. The auditors, who were just as involved in the
change without SOX expertise. scandal, were not convicted; instead, SOX rewards them
This may have an impact, and I would like to review by throwing them even more auditing business!
and approve any products, processes, and templates The CMMI approach is focused on changing the culture
before they are put into production. of SOX compliance toward a quality-based approach
This probably has an impact, and I want to directly that involves everyone and ensures that quality-based,
participate on the team that is researching and docu- independent audits are carried out efficiently. This arti-
menting the change. cle has offered some experience with this combination
and demonstrated its advantages, but it cannot answer
Even if the SOX compliance specialist decides not to the more fundamental questions about the ultimate
participate, all staff members have been trained in the value of SOX.
importance and principles of SOX and will be on the
lookout for potential risks. If any are uncovered, they
are then identified and reported for review. ADDITIONAL READING
Leeson, Peter. CMMI, SOX, and COBIT. Q:PIT, 14 June
2006 (www.qpit.ltd.uk/LinkedDocuments/CMMI-
CONCLUSION SOX-COBIT%2040.pdf).
This is the first year that Company X is required to
Laurent Janssens, CISA, is a Senior Consultant at Altran CIS
undergo the complete SOX exercise. The results of
(Consulting and Information Service) in Belgium, where he is
its first audit were encouraging. The PI program
the leader of the IT Governance practice. Mr. Janssens has 12
has enabled the company to determine the need for years’ experience in the IT management and IT audit world. He
improvement and to plan the path for improvement coordinated all SOX testing–related matters at the IT department
in all aspects of the development and management of a leading financial organization. Mr. Janssens can be reached at
processes. The sharing of practices and lessons learned ljanssens@altran-cis.be.
throughout the organization has allowed Company X to Peter Leeson of Q:PIT Ltd is a CMMI Appraiser and Instructor and a
significantly decrease the cost of the activities, increas- Visiting Scientist with the Software Engineering Institute. He assisted
ing productivity and reducing the time wasted. In the with the implementation of CMMI-compliant processes that satisfy
same manner, the cost of SOX compliance has also been and facilitate the business objectives of the organization being dis-
significantly reduced through the systematic implemen- cussed in this article. Mr. Leeson can be reached at Peter@qpit.ltd.uk.
tation of processes and the related controls.
7 CUTTER IT JOURNAL January 2007 ©2007 Cutter Information LLC