3. were dealing with IT risks, which we identified using of the SOX documentation. The company must there-
version 3.2 of COBIT, the international framework pub- fore keep track of the various versions of the control
lished by the IT Governance Institute. In practice, we description — including their validity dates — because
took all the control objectives of COBIT and documented this is important information for the external auditors.
them with regard to the practices used at Company X to
Consider the following example. Say you have CTRL-A,
check for specific risks. We called this our “matrix of con-
which is described as “obtain the vice president’s
trol activities.” For any gaps we detected, management
approval of all project plans.” Then imagine that on
defined an action plan and tracked it on a regular basis.
1 June a new policy is enacted in which the department
Before official tests or walkthroughs, we submitted the head now approves the project plan if the project in
matrix results to our external auditor, one of the “big question is less than 500 person-days in duration.
four” companies, to ensure that what we delivered at CTRL-A must now be changed, deployed, and com-
the end of the SOX exercise would be near our external municated. When you do the SOX test in August, you
auditor’s expectations. This resulted in a list of 23 will take a sample of projects started from 1 January to
key SOX controls to deploy to the entire organization 31 July. If you have not managed the validity date, your
(around 450 people). The point here is that it is vital to tests will fail, as you will find that the VP has approved
keep your external auditors informed as early as possi- plans for projects with fewer than 500 person-days. This
ble; otherwise, when they come for the walkthrough in small detail could be important.
June or July, it may be too late!
2. Process Documentation and Maintenance HOW CMMI HELPS SOX
Based on this control matrix, we identified a number The IT development department of Company X is run-
of controls as key. The key controls were clustered ning a CMMI-based PI program in order to improve the
in management processes, and a process owner was quality of its development. In November 2005, it was
assigned accordingly. The process owner then provided appraised at CMMI Maturity Level 2. This means that
a complete description of the control (who performs project management and control processes are system-
the control, how frequently it is performed, etc.). atically implemented and respected throughout the
IT development organization.2 The CMMI-compliant
3. Evaluation processes also allowed a stabilization of customer
requirements throughout the development lifecycle.
After those key controls are communicated to all the
Altran CIS (www.altran.com), an innovation consulting
operational teams, quality assurance (QA) assesses their
firm located in Brussels, participated in both programs.
efficiency through walkthroughs or compliance testing.
Q:PIT Ltd (www.qpit.ltd.uk), a UK-based SEI partner,
facilitated the change process.
4. Reporting
The SOX coordinator communicates the results of the Organization Structure
tests to senior management. This individual identifies
One of the first steps in Company X’s PI program was to
gaps and defines a remediation plan for the QA coordi-
define a structure with three independent departments,
nators to follow, according to the priorities set by
reporting directly to the CIO (see Figure 2):
management.
1. The Software Engineering Process Group (SEPG) is
5. External Audit responsible for the coherence of processes and their
alignment with business goals and stakeholders’
External auditors assess the effectiveness of those
needs. The SEPG participates in the definition of
controls.
pragmatic processes based on field experience.
An Iterative Cycle 2. Operational teams apply the processes and the
controls; they highlight improvement opportunities
This five-step approach is iterative. The gaps identified
based on field experiences.
during the evaluation process could lead to a redesign
2
An interesting by-product of the PI program was the reduced learning curve enjoyed by project managers who were completing their
PMP certifications.
Get The Cutter Edge free: www.cutter.com Vol. 20, No. 1 CUTTER IT JOURNAL
5. the approvals were done by e-mail, and they are are referenced in the control activities matrix and serve
included.” as an important basis for the exhaustive SOX control
documentation.
Changing the culture for SOX requires, first of all, that
people take responsibility for their actions and deci- The IT management processes are fully documented
sions. They must also have the wherewithal to back and available for the whole development community.
them up and to demonstrate that they know what The SEPG writes the processes, and for each one, a
they are doing. Previously, the culture at Company X developer can refer to precise procedures, templates,
focused more on blaming problems on other people or and so on. On the other hand, the SOX documentation
circumstances beyond one’s control. But as in dynamics, requirement is just a part of this complete process. It
it is easier to move a body that is in movement than one is crucial that the detailed process documentation and
that is at rest, because higher energy levels are needed the SOX documentation be completely aligned. The
to compensate for inertia. That energy had already been process owners themselves perform the verification of
used to get PI started. this alignment.
It is also interesting to note that by defining the organi- Auditors (internal and external) have other require-
zational structure and obtaining the commitment of IT ments for the process documentation, mainly for high-
senior management, the control environment required lighting the key controls. When we were in the SOX
by SOX is already partly present. Most of the pervasive control definition stage (with regular refinements and
controls — controls designed to manage and monitor minor updates), management decided to maintain two
the IT environment — are operating efficiently thanks sets of documentation. Now that the SOX documenta-
to CMMI. tion is stabilized, the SEPG has integrated the two sets
in the IT management processes according to a defined
Control Definition and Execution roadmap approved by our external auditor and follow-
ing a well-defined process of deployment.
COBIT defines 34 high-level control objectives, which are
divided into four domains: The main objective of the “development and mainte-
nance” ITGC at Company X is to ensure that every item
1. Plan and organize
put in production is under control. Based on the risk
2. Acquire and implement assessment, the IT development department defined
3. Deliver and support 23 key controls in five processes, as shown in Figure 3.
These 23 controls are common sense; there is no added
4. Monitor and evaluate complexity, just enforced management processes.
For Company X’s IT department, the most important Each of these SOX controls is linked to a phase of the
COBIT control objectives applied were identified as software development lifecycle. They are embedded in
those in the “plan and organize” and “acquire and the milestone review checklists, and (as with any other
implement” domains. These are largely covered by major issue in the project or application) if the expected
the CMMI practice areas. As IT management processes result of a control is not achieved, the next phase of
based on CMMI practice areas are established, they the project may not be started. The CMMI states that
Project management
lifecycle
IT governance Release
process management
Application
management lifecycle
Test process
Figure 3 — Five ITGC processes.
Get The Cutter Edge free: www.cutter.com Vol. 20, No. 1 CUTTER IT JOURNAL
7. teams to deliver, in their own way, according to their
own approaches, the results needed to deliver the
products (e.g., actuals) and to measure and report
those results (e.g., How do you measure the actuals?
How do you report them?). Projects are encouraged
to try different approaches within the context of the
detailed organizational policy (laying out manage-
ment’s needs and expectations) and the required quality
and reliability controls.
Naturally, the QA team, which needs to test the con-
trols, would have an easier job finding the appropriate
artifacts and evidence required if everyone did things
the same way. And as the organization progresses, best
practices can be identified in a “bottom-up” way and
the knowledge shared and standardized across the
board. The point of this approach to standardization is
to ensure that the organization does not blindly adopt
an “ideal” approach invented by some theoretician in
a university that does not correspond to the culture Figure 4 — COBIT IT governance focus areas.
and needs of the customers or the management of the
company. The sharing and standardization of best prac-
tices is the focus of the CMMI Maturity Level 3, which
to support ongoing continuous improvement, not to
Company X hopes to achieve by 2008.3 This should fur-
guarantee levels of quality. SOX, on the other hand,
ther reduce the cost of the SOX compliance.
requires that audits be performed on a yearly basis. The
level of quality achieved is a continuous requirement, to
PI vs. SOX
be respected at all times, even the week after the audit,
SOX testing seeks to ensure that controls are operating even during the holidays. You must stay SOX-compliant
efficiently. For example, SOX guarantees that the right from the first of January until the end of December!
business representative signs off on the test plan, but it
does not guarantee the quality of this test plan (in terms PI and SOX: Toward a Peaceful Coexistence
of effectiveness, completeness, and so on). SOX is there
Company X has defined a roadmap for its improvement
to limit the risks but not to improve the quality of the
program, laying out in time the different initiatives by
process or that of the product. That is the main differ-
focusing on the benefits to be achieved. This roadmap
ence between SOX and a PI program. In the latter, qual-
includes a number of improvements related to the busi-
ity should be embedded not only in a way of working,
ness needs and priorities, focusing first on known areas
but also as a kind of philosophy. In a PI program, you
of “lesser strength”; then on overall consistency in the
do not produce quality because you must, but because
processes, collaboration, and communication between
you “think” quality. In the case of SOX, you do it
teams (internal and external); then on known weak-
because you must be compliant! Quality and continuous
nesses and continuous improvement.
improvement are a mindset, while the SOX principles
are external audit-like requirements. Focusing on quality Areas of lesser strength are usually easier to correct.
will ensure that audits are easier to pass, as the needs, These are typically things that are implemented and
products, and controls are well defined to start with. understood, but not done systematically, or not done
completely. By starting with correcting some of the
CMMI does not offer certification. While a CMMI
easier items (i.e., “picking the low-hanging fruit”), an
appraisal’s “validity” is limited to three years, there is
organization can make rapid and visible improvements.
no requirement to perform a new appraisal or to main-
This will encourage and motivate the participants, as
tain the results achieved previously. The model is there
3
The time Company X needs to move up another level is longer than for most organizations mainly because of the size of its IT depart-
ment and the variance in the staffing. This is a company that has grown largely through acquisitions and mergers, combining a number
of different cultures, products, legacy systems, and locations, as well as working on a daily basis in three languages!
Get The Cutter Edge free: www.cutter.com Vol. 20, No. 1 CUTTER IT JOURNAL