• Save
Cutter Journal: Surfing the SOX wave thanks to CMMi ®, 2007
Upcoming SlideShare
Loading in...5

Cutter Journal: Surfing the SOX wave thanks to CMMi ®, 2007



How to link SOX and CMMI

How to link SOX and CMMI



Total Views
Views on SlideShare
Embed Views



1 Embed 4

http://www.linkedin.com 4


Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment

Cutter Journal: Surfing the SOX wave thanks to CMMi ®, 2007 Cutter Journal: Surfing the SOX wave thanks to CMMi ®, 2007 Document Transcript

  • Cutter The Journal of IT Journal Information Technology Management Vol. 20, No. 1 January 2007“The final area of debate is this: given the cost and the Sarbanes-Oxley: complexity of implementing SOX, at the end of the day, is What Have Companies Learned it worth it?” En Route to Compliance? — Robert N. Charette, Guest Editor SOX Stinks SOX Is Super SOX is an ongoing nightmare — a cost SOX is the small investor’s best friend. It creator and value destroyer. It needs radical may be costly to implement, but the cost change, if not outright repeal. is worth it to ensure trustworthy and transparent financial reporting. Opening Statement by Robert N. Charette . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Cut the SOX Clutter with IT Best Practices by Niel Nickolaisen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 The Voice of Experience: What One IT Executive Has Learned About SOX Compliance by Scott Stribrny . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12 Surfing the SOX Wave Thanks to CMMI by Laurent Janssens and Peter Leeson . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 Complying with Sarbanes-Oxley: Addressing the IT Issues and Risks by Mahesh Raisinghani and Bhuvan Unhelkar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
  • LIFE OF PI Surfing the SOX Wave Thanks to CMMI by Laurent Janssens and Peter Leeson In the wake of the Enron and WorldCom scandals, embracing the concept of process improvement (i.e., a new era of corporate governance has begun, focus- CMMI), while others were dragging behind and did not ing on accountability, responsibility, transparency, see why they needed to do this, for a variety of reasons. and ethical behavior. In the US, Congress passed the Then along came Sarbanes-Oxley. The requirement to Sarbanes-Oxley Act, the objective of which is to certify implement SOX meant that all areas were required to per- that corporate financial statements are reliable by plac- form a number of control and quality activities. This facil- ing increased personal responsibility on senior manage- itated the further implementation of CMMI-compliant ment and ensuring that their behavior matches the activities within the company and helped people under- responsibilities they have accepted. As IT supports stand the reasons behind some of the quality-related the business processes, IT is once again a major player activities they had been asked to implement. in the survival of the organization.1 IT management processes, through the IT general controls (ITGCs), In this article, we will explain first, at a high level, how must provide reasonable assurance that “undesired Sarbanes-Oxley was satisfied at Company X. Then we events will be prevented or detected.” will describe how its CMMI-based PI program facili- tated the SOX project and reduced its cost. We will also look at how the SOX project had positive impacts on COMPANY X the company’s PI program. The experiences we share in this article are based on As in any real-world case, not everything is running the implementation of CMMI, then SOX, within the as expected. So, later in the article, we will present some Belgian subsidiary of one of the largest European finan- suggestions for improvement. cial institutions. We will call this financial institution “Company X.” In addition to the usual challenges of process improvement (PI) and SOX initiatives, Company THE SOX STEPS X had to overcome the fact that the company was split The IT development department of Company X used a into several sites. Its people — many of whom had five-step approach for its SOX compliance project (see joined the company through a succession of mergers — Figure 1). spoke different languages, had different priorities, and worked in different businesses (including banking and 1. Risk Identification insurance). The difference in language and cultures throughout the company meant that some areas A risk is anything that can happen in a project to prevent were more willing to change than others. Some were you from reaching your objectives. In this context, we Process Risk documentation Evaluation Reporting External audit identification and maintenance Figure 1 — SOX steps. 1 IT has found itself in a similar position previously, with the e-commerce revolution, the introduction of the Euro, Y2K, and other such incidents.1 CUTTER IT JOURNAL January 2007 ©2007 Cutter Information LLC
  • were dealing with IT risks, which we identified using of the SOX documentation. The company must there-version 3.2 of COBIT, the international framework pub- fore keep track of the various versions of the controllished by the IT Governance Institute. In practice, we description — including their validity dates — becausetook all the control objectives of COBIT and documented this is important information for the external auditors.them with regard to the practices used at Company X to Consider the following example. Say you have CTRL-A,check for specific risks. We called this our “matrix of con- which is described as “obtain the vice president’strol activities.” For any gaps we detected, management approval of all project plans.” Then imagine that ondefined an action plan and tracked it on a regular basis. 1 June a new policy is enacted in which the departmentBefore official tests or walkthroughs, we submitted the head now approves the project plan if the project inmatrix results to our external auditor, one of the “big question is less than 500 person-days in duration.four” companies, to ensure that what we delivered at CTRL-A must now be changed, deployed, and com-the end of the SOX exercise would be near our external municated. When you do the SOX test in August, youauditor’s expectations. This resulted in a list of 23 will take a sample of projects started from 1 January tokey SOX controls to deploy to the entire organization 31 July. If you have not managed the validity date, your(around 450 people). The point here is that it is vital to tests will fail, as you will find that the VP has approvedkeep your external auditors informed as early as possi- plans for projects with fewer than 500 person-days. Thisble; otherwise, when they come for the walkthrough in small detail could be important.June or July, it may be too late!2. Process Documentation and Maintenance HOW CMMI HELPS SOXBased on this control matrix, we identified a number The IT development department of Company X is run-of controls as key. The key controls were clustered ning a CMMI-based PI program in order to improve thein management processes, and a process owner was quality of its development. In November 2005, it wasassigned accordingly. The process owner then provided appraised at CMMI Maturity Level 2. This means thata complete description of the control (who performs project management and control processes are system-the control, how frequently it is performed, etc.). atically implemented and respected throughout the IT development organization.2 The CMMI-compliant3. Evaluation processes also allowed a stabilization of customer requirements throughout the development lifecycle.After those key controls are communicated to all the Altran CIS (www.altran.com), an innovation consultingoperational teams, quality assurance (QA) assesses their firm located in Brussels, participated in both programs.efficiency through walkthroughs or compliance testing. Q:PIT Ltd (www.qpit.ltd.uk), a UK-based SEI partner, facilitated the change process.4. ReportingThe SOX coordinator communicates the results of the Organization Structuretests to senior management. This individual identifies One of the first steps in Company X’s PI program was togaps and defines a remediation plan for the QA coordi- define a structure with three independent departments,nators to follow, according to the priorities set by reporting directly to the CIO (see Figure 2):management. 1. The Software Engineering Process Group (SEPG) is5. External Audit responsible for the coherence of processes and their alignment with business goals and stakeholders’External auditors assess the effectiveness of those needs. The SEPG participates in the definition ofcontrols. pragmatic processes based on field experience.An Iterative Cycle 2. Operational teams apply the processes and the controls; they highlight improvement opportunitiesThis five-step approach is iterative. The gaps identified based on field experiences.during the evaluation process could lead to a redesign2An interesting by-product of the PI program was the reduced learning curve enjoyed by project managers who were completing theirPMP certifications.Get The Cutter Edge free: www.cutter.com Vol. 20, No. 1 CUTTER IT JOURNAL
  • 3. QA provides to management, operational teams, By clearly establishing the support of management and the SEPG an independent insight into the effec- and stakeholders in the PI program and considering tiveness and efficiency of the processes being used. Sarbanes-Oxley as a change request — albeit an impor- tant one — for the ongoing program, the cultural The CIO sets the objectives in the policy. The SEPG, like changes required on a short delay by SOX were imple- the other project teams, implements that policy through mented quite smoothly. And those required cultural the products it develops in order to respond to the cus- changes were considerable. The first audit (walk- tomers’ expressed needs. QA ensures that the policies, through) was planned at the end of June; our external strategies, processes, standards, and so on are being auditor had reviewed the 23 key controls at the end of correctly respected and implemented. Any deviations January. We had exactly six months to inform the proj- are reported and resolved as needed. ect teams, train the QA coordinators, organize the tests, Through this structure, roles and responsibilities are and perform the tests successfully. There were two phe- clearly defined. The SOX controls are embedded in the nomena that made the cultural changes most difficult: IT development management processes, and their coher- 1. Business partner intransigence. Business partners ence is ensured in all supporting documents (process asked us, “Why do I need to sign this document? description, trainings, etc.) as part of the quality checks This has to do with your internal controls; it’s not of the PI initiative. Moreover, SOX testing is done in a my problem.” When we began to implement CMMI, professional way by QA team members as part of their the business partners often considered it “overhead.” “business-as-usual” activities. As a result, the cost of Company X learned to live with this situation, but deploying and maintaining the SOX controls is largely when SOX arrived, there was no longer any choice. part of the PI program, and those controls are mainly If a test plan was not signed, the control would fail. institutionalized without significant additional effort. So the nature of the relationship between IT and the business partners changed: after all, we were now in IT Senior Management Commitment the same boat! Thanks to the generic practices required by CMMI for 2. Lack of discipline in evidence gathering. In the first a Maturity Level 2 rating, the major stakeholders of the weeks, we would say, “Show me the approval of the organization perceived the added value of the improve- test plan,” and we would hear, “Uh, I think it was ments. IT senior management established policies to approved in March. I’ll have a look at it.” Later on provide direction, state the need for improvement and they would say, “It got lost in the mail.” Afterwards, control, and define how the processes are to support once the SOX controls were well institutionalized, the business objectives. The necessary corresponding this became “Please find the test plan attached; embedded controls and reviews to be applied were also identified. CIO QA Execute Define Operational Test Software teams Engineering Process Group (SEPG) Figure 2 — The organization structure.3 CUTTER IT JOURNAL January 2007 ©2007 Cutter Information LLC
  • the approvals were done by e-mail, and they are are referenced in the control activities matrix and serve included.” as an important basis for the exhaustive SOX control documentation.Changing the culture for SOX requires, first of all, thatpeople take responsibility for their actions and deci- The IT management processes are fully documentedsions. They must also have the wherewithal to back and available for the whole development community.them up and to demonstrate that they know what The SEPG writes the processes, and for each one, athey are doing. Previously, the culture at Company X developer can refer to precise procedures, templates,focused more on blaming problems on other people or and so on. On the other hand, the SOX documentationcircumstances beyond one’s control. But as in dynamics, requirement is just a part of this complete process. Itit is easier to move a body that is in movement than one is crucial that the detailed process documentation andthat is at rest, because higher energy levels are needed the SOX documentation be completely aligned. Theto compensate for inertia. That energy had already been process owners themselves perform the verification ofused to get PI started. this alignment.It is also interesting to note that by defining the organi- Auditors (internal and external) have other require-zational structure and obtaining the commitment of IT ments for the process documentation, mainly for high-senior management, the control environment required lighting the key controls. When we were in the SOXby SOX is already partly present. Most of the pervasive control definition stage (with regular refinements andcontrols — controls designed to manage and monitor minor updates), management decided to maintain twothe IT environment — are operating efficiently thanks sets of documentation. Now that the SOX documenta-to CMMI. tion is stabilized, the SEPG has integrated the two sets in the IT management processes according to a definedControl Definition and Execution roadmap approved by our external auditor and follow- ing a well-defined process of deployment.COBIT defines 34 high-level control objectives, which aredivided into four domains: The main objective of the “development and mainte- nance” ITGC at Company X is to ensure that every item1. Plan and organize put in production is under control. Based on the risk2. Acquire and implement assessment, the IT development department defined3. Deliver and support 23 key controls in five processes, as shown in Figure 3. These 23 controls are common sense; there is no added4. Monitor and evaluate complexity, just enforced management processes.For Company X’s IT department, the most important Each of these SOX controls is linked to a phase of theCOBIT control objectives applied were identified as software development lifecycle. They are embedded inthose in the “plan and organize” and “acquire and the milestone review checklists, and (as with any otherimplement” domains. These are largely covered by major issue in the project or application) if the expectedthe CMMI practice areas. As IT management processes result of a control is not achieved, the next phase ofbased on CMMI practice areas are established, they the project may not be started. The CMMI states that Project management lifecycle IT governance Release process management Application management lifecycle Test process Figure 3 — Five ITGC processes.Get The Cutter Edge free: www.cutter.com Vol. 20, No. 1 CUTTER IT JOURNAL
  • conducting milestone reviews is part of monitoring from the field were “refreshed” on the roles everyone activities and is the responsibility of the project man- plays with regard to SOX. And we did it in a funny ager. Company X has institutionalized this milestone manner. Since the theme of the forum was around review process, demonstrating yet another way CMMI “SOX,” the invitation asked, “Which sox will you and SOX are interrelated. wear?” There was also a quiz in which, for each cor- rect answer, attendees received a sock with chocolate SOX Testing (Belgian, of course) inside. Through these communica- tion efforts, we rectified some bad perceived messages. Sarbanes-Oxley Section 404 states that the organization must report on the assessment of controls over financial reporting. This implies that the controls should be eval- POSITIVE IMPACTS OF SOX: uated in an independent way. Company X largely ful- SOME BENEFICIAL SIDE EFFECTS filled this requirement through ongoing and continuous “objective evaluation” of adherence to the processes, Implementing a CMMI-based process improvement plans, and standards by an authority that is identified effort is a serious challenge, even when you have the within the CMMI as responsible for “process and prod- full support of top management. First of all, it is not uct quality assurance.” As shown in the organization easy to instill the discipline required to transform a cul- structure (see Figure 2), QA team members are ideally ture in which project team members do what appears to placed to assess the SOX controls with regard to effi- be correct based on today’s pressures and priorities into cient operation. Once the training of QA has been a culture in which they understand, plan, document, adapted to include formal gathering of evidence and the and monitor their activities, even under pressure. The job assignments have been modified, QA performs SOX need to comply with Sarbanes-Oxley is a great motiva- testing in a business-as-usual mode. The overhead cost tor to respect the process, even in case of emergency. of SOX testing is thus kept low. Secondly, in order to stabilize project requirements, The more challenging point is to avoid a conflict situa- CMMI says that it is important to obtain stakeholders’ tion in the QA role. On one side, QA is a PI change commitment to the plans. For cultural/historical rea- facilitator, helping the teams reach the expected matu- sons, this was sometimes overlooked at Company X, rity in the defined processes. On the other side, as SOX with negative consequences as the projects progressed. tester, QA has a mandate to escalate SOX issues to Now, however, eight of the 23 SOX-defined ITGCs are senior IT management for direct action, which may lead related to obtaining formal approvals from stakeholders to stopping the project until issues are addressed. The during the different phases of a project. Thanks to SOX, change agent and the “SOX cop” roles are somewhat we were able to make business partners aware that difficult to combine. a formal sign-off is more than just bureaucracy and that SOX control failure in these cases is not only the Consider the history of QA at Company X. The QA responsibility of IT. function was created in 2004, and once the team was staffed (in 2005), the QA coordinator conducted Finally, three of the 23 SOX key controls are linked to two QA audits for training purposes. The project teams governance practices, enforcing management awareness perceived the QA staff as “police” who were always of the importance of maintaining the business case of saying, “You’re not doing this right,” “You are not OK a project. for this part of the process,” and the like. QA’s main role, however, was not to play the “enforcer” but to ONGOING CHALLENGES provide support for reaching CMMI Level 2 (and later, CMMI Level 3). QA staff were there to act as change Of course, there are limitations in the way things have facilitators, to inform the project teams of the good prac- been implemented in Company X. In this section, we tices used in other teams, to help in the deployment of would like to identify some of the challenges that are new processes, and so on. Eventually the project teams in the process of being managed or resolved. came to trust QA and to see the added value of this change facilitator role, which is an important success To Standardize, or Not to Standardize? factor in future PI steps. A CMMI Maturity Level 2 organization focuses on Now, with the advent of SOX, QA is again perceived project management activities and the stabilization as the auditor — as the “bad cop.” How did we tackle of project requirements and other practices. The idea this? By organizing voluntary IT forums where people behind Maturity Level 2 is to encourage projects and5 CUTTER IT JOURNAL January 2007 ©2007 Cutter Information LLC
  • teams to deliver, in their own way, according to theirown approaches, the results needed to deliver theproducts (e.g., actuals) and to measure and reportthose results (e.g., How do you measure the actuals?How do you report them?). Projects are encouragedto try different approaches within the context of thedetailed organizational policy (laying out manage-ment’s needs and expectations) and the required qualityand reliability controls.Naturally, the QA team, which needs to test the con-trols, would have an easier job finding the appropriateartifacts and evidence required if everyone did thingsthe same way. And as the organization progresses, bestpractices can be identified in a “bottom-up” way andthe knowledge shared and standardized across theboard. The point of this approach to standardization isto ensure that the organization does not blindly adoptan “ideal” approach invented by some theoretician ina university that does not correspond to the culture Figure 4 — COBIT IT governance focus areas.and needs of the customers or the management of thecompany. The sharing and standardization of best prac-tices is the focus of the CMMI Maturity Level 3, which to support ongoing continuous improvement, not toCompany X hopes to achieve by 2008.3 This should fur- guarantee levels of quality. SOX, on the other hand,ther reduce the cost of the SOX compliance. requires that audits be performed on a yearly basis. The level of quality achieved is a continuous requirement, toPI vs. SOX be respected at all times, even the week after the audit,SOX testing seeks to ensure that controls are operating even during the holidays. You must stay SOX-compliantefficiently. For example, SOX guarantees that the right from the first of January until the end of December!business representative signs off on the test plan, but itdoes not guarantee the quality of this test plan (in terms PI and SOX: Toward a Peaceful Coexistenceof effectiveness, completeness, and so on). SOX is there Company X has defined a roadmap for its improvementto limit the risks but not to improve the quality of the program, laying out in time the different initiatives byprocess or that of the product. That is the main differ- focusing on the benefits to be achieved. This roadmapence between SOX and a PI program. In the latter, qual- includes a number of improvements related to the busi-ity should be embedded not only in a way of working, ness needs and priorities, focusing first on known areasbut also as a kind of philosophy. In a PI program, you of “lesser strength”; then on overall consistency in thedo not produce quality because you must, but because processes, collaboration, and communication betweenyou “think” quality. In the case of SOX, you do it teams (internal and external); then on known weak-because you must be compliant! Quality and continuous nesses and continuous improvement.improvement are a mindset, while the SOX principlesare external audit-like requirements. Focusing on quality Areas of lesser strength are usually easier to correct.will ensure that audits are easier to pass, as the needs, These are typically things that are implemented andproducts, and controls are well defined to start with. understood, but not done systematically, or not done completely. By starting with correcting some of theCMMI does not offer certification. While a CMMI easier items (i.e., “picking the low-hanging fruit”), anappraisal’s “validity” is limited to three years, there is organization can make rapid and visible improvements.no requirement to perform a new appraisal or to main- This will encourage and motivate the participants, astain the results achieved previously. The model is there3The time Company X needs to move up another level is longer than for most organizations mainly because of the size of its IT depart-ment and the variance in the staffing. This is a company that has grown largely through acquisitions and mergers, combining a numberof different cultures, products, legacy systems, and locations, as well as working on a daily basis in three languages!Get The Cutter Edge free: www.cutter.com Vol. 20, No. 1 CUTTER IT JOURNAL
  • well as quickly free up some time and/or resources to While SOX is a legal requirement today, there are a focus on the more difficult areas. number of issues that remain open. Implementing the approach we have outlined above can assist companies Unfortunately, these successive improvements and with a number of problems, but it is not the solution to changes can seriously impact the results of SOX the problems that were the impetus for the law. The compliance efforts and the corresponding controls. As Enron scandal, to name one such problem, was largely a consequence, each new or improved process needs related to an illegal collaboration in masking data to go through a double SOX control. When the PI staff between the top management of the company and its wishes to start up a PI project, the SOX compliance spe- auditors. Since the implementation of SOX, this can only cialist performs an initial review. This specialist reviews now be done through an illegal collaboration in mask- the PI project’s summary and determines whether there ing data between top management and auditors! SOX is a known impact and whether it is large or small, or claims to place the responsibility for any fraud with the whether there is a potential impact. He or she might management; however, Enron’s management were rec- decide: ognized as responsible for the malfeasance without the This probably has no impact; go ahead with the law. The auditors, who were just as involved in the change without SOX expertise. scandal, were not convicted; instead, SOX rewards them This may have an impact, and I would like to review by throwing them even more auditing business! and approve any products, processes, and templates The CMMI approach is focused on changing the culture before they are put into production. of SOX compliance toward a quality-based approach This probably has an impact, and I want to directly that involves everyone and ensures that quality-based, participate on the team that is researching and docu- independent audits are carried out efficiently. This arti- menting the change. cle has offered some experience with this combination and demonstrated its advantages, but it cannot answer Even if the SOX compliance specialist decides not to the more fundamental questions about the ultimate participate, all staff members have been trained in the value of SOX. importance and principles of SOX and will be on the lookout for potential risks. If any are uncovered, they are then identified and reported for review. ADDITIONAL READING Leeson, Peter. CMMI, SOX, and COBIT. Q:PIT, 14 June 2006 (www.qpit.ltd.uk/LinkedDocuments/CMMI- CONCLUSION SOX-COBIT%2040.pdf). This is the first year that Company X is required to Laurent Janssens, CISA, is a Senior Consultant at Altran CIS undergo the complete SOX exercise. The results of (Consulting and Information Service) in Belgium, where he is its first audit were encouraging. The PI program the leader of the IT Governance practice. Mr. Janssens has 12 has enabled the company to determine the need for years’ experience in the IT management and IT audit world. He improvement and to plan the path for improvement coordinated all SOX testing–related matters at the IT department in all aspects of the development and management of a leading financial organization. Mr. Janssens can be reached at processes. The sharing of practices and lessons learned ljanssens@altran-cis.be. throughout the organization has allowed Company X to Peter Leeson of Q:PIT Ltd is a CMMI Appraiser and Instructor and a significantly decrease the cost of the activities, increas- Visiting Scientist with the Software Engineering Institute. He assisted ing productivity and reducing the time wasted. In the with the implementation of CMMI-compliant processes that satisfy same manner, the cost of SOX compliance has also been and facilitate the business objectives of the organization being dis- significantly reduced through the systematic implemen- cussed in this article. Mr. Leeson can be reached at Peter@qpit.ltd.uk. tation of processes and the related controls.7 CUTTER IT JOURNAL January 2007 ©2007 Cutter Information LLC