Properly Mobilizing the PCI Resistance: Lessons Learned From Fighting Prior Wars (SOX-404)" I have noticed that there is a growing wave of discontent and disenchantment from information security and compliance practitioners around the PCI DSS. Josh Corman has been an effective voice for these concerns, providing an intellectually honest and earnest analysis in his talk “Is PCI The No Child Left Behind Act For Infosec?” The problem are well-known and significant: too much ambiguity in the PCI DSS, Qualified Security Assessors (QSAs) and consultant using subjective interpretations, existing guidance either too prescriptive or too vague, scope missing critical systems that could risk cardholder data, overly broad scope and excessive testing costs, excessive subjectivity and inconsistency, poor use of scarce resources, no meaningful reduction in risk of data breaches, and so forth. For years, I have been studying the PCI DSS compliance problem, as well. I have noticed many similarities to the PCI compliance challenges and the “SOX-404 Is The Biggest IT Time Waster” wars in 2005. I was part of the leadership team at the Institute of Internal Auditors (IIA) where we did something about the it. We identified inability to accurately scope the IT portions of SOX-404 as the root cause of the billions of dollars of wasted time and effort, while not reducing the risk of financial misstatements. I propose to present the two-year success story of the IIA GAIT project and how we changed the state of the IT audit practice in support of SOX-404 financial reporting audits. We defined the four GAIT Principles, which could be used to correctly scope the IT portions of SOX-404. We mobilized over 100K internal auditors, the SEC and PCAOB regulatory and enforcement bodies, as well as the external auditors from the 8 big CPA firms (e.g, Big Four and other firms doing SOX advisory work). In short, we made a difference, in a highly political process that involved many constituencies. I am attempting to do something similar with the PCI Security Standards Council, through my work as part one of the leaders of the PCI Scoping SIG (Special Interest Group). My personal goal is to find a “third way” to better enable correct scoping of the PCI Cardholder Data Environment, and create a risk-based approach of substantiating the effective controls to ensure that cardholder data breaches can be prevented, and quickly detected and corrected when they do occur. My desired outcome is to find fellow travelers who also see the pile of dead bodies in PCI compliance efforts, and work with those practitioners to catalyze a similar movement to achieve the spirit and intent of PCI DSS.

1. Mobilizing The PCI Resistance:Lessons Learned From Previous Wars (SOX-404) Gene Kim, CISACTO, Tripwire@realgenekim, http://www.realgenekim.me#BSidesLV 2010

3. Problem Definition Success of any PCI DSS compliance initiative is very dependent on accurate definition and scoping of the Cardholder Data Environment. There is a wide variance in practice, experience and guidance in merchant and QSA community. These contribute to scoping errors that result in: Overly narrow scope that jeopardizes cardholder data Overly broad scope that adds unnecessary cost and effort for compliance Decreased confidence in and frustration with the PCI DSS standard

4. What This Really Means Incredible amount of discontent and growing disenchantment with PCI DSS Complaints that DSS is too specific or too vague Like Michelle Klinger, I have a love/hate relationship with PCI DSS The reach of PCI DSS is awesomely breathtaking, and is relevant to all PII But in the worst case, it's a total waste of time, at enormous cost to the organization

5. Agenda Describe the problems around SOX-404 What we did about it at the Institute of Internal Auditors The GAIT concepts, politics, tools and outcomes Show how we can use this as a model to change the state of the practice around PCI DSS Share with you the best formulation of the plan I have Get your help improving the plan And ideally… Share my biggest a-ha moments the GAIT experience Excite you enough to do something about it Tell you some interesting stories

6. Holy Crap. This Looks Familiar!

7. The Problem The IT portions of SOX-404 compliance has frustrated auditors and management Significant key controls reside inside IT and IT processes as well as in the business processes No well-established guidance for scoping IT work results in inconsistency and the process being overly subjective Sometimes result in overly broad scope and excessive testing costs Significant risks to financial assertions may be left unaddressed Suboptimal use of scarce resources

8. Why Is There A Problem? No clear guidance exists to define how IT processes and activities can invalidate financial application processing or financial assertions COSO provides an accepted construct for defining overall internal control objectives, assertions, risks and controls, but its application to the IT environmet is ambiguous COBIT doesn’t provide a clear mechanism to scope IT processes and controls to the achievement of specific internal control objectives (e.g., COSO objective for internal control over financial reporting) Something else is needed…

9. “OMG. 952 IT Deficiencies?!?”

10. Vision: Create Equivalence to Nine Firm Document on IT Control Exceptions GAIT takes the approach used in the nine firm document.GAIT represents the upfront scoping exercise to appropriately identify the IT controls work relevant to overall internal controls objectives Chart 3: Evaluating Information Technology General Control (ITGC) Deficiencies, “A Framework for Evaluating Control Exceptions and Deficiencies” (December 20, 2004)

11. What were/are people worried about? Holy cow!!! Enron wasn’t caused by a DBA. So, why are the auditors digging here?? --gk IT controls dominate the deficiencies, significant deficiencies, and material weaknesses identified through the S-O 404 assessment. The estimated percentage of deficiencies identified show IT controls accounting for the most (34 percent), followed distantly by revenue (13 percent), procure to pay (10 percent), and fixed assets (10 percent). The estimated percentage of significant deficiencies identified again shows IT controls leading the way (23 percent), followed by financial reporting and close (14 percent), procure to pay (13 percent), and revenue (12 percent). The estimated percentages of material weaknesses identified include IT controls (27 percent), revenue (18 percent), taxes (11 percent), and financial reporting and close (10 percent). It is important to note that the results presented here are based on self-reporting by the companies that participated in the survey. Conclusions may be affected by the differing methods companies use to report on various elements of Sarbanes-Oxley compliance.

13. Compliance effort.

14. Deficiencies.

15. Non Finance Apps.

16. Financial Statement Impact:

17. Indirect linkage

18. Least likely impact

20. Thought Experiment Auditors vs. Management We can agree that there are two extremes in spectrum of financial reporting risk eBay auction settlement business process Grain elevators Extremes are easy… Middle is hard…

21. PCI Scoping Exercises (Show Your Work!) Question 1: Is the Cardholder Data Environment (CDE) equivalent to the PCI Scope of Assessment? Question 2: Is a domain controller (e.g., Windows Active Directory server) that is being relied upon by CDE applications for authentication and security services in the PCI Scope Of Assessment? Question 3: How about a domain controller (e.g., Windows Active Directory server) that is not relied upon by any CDE applications? Question 4: Is a network attached stapler that happens to be on the same network segment as a CDE system component always also in the CDE? Question 5: Does it matter if a workstation that a customer service representative uses a thin- or thick-client? Question 6: When should it be acceptable that if a virtualization hypervisor hosting a production application in the CDE be also able to host another VM without it being part of the CDE, as well? Question 7: If you have a domain controller that is not in the CDE, but in the scope of PCI assessment, is a print server on the same network segment as that domain controller also in the scope of PCI assessment? Bonus Exercise: For each of the questions where you answered "in scope of the PCI assessment," describe a strategy to contain the scope, such that systems connected to that system are not in scope. (See Michelle Klinger's great post on the "PCI Contagion Dilemma.")

22. SOX-404 Value Network: Primary Constituencies

23. What Does PCI Value Network Look Like?

24. Language Is Often An Obstacle In Newton’s time, there were not concrete terms for several critical concepts: Force, acceleration, mass, inertia In the following slide, note how difficult it was for Newton to frame the “three laws of motion” without these concepts…

25. Early Drafts Of Three Laws Of Motion 1. If a quantity once move it will never rest unless hindered by some externall cause. 2. A quantity will always move on in the same straight line (not changing the determination nor celerity of its motion) unless some externall cause divert it. 3. There is exactly so much required and no more force to reduce a body to rest as there was to put it upon motion. Axiom 100: A body once moved will always keep the same celerity, quantity and determination of its motion Axiom 103: ...as the body (a) is to the body (b0), so must the power of efficacy vigor strength or virtue of the cause which begets the same quantity of velocity Source: Isaac Newton, James Gleick.

26. Benchmarks Pythagorean theorem: 24 words Archimedes' Principle: 67 words Newton’s Three Laws Of Motion: 91 words The 10 Commandments: 179 words GAIT Proposed Principles v3.0: 168 words The Gettysburg Address: 286 words The Declaration of Independence: 1,300 words GAIT Principles v1.3: 6,856 words GAIT Methodology v2.2: 11,348 words The US Government regulations on the sale of cabbage: 26,911 words

27. Solution: GAIT… Released in Feb 2007, Establishes four principles that Defines the relevance of IT infrastructure elements to financial reporting integrity Define the three types of IT processes that can affect them: change management and systems development, operations and security Defines an end-to-end process view of these three processes Defines an approach to defining objectives and key controls within those three processes Provides a methodology and thinking process that continues the top down, risk based approach started in AS2 to scope IT general controls Provides a common context for management and auditors to support and test management’s assessment that the necessary IT controls exist and are effective Initial target is internal control objectives for financial reporting, but should extend to operating effectiveness and complying with laws and regulations (as defined by COSO)

28. GAIT Principle #1 The only IT infrastructure elements (e.g., databases, operating systems, networks) relevant to ITGC assessment are those that support financially-significant applications and data. (“What are the relevant IT infrastructure elements?”)

29. GAIT Principle #2 The IT processes primarily relevant to ITGC assessment are those that directly impact the integrity of financially-significant applications and data: Change management and systems development: the processes around developing, implementing, and maintaining financially significant applications and supporting IT infrastructure Operations management: the processes around managing the integrity of production data and program execution Security management: the processes around limiting access to information assets (“What are the relevant end-to-end IT processes?”)

30. GAIT Principle #3 Implications to the reliability of financially-significant applications and data, including controls, are based upon the achievement or failure of IT process objectives, not the design and operating effectiveness of the individual controls within those processes. (“What are the relevant objectives of those IT processes? In other words, we shouldn’t get carried away when reaching a conclusion when testing a control.”)

31. GAIT Principle #4 The basis for identifying key controls in the three IT processes is based on: Inherent risk of not achieving the IT process objectives IT process risk indicators (“How do we select key controls within those IT processes?”)

32. GAIT Scoping: Step By Step AS2 begins here GAIT Starts Here

33. GAIT Tools Scenarios Online auction settlement process (high IT) Rebate approval process (med IT) Option expensing process (low IT) Ask Dr. GAIT

34. GAIT Evolution GAIT-R for Business Risk

35. Conclusions and Lessons Learned, Continued Improved audit comment wording helps to connect to things management cares about: “We noted poor change control procedures and were unable to obtain comfort that all changes were authorized and tested as required” -- vs. -- “Poor change control practices introduced the risk of unauthorized or untested changes to key data such as annual threshold amounts for toxic chemical releases. Given the level of precision applied to reviewing the final report downstream, it is unlikely management would detect such errors. Our testing disclosed numerous “break/fix” changes had been made to code or data without supervisory review and approval or notifying the users.”

36. GAIT Evolution Elements of GAIT was incorporated into PCAOB AS-5 GAIT-R for Business Risk To me, it's the first really well thought out way of linking IT to any COSO internal control objective Unlike ITIL, COBIT: it helps focus on what matters Which is very much unlike PCI… The Integrated Auditing Project (“Magic Glasses”)

37. Wait, You’re Lowering The PCI Bar! Until you get scoping right, you can't raise the bar Unless you correctly identify the scope of PCI assessment correctly, any work on the controls is potentially wasted

38. My PCI Mission And Crusade Create guidance to be able to scope correctly Enable a risk based way to not only scope, but to evaluate controls Prioritized PCI DSS is a disappointment What controls for the PCI Scope of Assessment? First, to earn the right to do all of this, we must enable correct scoping first

39. Participants Leads Kent Fox (Intermountain Healthcare) Brandon Green (T-Mobile) Gretchen Forsyth (Southwest Airlines) Mike Dahn (Verizon) Tabitha Greiner (Verizon) Ian White (Verizon) James Summers (Nike)

40. Extend Concepts In PCI DSS Page 4: DSS 1.2: “System components” are defined as any network component, server, or application that is included in or connected to the cardholder data environment.

41. Before vs. After Before: Prior to creating a structured method, we needed over 40 hours to come to a scoping conclusion. After: With the model under development, we generated consensus on 15 scoping conclusions in less than 2 hours.

42. Proposed Deliverables Define and deliver the following, in a manner that clarifies and supports the spirit and intent of protecting cardholder data: Scoping principles A structured scoping methodology A library of scoping scenarios demonstrating its usage for educational and clarification purposes Create useful tools and guidance that will assist in the scoping effort for both merchants and QSAs.

43. Decision Tree

44. Proposed Timeline Submit a set of guidance to the PCI SSC for approval before the PCI Community meeting in September 2010 Desired outcome: PCI SSC and Board of Advisors agree with problem and its significant, have confidence in the approach Assign a staff member to validate guidance and integrate it into the PCI practice

45. Also TODO Identify attributes of effective segmentation to contain PCI contagion Encrypted PIN device Citrix Thin Client Virtualization Where necessary, fix the words, "segment", "connected to,"

46. Next Up: Scoping Category vs. Control Consideration ????? ControlConsiderations

47. Next: Alternate Control Procedures Create a framework to evaluate alternate control procedures -- for that you need risk Right now, PCI is 220+ control activities: create the framework to state what the control objectives are, so you can evaluate whether the objective is being met COSO construct Objective, risk, control objective THEN control activities and controls!

48. Top A-Ha Moments Auditors rock: they have a comprehensive vocabulary that we need – otherwise, we’re stuck in Flatland We need more people who can see the sphere Auditors have seen the dead people longer than anyone These auditors will eventually go crazy, and need friends After a long detour into IT operations and audit, I’m returning to information security, in the guise of compliance

49. We Can Change The State Of The Practice It’s an important problem There are models we can replicate Do you want to get involved?

50. My New Twins

51. My Last Day At Tripwire

52. What I’m Working On 50% with my family 50% on When IT Fails: The Novel Figure out the methods, procedures and tools needed to enable the transformation Collaborate with communities of practice to help mobilize these transformations BSides, DevOps, ITIL, IIA, SEI

53. When IT Fails: The Novel: Day 1 Steve Masters, CEO Dick Landry, CFO Parts Unlimited$4B revenue/year

54. When IT Fails: The Novel: Day 2 Bill Palmer, VP IT Operations (new) Wes Davis, Director, Distributed Systems Patty McKee, Director, Support and Process Improvement

55. When IT Fails: The Novel: Day 3 Norman Merz, Chief Audit Executive John Kirkland, CISO

56. When IT Fails: The Novel: Day 4 Chris Anderson, VP Application Development Sarah Moulton, SVP Retail Products The outsourcing sales rep

57. When IT Fails: The Novel: Day 10 The Deployment

58. When IT Fails: The Novel: The Two Critical Projects Project Phoenix: designed to close the gap with the retail competition: $20M project Project Argo: designed to integrate POS systems with accounting systems to reduce time to close books, manufacturing order-to-cash, restock intervals