• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
2010 07 BSidesLV Mobilizing The PCI Resistance 1c

2010 07 BSidesLV Mobilizing The PCI Resistance 1c



Properly Mobilizing the PCI Resistance: Lessons Learned From Fighting Prior Wars (SOX-404)" ...

Properly Mobilizing the PCI Resistance: Lessons Learned From Fighting Prior Wars (SOX-404)"

I have noticed that there is a growing wave of discontent and disenchantment from information security and compliance practitioners around the PCI DSS. Josh Corman has been an effective voice for these concerns, providing an intellectually honest and earnest analysis in his talk “Is PCI The No Child Left Behind Act For Infosec?”

The problem are well-known and significant: too much ambiguity in the PCI DSS, Qualified Security Assessors (QSAs) and consultant using subjective interpretations, existing guidance either too prescriptive or too vague, scope missing critical systems that could risk cardholder data, overly broad scope and excessive testing costs, excessive subjectivity and inconsistency, poor use of scarce resources, no meaningful reduction in risk of data breaches, and so forth.

For years, I have been studying the PCI DSS compliance problem, as well. I have noticed many similarities to the PCI compliance challenges and the “SOX-404 Is The Biggest IT Time Waster” wars in 2005. I was part of the leadership team at the Institute of Internal Auditors (IIA) where we did something about the it. We identified inability to accurately scope the IT portions of SOX-404 as the root cause of the billions of dollars of wasted time and effort, while not reducing the risk of financial misstatements.

I propose to present the two-year success story of the IIA GAIT project and how we changed the state of the IT audit practice in support of SOX-404 financial reporting audits. We defined the four GAIT Principles, which could be used to correctly scope the IT portions of SOX-404. We mobilized over 100K internal auditors, the SEC and PCAOB regulatory and enforcement bodies, as well as the external auditors from the 8 big CPA firms (e.g, Big Four and other firms doing SOX advisory work). In short, we made a difference, in a highly political process that involved many constituencies.

I am attempting to do something similar with the PCI Security Standards Council, through my work as part one of the leaders of the PCI Scoping SIG (Special Interest Group). My personal goal is to find a “third way” to better enable correct scoping of the PCI Cardholder Data Environment, and create a risk-based approach of substantiating the effective controls to ensure that cardholder data breaches can be prevented, and quickly detected and corrected when they do occur.

My desired outcome is to find fellow travelers who also see the pile of dead bodies in PCI compliance efforts, and work with those practitioners to catalyze a similar movement to achieve the spirit and intent of PCI DSS.



Total Views
Views on SlideShare
Embed Views



2 Embeds 300

http://www.realgenekim.me 297
http://realgenekim.squarespace.com 3



Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
Post Comment
Edit your comment
  • There are many ways to react to this: like, fear, horror, trying to become invisible… All understandable, given the circumstances…

2010 07 BSidesLV Mobilizing The PCI Resistance 1c 2010 07 BSidesLV Mobilizing The PCI Resistance 1c Presentation Transcript

  • Mobilizing The PCI Resistance:Lessons Learned From Previous Wars (SOX-404)
    Gene Kim, CISACTO, Tripwire@realgenekim, http://www.realgenekim.me#BSidesLV 2010
  • Problem Definition
    Success of any PCI DSS compliance initiative is very dependent on accurate definition and scoping of the Cardholder Data Environment.
    There is a wide variance in practice, experience and guidance in merchant and QSA community.
    These contribute to scoping errors that result in:
    Overly narrow scope that jeopardizes cardholder data
    Overly broad scope that adds unnecessary cost and effort for compliance
    Decreased confidence in and frustration with the PCI DSS standard
  • What This Really Means
    Incredible amount of discontent and growing disenchantment with PCI DSS
    Complaints that DSS is too specific or too vague
    Like Michelle Klinger, I have a love/hate relationship with PCI DSS
    The reach of PCI DSS is awesomely breathtaking, and is relevant to all PII
    But in the worst case, it's a total waste of time, at enormous cost to the organization
  • Agenda
    Describe the problems around SOX-404
    What we did about it at the Institute of Internal Auditors
    The GAIT concepts, politics, tools and outcomes
    Show how we can use this as a model to change the state of the practice around PCI DSS
    Share with you the best formulation of the plan I have
    Get your help improving the plan
    And ideally…
    Share my biggest a-ha moments the GAIT experience
    Excite you enough to do something about it
    Tell you some interesting stories
  • Holy Crap. This Looks Familiar!
  • The Problem
    The IT portions of SOX-404 compliance has frustrated auditors and management
    Significant key controls reside inside IT and IT processes as well as in the business processes
    No well-established guidance for scoping IT work results in inconsistency and the process being overly subjective
    Sometimes result in overly broad scope and excessive testing costs
    Significant risks to financial assertions may be left unaddressed
    Suboptimal use of scarce resources
  • Why Is There A Problem?
    No clear guidance exists to define how IT processes and activities can invalidate financial application processing or financial assertions
    COSO provides an accepted construct for defining overall internal control objectives, assertions, risks and controls, but its application to the IT environmet is ambiguous
    COBIT doesn’t provide a clear mechanism to scope IT processes and controls to the achievement of specific internal control objectives (e.g., COSO objective for internal control over financial reporting)
    Something else is needed…
  • “OMG. 952 IT Deficiencies?!?”
  • Vision: Create Equivalence to Nine Firm Document on IT Control Exceptions
    GAIT takes the approach used in the nine firm document.GAIT represents the upfront scoping exercise to appropriately identify the IT controls work relevant to overall internal controls objectives
    Chart 3: Evaluating Information Technology General Control (ITGC) Deficiencies, “A Framework for Evaluating Control Exceptions and Deficiencies” (December 20, 2004)
  • What were/are people worried about?
    Holy cow!!! Enron wasn’t caused by a DBA. So, why are the auditors digging here?? --gk
    IT controls dominate the deficiencies, significant deficiencies, and material weaknesses identified through the S-O 404 assessment.
    The estimated percentage of deficiencies identified show IT controls accounting for the most (34 percent), followed distantly by revenue (13 percent), procure to pay (10 percent), and fixed assets (10 percent). 
    The estimated percentage of significant deficiencies identified again shows IT controls leading the way (23 percent), followed by financial reporting and close (14 percent), procure to pay (13 percent), and revenue (12 percent).  
    The estimated percentages of material weaknesses identified include IT controls (27 percent), revenue (18 percent), taxes (11 percent), and financial reporting and close (10 percent).  
    It is important to note that the results presented here are based on self-reporting by the companies that participated in the survey. Conclusions may be affected by the differing methods companies use to report on various elements of Sarbanes-Oxley compliance.
  • February 2006
    Corporate Finance
    Again, holy cow!!! If the risk isn’t in IT, then auditors are not only generating efforts, but finding deficiencies that don’t matters… --gk
    • Disproportionate Share:
    • Compliance effort.
    • Deficiencies.
    • Non Finance Apps.
    • Financial Statement Impact:
    • Indirect linkage
    • Least likely impact
    • Business & IT integration.
  • Why We Knew There Was A Better Way
    All the work using Chart 3 is linking controls to risks that actually mattered
    COSO describes objectives, risks, controls and assertions
    COBIT is an exhaustive list of controls
    This is called scoping, which is critical to getting the right outcomes
    Comes before control design, implementation and testing
  • Thought Experiment
    Auditors vs. Management
    We can agree that there are two extremes in spectrum of financial reporting risk
    eBay auction settlement business process
    Grain elevators
    Extremes are easy… Middle is hard…
  • PCI Scoping Exercises (Show Your Work!)
    Question 1: Is the Cardholder Data Environment (CDE) equivalent to the PCI Scope of Assessment?
    Question 2: Is a domain controller (e.g., Windows Active Directory server) that is being relied upon by CDE applications for authentication and security services in the PCI Scope Of Assessment?
    Question 3: How about a domain controller (e.g., Windows Active Directory server) that is not relied upon by any CDE applications?
    Question 4: Is a network attached stapler that happens to be on the same network segment as a CDE system component always also in the CDE?
    Question 5: Does it matter if a workstation that a customer service representative uses a thin- or thick-client?
    Question 6: When should it be acceptable that if a virtualization hypervisor hosting a production application in the CDE be also able to host another VM without it being part of the CDE, as well?
    Question 7: If you have a domain controller that is not in the CDE, but in the scope of PCI assessment, is a print server on the same network segment as that domain controller also in the scope of PCI assessment?
    Bonus Exercise: For each of the questions where you answered "in scope of the PCI assessment," describe a strategy to contain the scope, such that systems connected to that system are not in scope. (See Michelle Klinger's great post on the "PCI Contagion Dilemma.")
  • SOX-404 Value Network: Primary Constituencies
  • What Does PCI Value Network Look Like?
  • Language Is Often An Obstacle
    In Newton’s time, there were not concrete terms for several critical concepts:
    Force, acceleration, mass, inertia
    In the following slide, note how difficult it was for Newton to frame the “three laws of motion” without these concepts…
  • Early Drafts Of Three Laws Of Motion
    1. If a quantity once move it will never rest unless hindered by some externall cause.
    2. A quantity will always move on in the same straight line (not changing the determination nor celerity of its motion) unless some externall cause divert it.
    3. There is exactly so much required and no more force to reduce a body to rest as there was to put it upon motion.
    Axiom 100: A body once moved will always keep the same celerity, quantity and determination of its motion
    Axiom 103: ...as the body (a) is to the body (b0), so must the power of efficacy vigor strength or virtue of the cause which begets the same quantity of velocity
    Source: Isaac Newton, James Gleick.
  • Benchmarks
    Pythagorean theorem: 24 words
    Archimedes' Principle: 67 words
    Newton’s Three Laws Of Motion: 91 words
    The 10 Commandments: 179 words
    GAIT Proposed Principles v3.0: 168 words
    The Gettysburg Address: 286 words
    The Declaration of Independence: 1,300 words
    GAIT Principles v1.3: 6,856 words
    GAIT Methodology v2.2: 11,348 words
    The US Government regulations on the sale of cabbage: 26,911 words
  • Solution: GAIT…
    Released in Feb 2007, Establishes four principles that
    Defines the relevance of IT infrastructure elements to financial reporting integrity
    Define the three types of IT processes that can affect them: change management and systems development, operations and security
    Defines an end-to-end process view of these three processes
    Defines an approach to defining objectives and key controls within those three processes
    Provides a methodology and thinking process that continues the top down, risk based approach started in AS2 to scope IT general controls
    Provides a common context for management and auditors to support and test management’s assessment that the necessary IT controls exist and are effective
    Initial target is internal control objectives for financial reporting, but should extend to operating effectiveness and complying with laws and regulations (as defined by COSO)
  • GAIT Principle #1
    The only IT infrastructure elements (e.g., databases, operating systems, networks) relevant to ITGC assessment are those that support financially-significant applications and data.
    (“What are the relevant IT infrastructure elements?”)
  • GAIT Principle #2
    The IT processes primarily relevant to ITGC assessment are those that directly impact the integrity of financially-significant applications and data:
    Change management and systems development: the processes around developing, implementing, and maintaining financially significant applications and supporting IT infrastructure
    Operations management: the processes around managing the integrity of production data and program execution
    Security management: the processes around limiting access to information assets
    (“What are the relevant end-to-end IT processes?”)
  • GAIT Principle #3
    Implications to the reliability of financially-significant applications and data, including controls, are based upon the achievement or failure of IT process objectives, not the design and operating effectiveness of the individual controls within those processes.
    (“What are the relevant objectives of those IT processes? In other words, we shouldn’t get carried away when reaching a conclusion when testing a control.”)
  • GAIT Principle #4
    The basis for identifying key controls in the three IT processes is based on:
    Inherent risk of not achieving the IT process objectives
    IT process risk indicators
    (“How do we select key controls within those IT processes?”)
  • GAIT Scoping: Step By Step
    AS2 begins here
    GAIT Starts Here
  • GAIT Tools
    Online auction settlement process (high IT)
    Rebate approval process (med IT)
    Option expensing process (low IT)
    Ask Dr. GAIT
  • GAIT Evolution
    GAIT-R for Business Risk
  • Conclusions and Lessons Learned, Continued
    Improved audit comment wording helps to connect to things management cares about:
    “We noted poor change control procedures and were unable to obtain comfort that all changes were authorized and tested as required”
    -- vs. --
    “Poor change control practices introduced the risk of unauthorized or untested changes to key data such as annual threshold amounts for toxic chemical releases. Given the level of precision applied to reviewing the final report downstream, it is unlikely management would detect such errors. Our testing disclosed numerous “break/fix” changes had been made to code or data without supervisory review and approval or notifying the users.”
  • GAIT Evolution
    Elements of GAIT was incorporated into PCAOB AS-5
    GAIT-R for Business Risk
    To me, it's the first really well thought out way of linking IT to any COSO internal control objective
    Unlike ITIL, COBIT: it helps focus on what matters
    Which is very much unlike PCI…
    The Integrated Auditing Project (“Magic Glasses”)
  • Wait, You’re Lowering The PCI Bar!
    Until you get scoping right, you can't raise the bar
    Unless you correctly identify the scope of PCI assessment correctly, any work on the controls is potentially wasted
  • My PCI Mission And Crusade
    Create guidance to be able to scope correctly
    Enable a risk based way to not only scope, but to evaluate controls
    Prioritized PCI DSS is a disappointment
    What controls for the PCI Scope of Assessment?
    First, to earn the right to do all of this, we must enable correct scoping first
  • Participants
    Kent Fox (Intermountain Healthcare)
    Brandon Green (T-Mobile)
    Gretchen Forsyth (Southwest Airlines)
    Mike Dahn (Verizon)
    Tabitha Greiner (Verizon)
    Ian White (Verizon)
    James Summers (Nike)
  • Extend Concepts In PCI DSS
    Page 4: DSS 1.2: “System components” are defined as any network component, server, or application that is included in or connected to the cardholder data environment.
  • Before vs. After
    Before: Prior to creating a structured method, we needed over 40 hours to come to a scoping conclusion.
    After: With the model under development, we generated consensus on 15 scoping conclusions in less than 2 hours.
  • Proposed Deliverables
    Define and deliver the following, in a manner that clarifies and supports the spirit and intent of protecting cardholder data:
    Scoping principles
    A structured scoping methodology
    A library of scoping scenarios demonstrating its usage for educational and clarification purposes
    Create useful tools and guidance that will assist in the scoping effort for both merchants and QSAs.
  • Decision Tree
  • Proposed Timeline
    Submit a set of guidance to the PCI SSC for approval before the PCI Community meeting in September 2010
    Desired outcome:
    PCI SSC and Board of Advisors agree with problem and its significant, have confidence in the approach
    Assign a staff member to validate guidance and integrate it into the PCI practice
  • Also TODO
    Identify attributes of effective segmentation to contain PCI contagion
    Encrypted PIN device
    Citrix Thin Client
    Where necessary, fix the words, "segment", "connected to,"
  • Next Up: Scoping Category vs. Control Consideration
  • Next: Alternate Control Procedures
    Create a framework to evaluate alternate control procedures -- for that you need risk
    Right now, PCI is 220+ control activities: create the framework to state what the control objectives are, so you can evaluate whether the objective is being met
    COSO construct
    Objective, risk, control objective
    THEN control activities and controls!
  • Top A-Ha Moments
    Auditors rock: they have a comprehensive vocabulary that we need – otherwise, we’re stuck in Flatland
    We need more people who can see the sphere
    Auditors have seen the dead people longer than anyone
    These auditors will eventually go crazy, and need friends
    After a long detour into IT operations and audit, I’m returning to information security, in the guise of compliance
  • We Can Change The State Of The Practice
    It’s an important problem
    There are models we can replicate
    Do you want to get involved?
  • My New Twins
  • My Last Day At Tripwire
  • What I’m Working On
    50% with my family
    50% on
    When IT Fails: The Novel
    Figure out the methods, procedures and tools needed to enable the transformation
    Collaborate with communities of practice to help mobilize these transformations
    BSides, DevOps, ITIL, IIA, SEI
  • When IT Fails: The Novel: Day 1
    Steve Masters, CEO
    Dick Landry, CFO
    Parts Unlimited$4B revenue/year
  • When IT Fails: The Novel: Day 2
    Bill Palmer, VP IT Operations (new)
    Wes Davis, Director, Distributed Systems
    Patty McKee, Director, Support and Process Improvement
  • When IT Fails: The Novel: Day 3
    Norman Merz, Chief Audit Executive
    John Kirkland, CISO
  • When IT Fails: The Novel: Day 4
    Chris Anderson, VP Application Development
    Sarah Moulton, SVP Retail Products
    The outsourcing sales rep
  • When IT Fails: The Novel: Day 10
    The Deployment
  • When IT Fails: The Novel: The Two Critical Projects
    Project Phoenix: designed to close the gap with the retail competition: $20M project
    Project Argo: designed to integrate POS systems with accounting systems to reduce time to close books, manufacturing order-to-cash, restock intervals