• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Application of Q methodology in critical success factors of information security risk management
 

Application of Q methodology in critical success factors of information security risk management

on

  • 1,057 views

 

Statistics

Views

Total Views
1,057
Views on SlideShare
1,041
Embed Views
16

Actions

Likes
0
Downloads
0
Comments
0

1 Embed 16

http://www.personal.psu.edu 16

Accessibility

Categories

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • Every year for the last 13 years, Computer Security Institute conducts a nation-wide survey on Computer Crime and Security statistics. The survey asks several hundred security professionals all over the country to find out the type of incidents that affected the organizations. This table shows the type of incidents, and the % of participants acknowledging their organization being affected by each incident type between 2004 and 2008. All these types of incidents show a decrease in the numbers, except few: unauthorized access, misuse of web application, and DNS attack, but these numbers are still quite significant. The survey finds that the decrease in these numbers can be explained by increasing awareness of information security in the organizations. Why the numbers are still significant can be understood by the study of Partida and Ezingeard (2007). They found a lack of strategic investment in information security. More reactive and tactical, instead of being more proactive and strategic. So how can we make things better? Partida and Ezingeard stress that what we need is a solid understanding of the benefits of a well-developed approach to information security, and an understanding of the critical success factors to achieve those benefits. Past studies have attempted to address this need by identifying a list of general critical success factors by means of interviews, case studies, and large surveys, involving qualitative data. However, none of those studies attempted to study the subjectivity or viewpoint of those participants regarding those critical success factors. That’s exactly the focus of our research.
  • So, what motivated us for this research? The study of human subjectivity has been successfully conducted in various disciplines including nursing, veterinary medicine, public health, transportation, education, etc. The research on human subjectivity is yet to gain popularity in the field of information security. This point is evident by the fact that vast majority of the past researches on critical success factors of information security are based on qualitative data only, as mentioned earlier. With more organizations undertaking information security initiatives, there is increasing awareness among the IT professionals on issues like risk management, risk assessment, vulnerability analysis, etc. These are subjective areas that can be better studied using a research tool that combines the strengths of both qualitative and quantitative research. That’s where Q-methodology comes into picture. Within Q methodology, participants are given a question, and a deck of stimulating statements. Participants are asked to rank-order those statements (aka Q-sort), which are then inter-correlated and subjected for factor analysis. In this way, groups of individuals holding similar viewpoints or opinions are identified. The factors are then interpreted to provide understanding of their underlying subjectivities. So, the motivating element of our research is to explore the theoretical principles of Q-methodology and its application as a research method in the field of information security.
  • The concept of critical success factors was first presented by John Rockart in 1979 when he wrote an article in Harvard business Review called “Chief executives define their own data needs”. The focus of his CSF analysis was on management. Although such a concept was introduced almost 30 years ago, there’s still not many scholarly literature on CSF affecting information security risk management. There’s also a lack of experimental research in the field of risk management. So how did we address this gap? We addressed this gap by consulting the literature and identified the items that may affect successful implementation of information security practices, such as risk management and risk assessment. 24 such items were identified for our study. We call these 24 items as Q-set.
  • These are the first 6 of these 24 statements.
  • These are item numbers 7 through 12.
  • 13 through 18.
  • And finally, from 19 through 24.
  • Discuss difference between associates and senior associates . This distinction was based on the seniority of the title/position of the participants, not based on their length of service.
  • Prep work includes sending initial communication to an organization asking for participants for this study. We had to explain the purpose of the study, brief description of what is expected from the participants, and confirmation of their confidentiality throughout the study. Once the participants are identified, then we meet each person in person to conduct the Q-sort exercise. Give them 1 card with the research question written on it. Give them 5 cards, each indicating a pile or “degree of agreement”. Give them 24 cards for 24 statements. After a participant is done finishing the Q-sort exercise, then result looks like this picture:
  • The first step of data analysis is constructing the correlation matrix between the Q-sorts. This is the Pearson product-moment correlation (r ). Sub01 correlates highly with sub29 (.87), and correlates weakly with sub20 (.04). One key point to mention here is the fact that the purpose of Q methodology is not finding out how closely two participants correlate. Therefore, correlation matrix is simply an intermediate step before the data is used for factor analysis.
  • The first step of data analysis is constructing the correlation matrix between the Q-sorts. This is the Pearson product-moment correlation (r ). Sub01 correlates highly with sub29 (.87), and correlates weakly with sub20 (.04). One key point to mention here is the fact that the purpose of Q methodology is not finding out how closely two participants correlate. Therefore, correlation matrix is simply an intermediate step before the data is used for factor analysis.
  • Next, the factor analysis is performed in order to search for resemblance among the Q-sorts. How does that work? Factor analysis takes the correlations between these variables, and reduce the multivariate data down to a small number of factors. Thus, factor analysis helps in analysis and interpretation of the data. You can see that 50 variables have been reduced to 8 factors by PQMethod software which calculated the factor analysis values. An important part of factor analysis is computing the eigenvalues. Eigenvalues reflect the amount of variance accounted for by each factor . Eigenvalues can also be used to determine the importance of each factor. You can see that factor 1 accounts for 28% of variance. Eigenvalues are expressed by the greek letter Lambda and are frequently used in matrix algebra. Eigenvalues are computed by performing a summation of all the squared values in the column of a factor matrix . A(ik) = factor loading of variable I on factor k. m = number of variables. From here, the next step is to determine the optimal number of factor. We choose 3 factors according to skree value. Skree value is the number of factor at which the eigenvalues kind of levels off.
  • The next steps are to perform a varimax rotation and determine the factor loading values for each Q-sort. Varimax rotation is a statistical technique in which the relation between q-sorts can be examined from different angles. The factor loading values display the extent to which a Q-sort is associated with the viewpoint of a particular Factor . The q-sort that loads significantly on a factor is marked by an X. PQMethod software automatically does that for you. You can see that 12 q-sorts loaded significantly in factor 1, 13 q-sorts loaded significantly on factor 2, and 13 q-sorts loaded significantly on factor 3. 12 q-sorts did not significantly load on any factor. That’s because there may be an error in understanding the statements, or their viewpoints are idiosyncratic with respect to other participants. These 12 q-sorts are left out from further analysis.
  • The next step is to calculate the factor scores based on the defining sorts for each factor. Each q-sort is given a factor score in terms of the original values used in the Q-sort (-2 for Definitely Not, -1 for probably not, 0 for neutral, 2 for definitely, etc.) The factor scores illustrate how each statement agrees within a factor, thus, helps to determine the areas of agreement.
  • The consensus statement does not mean that all the respondents considered competence of the team members as unnecessary. It simply means that the participant’s thinking on the subject did not distinguish it from the others.

Application of Q methodology in critical success factors of information security risk management Application of Q methodology in critical success factors of information security risk management Presentation Transcript

  • APPLICATION OF Q-METHODOLOGY IN CRITICAL SUCCESS FACTORS OF INFORMATION SECURITY RISK MANAGEMENT Master’s Thesis Defense Candidate Sohel M. Imroz Advisors Dr. Leah R. Pietron Dr. Dwight A. Haworth April 2, 2009
    • Section 1: Background
      • Problem Description
      • Motivation
      • Purpose of Study
    • Section 2: Problem Statement
    • Section 3: Literature Review
    • Section 4: Research Design
      • Research Question
      • Source of Data
      • Participant Selection
      • Data Collection Procedure
    • Section 5: Data Analysis
      • Correlation
      • Factor Analysis
      • Varimax Rotation and Factor Loadings
      • Factor Scores
      • Areas of Agreement
      • Distinguishing Statements
      • Consensus Statements
    • Section 6: Results and Conclusion
      • Significant Findings
      • Limitations of Study
      • Future Research
    Outline
    • Problem Description
      • Percentages of reported incident types decreased between 2004 – 2008, but still significant.
      • Increased security awareness.
      • Past studies on information security CSF:
        • Interviews, case studies, and surveys.
        • Qualitative data.
    Section 1: Introduction Source: http://i.cmpnet.com/v2.gocsi.com/pdf/CSIsurvey2008.pdf
    • Motivation
      • An alternate approach is needed which:
        • Provides insight into individual subjectivity
        • Combines both qualitative and quantitative analysis
      • Q-methodology:
        • Rank-order statements (Q-sort)
        • Factor analysis
        • Identify similar viewpoints or opinions
        • Interpret underlying subjectivities
    Section 1: Introduction (cont’d)
    • Purpose of the study
      • To identify the critical success factors in information security risk management;
      • To reduce many individual viewpoints on those critical success factors to few “perspectives”, which represent shared ways of thinking;
      • To investigate the existence of any gap or disconnect between various groups of participants in understanding the criticality of any critical success factor.
    Section 1: Introduction (cont’d)
    • Successful implementation of a well-developed information security program depends on identifying the critical success factors (CSF) of the risk management process. An understanding of experiences and perspectives of IT professionals regarding the CSF is central to the delivery of quality risk management. Implicit with this is a need to understand the attributes and characteristics of these individuals, their subjectivity.
    • We propose to apply the theoretical underpinnings of Q-methodology as an alternate research approach to provide better insight into individual subjectivity in the field of information security.
    Section 2: Problem Statement
    • Not many scholarly literature on CSF affecting information security risk management (Yeo, Rahim, & Miri, 2007).
    • Lack of experimental research in the area of information security risk management (Kotulic & Clark, 2004).
    • Consulted literature on items that may affect successful implementation of information security practices, such as risk management and risk assessment (Yeo, Rahim, & Miri, 2007).
    • 24 such items were identified for our study.
    Section 3: Literature Review
    • 1. SENIOR MANAGEMENT SUPPORT: Ensure senior management’s commitment, support, and active participation.
    • 2. DEFINED PROCEDURES: Have each security policy accompanied with a set of clearly defined and documented procedures showing how to implement the policy.
    • 3. DESIGNATED FOCAL POINTS: Designate groups or individuals as focal points to oversee and guide the organization’s risk assessment processes.
    • 4. KEEPING POLICIES AND PROCEDURES UP-TO-DATE: Ensure that someone (or a group of people) owns the responsibility of maintaining the policies and procedures and keep them up-to-date.
    • 5. INVOLVEMENT OF BUSINESS OWNERS AND TECHNICAL EXPERTS: Ensure that both the business process owners and technical experts are involved in developing the security policies and procedures, and also in evaluating the security activities.
    • 6. ROLES, RESPONSIBILITIES, AND ACCOUNTABILITY: Ensure that security policies and procedures clearly define roles and responsibilities for all operations and tasks. Have only the appropriately trained individuals carry out the security responsibilities, and hold them accountable for their actions or lack of actions.
    Section 3: Literature Review (cont’d)
    • 7. LIMITED SCOPE OF EACH ASSESSMENT: Limit scopes of individual risk assessments and define the extent of each risk evaluation. Include guidelines to decide which operational areas (business units) to include in the evaluation.
    • 8. DOCUMENTATION AND MAINTENANCE OF SECURITY INFORMATION: Provide a means of documenting and maintaining all security risk related information in a consistent format. Define and document the procedures of performing each evaluation activity, the artifacts (worksheets, catalogs, etc.) used during each activity, and the results of the evaluation.
    • 9. SECURITY POLICY AND STRATEGY: Develop and implement a comprehensive enterprise-wide security policy.
    • 10. ORGANIZATIONAL POLICY AND STRATEGY: Have information security policies and strategies consistent with the organizational policies and strategies.
    • 11. BUSINESS REQUIREMENTS: Have the business requirements of the organization as the basis of the security policy and strategy.
    • 12. TRAINING AND AWARENESS PROGRAM: Develop and implement regular enterprise-wide information security training and awareness programs that efficiently elicit security related policies, procedures, controls, and recommended best practices to staff personnel, contractors, and other users.
    Section 3: Literature Review (cont’d)
    • 13. INTERACTION, COLLABORATION, AND TEAM WORK: Ensure interaction, collaboration, and team-work from all participating individuals and business partners to apply necessary skills and knowledge to complete evaluation process satisfactorily.
    • 14. COMPETENCE OF TEAM MEMBERS: Ensure that the risk assessment team members are competent and have the expertise to apply the risk assessment methodology to a specific site and system, identify mission risks, and provide cost-effective safeguards that meet the needs of the organization.
    • 15. CONTINGENCY AND DISASTER RECOVERY: Develop and implement adequate contingency and disaster recovery plans and procedures to ensure continuity of operations and assets of the organization.
    • 16. COMMUNICATION / INCIDENT RESPONSE: Establish a clear and open communication system for the staff members to detect, report, and respond to security incidents in a timely manner.
    • 17. SECURITY PRIORITIES: Communicate security priorities to stakeholders of all levels within an organization.
    • 18. INTEGRATION OF RISK MANAGEMENT PRACTICES WITH MANAGEMENT PRACTICES: Integrate information security risk management practices with management activities of the organization, including strategic planning, capital planning, and enterprise architecture.
    Section 3: Literature Review (cont’d)
    • 19. COMPLIANCE: Have the information security risk management goals, policies, and practices consistent with all laws, regulations, and standards of due care with which the organization must comply.
    • 20. CORPORATE CULTURE: Make sure the culture of the company influences the success of the security processes within the organization.
    • 21. ASPECTS OF CRITICAL INFRASTRUCTURES: Consider the unique aspect of each critical infrastructure within the organization. Each critical infrastructure needs to be studied and their unique vulnerabilities need to be identified.
    • 22. AN ONGOING PROCESS: Have an ongoing risk evaluation and assessment process that facilitates and allows the business process owners to control and manage their exposure.
    • 23. PRE-SELECTION OF A RISK ASSESSMENT METHOD: Pre-select the risk assessment method that is suitable for the specific needs of the organization. Some of the leading risk assessment methods are Common Criteria, OCTAVE, COBRA, EESA, etc.
    • 24. PERFORMANCE MONITORING / EFFECTIVENESS EVALUATION: Have information security managers regularly monitor the performance and effectiveness of the security program/effort, policies, procedures, management and operational practices, and technical controls for which they are responsible.
    Section 3: Literature Review (cont’d)
    • Research Question: Which of the following 24 items can be considered essential for the success of information security risk management program in organizations?
    • ◦ Senior management support ◦ Defined procedures ◦ Designated focal points ◦ Communication of incident response ◦ Limited scope of each assessment ◦ Security priorities
    • ◦ Security policy and strategy ◦ Organizational policy and strategy ◦ Business requirements ◦ Training and awareness program ◦ Competence of team members ◦ Contingency and disaster recovery ◦ Compliance ◦ Corporate culture ◦ Aspects of critical infrastructures ◦ An ongoing process ◦ Keeping policies and procedures up-to-date ◦ Roles, responsibilities, and accountability
    • ◦ Interaction, collaboration, and team-work ◦ Pre-selection of a risk assessment method ◦ Performance monitoring / effective evaluation
    • ◦ Documentation and maintenance of security information ◦ Involvement of business owners and technical experts ◦ Integration of risk management practices with management practices
    Section 4: Research Design
    • Source of Data:
      • Total 18 organizations.
        • For-profit
        • Non-profit
      • All organizations are located in Omaha, NE.
      • Organizations are selected from private, public, academic, and government sectors.
      • Organizations are selected from various industries to avoid any industry-specific predisposition.
    Section 4: Research Design (cont’d)
    • Participant Selection:
      • Total 50 participants are selected.
    Section 4: Research Design (cont’d) For-profit Non-profit Students (Full-time, Part-time) - 5 Faculty Members (Full-time, Part-time) - 3 Associates (Intern, Security Analyst, Programmer, System Architect, Network Admin, Business Analyst, Internal Auditor) 12 9 Senior Associates (Executive Director, IT Manager, Senior Business Analyst, Senior Security Analyst, Senior Architect) 14 7 Total 26 24
    • Data Collection Procedures:
      • Conduct prep-work.
      • Meet each participant in person.
        • 1 card for the research question.
        • 5 cards for the Q-sort categories.
          • Definitely Not (-2)
          • Probably Not (-1)
          • Neutral (0)
          • Probably (1)
          • Definitely (2)
        • A deck of 24 cards for the 24 items selected from literature review.
        • Perform Q-sort exercise.
    Section 4: Research Design (cont’d)
  • Section 4: Research Design (cont’d)
    • Data Collection Procedures:
      • Conduct prep-work.
      • Meet each participant in person.
        • 1 card for the research question.
        • 5 cards for the Q-sort categories.
          • Definitely Not (-2)
          • Probably Not (-1)
          • Neutral (0)
          • Probably (1)
          • Definitely (2)
        • A deck of 24 cards for the 24 items selected from literature review.
        • Perform Q-sort exercise.
    • Participant Response Data Set:
    Section 5: Data Analysis
    • Correlation:
    Section 5: Data Analysis
  • Section 5: Data Analysis (Cont’d)
    • Factor Analysis:
      • Reduces multivariate data to small number of dimensions
      • or factors.
      • Eigenvalues:
        • Amount of variation accounted for by corresponding factor (Brown, 2004, p. 10).
      • Number of factors to extract:
        • First 3 factors are selected.
        • Accounts for 49% of total variance.
    Unrotated Factor Loadings
    • Varimax Rotation:
      • A statistical technique in which the relation between Q-sorts can be examined from different angles.
    • Factor Loadings:
      • The extent to which a respondent (an individual Q-sort) is associated with the viewpoint of a particular factor.
      • The sorts which load significantly on a factor are marked by an “X”.
        • Factor 1 has 12 significant sorts.
        • Factor 2 has 13 significant sorts.
        • Factor 3 has 13 significant sorts.
        • Remaining 12 sorts are left out.
    Section 5: Data Analysis (Cont’d) Factor Matrix with an “X” Indicating a Defining Sort
    • Factor Scores:
      • Scores given to statements (items) in the composite Q-sort corresponding to the original values used in the Q-sort
      • (-2, -1, 0, 1, 2).
      • Illustrate how the level of agreement / disagreement among statements (items) within each identified opinion cluster.
    Section 5: Data Analysis (Cont’d) Factor Q-sort Scores For Each Statement (Item)
    • Areas of Agreement:
      • Senior management support:
        • “ Definitely” critical in two factor arrays.
        • Q-sort value of 2.
      • Pre-selection of a risk assessment method:
        • “ Definitely Not” critical in all three factor arrays.
        • Q-sort value of -2.
    Section 5: Data Analysis (Cont’d)
    • Distinguishing Statements for Factor 1:
      • Contingency and disaster recovery (Score = 2.31)
      • Compliance (Score = 1.29)
    • Objectives:
      • Ensure continuity of operations;
      • Create understandable, accessible, and written disaster recovery plans and procedures;
      • Demonstrate due diligence to Comply with required laws, regulations, and standards.
    • Characteristics:
      • Survival and protective.
    Section 5: Data Analysis (Cont’d) Distinguishing statements for factor 1 Factor 1: Contingency, disaster recovery, and compliance
    • Distinguishing Statements
    • for Factor 2:
      • Business requirements
      • (Score = 1.82)
    • Objectives:
      • Have business requirements as the basis of security policies and strategies;
      • Ensure cooperation between business leaders and security team;
      • Prioritize business requirements.
    • Characteristics:
      • Supportive and cooperative.
    Section 5: Data Analysis (Cont’d) Distinguishing statements for factor 2 Factor 2: Business requirements
    • Distinguishing Statements for Factor 3:
      • Involvement of business owners and technical experts
      • (Score = 1.75)
    • Objectives:
      • Involve business owners and security experts in developing and evaluating security policies and procedures;
      • Participate in creating business impact analysis and threat scenario.
    • Characteristics:
      • Participating and contributing.
    Section 5: Data Analysis (Cont’d) Distinguishing statements for factor 3 Factor 3: Involvement of business owners and technical experts
    • Consensus Statement:
      • Competence of team members (Score = 0.5 in Factor 1, 0.33 in Factor 2, 0.26 in Factor 3)
    • Objectives:
      • Ensure competency and expertise of the team members;
      • Follow a rigorous hiring practice;
      • Consider experienced candidates.
    • Characteristics:
      • Does not distinguish between any pair of factor.
    Section 5: Data Analysis (Cont’d) Consensus statement
    • Significant Findings:
      • Three distinct types of perspectives can be identified regarding the critical success factors in information security risk management initiatives.
        • The first perspective can be viewed as survival and protective in nature.
        • The second perspective can be described as supportive and cooperative.
        • The third perspective can be considered as participating and contributing.
      • Senior management support is considered to be the most critical in information security risk management program in the for-profit organizations, but not in the non-profit organizations.
      • Furthermore, there is a gap of understanding regarding the criticality of senior management support between associates and senior associates in the for-profit organizations.
      • Pre-selecting a risk assessment method is considered to be the least critical in information security risk management program.
    Section 6: Results and Conclusion
    • Limitations of the Study:
      • All the organizations in this study are located in Omaha.
      • Total number of participants may be too small.
      • Not enough representation from all types of participants.
      • Some participants took longer time to complete the Q-sort.
      • No explanation was provided to the participants about any of the Q-set statements.
    • Future Research:
      • Use larger sample size for each type of participant.
      • Use organizations from different geographic areas in United States.
      • Determine what kinds of effects geographic areas may have on organization’s information security policies and procedures.
    Section 6: Results and Conclusion (cont’d)
    • Brown, M. (October 2004). Illuminating patterns of perception: An overview of Q Methodology [PDF Document]. TECHNICAL NOTE CMU/SEI-2004-TN-026. Retrieved January 15, 2009, from http://www.sei.cmu.edu/pub/documents/04.reports/pdf/04tn026.pdf.
    • Kotulic, A. & Clark, J. (2004). Why there aren't more information security research studies. Information & Management, 41 (2), 597-607.
    • Rockart, J. F. (1979). Chief executives define their own data needs. Harvard Business Review, Vol. 2 , pp. 81-93.
    • Yeo, A. C., Rahim, M. M., & Miri, L. (2007). Understanding factors affecting success of information security risk assessment: The case of an Australian higher education institution [PDF Document]. Retrieved December 29, 2008 from http://www.pacis-net.org/file/2007/1272.pdf .
    • http://i.cmpnet.com/v2.gocsi.com/pdf/CSIsurvey2008.pdf
    References
  • APPLICATION OF Q-METHODOLOGY IN CRITICAL SUCCESS FACTORS OF INFORMATION SECURITY RISK MANAGEMENT
    • THE END