2. Area Objective
• Ensure that the IS Auditor understand and able to
provide assurance that the management practices
for the development/acquisition, testing,
implementation, maintenance, and disposal of
systems and infrastructure will meet the
organization’s objectives.
This area will represent approximately 16% of the CISA
examination (approximately 32 questions)
11/09/2013 2
3. Topic Covered
• Business Realization
• Project Management Structure & Practices
• Business Application Development & Alternative
Approaches
• Alternative Software Project Organization & Development
Methods
• Infrastructure Development/Acquisition Approaches
• Information Systems Maintenance Practices
• System Development Tools & Productivity Aids
• Process Improvement Practices
• Application Controls & Auditing Application Controls
• Auditing System Development, Acquisition and Maintenance
• Business Application
11/09/2013 3
4. Business Realization
• Portfolio/Program Management
– Program is a group of projects and time-bound tasks that are closely
linked together through common objectives, a common budget,
intertwined schedules and strategies. Program is more complex,
longer duration, higher budget, higher risk and higher importance,
than project.
– Program Management Objectives : successful execution of program
scope, financial, schedules, objectives, deliverables, context,
environment, communication, culture, organization.
– Program Organization : Program Owner/Sponsor, Program Manager,
Program Team, Program Office
– Project Portfolio : All the projects being carried out in an organization
at a given point in time (snapshot).
– Project Portfolio Objectives : Optimization of result of project
portfolio, prioritizing & scheduling projects, resource coordination,
knowledge transfer
– Project Portfolio requires : Database & Reports
11/09/2013 4
5. Business Realization
• Business Case Development & Approval
– Should be developed before project commencement
– Derived from Feasibility Study :
• Scope the problem
• Identify & explore a number of solutions
• Make recommendation on what action to take
– Calculate and outline business case for each of aspect of
comparison
– Should be justifying the project and answer the question of
“Why ?”
– Business case may become no longer valid, therefore a project
should has some Decision Points / Stage Gate / Kills Points,
where a business case is reviewed.
– If the business case changes during project, the project should
be reapproved through approval process.
11/09/2013 5
6. Business Realization
• Business Realization Techniques
– Benefits Management or Benefit Realization requires :
• Validating the benefits predicted in the business
• Planning and describing the benefit plan that is to be
realized
• Assigning a measure and target
• Documenting the assumptions
• Establishing key responsibilities for realizations
• Establishing a tracking/measuring regime
– Usually includes a Post-Implementation Review at 6-18
months after implementation.
– There must be a periodic review of benefits
11/09/2013 6
7. Project Management Structure
• Standards : PMBOK & PRINCE2
• Organizations : PMI & IPMA
• General Aspects
• Project Context & Environment :
– Contents, Time and Social
• Project Organizational Forms :
– Influence, Pure, Matrix
• Project Communication & Culture :
– One-on-one meetings, Kick-off meetings, project start workshops, or a
combination, project mission statement, project name & logo, project
team meeting rules & communication protocol, and project specific social
events.
• Project Objectives
– Main Objectives, Additional Objectives, Non-Objectives
– Object Breakdown Structure (OBS) Work Breakdown Structure (WBS)
Work Packages To-do List
11/09/2013 7
8. Project Management Structure
• Project Roles & Responsibilities
– Senior Management
– User Management
– Project Steering Committee
– Project Sponsor
– System Development Management
– Project Manager
– System Development Project Team
– User Project Team
– Security Officer
– Quality Assurance
11/09/2013 8
10. Business Application Development
• An individual application or project is initiated by:
– A new opportunity that relates to a new or existing business process
– A problem that relates to an existing business process
– A new opportunity that will enable the organization to take advantage
of Technology
– A problem with the current technology
• The Traditional System Development Life Cycle Approach:
– Phase 1 Feasibility
– Phase 2 Requirements
– Phase 3A Design
– Phase 3B Selection
– Phase 4A Development
– Phase 4B Configuration
– Phase 5 Implementation
– Phase 6 Post-implementation
11/09/2013 10
11. Business Application Development
• The Traditional System Development Life Cycle Approach:
– Feasibility Study
• Issue to be addressed
• Factors impacting
– Requirement Definition
• Identify & Analyze
• Record & Verify
• Resolve Conflicts
– Entity Relationship Diagram vs Object-Oriented
– Software Acquisition
• Request For Proposal (RFP) or Invitation To Tender (ITT)
• Required HW, supported OS, additional tools, supported DB
• Reliability, Commitment to service, training, technical support &
documentation
• Details of Contract
11/09/2013 11
13. Business Application Development
• The Traditional System Development Life Cycle Approach:
– Design
• User involvement in the design
• Software baselining
• End of design phase
• IS auditor involvement
– Development
• Programming methods and techniques
• Online programming facilities (integrated development environment –
IDE)
• Programming language
• Program debugging
• Testing
• Elements of a software testing phase
• Testing Classification
• Other types of testing-related terminology
• Automated application testing
11/09/2013 13
14. Business Application Development
• The Traditional System Development Life Cycle Approach:
– Implementation
• Implementation Planning
• Phase 1 : Gap Analysis, Role Definitions
• Phase 2 : Service Level Agreement, Knowledge Transfer Plan,
Training Plans
• End-user Training
• Data Conversion
• Refining Migration Scenario
• Fallback Scenario
• Cutover (Go-Live) Techniques
• Parallel Changeover
• Phased Changeover
• Absurd Changeover
• Certification/Accreditation
– Post-Implementation Review
11/09/2013 14
15. Business Application Development
• Risks Associated with Software Development
– Within the project
– With suppliers
– Within the organization
– With the external environment
• Use of Structured Analysis Design and Development
Techniques
– Develop system context diagrams.
– Perform hierarchical data flow/control flow decomposition.
– Develop control transformations.
– Develop mini-specifications.
– Develop data dictionaries.
– Define all external events—inputs from external environment.
– Define single transformation data flow diagrams from each
external event.
11/09/2013 15
16. Alternative Application Development Approach
• Alternative Approaches
– Approaches an IS auditor may encounter:
• Incremental or progressive development
• Iterative development
– Data-Oriented System Development
– Object-Oriented System Development
– Component-Based Development
– Web-based Application Development
– Prototyping
– Rapid Application Development
– Agile Development
– Reengineering & Reverse Engineering
11/09/2013 16
17. Infrastructure Development/Acquisition Practices
• Physical Architecture Analysis
– Goals :
• To analyze existing system
• To design a new architecture
• To write functional requirement of new architecture
• To develop proof of concept based on functional requirements
– Project Phases :
• Review of existing system
• Analysis and Design
• Draft Functional Requirements
• Vendor & Product Selection
• Writing Functional Requirements
• Proof of Concept
11/09/2013 17
18. Infrastructure Development/Acquisition Practices
• Planning the Implementation of Infrastructure
– Procurement Phase
• Develop vendor evaluation criteria
• Develop vendor long list & short list
• Select preferred vendor & define partnership
– Delivery Time
• Develop delivery plan
• Review delivery plan
– Installation Plan
• Develop installation plan
• Review installation plan
– Installation Test Plan
• Develop test plan
• Review test plan
11/09/2013 18
19. Infrastructure Development/Acquisition Practices
• Hardware Acquisition
– Invitation to Tender (ITT)
• Organizational description indicating whether the computer
facilities are centralized or decentralized, distributed or
outsourced
• Information processing requirements
• Hardware requirements
• System software requirements
• Support requirements
• Adaptability requirements
• Constraints
• Conversion requirements
11/09/2013 19
20. Infrastructure Development/Acquisition Practices
• Hardware Acquisition
– Acquisition Steps
• Testimonials/visits to other users
• Provision for competitive bidding, analysis of bids against
requirements, and bids comparison against each other
• Analysis of vendor’s financial condition, capability to provide
maintenance, support, training
• Review of delivery schedules against requirement
• Analysis of product’s upgrade capability, and security & control
facilities
• Evaluation of performance against requirements
• Review and negotiation of price, review of contract terms (incl.
right to audit)
• Preparation of formal report
11/09/2013 20
21. Infrastructure Development/Acquisition Practices
• Hardware Acquisition
– Criteria that should be considered in the evaluation process:
• Turnaround time
• Response time
• System reaction time
• Throughput
• Workload
• Compatibility
• Capacity
• Utilization
11/09/2013 21
22. Infrastructure Development/Acquisition Practices
• System Software
– System Software Acquisition
• Business, functional and technical needs and specifications.
• Cost / benefits
• Obsolescence
• Compatibility with existing systems
• Security
• Demands on existing staff
• Training and hiring requirements
• Future growth needs
• Impact on system performance and the network
– System Software Implementation
– System Software Change Control Procedures
11/09/2013 22
23. Infrastructure Development/Acquisition Practices
• Change Management Process Overview
– Deploying changes
– Documentation
– Testing changed programs
– Auditing program changes
– Emergency changes
– Deploying changes back into production
– Change exposure (unauthorized changes)
• Configuration Management
– Develop the configuration management plan
– Baseline the code and associated documents
– Analyze and report on the results of configuration control
– Develop the reports that provide configuration status information
– Develop release procedures
– Perform configuration control activities
– Update the configuration status accounting database
11/09/2013 23
25. Process Improvement Practices
• Business Process Reengineering (BPR)
– Steps :
• Define the areas to be reviewed.
• Develop a project plan.
• Gain an understanding of the process under review.
• Redesign and streamline the process.
• Implement and monitor the new process.
• Establish a continuous improvement process.
– Results :
• New business priorities
• Concentration on process
• New approaches to organizing and motivating people
• New approaches to the use of technology
• New approaches to the use of information
• Redefined rules for suppliers
• Often, redefined rules for clients and customers
11/09/2013 25
26. Process Improvement Practices
• Business Process Reengineering (BPR)
– Process :
• Plan
• Research
• Observe
• Analyze
• Adapt
• Improve
– Audit & Evaluation :
• The organization’s change efforts are consistent with the overall
culture and strategic plan
• The reengineering team is making an effort to minimize any
negative impact
• The change management team has documented lessons to be
learned after the completion of the BPR
11/09/2013 26
27. Process Improvement Practices
• ISO 9126
– Provides the definition of the characteristics and associated
quality evaluation process to be used when specifying the
requirements for and evaluating the quality of software
products throughout their life cycle
– Evaluation attributes :
• Functionality
• Reliability
• Usability
• Efficiency
• Maintainability
• Portability
11/09/2013 27
28. Process Improvement Practices
• Software Capability Maturity Model (CMM)
– Initial
– Repeatable
– Defined
– Managed
– Optimizing
• Capability Maturity Model Integration (CMMI)
– Iterative development
– Early definition of architecture
– Model based design notation
– Component based development
– Demonstration based assessment of intermediate development
products
– Use of scalable, configurable processes
11/09/2013 28
29. Process Improvement Practices
• ISO 15504
– Also known as SPICE (Software Process Improvement and
Capability Determination)
– Reference model :
• Software life cycle processes
• System life cycle processes
• Human-centered life cycle processes
• Component-based development processes
• IT service management system processes
• Quality management system processes
• Automotive embedded software
• Medical device software
11/09/2013 29
30. Application Controls
• For ensuring that:
– Only complete, accurate and valid data are entered and updated in a
computersystem
– Processing accomplishes the correct task
– Processing results meet expectations
– Data are maintained
• IS auditor’s tasks:
– Identifying the significant application components and the flow of
transactions through the system and gaining detailed understanding
– Identifying the application control strengths & evaluating the impact
of control weaknesses
– Testing the controls to ensure their functionality and effectiveness by
applying appropriate audit procedures
– Evaluating the control environment to determine that control
objectives were Achieved
– Considering the operational aspects of the application to ensure its
activity and effectiveness
11/09/2013 30
31. Application Controls
• Input/origination controls
– Input Authorization
• Signatures on batch forms or source documents
• Online access controls
• Unique passwords
• Terminal or client workstation identification
• Source documents
– Batch Controls
• Total monetary amount
• Total items
• Total documents
• Hash totals
– Batch Balancing
• Batch registers
• Control accounts
• Computer agreement
11/09/2013 31
32. Application Controls
• Input/origination controls
– Error Reporting & Handling
• Rejecting only transaction with errors
• Rejecting the whole batch of transactions
• Holding batch in suspense
• Accepting batch and flagging error transactions
– Input Controls Techniques
• Transaction log
• Reconciliation of data
• Documentation
• Error correction procedures
• Anticipation
• Transmittal log
• Cancellation of source documents
– Batch integrity in online or database systems
11/09/2013 32
33. Application Controls
• Processing Procedures and Controls
– Data Validation and Editing
• Data validation identifies data errors, incomplete/ missing data
and inconsistencies among related data items.
• Edit controls are preventive controls that are used in a program,
before data are processed.
– Techniques
Sequence check Existence check
Limit check Key verification
Range check Check digit
Validity check Completeness check
Reasonableness check Duplicate check
Table look-ups Logical relationship check
11/09/2013 33
35. Application Controls
• Processing Procedures and Controls
– Data File Controls Procedures
• Before and after image reporting
• Maintenance error reporting and handling
• Source documentation retention
• Internal and external labeling
• Version usage
• Data file security
• One-for-one checking
• Prerecorded input
• Transaction logs
• File updating and maintenance authorization
• Parity checking
11/09/2013 35
36. Application Controls
• Output Controls
– Logging and storage of negotiable, sensitive and critical forms in
a secure place
– Computer generation of negotiable instruments, forms and
signatures
– Report distribution
– Balancing and reconciling
– Output error handling
– Output report retention
– Verification of receipt of reports
11/09/2013 36
37. Application Controls
• Business Process Control Assurance
– Evaluating controls at the process and activity level
– Combination of management, programmed and manual
controls
– Considerations :
• Process maps
• Process controls
• Assess business risks within the best practices
• Roles and responsibilities
• Activities and tasks
• Data restrictions
11/09/2013 37
38. Auditing Application Controls
• Review the following document :
– System development methodology documents
– Functional design specifications
– Program changes
– User manuals
– Technical reference documentation
• Analyze the flow of transaction through the system
• Prepare a risk assessment model to analyze the application’s control
• Observe and test user’s performing procedures:
– Separation of duties
– Authorization of input
– Balancing
– Error control and correction
– Distribution of reports
– Review and test access authorizations and capabilities
11/09/2013 38
39. Auditing Application Controls
• Data Integrity Testing
– Relational integrity
– Referential integrity
• Data integrity in online transaction processing systems
– Atomicity
– Consistency
– Isolation
– Durability
11/09/2013 39
40. Auditing Application Controls
• Test Application System
– Analyzing Computer Application Controls
• Snapshot
• Mapping
• Tracing & tagging
• Test data/deck
• Base case system evaluation
• Parallel operation
• Integrated testing facility
• Parallel simulation
• Transaction selection programs
• Embedded audit data collection
• Extended records
11/09/2013 40
41. Auditing Application Controls
• Continuous online auditing
– Online auditing techniques
• Systems control audit review file and embedded audit
modules (SCARF/EAM)
• Snapshots
• Audit hooks
• Integrated test facilities (ITF)
• Continuous and intermittent simulation (CIS)
11/09/2013 41
42. Auditing System Development, Acquisition and Maintenance
• Determine main components, objectives and user
requirements
• Determine and rank major risks
• Identify controls to mitigate risks
• Advise the project team regarding the design of the system
and implementation of controls
• Monitor the systems development process
• Participate in post-implementation reviews
• Evaluate system maintenance standards and procedures
• Test system maintenance procedures
• Evaluate the system maintenance process
• Determine the adequacy of production library security
11/09/2013 42
43. Auditing System Development, Acquisition and Maintenance
• Project Management
• Feasibility Study
• Requirements Definition
• Software Acquisition Process
• Detailed Design and Development
• Testing
• Implementation Phase
• Post-implementation Review
• System Change Procedures and the Program Migration
Process
11/09/2013 43
45. Business Aplication Systems
• Electronic Commerce
– E-commerce requirements
• Top-level commitment
• Business process reconfiguration
• Links to legacy systems
– E-commerce audit and control issues (best practices)
• A set of security mechanism and procedure (e.g., internet firewalls, PKI,
etc.)
• Firewall mechanism placing to mediate the public network and
organization’s private network
• Process whereby participants in an e-commerce transaction can be
identified uniquely and positively
• Digital signatures, attributes include:
• Unique to the person using it
• Verifiable
• Mechanism for generating & affixing is under sole control of person
using it
• Linked to data, if data are changed, it is invalidated
11/09/2013 45
46. Business Aplication Systems
• Electronic Commerce
– E-commerce audit and control issues (best practices)
• The procedures in place Logs of e-commerce applications
• Methods & procedures
• Features in e-commerce applications
• Protections in place
• Means to ensure confidentiality of data between customers &
vendors
• Features within e-commerce architecture
• Plan and procedure to continue e-commerce activities
• Commonly understood set of practices & procedures
• Shared responsibility within org for e-commerce security
• Regular program of audit & assessment of the security
11/09/2013 46
47. Business Aplication Systems
• Electronic Data Interchange
– General requirements
– Traditional EDI
– Web-based EDI
• EDI Risks and Controls
– Unauthorized access
– Deletion or manipulation
– Loss or duplication
– Loss of confidentiality and improper distribution
• Controls in EDI Environment
– Receipt of inbound transactions
– Outbound transactions
– Auditing EDI
• Audit monitors
• Expert systems
11/09/2013 47
48. Business Aplication Systems
• Electronic mail
– The most heavily used feature of the internet or LANs
– Two principal components
• Mail servers
• Clients
• Security issues of e-mail
– Flaws in the configuration of mail server application
– Denial-of-service (DoS) attacks
– Sensitive information transmitted unencrypted
– Information within the e-mail may be altered
– Viruses and malicious code
– Legal exposure
• Standards for e-mail security
– Digital signatures
– The signature can not be forged
– The signature is authentic and encrypted
– The signature can not be reused
– The signed document can not be altered
11/09/2013 48
49. Business Aplication Systems
• Electronic Banking
– Major risks : Strategic, Reputational, Transactional, Credit, Price,
Foreign exchange, Interest rate, Liquidity
– Risk management
• Risk management
• Implementing technology
• Measuring & monitoring risk
– Risk management challenges in electronic banking
• Speed of change relating to technological and service innovation
• Integrated transactional electronic banking
• Bank’s dependence on information technology
• The internet
– Risk management controls for electronic banking
• Board and management oversight
• Security controls
• Legal and reputational risk management
11/09/2013 49
50. Business Aplication Systems
• E-Finance
– Payment Systems
• The electronic money model of pay system
• The electronic checks model of pay system
• The electronic transfer model of pay system
– Integrated Manufacturing Systems (IMS)
– Electronic Funds Transfer (EFT)
• Controls in EFT Environment
– Integrated Customer File
– Office Automation (OA)
– Automated Teller Machine (ATM)
• Audit of ATM
– Cooperative Processing Systems
– Voice Response Ordering System
– Purchase Accounting System
– Image Processing
11/09/2013 50
51. Business Aplication Systems
• Artificial Intelligence (AI) & Expert Systems
– Benefits of expert systems
– Capturing the knowledge & experience of individuals
– Sharing knowledge & experience
– Facilitating consistent & efficient quality decisions
– Enhancing personnel productivity & performance
– Automating highly repetitive tasks
– Operating in environtments where a human expert is not
available
11/09/2013 51
53. Business Aplication Systems
• Business Intelligent (BI)
– Various layers/component:
– Presentation/desktop access layer
– Data source layer
– Core data warehouse
– Data mart layer
– Data staging and quality layer
– Data access layer
– Data preparation layer
– Metadata repository layer
– Warehouse management layer
– Application messaging layer
– Internet/intranet layer
11/09/2013 53
