This document discusses system and infrastructure lifecycle management as it relates to the CISA examination. It covers topics such as business realization, project management structures and practices, application development approaches, infrastructure development, maintenance practices, and auditing controls. The key areas covered include ensuring management practices meet organizational objectives, understanding different development approaches, auditing system development and maintenance, and providing assurance on controls.

1. System and Infrastructure
Lifecycle Management
Power Point ini diadopsi dari Auditindo Education
11/09/2013 1

2. Area Objective
• Ensure that the IS Auditor understand and able to
provide assurance that the management practices
for the development/acquisition, testing,
implementation, maintenance, and disposal of
systems and infrastructure will meet the
organization’s objectives.
This area will represent approximately 16% of the CISA
examination (approximately 32 questions)
11/09/2013 2

3. Topic Covered
• Business Realization
• Project Management Structure & Practices
• Business Application Development & Alternative
Approaches
• Alternative Software Project Organization & Development
Methods
• Infrastructure Development/Acquisition Approaches
• Information Systems Maintenance Practices
• System Development Tools & Productivity Aids
• Process Improvement Practices
• Application Controls & Auditing Application Controls
• Auditing System Development, Acquisition and Maintenance
• Business Application
11/09/2013 3

4. Business Realization
• Portfolio/Program Management
– Program is a group of projects and time-bound tasks that are closely
linked together through common objectives, a common budget,
intertwined schedules and strategies. Program is more complex,
longer duration, higher budget, higher risk and higher importance,
than project.
– Program Management Objectives : successful execution of program
scope, financial, schedules, objectives, deliverables, context,
environment, communication, culture, organization.
– Program Organization : Program Owner/Sponsor, Program Manager,
Program Team, Program Office
– Project Portfolio : All the projects being carried out in an organization
at a given point in time (snapshot).
– Project Portfolio Objectives : Optimization of result of project
portfolio, prioritizing & scheduling projects, resource coordination,
knowledge transfer
– Project Portfolio requires : Database & Reports
11/09/2013 4

5. Business Realization
• Business Case Development & Approval
– Should be developed before project commencement
– Derived from Feasibility Study :
• Scope the problem
• Identify & explore a number of solutions
• Make recommendation on what action to take
– Calculate and outline business case for each of aspect of
comparison
– Should be justifying the project and answer the question of
“Why ?”
– Business case may become no longer valid, therefore a project
should has some Decision Points / Stage Gate / Kills Points,
where a business case is reviewed.
– If the business case changes during project, the project should
be reapproved through approval process.
11/09/2013 5

6. Business Realization
• Business Realization Techniques
– Benefits Management or Benefit Realization requires :
• Validating the benefits predicted in the business
• Planning and describing the benefit plan that is to be
realized
• Assigning a measure and target
• Documenting the assumptions
• Establishing key responsibilities for realizations
• Establishing a tracking/measuring regime
– Usually includes a Post-Implementation Review at 6-18
months after implementation.
– There must be a periodic review of benefits
11/09/2013 6

7. Project Management Structure
• Standards : PMBOK & PRINCE2
• Organizations : PMI & IPMA
• General Aspects
• Project Context & Environment :
– Contents, Time and Social
• Project Organizational Forms :
– Influence, Pure, Matrix
• Project Communication & Culture :
– One-on-one meetings, Kick-off meetings, project start workshops, or a
combination, project mission statement, project name & logo, project
team meeting rules & communication protocol, and project specific social
events.
• Project Objectives
– Main Objectives, Additional Objectives, Non-Objectives
– Object Breakdown Structure (OBS)  Work Breakdown Structure (WBS)
 Work Packages To-do List
11/09/2013 7

8. Project Management Structure
• Project Roles & Responsibilities
– Senior Management
– User Management
– Project Steering Committee
– Project Sponsor
– System Development Management
– Project Manager
– System Development Project Team
– User Project Team
– Security Officer
– Quality Assurance
11/09/2013 8

9. Project Management Practises
11/09/2013 9

10. Business Application Development
• An individual application or project is initiated by:
– A new opportunity that relates to a new or existing business process
– A problem that relates to an existing business process
– A new opportunity that will enable the organization to take advantage
of Technology
– A problem with the current technology
• The Traditional System Development Life Cycle Approach:
– Phase 1 Feasibility
– Phase 2 Requirements
– Phase 3A Design
– Phase 3B Selection
– Phase 4A Development
– Phase 4B Configuration
– Phase 5 Implementation
– Phase 6 Post-implementation
11/09/2013 10

11. Business Application Development
• The Traditional System Development Life Cycle Approach:
– Feasibility Study
• Issue to be addressed
• Factors impacting
– Requirement Definition
• Identify & Analyze
• Record & Verify
• Resolve Conflicts
– Entity Relationship Diagram vs Object-Oriented
– Software Acquisition
• Request For Proposal (RFP) or Invitation To Tender (ITT)
• Required HW, supported OS, additional tools, supported DB
• Reliability, Commitment to service, training, technical support &
documentation
• Details of Contract
11/09/2013 11

12. Business Application Development
• The Traditional System Development Life Cycle Approach:
11/09/2013 12

13. Business Application Development
• The Traditional System Development Life Cycle Approach:
– Design
• User involvement in the design
• Software baselining
• End of design phase
• IS auditor involvement
– Development
• Programming methods and techniques
• Online programming facilities (integrated development environment –
IDE)
• Programming language
• Program debugging
• Testing
• Elements of a software testing phase
• Testing Classification
• Other types of testing-related terminology
• Automated application testing
11/09/2013 13

14. Business Application Development
• The Traditional System Development Life Cycle Approach:
– Implementation
• Implementation Planning
• Phase 1 : Gap Analysis, Role Definitions
• Phase 2 : Service Level Agreement, Knowledge Transfer Plan,
Training Plans
• End-user Training
• Data Conversion
• Refining Migration Scenario
• Fallback Scenario
• Cutover (Go-Live) Techniques
• Parallel Changeover
• Phased Changeover
• Absurd Changeover
• Certification/Accreditation
– Post-Implementation Review
11/09/2013 14

15. Business Application Development
• Risks Associated with Software Development
– Within the project
– With suppliers
– Within the organization
– With the external environment
• Use of Structured Analysis Design and Development
Techniques
– Develop system context diagrams.
– Perform hierarchical data flow/control flow decomposition.
– Develop control transformations.
– Develop mini-specifications.
– Develop data dictionaries.
– Define all external events—inputs from external environment.
– Define single transformation data flow diagrams from each
external event.
11/09/2013 15

16. Alternative Application Development Approach
• Alternative Approaches
– Approaches an IS auditor may encounter:
• Incremental or progressive development
• Iterative development
– Data-Oriented System Development
– Object-Oriented System Development
– Component-Based Development
– Web-based Application Development
– Prototyping
– Rapid Application Development
– Agile Development
– Reengineering & Reverse Engineering
11/09/2013 16

17. Infrastructure Development/Acquisition Practices
• Physical Architecture Analysis
– Goals :
• To analyze existing system
• To design a new architecture
• To write functional requirement of new architecture
• To develop proof of concept based on functional requirements
– Project Phases :
• Review of existing system
• Analysis and Design
• Draft Functional Requirements
• Vendor & Product Selection
• Writing Functional Requirements
• Proof of Concept
11/09/2013 17

18. Infrastructure Development/Acquisition Practices
• Planning the Implementation of Infrastructure
– Procurement Phase
• Develop vendor evaluation criteria
• Develop vendor long list & short list
• Select preferred vendor & define partnership
– Delivery Time
• Develop delivery plan
• Review delivery plan
– Installation Plan
• Develop installation plan
• Review installation plan
– Installation Test Plan
• Develop test plan
• Review test plan
11/09/2013 18

19. Infrastructure Development/Acquisition Practices
• Hardware Acquisition
– Invitation to Tender (ITT)
• Organizational description indicating whether the computer
facilities are centralized or decentralized, distributed or
outsourced
• Information processing requirements
• Hardware requirements
• System software requirements
• Support requirements
• Adaptability requirements
• Constraints
• Conversion requirements
11/09/2013 19

20. Infrastructure Development/Acquisition Practices
• Hardware Acquisition
– Acquisition Steps
• Testimonials/visits to other users
• Provision for competitive bidding, analysis of bids against
requirements, and bids comparison against each other
• Analysis of vendor’s financial condition, capability to provide
maintenance, support, training
• Review of delivery schedules against requirement
• Analysis of product’s upgrade capability, and security & control
facilities
• Evaluation of performance against requirements
• Review and negotiation of price, review of contract terms (incl.
right to audit)
• Preparation of formal report
11/09/2013 20

21. Infrastructure Development/Acquisition Practices
• Hardware Acquisition
– Criteria that should be considered in the evaluation process:
• Turnaround time
• Response time
• System reaction time
• Throughput
• Workload
• Compatibility
• Capacity
• Utilization
11/09/2013 21

22. Infrastructure Development/Acquisition Practices
• System Software
– System Software Acquisition
• Business, functional and technical needs and specifications.
• Cost / benefits
• Obsolescence
• Compatibility with existing systems
• Security
• Demands on existing staff
• Training and hiring requirements
• Future growth needs
• Impact on system performance and the network
– System Software Implementation
– System Software Change Control Procedures
11/09/2013 22

23. Infrastructure Development/Acquisition Practices
• Change Management Process Overview
– Deploying changes
– Documentation
– Testing changed programs
– Auditing program changes
– Emergency changes
– Deploying changes back into production
– Change exposure (unauthorized changes)
• Configuration Management
– Develop the configuration management plan
– Baseline the code and associated documents
– Analyze and report on the results of configuration control
– Develop the reports that provide configuration status information
– Develop release procedures
– Perform configuration control activities
– Update the configuration status accounting database
11/09/2013 23

24. Infrastructure Development/Acquisition Practices
• Code Generators
• Computer-aided Software Engineering (CASE)
– Upper CASE, Middle CASE, Lower CASE
• Fourth-generation Languages (4GLs)
– Characteristic
• Nonprocedural language
• Environmental independence (portability)
• Software facilities
• Programmer workbench concepts
• Simple language subsets
– Classified
• Query and report generators
• Embedded database 4GLs
• Relational database 4GLs
• Application generators
11/09/2013 24

25. Process Improvement Practices
• Business Process Reengineering (BPR)
– Steps :
• Define the areas to be reviewed.
• Develop a project plan.
• Gain an understanding of the process under review.
• Redesign and streamline the process.
• Implement and monitor the new process.
• Establish a continuous improvement process.
– Results :
• New business priorities
• Concentration on process
• New approaches to organizing and motivating people
• New approaches to the use of technology
• New approaches to the use of information
• Redefined rules for suppliers
• Often, redefined rules for clients and customers
11/09/2013 25

26. Process Improvement Practices
• Business Process Reengineering (BPR)
– Process :
• Plan
• Research
• Observe
• Analyze
• Adapt
• Improve
– Audit & Evaluation :
• The organization’s change efforts are consistent with the overall
culture and strategic plan
• The reengineering team is making an effort to minimize any
negative impact
• The change management team has documented lessons to be
learned after the completion of the BPR
11/09/2013 26

27. Process Improvement Practices
• ISO 9126
– Provides the definition of the characteristics and associated
quality evaluation process to be used when specifying the
requirements for and evaluating the quality of software
products throughout their life cycle
– Evaluation attributes :
• Functionality
• Reliability
• Usability
• Efficiency
• Maintainability
• Portability
11/09/2013 27

28. Process Improvement Practices
• Software Capability Maturity Model (CMM)
– Initial
– Repeatable
– Defined
– Managed
– Optimizing
• Capability Maturity Model Integration (CMMI)
– Iterative development
– Early definition of architecture
– Model based design notation
– Component based development
– Demonstration based assessment of intermediate development
products
– Use of scalable, configurable processes
11/09/2013 28

29. Process Improvement Practices
• ISO 15504
– Also known as SPICE (Software Process Improvement and
Capability Determination)
– Reference model :
• Software life cycle processes
• System life cycle processes
• Human-centered life cycle processes
• Component-based development processes
• IT service management system processes
• Quality management system processes
• Automotive embedded software
• Medical device software
11/09/2013 29

30. Application Controls
• For ensuring that:
– Only complete, accurate and valid data are entered and updated in a
computersystem
– Processing accomplishes the correct task
– Processing results meet expectations
– Data are maintained
• IS auditor’s tasks:
– Identifying the significant application components and the flow of
transactions through the system and gaining detailed understanding
– Identifying the application control strengths & evaluating the impact
of control weaknesses
– Testing the controls to ensure their functionality and effectiveness by
applying appropriate audit procedures
– Evaluating the control environment to determine that control
objectives were Achieved
– Considering the operational aspects of the application to ensure its
activity and effectiveness
11/09/2013 30

31. Application Controls
• Input/origination controls
– Input Authorization
• Signatures on batch forms or source documents
• Online access controls
• Unique passwords
• Terminal or client workstation identification
• Source documents
– Batch Controls
• Total monetary amount
• Total items
• Total documents
• Hash totals
– Batch Balancing
• Batch registers
• Control accounts
• Computer agreement
11/09/2013 31

32. Application Controls
• Input/origination controls
– Error Reporting & Handling
• Rejecting only transaction with errors
• Rejecting the whole batch of transactions
• Holding batch in suspense
• Accepting batch and flagging error transactions
– Input Controls Techniques
• Transaction log
• Reconciliation of data
• Documentation
• Error correction procedures
• Anticipation
• Transmittal log
• Cancellation of source documents
– Batch integrity in online or database systems
11/09/2013 32

33. Application Controls
• Processing Procedures and Controls
– Data Validation and Editing
• Data validation identifies data errors, incomplete/ missing data
and inconsistencies among related data items.
• Edit controls are preventive controls that are used in a program,
before data are processed.
– Techniques
Sequence check Existence check
Limit check Key verification
Range check Check digit
Validity check Completeness check
Reasonableness check Duplicate check
Table look-ups Logical relationship check
11/09/2013 33

34. Application Controls
• Processing Procedures and Controls
– Processing Controls
• Techniques (completeness & accuracy of accumulated
data) :
• Manual recalculation
• Editing
• Run-to-run totals
• Programmed controls
• Reasonableness verification of calculated amounts
• Limit checks on calculated amounts
• Reconciliation of file totals
• Exception reports
11/09/2013 34

35. Application Controls
• Processing Procedures and Controls
– Data File Controls Procedures
• Before and after image reporting
• Maintenance error reporting and handling
• Source documentation retention
• Internal and external labeling
• Version usage
• Data file security
• One-for-one checking
• Prerecorded input
• Transaction logs
• File updating and maintenance authorization
• Parity checking
11/09/2013 35

36. Application Controls
• Output Controls
– Logging and storage of negotiable, sensitive and critical forms in
a secure place
– Computer generation of negotiable instruments, forms and
signatures
– Report distribution
– Balancing and reconciling
– Output error handling
– Output report retention
– Verification of receipt of reports
11/09/2013 36

37. Application Controls
• Business Process Control Assurance
– Evaluating controls at the process and activity level
– Combination of management, programmed and manual
controls
– Considerations :
• Process maps
• Process controls
• Assess business risks within the best practices
• Roles and responsibilities
• Activities and tasks
• Data restrictions
11/09/2013 37

38. Auditing Application Controls
• Review the following document :
– System development methodology documents
– Functional design specifications
– Program changes
– User manuals
– Technical reference documentation
• Analyze the flow of transaction through the system
• Prepare a risk assessment model to analyze the application’s control
• Observe and test user’s performing procedures:
– Separation of duties
– Authorization of input
– Balancing
– Error control and correction
– Distribution of reports
– Review and test access authorizations and capabilities
11/09/2013 38

39. Auditing Application Controls
• Data Integrity Testing
– Relational integrity
– Referential integrity
• Data integrity in online transaction processing systems
– Atomicity
– Consistency
– Isolation
– Durability
11/09/2013 39

40. Auditing Application Controls
• Test Application System
– Analyzing Computer Application Controls
• Snapshot
• Mapping
• Tracing & tagging
• Test data/deck
• Base case system evaluation
• Parallel operation
• Integrated testing facility
• Parallel simulation
• Transaction selection programs
• Embedded audit data collection
• Extended records
11/09/2013 40

41. Auditing Application Controls
• Continuous online auditing
– Online auditing techniques
• Systems control audit review file and embedded audit
modules (SCARF/EAM)
• Snapshots
• Audit hooks
• Integrated test facilities (ITF)
• Continuous and intermittent simulation (CIS)
11/09/2013 41

42. Auditing System Development, Acquisition and Maintenance
• Determine main components, objectives and user
requirements
• Determine and rank major risks
• Identify controls to mitigate risks
• Advise the project team regarding the design of the system
and implementation of controls
• Monitor the systems development process
• Participate in post-implementation reviews
• Evaluate system maintenance standards and procedures
• Test system maintenance procedures
• Evaluate the system maintenance process
• Determine the adequacy of production library security
11/09/2013 42

43. Auditing System Development, Acquisition and Maintenance
• Project Management
• Feasibility Study
• Requirements Definition
• Software Acquisition Process
• Detailed Design and Development
• Testing
• Implementation Phase
• Post-implementation Review
• System Change Procedures and the Program Migration
Process
11/09/2013 43

44. Business Aplication Systems
• Electronic Commerce
– E-commerce models
• Business-to-customer (B-to-C) relationships
• Business-to-business (B-to-B) relationships
• Business-to-employee (B-to-E) relationships
• Business-to-government (B-to-G) relationships
• Consumer-to-government (C-to-G) relationship
• Exchange-to-exchange (X-to-X) relationships
– E-commerce architectures
– E-commerce Risks
• Confidentiality
• Integrity
• Availability
• Authentication and non-repudiation
• Power shift to customer
11/09/2013 44

45. Business Aplication Systems
• Electronic Commerce
– E-commerce requirements
• Top-level commitment
• Business process reconfiguration
• Links to legacy systems
– E-commerce audit and control issues (best practices)
• A set of security mechanism and procedure (e.g., internet firewalls, PKI,
etc.)
• Firewall mechanism placing to mediate the public network and
organization’s private network
• Process whereby participants in an e-commerce transaction can be
identified uniquely and positively
• Digital signatures, attributes include:
• Unique to the person using it
• Verifiable
• Mechanism for generating & affixing is under sole control of person
using it
• Linked to data, if data are changed, it is invalidated
11/09/2013 45

46. Business Aplication Systems
• Electronic Commerce
– E-commerce audit and control issues (best practices)
• The procedures in place Logs of e-commerce applications
• Methods & procedures
• Features in e-commerce applications
• Protections in place
• Means to ensure confidentiality of data between customers &
vendors
• Features within e-commerce architecture
• Plan and procedure to continue e-commerce activities
• Commonly understood set of practices & procedures
• Shared responsibility within org for e-commerce security
• Regular program of audit & assessment of the security
11/09/2013 46

47. Business Aplication Systems
• Electronic Data Interchange
– General requirements
– Traditional EDI
– Web-based EDI
• EDI Risks and Controls
– Unauthorized access
– Deletion or manipulation
– Loss or duplication
– Loss of confidentiality and improper distribution
• Controls in EDI Environment
– Receipt of inbound transactions
– Outbound transactions
– Auditing EDI
• Audit monitors
• Expert systems
11/09/2013 47

48. Business Aplication Systems
• Electronic mail
– The most heavily used feature of the internet or LANs
– Two principal components
• Mail servers
• Clients
• Security issues of e-mail
– Flaws in the configuration of mail server application
– Denial-of-service (DoS) attacks
– Sensitive information transmitted unencrypted
– Information within the e-mail may be altered
– Viruses and malicious code
– Legal exposure
• Standards for e-mail security
– Digital signatures
– The signature can not be forged
– The signature is authentic and encrypted
– The signature can not be reused
– The signed document can not be altered
11/09/2013 48

49. Business Aplication Systems
• Electronic Banking
– Major risks : Strategic, Reputational, Transactional, Credit, Price,
Foreign exchange, Interest rate, Liquidity
– Risk management
• Risk management
• Implementing technology
• Measuring & monitoring risk
– Risk management challenges in electronic banking
• Speed of change relating to technological and service innovation
• Integrated transactional electronic banking
• Bank’s dependence on information technology
• The internet
– Risk management controls for electronic banking
• Board and management oversight
• Security controls
• Legal and reputational risk management
11/09/2013 49

50. Business Aplication Systems
• E-Finance
– Payment Systems
• The electronic money model of pay system
• The electronic checks model of pay system
• The electronic transfer model of pay system
– Integrated Manufacturing Systems (IMS)
– Electronic Funds Transfer (EFT)
• Controls in EFT Environment
– Integrated Customer File
– Office Automation (OA)
– Automated Teller Machine (ATM)
• Audit of ATM
– Cooperative Processing Systems
– Voice Response Ordering System
– Purchase Accounting System
– Image Processing
11/09/2013 50

51. Business Aplication Systems
• Artificial Intelligence (AI) & Expert Systems
– Benefits of expert systems
– Capturing the knowledge & experience of individuals
– Sharing knowledge & experience
– Facilitating consistent & efficient quality decisions
– Enhancing personnel productivity & performance
– Automating highly repetitive tasks
– Operating in environtments where a human expert is not
available
11/09/2013 51

52. Business Aplication Systems
• Artificial Intelligence (AI) & Expert Systems
11/09/2013 52

53. Business Aplication Systems
• Business Intelligent (BI)
– Various layers/component:
– Presentation/desktop access layer
– Data source layer
– Core data warehouse
– Data mart layer
– Data staging and quality layer
– Data access layer
– Data preparation layer
– Metadata repository layer
– Warehouse management layer
– Application messaging layer
– Internet/intranet layer
11/09/2013 53

54. Business Aplication Systems
• Business Intelligent (BI)
11/09/2013 54

55. Business Aplication Systems
• Decision Support System (DSS)
– Efficiency VS effectiveness
– Decision focus
– DSS frameworks
– Design & Development
– Implementation & use
– Risk factors
– Implementation strategies
– Assessment & evaluation
– DSS Common Characteristic
– DSS trends
• Customer Relationship Management (CRM)
• Supply Chain Management (SCM)
11/09/2013 55