1) The document discusses evaluating the effectiveness of information systems by assessing whether they are achieving planned goals, using resources properly, and having the right controls selected. 2) Effectiveness can be evaluated through both relative and absolute approaches by comparing performance before and after implementation or directly assessing goal accomplishment. 3) Key factors that may indicate ineffectiveness include excessive downtime, slow response time, high maintenance costs, inability to interface with new software/hardware, unreliable outputs, and frequent need for maintenance/modifications.

1. INFORMATION SYSTEM
AUDIT-II
CHP 5. EVALUATING SYSTEM EFFECTIVENESS
MR. JAYANT. P. DALVI

2. INTRODUCTION
Need of Evaluation:
= to know if processes or systems of organization are suitable or
effective as input to their improvement process
Definition of Effectiveness and Suitablity:
1) Effectiveness: process or system is achieving planned goals or results
in capable manner with optimum use of resources
2) Suitability: chosen process or system and associated controls are
consistent with internal or external requirements or need

3. AUDIT QUALITY TEST
• 4 things to test during quality audit:
1) Is activity getting desired results?
2) Are resources(man, money, material) being used properly? Are they
cost-effective?
3) Are people be able to do it Right for first time?
4) Have right resources and controls been selected?

4. EFFECTIVENESS EVALUATION PROCESS
• Indicators of ineffectiveness:
- Excessive down time or idle time
- slow system response time
- excessive maintenance cost
- inable to interface with new software or hardware
- unreliable system outputs
- data loss
- frequent need for maintenance and modification

5. How to make system effective
• 2 approaches :
1) Goal centered view: to complete goals/objectives to achieve system
effectiveness, sacrificing of fundamental system qualities
2) System Resource view: identification of desirable qualities of system
and measurement of their levels.

6. TYPES OF SYSTEM EFFECTIVENESS EVALUATION
• Relative Evaluation:
- comparing of status of goal accomplished before implementation with
after implementation, improved task accomplishment and improved
quality of working life
Absolute Evaluation:
-assesment of size of goal accomplish after implementation of system

7. MEETING NEEDS FOR EFFECTIVENESS EVALUATION
• well managed organizations requires continous improvement with the
help of planning, implementing, checking and improving
• evaluations to be done to meet identified needs, execute plans, check
to see that needs were met and made improvements where required
• Benefits:
• Need for effectiveness evaluations continues to grow
• Departments have systematic processes to plan effectiveness
evaluation
• Departments complete most planned evaluations

8. QUALITY ASSURANCE PROCESSES
• It is designed to ensure that quality requirements are being met.
• helps to ensure that reports satisfy defined standards and provide
decision makers with reliable and useful evaluation findings.
• Example: Treasury Board of India orders 6 departments to perform
effectiveness valuation for their IS and give reports with
recommendations or response in it.

9. EVALUATING SYSTEM QUALITY
• Quality System Evaluation is:
• assessment of quality programs and processes continually to fulfill
internal and external expectations
• ensure to minimize risk and consistently produce high quality risk
• maximize profit and minimize losses
• verifies preparation of documentations for global system audits
• satisfy customer requirements and ensure their specification to be
meet consistently
• train employees to optimize quality of product within limits of
production process

10. • IS must be developed in accordance with all informational flows and
circuits and needs of decision systems
• Its quality evaluation can be made for systems that are in use or for
the systems that are in development stage.
• The results of audit process development must be analyzed and
conclusions must be translated into procedure and actions
iplemented in audited information systems.

11. EVALUATING INFORMATION QUALITY
• auditor must plan and perform audit procedures to obtain sufficient
appropriate audit evidence to provide reasonable basis for his opinion
• Sufficiency is measure of quantity of audit evidence
• Appropriateness is measures of quality of audit evidence i.e its
relevance and reliability.
• To be appropriate, audit evidence must be both relevant and reliable
to provide support for conclusions on which auditor's opinion is
based.

12. 1) Relevance: refers to objective of control being tested
it is depends on:
- design of audit procedure used to test the control
- timing of audit procedure used to test the control
2) Reliability: depends on nature and source of evidence and
circumstances under which it is obtained
- evidence obtained from external knowledge source is reliable than
internal company sources
- evidence obtained directly by auditor is more reliable.
- evidence obtained from original documents is more reliable than
photocopies, scanned copies, electronic copies

13. 3) Information Quality: describe quality of content of IS.
it is measure of value which information provides to user of that information
4 classes of IQ:
• Intrinsic IQ: accuracy, objectivity, believable, reputation
• Contexual IQ: relevancy, value added, timeliness, completeness, amt of info
• Representational IQ: Interpretability, Compatibility
• Accessibility IQ: accessibility, access security

14. EVALUATING PERCEIVED USEFULNESS
• Perceived usefulness is user's probability of using specific application will
increase his or her job performance within organizational context
• It is linked with whether user will get reward after using the system and
attitude they have toward the system.
• If they find it favourable they will use system most frequently otherwise not.
• Items to measure perceived usefulness:
users perceive that IS enables them to accomplish task associated with their job
more quickly, improve their job performance, increase their productivity and
effectiveness, useful.
• Auditors can include this items in questionaire or interviews if they want to
evaluate perceived usefulness

15. EVALUATING INFORMATION SYSTEM
SATISFACTION
• end user satifies if the characteristics of sytem are accurace and
reliable.
• end user satisfies when IS consists of web-based products and
services

16. EVALUATING INDIVIDUAL IMPACT
• impact of IT on individual enhances the performance of individual in
terms of productivity, efficiency and effectiveness.
• enhances task productivity, task innovation and individual sense of
accomplishment
• helps individual to create and try new ideas in their work
• enhances sense of togetherness and closeness

17. EVALUATING ORGANIZATIONAL IMPACT
• IT provides advantages to IS like ability of IT to link and enable
employees , communication increased.
• also various people from different department from same or different
organizations can communicate easily.
• quicker availability of information
• eliminates repetition of information because of boundary spanning.
• information can be stored, retrieved and transferred easily and
quickly.

18. EVALUATING COMPUTER SELF-EFFICACY
• it refers to individual's ability in his or her ability to apply computer
skills to wide range of tasks.
• it includes determinants and consequences.
• consequences are grouped into outcomes, beliefs and behaviours.
• here determinant is simple use of computer or frequency of using a
computer
• individual with high self-efficacy will perceive the system to be easy
and useful while individual with low self efficacy will avoid to use the
system
• it affects system usage directly and indirectly through perceived ease
of use and perceived usefullness

19. EVALUATING IS USE
• IS is combination of IT and people's activities that supports
operations, management and decision making.
• IS also referred as interaction between people, process, data and
technology
• it is a form of communication system in which data is processed and
presented to the user.
• IS is essential and beneficial if user resists to change.
• Ethics of IS- privacy, property, accuracy, access, data processing and
decision making, security, disaster recovery.
• usefulness of IS in following fields: finance and accounting, sales and
marketing, HRM, etc.ss