Evaluating Vendor Risks - slides

Evaluating Vendor Risks
Do you know if they have
controls?
May 5, 2010

Introductions

• Relevant Participant Experiences
• Participant Objectives for this class

Copyright 2010 Riebeeck Stevens Ltd

Page 2

Course Objective

To educate participants regarding the nature
of vendor risks and the mechanisms to
effectively assess, manage and control those
risks by providing a learning forum where
individuals with greater audit and third party
individuals with greater audit and third party
assurance experience can share their
knowledge with peers who are interested in
learning about third party assurance and the
different mechanisms and standards available
to accomplish it.

Page 3

1

Today’s Discussion Topics
• Overview of outsourcing arrangements
• Rights to audit
• Diversity of service organizations
• Assessment mechanisms
o SAS 70
SAS 70
o Shared Assessments
o ISAE 3402
• SAS 70 No More
• Conducting an assessment engagement
• Using a third party assessment
• Project management considerations

Page  4

Outsourcing Business Processes


Page  5

Background

• Many entities use outside service organizations
to accomplish tasks that affect the entity’s
management and information system
• In recent years, there has been an increase in
the use of service organizations
the use of service organizations
• Why do you think BPO (business process
outsourcing) has increased so much?
• “Practical IT Auditing” Checklist to evaluate
candidates for outsourcing


Page  6

2

Typical Service Organizations

• Fund accounting agents/Fund administrators
• Custodians/Trustees/Investment advisors
• Transfer agents/Retirement plan record keepers
• Claims processors
• ASPs
• ISPs
• Payroll processors
• Network/Security management
• Thoughts on Cloud Computing Providers?


Page  7

Outsourcing Arrangements

• Total outsourcing – complete business or
business function
• Production outsourcing – Call centers
• Processing outsourcing – Payroll
• Recordkeeping outsourcing – Transfer agent
• Reporting outsourcing – FISERV and Crawford
Technologies
• Physical Facilities outsourcing – Hosting/Co‐
location

Page  8

Sample Outsourcing Agreements

• 2002: $4 billion / 7‐year utility based deal between
American Express and IBM
• 1998: $3 billion application development and
maintenance agreement between BellSouth and
Andersen Consulting
• 1998: $4 billion infrastructure outsourcing agreement
between BellSouth and EDS
• 1996: $4.5 billion / 10 year outsourcing and strategic
alliance agreements between Dupont and CSC and
Andersen Consulting
• 1994: $3 billion / 10‐year IT services between Xerox and
EDS

Page  9

3

Classification of Vendor Risks

• Operational Risk
• Reputation Risk
• Strategic Risk
• Compliance Risk
Compliance Risk
• Financial Risk
• Support Risk


Page  10


• Operational Risk ‐ Operational risk not only
includes operations and transaction
processing, but also areas such as customer
service, Information Technology security and
the protection of non‐public data, systems
development and support programs, internal
control processes, and capacity and
contingency planning.


Page  11


• Reputation Risk – Errors, delays, or omissions
in outsourced services that become public
knowledge or directly affect the company's
customers can significantly affect reputation.
For example, a vendor s failure to maintain
For example a vendor's failure to maintain
adequate service levels and contingencies for
key items such as cash deliveries, network
hardware devices or ATM servicing could
disrupt the ability to deliver service to
customers.

Page  12

4


• Strategic Risk – Inadequate management
experience and expertise can lead to a lack of
understanding of key risks facing the industry
today and into the future. Additionally,
inaccurate information from vendors can
cause the company's management and board
of directors to make poor strategic decisions.


Page  13


• Compliance Risk – Outsourced activities that
fail to comply with legal or regulatory
requirements can subject the company to
legal sanctions. For example, inaccurate or
untimely consumer compliance disclosures
or unauthorized disclosure of confidential
customer information could expose the
company to civil money penalties or
litigation.

Page  14


• Financial Risk – financial strength of the
vendor, cash position, credit rating,
bankruptcy history, historical financial
performance indicators – return on equity,
return on investment, return on assets


Page  15

5


• Support Risk – ability to perform according to
service level agreements, professional
diversity and capacity of staff, experienced of
workers, staff rotation policy, operational
performance in the market – are they losing
customers, is their quality falling


Page  16

Rights to Audit

• Contract clause allowing the user
organization to audit or have access to audits
of the services contracted
• Should be a standard part of every
outsourcing contract
outsourcing contract
• Use more frequently
• Demanding specific types of audits
• Make sure you are specific in terms of period
of audits

Page  17

Case Study
New York ‐ 30 Dec 2002: J.P. Morgan Chase & Co. today finalized with IBM
a groundbreaking seven‐year outsourcing agreement, in excess of $5
billion, the largest of its kind. The agreement will enable JPMorgan Chase
to transform its technology infrastructure through absolute costs savings,
increased cost variability, access to the best research and innovation, and
improved service levels. By moving from a traditional fixed‐cost approach
to one with increased capacity and cost variability, JPMorgan Chase will be
able to respond more quickly to changing market conditions.
able to respond more quickly to changing market conditions

JPMorgan Chase will outsource a significant portion of its data processing
technology infrastructure, including data centers, help desks, distributed
computing, data networks and voice networks. The agreement includes
the transfer of approximately 4,000 JPMorgan Chase employees and
contractors as well as selected resources and systems to IBM in the first
half of 2003. Application delivery and development, desktop support and
other core competencies will largely be retained inside JPMorgan Chase.

Page  18

6

Case Study ‐ Instructions

• Study the JPM/IBM press release
• Identify the key risks faced by JPM when
transferring functions to IBM
• Discuss methods JPM can use to stay informed
of controls at IBM to address those risks
• Discuss impact to security, audit and compliance
• Should JPM require IBM to include a right to
audit clause in their contract? Why?


Page  19

Summary

After completing this module, you should now:
• Understand the business drivers behind the
outsourcing decision
• Understand the various types of outsourcing
arrangements
• Understand the key classes of vendor risk
• Begin to understand the need to evaluate
controls at service organizations


Page  20

Assessment Mechanisms


Page  21

7

Definition of Key Players

Service Organization – The entity that provides
services to a user organization
Subservice Organization – An entity that is a
service organization of another service
o ga at o
organization
Service Auditor – Reports on the processing of
transactions by a service organization
User Organization – The entity that has engaged
a service organization
User Auditor – Auditor of a user organization


Page  22

Key Players

User Organization Service Auditor

Service Organization

Subservice
User Auditor Organization


Page  23

Evaluating Internal Control
at Service Organizations

• How can a user of a service organization (and its
internal/external auditor) obtain a sufficient
level of comfort that there is an effective control
environment at the service organization?
• How can user management ensure that
How can user management ensure that
outsourced processes are managed following
policies, procedures and practices that are
aligned with those of his/her own company?


Page  24

8

Assessment Mechanism:
Traditional Approach

• User management submits an internal
control questionnaire to service organization
• Service organization provides a self‐
assessment report to clients
• User organization management (internal
audit) performs audit procedures at service
organization
• User auditor performs audit procedures at
service organizations

Page  25

Assessment Mechanisms:
Third Party Assurance Approach

• One independent firm (third party) is
brought in to issue an opinion as to
whether management’s description of
the control environment is presented
fairly.
• In many cases, the independent firm is
also engaged to perform tests of specific
controls and report on the result of
those tests.

Page  26


• Agreed‐Upon Procedures
• Shared Assessments
• Standard Compliance Audit
• SAS 70
SAS 70
• Attestation
• Who can issue reports using these
mechanisms?


Page  27

9


• Agreed‐Upon Procedures
Issued by independent CPA
• Shared Assessments
Issued by independent  CPA or assessment firm
• Standard Compliance Audit
Standard Compliance Audit
Issued by certified party – i.e. PCI and ISO
• SAS 70
Issued by CPA or CA
• Attestation
Issued by CPA or CA


Page  28

Module Summary

• Understand the process to evaluate internal
controls at Service Organizations
• Understand the basic concepts of Third Party
Assurance (TPA)
( )
• Identify different mechanisms for conducting
TPA engagements
• Understand who can issue third party
assurance reports

Page  29

Agreed‐Upon Procedures


Page  30

10

What are Agreed Upon Procedures

• Section 201 of the AICPA Statements on Standards
for Attestation Engagements (SSAE)
• An agreed‐upon procedures engagement is one in
which a practitioner is engaged by a Responsible
Party to issue a report of findings based on
Party to issue a report of findings based on
specific procedures performed on subject matter.
The Responsible Party engages the practitioner to
assist Specified Parties in evaluating subject
matter or an assertion as a result of a need or
needs of the Specified Parties.

Page  31

What is an AUP Report

• An AUP Report is a report issued according to
SSAE 10 Section 201
• An AUP Report contains the procedures
agreed‐upon by the parties and the findings
identified by the auditor
• An AUP Report does not contain an opinion
from the auditor just the facts of the results


Page  32

Who Uses a AUP report

• Agreed‐Upon procedures are used by the
service organization, user management,
external auditors and regulators
• Internal users include senior management,
compliance, internal audit, security and risk
management
• External users typically limited to external
auditors and regulators


Page  33

11

Distribution of the Report

• As an Attestation report, AUP reports have
limited distribution
• The Service Organization and the specified
parties can have access to the report
• Other parties interested in the report need
to agree as to the sufficiency of the
procedures with respect to the subject
matter or assertion prior to receiving the
report

Page  34

AUP Auditor’s Responsibilities

• Carry out the procedures
• Report the findings in accordance with the
professional standards (general, fieldwork
and reporting)
• Adequately plan and supervise the audit and
exercise due professional care in performing
the procedures, determining the findings,
and preparing the report


Page  35

AUP Auditor’s Responsibilities

• Risk that misapplication of the procedures may
result in inappropriate findings being reported
• Risk that appropriate findings may not be
reported or may be reported inaccurately
• These risks are reduced by becoming
These risks are reduced by becoming
knowledgeable about the subject matter and
thoroughly planning and executing the work
• The AUP Auditor has no responsibility to
determine completeness or adequacy of the
agreed‐upon procedures

Page  36

12

Layout of a Typical AUP Report

• A title that includes the word independent
• Identification of the specified parties
• Identification of the subject matter (or the
written assertion related thereto) and the
written assertion related thereto) and the
character of the engagement
• Identification of the responsible party
• A statement that the subject matter is the
responsibility of the responsible party

Extracted from “AICPA Attestation Standards Section 201”
Page  37


• A statement that the procedures performed were
those agreed to by the specified parties identified
in the report
• A statement that the agreed‐upon procedures
engagement was conducted in accordance with
engagement was conducted in accordance with
attestation standards established by the AICPA
• A statement that the sufficiency of the procedures
is solely the responsibility of the specified parties
and a disclaimer of responsibility for the
sufficiency of those procedures
Page  38


• A list of the procedures performed (or reference
thereto) and related findings (The practitioner
should not provide negative assurance
• Where applicable, a description of any agreed‐upon
materiality limits
materiality limits

Page  39

13


• A statement that the practitioner was not engaged
to and did not conduct an examination of the
subject matter, the objective of which would be the
expression of an opinion, a disclaimer of opinion on
the subject matter, and a statement that if the
the subject matter, and a statement that if the
practitioner had performed additional procedures,
other matters might have come to his or her
attention that would have been reported

Page  40


• A statement of restrictions on the use of the report
because it is intended to be used solely by the specified
parties
• Where applicable, reservations or restrictions
concerning procedures or findings.
• For an agreed‐upon procedures engagement on
prospective financial information.
• Where applicable, a description of the nature of the
assistance provided by a specialist.
• The manual or printed signature of the practitioner's
firm
• The date of the report
Page  41

Procedures to be Performed

• Can be as limited or as extensive as the specified
parties desire
• Mere description of assertion or subject matter
does not constitute a valid procedure
• There is flexibility in determining the procedures
There is flexibility in determining the procedures
• Changes to the procedures are acceptable as long
as the specified parties accept responsibility for the
sufficiency of the procedures
• Matters that need to be agreed upon include the
nature, timing and extent of the procedures


Page  42

14

Procedures to be Performed

• Procedures should not be subjective and
open to interpretations
• Terms of uncertain meaning (such as general
review, limited review or check) should be
avoided
• For each procedure, there should be
evidential matter supporting the finding or
findings
Let’s explore the Q‐Services report

Page  43

Project Management Considerations

• Use Of a Specialist
• Internal Auditors and Other Personnel
• Findings
• Working Papers
Working Papers


Page  44

AUP Sample Findings

• Procedure: Inspect the shipment dates for a
sample (agreed‐upon) of specified shipping
documents, and determine whether any such
dates were subsequent to December 31, 20XX.
• Finding (Appropriate description): No shipment
dates shown on the sample of shipping
documents were subsequent to December 31,
20XX.
• Finding (Inappropriate description): Nothing came
to my attention as a result of applying that
procedure.
• Sample findings matrix from AT 201

Page  45

15

AUP Auditor Considerations

• Validate that the Specified Parties have agree to the
procedures
• Document the steps taken in performing the
procedures
• Obtain and maintain appropriate evidence of the
Obtain and maintain appropriate evidence of the
work conducted
• Ensure all changes to the procedures are approved
by the Specified Parties
• Obtain representations from management


Page  46

Using a AUP Report

• A AUP Report contains the results of applying
the procedures only – No Opinion
• Each procedure and related result must be
evaluated by the user in the context of its
entity’s internal control
• Be careful not to extrapolate the findings to
systems or dates not related to the AUPs


Page  47

AUP Exercise

• With the JPM/IBM agreement, multiple systems are
being processed and supported at IBM
• You work for JPM and some of your clients (your team
members) want to audit the system at IBM to evaluate
the security controls at IBM
• Identify and describe 5 audit procedures and discuss
them in your group until everyone agrees they are
sufficient to meet your objective
• Ensure the wording of the procedures is specific and
avoid vague terms
• Draft the result of applying the procedure and share
them with the group

Page  48

16

Module Summary

After completing this module, you now have an
understanding of:
• What Agreed‐Upon Procedures are
• What an AUP Report is
• The content of AUPs
• The responsibilities of the AUP Auditor
• Key considerations of managing an AUP
project
• The usability of AUP reports

Page  49

Shared Assessments


Page  50

Shared Assessments

• Special application of the AICPA AUP
standard
• Shared Assessments is a program created by
BITS, a division of the Financial Services
Roundtable
• Initially targeted the financial services
industry, it is quickly expanding to other
industries such as health care
• Program managed by the Santa Fe Group

Page  51

17

Shared Assessments

• Standardized Information Gathering (SIG)
Questionnaire
• Agreed‐Upon Procedures (AUP)
• Created under the principle of getting
Created under the principle of getting
everyone involved
• Sort of like Skype and IP telephony, when
everyone is connected, there is no need to
pay for phone service


Page  52

Who uses a Shared Assessments Report?

• SIG is used by the Service Organization and
the Outsourcer
• AUP report can be used by all related parties
who approved the procedures
• Limited distribution report – others can use it
but need to agree to the sufficiency of the
procedures to evaluate the related controls


Page  53

Shared Assessments Risk Domains

• Information security policy
• Organization of information security
• Asset management
• Human resources security
• Physical and environmental security
• Communications and operations management
Communications and operations management
• Access control
• Information systems acquisition, development and
maintenance
• Information security incident management
• Business continuity management
• Compliance
• Privacy

Page  54

18

Shared Assessments Project

• Scoping questions – determine:
• Service provider and its business model
• Target systems and processes
• Data that it collects, stores, uses, shares, transports,
retains, secures and/or deletes:
retains, secures and/or deletes:
o Target Data
o Protected Target Data
o Privacy Target Data
o Protected Privacy Target Data
• Based on this information, identify hardware,
software and procedures to be tested.

Page  55

Shared Assessments Lite

• SIG v5 Level 1
• Contains 91 questions
• Intended for low risk scenarios
• Inquiry of Service Organization management
Inquiry of Service Organization management
• No testing is involved

SIG v5 L1 Questions


Page  56

Shared Assessments AUP

• Full SIG v5 and management tools
• AUP v5
• 12 Risk Domains
• Specific procedures to be executed by assessor
• Each AUP control area contains:
o Objective(s): Statement(s) describing the business interest
Objec e(s) S a e e (s) desc b g e bus ess e es
behind assessing the Domain
o Control(s): Statement(s) about the controls service
providers should have in place
o Procedure(s): The action or actions a practitioner will
perform to test each control Area
o Industry Relevance: Reference(s) to other standards that
apply to the same objective and control as the procedure


Page  57

19

Shared Assessments Sample Procedure

F.5 Secure Workspace Access Reporting
Objective:
An organization should maintain access and
incident reports.
incident reports.
Control:
Access to Secure Workplace is logged and
incident reports are maintained.

.

Extracted from the Shared Assessments AUP document
Page  58


Procedures:
a. Obtain the access and incident logs (physical or electronic)
from the service provider for the Secure Workspace Perimeter,
and inspect for evidence of the following attributes:
Access Logs (Staff):
1. Name
2. Date and time
2 d i
3. Point of access
4. Date of last update
Access Logs (Visitor):
1. Name
2. Date and time
3. Point of access
Page  59


4. Company name
5. Visiting
6. Equipment
7. Sign out and return of badge
Incident Logs:
1. Name
2. Date and time
3. Company name
4. Incident type
b. Report the attributes listed in step a not in evidence, the
date the access logs and incident log was last updated, or
the nonexistence of the access log or incident log.
Page  60

20

Shared Assessments

Exercise
• Review the JPM/IBM outsourcing
arrangement and based on the limited
information provided, review the questions
on Section C2.2 of SIG v5 and the
corresponding procedures in Section C of
Shared Assessments AUP v5
• Could this provide any comfort when
performed by a trusted party?

Page  61

Shared Assessments Report Layout

• The Shared Assessments report follows the
AUP standard of the AICPA
• Description of scope
• Domain area
• Control objective
• Control
• Procedure
• Results of applying the procedure


Page  62

Using a Shared Assessments Report

• The Shared Assessments report does not
provide assurance just attestation of the result
• Each user of the report must evaluate the
results in the context of their own risk universe
• Some controls may be applicable others may
Some controls may be applicable others may
not
• The absence of certain controls may not be
relevant to the user’s environment
• Do not extrapolate in time and space


Page  63

21

Using a Shared Assessments Report

• Limitations of the Shared Assessment Report
• Limited to Security, business continuity and
privacy
• No third party opinion
• Can it be relied upon for purposes of an audit of
C it b li d f f dit f
financial statements? Only if issued by CPA?
What about internal audit of the user
organization?
• What about sub‐service organizations? What
options are there to report on that relationship?

Page  64

Module Summary

After completing this module, you should now
understand:
• What are Shared Assessments
• What is a Shared Assessments Report
• The content of a Shared Assessments Report
The content of a Shared Assessments Report
• The responsibilities of the Shared Assessments
Auditor
• Key considerations of managing a Shared
Assessments project
• The usability of Shared Assessments reports

Page  65

SAS 70 Audits


Page  66

22

What is “SAS 70”?
• Statement on Auditing Standards (SAS) No. 70,
Service Organizations, as amended
• Issued by the American Institute of Certified
Public Accountants (AICPA)


Page  67

What is a “SAS 70” Report?
A report containing:

• Description of the control environment
• Description of management’s control objectives
• Description of specific controls, policies and
procedures
• Description of tests of those specific controls,
policies and procedures
• Results of those tests
• Independent auditor’s opinion
• Supplemental information provided by the Service
Organization (optional)


Page  68

Who uses the SAS 70 report?

Primary external users (outside of service organization)
• Clients of service organizations and their auditors
• Auditors of service organization
• Prospective clients of service organizations


Page  69

23


Benefits of the report to external users
• Enhanced understanding of the control
environment
• Additional level of comfort
Additional level of comfort
• Contained audit costs
• Ability to compare service organizations
• Reliance on controls


Page  70


Primary internal users (within service organization)
• Management
• Internal Audit
• Legal and Compliance
• Risk Management
• Marketing


Page  71


Benefits of the report to internal users
• Independent evaluation of processes and controls
• Standard documentation of processes and controls for
future evaluation of efficiencies
• Improved risk management
I d ik t
• Potential reduction of coordination with your client’s
auditors
• Marketing


Page  72

24

Distribution of the Report

Controlled by service organization
Generally limited to:
• Service organization
• Clients of service organization
• Auditors of clients of service organization
• Prospective clients of service organization


Page  73

Types of Reports

• Type I  – Report on Controls placed in
Operation as of a specified date

• Type II – Report on Controls placed in
Type II Report on Controls placed in
Operation as of a specified date
AND
Results of Tests of Operating Effectiveness
during a specified period


Page  74

Service Auditor’s Responsibilities:
Type I  Engagement

• Determine whether the description of controls
presents fairly the relevant aspects of the
controls placed in operation as of the date of
report

• Determine whether the controls are suitably
designed to achieve the specified control
objectives


Page  75

25

Service Auditor’s Responsibilities :
Type II Engagement

• Same as in Type I Engagement
AND
• Determine whether the controls that were
tested were operating with sufficient
tested were operating with sufficient
effectiveness to achieve control objectives
for the specified period of the report


Page  76

Sub‐Service Organizations: Carve‐out

• Exclude sub‐service organization’s relevant controls and
control objectives from report and from auditor’s scope
• If Carve‐Out sub‐servicer, then:
 Modify scope paragraph in the auditor’s report for the controls of
the sub‐service organization
o Describe the functions and nature of processing performed by sub‐
service organization
i i ti
o That the description of the controls includes only the controls and
related control objectives of the service organization
o That our examination does not extend to the controls at the sub‐service organization
 Service Organization modifies description of controls to summarize
the functions and nature of the processing performed by the sub‐
service organization that are omitted from the report
• May be necessary to modify opinion paragraph in auditor’s
report

Page  77

Sub‐Service Organizations: Inclusive

• Include sub‐service organization’s relevant controls and
control objectives in report and in auditor’s scope
• Ensure description of controls and control objective
discussion in report clearly differentiates controls at service
organization and at sub‐service organization, but includes
both in reporting
• Modify auditor s report throughout (scope, opinion, Company
Modify auditor’s report throughout (scope opinion Company
references) to include sub‐service organization (and its
related controls, etc.)
• Perform procedures at the sub‐servicer to determine
whether:
 controls (functions/nature of processing and controls)  are fairly
presented
 controls are suitably designed to achieve the related control objectives
 controls are operating with sufficient effectiveness (For Type II
engagements)

Page  78

26

User Control Considerations

• Complementary Controls that may be
required at the User Organization
• Include in report’s description of controls
Include in report s description of controls
• Include in auditor’s report
• Sample UCC: User Organization should
remove terminated employees when access
no longer needed


Page  79

Service Auditor’s Responsibilities

• Addressing the representations in the service
auditor’s report
• Adhere to the AICPA general standards and
Adhere to the AICPA general standards and
with the relevant AICPA fieldwork and
reporting standards


Page  80

Layout of Typical SAS 70 Report

Opinion
Section I – Information provided by the Service Organization
 Overview of the business
 Control Environment
 Applicability of Report
 Description of Controls
D i ti fC t l
Section II – Information Provided by the Service Auditor
Section III – Controls, Control Objectives and Tests of
Operating Effectiveness
Section IV – Other information provided by the Service
Organization


Page  81

27

Module Summary

After completing this module, you should now be
able to:
• Understand the basic SAS 70‐related terms and
definitions
• Understand the basic overview of SAS 70
• Understand who uses SAS 70 reports and why


Page  82

Project Management:
Useful information for the
Useful information for the
Service Auditor Engagement Team


Page  83

Define and Understand
Engagement/Report Scope

Collaborative process with the Client
 Scope should be driven by USER needs and
requirements
o Include Core Areas
o Include desired Locations


Page  84

28

Engagement Time Management

Time Management
• Activity Definition
• Activity Sequencing
• Activity Duration Estimating
• Schedule Development
S h d l D l t
• Schedule Control


Page  85

Service Organization Involvement

• Project Sponsor (leader/owner) of the
Process
• Project Coordinator (daily task
management)
• Internal Pre‐Assessment and Remediation
• “Buy‐In” of Senior Management within all
functional departments/areas


Page  86

Senior Management Buy‐In

• Assists in obtaining information timely
• Ensures right personnel/contacts are met
• Ensures personnel/contacts will provide all
necessary assistance
• Ensures personnel/contacts know the
importance of the project to their department
leaders


Page  87

29

Responsibilities

May impact:
• Timing
• Deadlines
• Budgets/fees
• Staffing mix
• Expectations set by client or by auditor
• Satisfaction with meeting expectations and
• The ability to manage expectations


Page  88

Reporting Responsibilities

Generally, Client should draft most areas the Report
• Overview of Operations (Organization Definition)
• Description of Controls and Control Environment
• Control Objectives and Controls
• Other Information provided by the Service Organization
Other Information provided by the Service Organization
Generally, the Service Auditor should focus on:
• Opinion
• Information Provided by Service Auditor
• Testing of Controls and Results of Testing


Page  89

Managing Expectations

• Expectations of Significant Changes During Report
Period (mid‐year significant changes in
controls/processes to consider)
• Presence of Exceptions in the Report
• Multi‐location Considerations
• Report is evolving
• Recommendations to be Provided to Client
• Regular Status Meetings with Project Champion and
Day‐to‐Day Contact Person is important


Page  90

30

Managing Expectations

• Timeline/Deadline for Stages of Engagement
 Setting project milestones minimizes time overages
• Detailed Project Plan by Control Objective
 Breaking down project plan to task level increases
Breaking down project plan to task level increases
accuracy of cost estimation and subsequent budgeting
• Monitor Timing/Fees (budget to actual)
 Enhanced cost control through frequent budget to actual
monitoring


Page  91

Module Summary

• Understand key aspects of managing a SAS 70
project effectively and efficiently.
• Understand common pitfalls/challenges and
p / g
successes that we have encountered in our
experience with SAS 70 engagements.


Page  92

Service Auditor Considerations


Page  93

31

Service Auditor Considerations

• Workpaper documentation
• Design of Tests
• Types of tests
• p g
Sampling
• Findings
• Testing strategies


Page  94

Design of Tests

Control           Test


Page  95

Types of Tests

• Inquiry
• Inspection
• Observation
• p
Re‐performance of the control


Page  96

32

Sample Sizes

• No definitive guidance
• Driven by four variables
 Significance of control
 Frequency
q y
 Past experience
 Client expectation


Page  97

Sample Sizes (continued)
• Frequently used numbers (influenced
primarily by SOX developments):

Type of Control
Primary Secondary Other

25 15 5

Page  98

Findings

Findings should be classified into:
• Nominal
• Management Letter Comment (“MLC”)
• Exceptions


Page  99

33

Findings (continued)

• Quantitative materiality thresholds do not
apply
• How to deal with exceptions
 Identify compensating controls
y p g
 Redefine control objectives
 Timely validation


Page  100

Testing Strategies

• Report must be applicable to internal
controls in place during the entire testing
period.
• Narrative update can occur at six month
point
• Controls can be tested at any time during the
testing period


Page  101

Module Summary

• Understand important items to consider when
performing a SAS 70 engagement including
sample sizes, testing strategies and addressing
findings.


Page  102

34

User Auditor Considerations:
How to Use a SAS 70 Report
Ho to Use a SAS 70 Report


Page  103

Is the SAS 70 Useful?

• Address the applications and/or locations used by
the Service Organization that are relevant to
financial statement assertions?
• Adequate to understand flow of transactions?
• Sufficient detail of controls that prevent or detect
p
possible errors?
• Are there findings within control tests?
• Does opinion address any exceptions?
• Are any areas being carved‐out?


Page  104

Procedures when using a SAS 70 Report

• Read report to:
• Understand the flow of transactions and the controls
• Determine that controls were operating as intended
• Determine whether significant control deficiencies
were noted
• Inquire of client as to changes since date of SAS 70
• Consider whether additional procedures are
necessary


Page  105

35

Assessing User Control Considerations

• Read service auditor’s report to determine:
 Whether the considerations are relevant to your
client
o If relevant, ensure during your planning that the
controls have been implemented by the client
controls have been implemented by the client
 Nature of complementary controls that should
be in place at our client


Page  106

Updating a SAS 70

When date of SAS 70 report is within the client’s
fiscal year (and assessed controls as effective):
• Update through client discussions
When date of SAS 70 is outside of our client’s
fiscal year (and anticipate assessing controls as
effective):
• Can use the report as a starting point in gaining
an understanding of the control environment
• You may not rely on this report as audit evidence

Page  107

Using a SAS 70 Report

READ IT!
READ IT!
READ IT!
READ IT!

Page  108

36


• Make sure you understand which significant
processes are covered
• Can you rely on the testing which was
performed?
• Determine the results of any testing that was
performed


Page  109


• If the report does not cover the entire period
of the user organization’s fiscal year, gain an
understanding for the period not covered.


Page  110

Module Summary

• Understand when you can rely on a SAS 70
report.
• Understand the documentation requirements
q
when leveraging a SAS 70 report.
• Understand how you can benefit from a SAS
70 report.
Discuss the SAS 70 Reliance Decision Tree


Page  111

37

Attest Engagement


Page  112

What is an Attest Engagement?

• Examination, audit or review of subject
matter or management assertion
• Higher level of assurance
• Generally includes an opinion of the auditor
Generally includes an opinion of the auditor
• Follows the Statement on Standards for
Attestation Engagements of the AICPA


Page  113

Why Do We Need Attest Reports?

• Many financial situations require an attest
report
• In the controls space, they can cover areas
that are not possible to cover in SAS 70 or
other reports
• An example is business continuity planning
and the availability principle


Page  114

38

Who uses Attest Reports?

• Attest reports are limited distribution reports
• Can be used by external auditors for
evaluating audit risk
• Can be used by the service organization
Can be used by the service organization
management
• Can be used by the user organization
management


Page  115

Attest Engagements

Definition and Underlying Concepts
• Subject matter
• Assertion
• Responsible party


Page  116

Attest Engagements

• Suitability of Criteria
 Objectivity
 Measurability
 Completeness
 Relevance
• Availability of Criteria


Page  117

39

Attest Auditor Responsibilities

• Training and proficiency
• Adequate knowledge of the subject matter
• Independence
• Due professional care
Due professional care
• If report issued according to the AICPA
standard then auditor should be a CPA


Page  118

Layout of Attest Report

• Differences in content for an Examination
and a Review report
• Considerations as to whether opining on
subject matter or management assertion
• Statement that the work conducted supports
the opinion provided
• Compliance with AICPA standards


Page  119

Project Management Considerations

• Obtain clear management assertion
• Ensure there are suitable criteria
• Delineate an plan every activity
• Discuss and walkthrough every risk and area
Discuss and walkthrough every risk and area
of control
• Establish a clearly defined timeline
• Obtain concurrence from management on all
identified findings

Page  120

40

Attest Auditor Considerations

• Planning and supervision
• Obtaining sufficient evidence
• Management representations
• Reporting
• Analysis of other information presented by
management


Page  121

Using an Attest Report

• Ensure focus and scope are relevant
• Review criteria
• Evaluate findings
• Consider period of the attestation
Consider period of the attestation
• Determine whether subsequent events
occurred
• Integrate controls in the report with risks in
your organization

Page  122

Module Summary

able to understand:
• What are Attest engagements
• What is an Attestation Report
• The content of an Attestation Report
• The responsibilities of the Attest Auditor
• Key considerations of managing a Attest
project
• The usability of Attest reports

Page  123

41

Good Bye SAS 70


Page  124

SAS 70 No More

• Recent Developments
• International Demand
• IFAC ‐ ISAE 3402
• AICPA SSAE 16 – Reporting on Controls at a
AICPA SSAE 16 – Reporting on Controls at a
Service Organization
• New SAS – Audit Considerations Relating to
an Entity Using a Service Organization


Page  125

SAS 70 No More

• New Standards do not affect inquiries of
management
• New Standards do not affect AUP/Shared
Assessments
• New Standards do not affect the Attest
Engagements


Page  126

42

AICPA SSAE 16

• Separates Service Audit from existing SAS
• Falls under different family of standards
• Instead of an audit standard, it is an attest
standard
• Requires a written management assertion
• And suitable criteria
• Does not consider the usability in a financial
statement audit ONLY

Page  127

SSAE 16 – Impact

• Management of the service organization required
to provide the service auditor with a written
assertion about
1. The fairness of the presentation of the description of
the service organization’s system
g y
2. The suitability of the design of the controls to
achieve the related control objectives stated in the
description, and, in a type 2 engagement
3. The operating effectiveness of those controls to
achieve the related control objectives stated in the
description.


Page  128

SSAE 16 – Impact

• A service auditor is able to report on controls
at a service organization other than controls
that are relevant to user entities’ financial
reporting, for example, controls related to
user entities’ regulatory compliance,
production, or quality control.
• This is probably the greatest benefit of all!


Page  129

43

SSAE 16 – Impact

• In a type 2 report, the service auditor’s
opinion on the fairness of the presentation of
the description of the service organization’s
system and on the suitability of the design of
the controls is for a period of time rather
than as of a specified date, as is the case in
the current standard


Page  130

SSAE 16 – Impact

• When obtaining an understanding of the
service organization‘s system, the service
auditor would be required to obtain
information to identify risks that the
description of the service organization’s
system is not fairly presented or that the
control objectives stated in the description
were not achieved due to intentional acts by
service organization personnel.

Page  131

SSAE 16 – Impact

• Indicates that when assessing the operating
effectiveness of controls in a type 2
engagement, evidence obtained in prior
engagements about the satisfactory
operation of controls in prior periods does
not provide a basis for a reduction in testing,
even if supplemented with evidence
obtained during the current period.


Page  132

44

SSAE 16 – Impact

• A service auditor’s type 2 report would
identify the customers to whom use of the
report is restricted as "customers of the
service organization’s system during some or
all of the period covered by the service
auditor’s report,"and in a service auditor’s
type 1 report, as, "customers as of the date
of the service organization’s description
covered by the report."

Page  133

SSAE 16 – Key Considerations

• Effective date – the AICPA/ASB has proposed
making the SSAE effective concurrently with
the new ISAE 3402
• Management assertion – An assertion‐based
engagement includes an explicit
acknowledgement by management of its
responsibility for the matters addressed in its
assertion
• Convergence with International Standards

Page  134

IFAC – ISAE 3402

• ISAE 3402 – Assurance Reports on Controls at
a Service Organization
• Based on original structure of SAS 70 but very
similar to the New SSAE
• Applies to all countries where IFAC is
li ll i h Ci
recognized
• Scope – applies to engagements that convey
reasonable assurance when the service
organization is responsible for the suitable
design of controls

Page  135

45

ISAE 3402

• The standard deals with assurance
engagements by professional accountants in
public practice to provide a report for use by
the user entities and their auditors on the
controls at a service organization that
provides a service to user entities that is
likely to be relevant to user entities’ internal
control, as it relates to financial reporting.


Page  136

ISAE 3402

The standard does not deal with assurance
engagements:
• To report on whether controls at a service
organization operated as described, or
• To report ONLY on controls at a service
organization that are not related to a service
that is likely to be relevant to user entities’
internal controls as it relates to financial
reporting

Page  137

Why is ISAE 3402 Important

• Impact at domestic and international levels
• It updates/replaces (potentially)/complements:
• US ‐ Statement on Auditing Standards (SAS) No. 70
• CA ‐ Canadian Institute of Chartered Accountants
(CICA) 5970
• UK ‐ Audit and Assurance Faculty Standard (AAF)
01/06
• AU ‐ Guidance Statement (GS) 007
• HK ‐ HKSA Statements – Auditing Practice Note 860.2
• JP ‐ Audit Standards Committee Report No. 18
• DE (Germany) ‐ IDW PS 951


Page  138

46

IFAC – ISAE 3402

• Introduces the concept of materiality
• Not with respect to the financial statements
but with respect to the system
 The concept of materiality takes into account that
the service auditor’s assurance report provides
the service auditor’s assurance report provides
information about the service organization’s system
to meet the common information needs of a broad
range of user entities and their auditors who have an
understanding of the manner in which that system
has been used.


Page  139

IFAC – ISAE 3402

• Materiality with respect to the fair presentation of
the service organization’s description of its system,
and with respect to the design of controls, includes
primarily the consideration of qualitative factors,
for example: whether the description includes the
for example: whether the description includes the
significant aspects of processing significant
transactions; whether the description omits or
distorts relevant information; and the ability of
controls, as designed, to provide reasonable
assurance that control objectives would be
achieved.

Page  140

IFAC – ISAE 3402

• Materiality with respect to the service
auditor’s opinion on the operating
effectiveness of controls includes the
consideration of both quantitative and
qualitative factors, for example, the tolerable
rate and observed rate of deviation (a
quantitative matter), and the nature and
cause of any observed deviation (a
qualitative matter).

Page  141

47

Critical Steps in Assurance Reporting
Under ISAE 3402

• Assessing the Suitability of the Criteria
• Obtaining an Understanding of the Service
Organization’s System
• Obtaining Evidence Regarding the
Description
i i
• Obtaining Evidence Regarding Design of
Controls
• Obtaining Evidence Regarding the Operating
Effectiveness of Controls

Page  142

Critical Steps in Assurance Reporting
Under ISAE 3402

• The Work of an Internal Audit Function
• Other Information
• Preparing the Service Auditor’s Assurance
Report
• Other Communication Responsibilities


Page  143

Comparison of SAS 70 with ISAE/SSAE
Topic Existing SAS 70 Standard ISAE 3402 / SSAE
Scope SAS 70 is limited to controls Report can be extended
over the processing of beyond financial
financial transactions by a reporting.
service organization.

Opinion /
p The auditor provides an
p In addition to the
Assertion opinion based directly on auditor's opinion,
the subject matter with no management of the
formal management service organization
assertion. provides a formal
assertion affirming its
responsibilities for the
controls in the report.
Extracted from “Good‐bye SAS 70” by Fiona Gaskin
Page  144

48

Disclosure Work performed by internal Work performed by internal audit
requirements audit to support the service used in part to form the service
for use of IA auditor's opinion is not auditor’s opinion shall include a
disclosed. description of the internal
auditor’s work and of the service
auditor’s procedures with respect
to that work.
Audit Guidance Guidance is provided in an Guidance for the service auditor
annually updated Audit will be solely contained in the
Guide, which includes ISAE itself and will not contain
illustrative control objectives illustrative control objectives.
for various types of service The US will continue to provide
organizations. audit guidance to support the
SSAE/SAS 70
standards.

Page  145

Example of Type I - report on the Type 1 - report on the
Terminology fairness of the fairness of the description
Differences description of controls of controls and whether
and whether those those controls were suitably
controls were suitably designed.
designed.
designed

Type II - report also Type 2 - report also includes
includes an opinion on an opinion on the operating
the operating effectiveness of the controls.
effectiveness of the
controls.
Page  146

ISAE 3402 Report

• Internal control is a process designed to provide
reasonable assurance regarding the achievement of
objectives related to the reliability of financial
reporting, effectiveness and efficiency of operations
and compliance with applicable laws and regulations.
p pp g
• Control objectives and controls at the User
Organizations
• Control objectives and controls at the Service
Organization
• Controls at the Service Organization that need to be
complemented at User Organizations

Page  147

49

Module Summary

able to understand:
• The latest developments in Third Party Assurance
Standards
• The impact of new Standards
p
• The benefits of the new Standards
• Key differences and similarities between domestic
and international standards
• Key considerations and responsibilities of a
service auditor and the user of a third party
assurance report

Page  148

Wrap-Up

Wrap‐Up and Summary


Page  149

Using Third Party Reports

• A report is not relevant if it does not address your
company’s risks
• Prepare your own ICQ or use a standard one as a
pre‐audit tool
• Use your company’s risk and control matrices as
the basis to evaluate ICQ, AUP, SAS 70, ISAE and
SSAE findings
• Starting point is your company’s risks not what is
in the reports


Page  150

50

Third Party Assurance – Final Comments

• Businesses will continue to look for opportunities
to increase efficiency and effectiveness of
business processes
• Globalization will not stop
• Cloud Computing will make this field more
Cloud Computing will make this field more
interesting and complex
• Third party assurance practice will continue to
grow
• We will be either auditing or will be audited by a
service auditor …


Page  151

Contact

Felix Ramirez
(W) 646‐290‐8998
(C)  908‐230‐4562
(C) 908‐230‐4562
(e) felix.ramirez@riebeeckstevens.com


Page  152

51

Evaluating Vendor Risks - slides

Recommended

Recommended

More Related Content

Similar to Evaluating Vendor Risks - slides

Similar to Evaluating Vendor Risks - slides (20)

Recently uploaded

Recently uploaded (20)

Evaluating Vendor Risks - slides