NICSA's 2016 Strategic Leadership Forum Breakout Session | Sponsored by Delta Data

1. How to use Quantitative Analytics to
Identify and Effectively Manage Risk

2. Why Use Data to Manage Risk
• SEC page on Distribution in Guise
• FINRA guidance aimed at Broker Dealers
• Your Oversight Program should do more than just accept
FICCA’s / Questionnaires
Use the data available to you to rate your distributors efficiency
in their controls.

3. Research Findings
• An oversight program can take flight on existing relationships and smartly
align itself on newly on-boarded counterparties using quantitative
weighting of demographic attributes to estimate a risk tier or profile for a
firm that has not worked through a periodic evaluation
• The annual or semi-annual review should leverage quantitative analytics
of measured KPIs tied to specific control objectives to observe and
quantify capability of control in a normalized fashion
• Once established this approach will provide the platform from which
predictive analytics will provide intelligence and efficiency in the execution
of the oversight program

4. Analytical Approach
Findings show two applicable types of quantitative analysis are
in use and can be leveraged to provide insight to your
counterparty
• The first will help estimate the risk that a given counterparty will likely
provide and set broadly the amount of time spent in the first year on that
entity
• The second will help an oversight program measure the effectiveness of
the counterparty’s control objectives and therefore further qualify their
risk in their profile risk ranking

5. Quantitative Methods Employed
Two Primary Quantitative Methods Employed
• Profile Risk Ranking
– Various key metrics and demographic elements weighted to estimate a
distribution partner’s risk
• Individual Control Objective Analytics
– Individual KPIs driven from actual operational interaction with the
intermediary

6. Discussion Approach
Most standard Oversight Programs consist of core requirements
that can be found in five pillars:
1. Counterparty Deal Management
2. Product Setup/Maintenance
3. Transparency Data Management
4. Compliance
5. Fee / Services Management

7. Counterparty Deal Management
 Management of counterparty relationships in one single portal
 Join and summarize data from all of your operational silos with
demographic and contract level data provided by your
counterparties
 Information captured here will all role up to the counterparty risk
ranking

8. Counterparty Deal Management
Relevant Control Objectives
• Third Party Oversight
• Document Retention & Recordkeeping
• Shareholder Communications
• Business Continuity / DR
Potential Quantitative Elements
• AUM
• # of Accounts
• Age of Relationship
• Age of Last Legal Review
• % growth of relationship over period
• % growth of counterparty over period

9. Product Setup / Maintenance
 Fund parameter comparison rules (Load, 12b1, Fee, etc…)
 Compare TA, Sub-TA and MFPII Data
 Identify, resolve or retain discrepancies with comments

10. Product Setup / Maintenance
Relevant Control Objectives
• Security Master Setup & Maintenance
• Transaction Processing
Potential Quantitative Elements
• Parameter Comparison Annotations

11. Transparency Data Management
 Centralized and Secure infrastructure for storing Transparency Data
 Ability to perform advanced analytics:
 Inflow and Outflow Analysis
 Market Timing Monitoring
 Prospectus Compliance Analysis
 Blue Sky Analysis
 Sales By Social Code
 Initial vs. Subsequent

12. Transparency Data Management
Relevant Control Objectives
• Transaction Processing
• Blue Sky Reporting
• Cash & Share Reconciliation
• Lost & Missing Shareholders
Potential Quantitative Elements
• Blue Sky Sales by State
• Reconciliation of Account Balances
• Account Monitoring
• Inflows / Outflows
• Measure # of Trade Breaks
• # of Trade Violations associated with
rule setup

13. Compliance
 Demographic and policy level metrics that show your counterparty
has the controls and processes in place to meet standards defined
by the ICI.
 Fund Companies should review these controls and policies on a
scheduled basis by conducting reviews that satisfy the board’s
request.

14. Compliance
Relevant Control Objectives
• Risk Government
• Code of Ethics
• Information Security Program
• Anti-Money Laundering and the
Prevention of Terrorist Financing
Potential Quantitative Elements
• Flags / Indicator showing certain
policies exist:
– AML
– KYC
• Scores / ratings from previous annual
reviews

15. Fee and Service Management
 Analyze the rates you are paying distributors vs. the actual
services they are offering
 Calculation and validation of invoicing
 Settlement of Fee Invoicing
 Leveraging transparency data for invoice reconciliation
 Use fees from your TA services to baseline what you should pay
the Sub-TA platforms

16. Fee and Services Management
Relevant Control Objectives
• Sub account Billing & Invoice
Processing
• Fee Calculations
Potential Quantitative Elements
• Invoiced vs. Calculated Fee Discrepancies
• Comparison of rates across distribution
partners
• Validation of Fee Amounts
– Position Based
– Asset Based
• Payment Funding Allocations
• Tolerance Checks
– Change in accounts month over month
– % fee variance vs. change in AUM

17. Profile Risk Ranking
Quantitative metrics from all pillars
+
Qualitative demographic attributes weighted by
importance

18. Example Application
Profile Attribute Values Weighting Firm 1 Score Firm 2 Score
Previous Review Score 1-100 30% 50 15 82 23
Sifi Designation Y/N 3% N 3 Y 3
Sub Accounts Y/N 15% Y 15 Y 15
AUM Tier 0-15% $4B 15 $4B 15
Trading Volume Tier 0-5% 220/Day 5 230/Day 5
International Accts Y/N 5% N 0 Y 5
Provides FICCA Y/N 12% Y 0 N 12
Servicing Firm Y/N 8% N 0 Y 8
Contract Issues Y/N 10% Y 0 N 10
53 = Tier 2 96 = Tier 4

19. Individual Control Objective Analytics
• Association of specific control objectives in the oversight
framework with KPIs from the operational tools that support
the process
• Normalization using key metrics like AUM, number of
accounts or number of RIAs is key for use in the overall
program analytics

20. Examples
Fees Paid vs. Finances Independent Calculation
• A competent Fee Management process should produce a KPI that
illustrates the difference between the amount invoiced and the amount
the fund company believes should have been paid
• This variance is a KPI that can be applied to the invoicing control objective
in the FICCA framework to provide an “as observed” quantitative
assessment of the performance of the counterparty’s controls
22c2 Market Timing Events Observed
• A key KPI that is a result of the 22c2 process is the number of accounts
where market timing was identified. This metric can be tied to the control
objectives for trade monitoring and market timing

21. Normalization
The raw KPI results from the underlying oversight processes
should be normalized for using quantitative demographic
counterparty metrics. This ensures quantitative findings are kept
in perspective.
• In our first example, the total amount of fees paid should be used to
normalize the variance of the paid/calculated percentage
• Similarly, in example two, the KPI of number of accounts where market
timing was observed in a specific time period should be normalized by
dividing by the total number of sub accounts registered for the time
period on the sub account platform

22. Predictive Analytics – the Next Step
• Once your mature quantitative program has taken root,
Predictive Analytics should be leveraged to illustrate where
actual risk is found
• Observed measurement tied to the quantitative oversight
process will provide the quality data needed for Predictive
Intelligence

23. Contributing Research
• Direct insight from our panelists’ experience with mature,
large scale oversight programs
• Observations from several major oversight programs that are
not Delta Data clients or panelists’ funds