Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

How to Measure the Effectiveness of Your Ethics & Compliance Program (2014 Compliance & Ethics Institute 206)


Published on

Learn actual tools you can use to identify local risks, objectively evaluate tone at the top, and measure the effectiveness of the implementation of your ethics and compliance program. Topics include how to create and use effective surveys, conduct meaningful focus groups, and perform deep dive site visits to learn what's really going on throughout your business. Discuss real-life examples, successes, and learning opportunities and come away with real tools you can use across different cultures and throughout a geographically diverse organization.

Published in: Education

How to Measure the Effectiveness of Your Ethics & Compliance Program (2014 Compliance & Ethics Institute 206)

  1. 1. Measuring the Effectiveness of Your Ethics and Compliance Program SCCE Compliance & Ethics Institute, Chicago, Illinois September 15, 2014 Michael Ward Atlas Compliance Solutions
  2. 2. First things First . . • What exactly are you trying to measure? • What does “effectiveness” mean? • Effectiveness at doing what? • Who is your audience/stakeholders? • What are their effectiveness criteria?
  3. 3. Why measure program effectiveness? • Demonstrate achievement of minimum legal requirements • Track attainment of internal program objectives • Demonstrate efficient allocation of program resources and/or justify additional resources • Provide feedback/motivate employees • Diagnose opportunities for improvement
  4. 4. Key Stakeholders and Criteria Enforcement Agencies – Priorities: • Your success in preventing non-compliant outcomes • Your best efforts in trying to prevent non-compliance • Your awareness/response to previous non-compliance – Criteria: • US Federal Sentencing Guidelines or similar published stds • Adherence to company’s own procedures/policies – Relevant Metrics: • Program assessment (alignment to FSG elements) • Previous incidents and response • More concerned with effort expended than efficiency
  5. 5. Key Stakeholders and Criteria Company Board and Senior Management – Priorities: • Success in preventing non-compliant outcomes • Prudent balancing of company resources against risk • Reliable process to identify/escalate issues or risks – Criteria: • US Federal Sentencing Guidelines or similar stds • Efficient and prudent allocation of resources – Relevant Metrics: • Program and risk assessment (alignment to FSG elements) • Trend and benchmark data (within and outside company)
  6. 6. Measurement Challenges • Availability of data • Unintended consequences • Inconsistent, incompatible and siloed information • Measuring a negative • “Black swan” infrequent events • "Not everything that counts can be counted, and not everything that can be counted counts.”
  7. 7. Common Compliance Program Metrics Hotline – No. of allegations – Types of allegations – Locations/business units – Substantiation rate – Percentage of anonymous reports – Case cycle time – Method/source of report? – Level of offender? – Repeat reporter? – Satisfied reporter?
  8. 8. Not everything that can be counted counts . . . Program – No. of FTEs assigned to compliance? – Compliance budget dollars – Average annual hours of compliance training/FTE – Code of Conduct delivery frequency – Average no. of days since last compliance certification – Employee perceived value of training
  9. 9. 1/1/2015 5:01 PM 9 Activity-Based Indicators Results-Based Indicators • Compliance program investment • Total dollars • FTE • Number of investigations • Number of training sessions conducted • Number of employees trained • Hours of employee training delivered • Mean number of days to resolve an allegation • Number of disciplinary actions • % of employees who actually understand specific policies • % of employees who observed misconduct • % of employees who fear retaliation • Cost of non-compliance • Litigation costs • Claims paid • % of high risk areas addressed by control improvements • Number of complaints received? • Percentage of complaints anonymous? Are you measuring effort or results?
  10. 10. 1/1/2015 5:01 PM 10 Trailing Indicators Leading Indicators • Integrity Hotline calls received • Total allegations • Types of allegations • % of anonymous calls • Substantiated allegations • Case cycle time • Employee disciplinary actions • Claims filed • Days lost to injury • Risk Assessments • Inherently likely events • Objectively immature controls • Employee surveys • Report issues w/out fear of retaliation • Awareness of resources • Ethical attitudes • Policy understanding • Predictive Analytics • Real time flagging of potentially non-compliant activity Reporting the past or being proactive?
  11. 11. Proactive/Leading Indicators Compliance Risk Assessment Process • Proactive effort to inventory and prioritize compliance risks to organization • Consistent methodology across risks areas allows comparison to one another and prioritization • Identification of risk areas with higher likelihood or weaker controls before a serious compliance breach • Ensures limited compliance budget is consistently applied to reducing the most risk • Enables board and senior management to fulfill their program oversight responsibilities and see the big picture • Reduces the consequence of a compliance issue by demonstrating company best efforts to prevent • Risk assessment is element of FSG Compliance Program Standards
  12. 12. EXAMPLE – Acme Co. Alignment to FSG Standards Federal Sentencing Guideline Requirement Best Practices Current Acme Co. Action Items Compliance Program Oversight  Board shall be knowledgeable and exercise reasonable oversight of compliance program. [8B2.1(b)(2)(A)]  A specific senior officer shall be assigned “overall responsibility for the compliance and ethics program.” [8B2.1(b)(2)(B)] Assign Day-to-Day Responsibility for Compliance [8B2.1(b)(2)(C)] “Specific individuals shall be delegated day-to- day operational responsibility for the compliance and ethics program.”  Such individual(s) shall report periodically to senior officers and the Board of Directors on the effectiveness of the compliance and ethics program.  Such individual(s) shall be given adequate resources, appropriate authority and direct access to the Board of Directors.  To obtain full benefit of compliance program, individual with operational compliance responsibility shall have formal reporting relationship to Board. • Audit Committee has direct oversight. • Gov’t consent decrees often separate CCO and General Counsel role. in highly regulated industries. • Regular compliance content on agenda • Formal independent CCO reporting relationship to Board. • CCO is senior level executive evidencing importance of compliance program. • Audit Committee has direct oversight. • General Counsel assigned overall responsibility. • Senior level company executive coordinating ethics and compliance program. • Sufficient financial resources. • Added key personnel to bolster team capabilities. • Compliance program updates are regular agenda item of Audit Committee. • Regular 1:1 meetings of Audit Committee and Compliance Officer. • No specific gaps to address. • No specific gaps to address. 12PRIVILEGED & CONFIDENTIAL ATTORNEY-CLIENT COMMUNICATIONS
  13. 13. Federal Sentencing Guideline Requirement Best Practices Current Acme Co. Action Items Establish policies, standards and internal controls to prevent and detect misconduct and noncompliance. [8B2.1(b)(1)] Continuing communication and training of compliance program components to employees and third party agents. [8B2.1(b)(4)(A)]  The organization shall take reasonable steps to periodically communicate its standards and procedures through training and other means.  Audience shall include Board members, senior executives, employees and agents/ partners. • Paper and on-line Code of Conduct with learning aides and FAQs. • Corporate policy portal with related compliance resources. • Deliver both on-line and live training. • Training content should be risk-based according to role/responsibilities. • Periodic refresher messaging outside of training content. • Business partners and agents should be trained. • Employee Code of Business Conduct. • Business Partner Code of Conduct distributed to third parties. • Corporate compliance and policy website. • Employee COI Questionnaire. • Gifts/Hospitality policy and reporting. • Code of Conduct course and additional courses delivered on- line and translated to local languages. • Live training delivered annually to sales force. • FCPA training made available to resellers. • Compliance standards provided to all resellers, agents and suppliers. • Refresh of Code of Business Conduct. • Scheduled update of FCPA, COI and other policies. • Policy portal update to deliver key policies to mobile devices. • Implement “just in time” and on demand training program for key compliance processes. 13PRIVILEGED & CONFIDENTIAL ATTORNEY-CLIENT COMMUNICATIONS EXAMPLE - Acme Co. Alignment to FSG Standards
  14. 14. Acme Software Co. Compliance Risk Dashboard Key Legal Risks Inherent Risk (FY11) Controls Rating Residual Risk FY11 FY12 FY11 FY12  Anti-Corruption HIGH HIGH HIGH MEDIUM MEDIUM  Antitrust/ Competition MED-HIGH MEDIUM MEDIUM MEDIUM MEDIUM  Online Services) HIGH MEDIUM MEDIUM MEDIUM MED-HIGH  Conflicts of Interest HIGH MEDIUM HIGH MEDIUM MEDIUM  Employment HIGH HIGH HIGH MEDIUM MEDIUM  Export Controls HIGH MEDIUM MEDIUM MEDIUM MEDIUM  Government Contracting MED-HIGH LOW MEDIUM HIGH MEDIUM  Government Relations MEDIUM MEDIUM MEDIUM MEDIUM MEDIUM  Information Privacy: Corporate HIGH MEDIUM MEDIUM MEDIUM MEDIUM  Information Privacy: Products HIGH MEDIUM MEDIUM HIGH MED-HIGH  Intellectual Property HIGH MEDIUM MEDIUM MEDIUM MEDIUM  Marketing/ Trademarks MEDIUM HIGH HIGH LOW LOW  Records Retention & Information Management (RIM) MEDIUM MED-LOW MED-LOW MEDIUM MEDIUM  Revenue Recognition/ Side Letters HIGH HIGH HIGH MEDIUM MEDIUM  Securities/Insider Trading MEDIUM HIGH HIGH LOW LOW 14PRIVILEGED & CONFIDENTIAL ATTORNEY-CLIENT COMMUNICATIONS
  15. 15. Expected Consequence/Impact of Event InherentLikelihoodofEvent LOWER HIGHER HIGHERLOWER HIGH MEDIUM MEDIUM 1. Anti-Corruption (FCPA) 2. Antitrust/Competition 3. Online Services 4. Conflicts of Interest 5. Employment 6. Export Controls 7. Government Contracting 8. Government Relations 9. Information Privacy: 10. Intellectual Property 11. Marketing/Trademarks 12. Records Retention & Information Management (RIM) 13. Revenue Recognition/Side Letters 14. Securities/Insider Trading 2 3 12 5 10 11 8 Compliance Risks 13 14 1 6 9a Acme Software Co. Compliance Risk Heat Map 4 7 15PRIVILEGED & CONFIDENTIAL ATTORNEY-CLIENT COMMUNICATIONS Mature and optimized. Key controls present , opportunity to improve. Assessed Effectiveness of Controls Controls are immature
  16. 16. Residual Risk: Risk Assessment Criteria Likelihood: This section describes the inherent likelihood (likelihood excluding the impact of any controls) of the particular compliance risk by these criteria:  The number of opportunities for non-compliance  The degree of individual incentives for non-compliance  The complexity and number of dependencies in achieving compliance  The rate of change in the environment (expansion or contraction of people, processes and systems)  Recent regulatory and enforcement trends  Observed non-compliance by similarly situated companies Severity: The criteria for the expected severity or impact of non-compliance are:  Civil v. criminal enforcement  Private v. government enforcement  Potential termination or suspension of operations  Potential for class action lawsuit  Impact on reputation  Impact on customers  Employee recruiting or retention consequence  Increased future cost of compliance Controls: Standard controls (below) as well as risk specific controls are considered: • Have preventative controls been embedded in the business process? • Has a written compliance standard been provided to employees? • Is there periodic compliance training of the affected employees? • Is a specific person assigned accountability for achieving compliance? • Is there a defined process to respond to detected noncompliance? • Is there a periodic risk assessment by an SME of both the compliance obligations and the existing controls? The residual risk rating is the net assessment of the risk to the enterprise from the specific risk. It is the combination of both inherent risk (likelihood and severity) and the assessed state of controls. Key Processes/Owners: • This section identifies the specific business processes that generate the risk under assessment. • Other primarily control processes that are central to managing the risk should also be identified. Legal Owner: The Legal owner is typically the company’s legal subject matter expert for the specific risk. Action Item(s) Owner Status This section will identify recommended or ongoing risk remediation projects (process changes, policy updates, training initiatives) that are expected to improve the control state or reduce the likelihood of the specific risk. Compliance Q4 2014 16PRIVILEGED & CONFIDENTIAL ATTORNEY-CLIENT COMMUNICATIONS
  17. 17. Residual Risk: Anti-Corruption Likelihood: The inherent likelihood (excluding the impact of any controls) of an anti- corruption issue is assessed to be relatively HIGHER. This is due to the level of sales activities to foreign government customers (both traditional government entities as well as education, healthcare and state-owned commercial entities), the involvement of multiple partners and tiers in channel transactions, and the inherent variability and complexity in software pricing and incentives. The new UK Bribery Act also increases exposure for commercial bribery. Finally, there is potential for criminal liability to Acme for the acts of third party agents. Severity: The expected severity or impact on the enterprise from non-compliance in the anti-corruption area is relatively HIGH. Noncompliance is ordinarily criminally prosecuted both against the company and individual executives. The financial penalties are likely to be in the millions of dollars including disgorgement of any improperly obtained profits. Finally, even the investigation alone would generate great negative publicity and a conviction could result in the company’s suspension disbarment from federal government contracts. Controls: It is assessed that control state for this risk is HIGH. Acme has a comprehensive third party due diligence process and channel on-boarding process to mitigate the likelihood and severity of third party risks. Acme has also implemented a comprehensive country gift matrix providing employees with guidance on local legal standards and a gift and hospitality reporting and approval tool for exception management. We are also implementing a supply chain and business partner due diligence process. Even with the significant improvement of controls, the residual anti-corruption risk remains MEDUM. The new Dodd-Frank financial bounties for whistleblowers increases the likelihood of a report and the new UK Bribery law expands attention on commercial non- governmental corruption. Key Processes/Owners: • Retention of channel partners and any professional service providers or agents who interact with government. • Any provision of gifts or hospitality to government employees. Legal Owner: Mark Ericksen Action Item(s) Owner Status Continued expansion of on boarding/due diligence process beyond channel partners to others at risk intermediary relationships. Compliance Ongoing Revise/update FCPA policy to cover newly prohibited commercial bribery risks. Compliance Q4 2012 Publish gift/hospitality standards matrix and launch reporting and approval tool. Compliance Done HIGHER 2011- H HIGHER 2011- H MEDIUM 2011- M HIGHER 2011 - H 17PRIVILEGED & CONFIDENTIAL ATTORNEY-CLIENT COMMUNICATIONS
  18. 18. Proactive/Leading Indicators Diagnostic Integrity Surveys of Employees • Allows direct measurement of employee attitudes and perceptions • “Pushed” to employees who might otherwise not report issues or provide feedback • Anonymity used to encourage candor • Combination of data points allows correlation and other analytical tools to expand insights • Can be standalone, integrated with general employee survey, or both. • Allows detection and remediation of undisclosed issues and attitudes before they become allegations • Supports tailored and targeted response to issues instead of “one size fits all” uniform approach
  19. 19. 1/1/2015 5:01 PM 19 Employee Integrity Surveys – “I have observed misconduct at Company X in the past year.” • Yes 18.92% • No 81.08% – “Did you report the misconduct or raise a concern?” • Yes 79.27% • No 20.73% – “To whom or how did you report the misconduct or concern?” • My supervisor/manager 34.88% • Human Resources 27.91% • Corporate Security 19.38% • Ethics Line 2.33% • All other 15.51%
  20. 20. 1/1/2015 5:01 PM 20 Reason for Not Reporting (Percentage of Non-Reporting Explanations) Employee Integrity Surveys Don’t know why Assumed someone else would report Did not want to get anyone fired Person who committed it was senior Not certain it was a violation Did not think they had enough information Fear of retaliation Did not want to become involved Did not think the company would do anything Other Knew the person Did not think anyone would believe them 6.0% 1.2% 2.4% 2.4% 3.4% 7.1% 7.1% 9.2% 8.3% 16.7% 22.6% 14.3%
  21. 21. 1/1/2015 5:01 PM 21 Employee Integrity Surveys Observed Misconduct by Type Percentage of Survey Respondents 7.4% 6.3% 4.5% 3.7% 2.7% 2.5% 2.2% 1.6% 0.7% 0.6% 0.4% 0.2% 0.2% 0.0% 3.2% Stealing ofC om pany PropertyH arassm ent M isuse ofO rganization's Tim e orR esourcesD iscrim ination C onflicts ofInterest Inappropriate G iving orR eceiving ofG ifts Violation ofH ealth orSafety Policy Fraud Accounting Irregularities Violation ofEnvironm entalR egulation Im properPaym ents Im properSales Business Inform ation ViolationInsiderTrading O ther
  22. 22. 22 Employee Integrity Surveys “I can report unethical behavior or practices without fear of retaliation at Company X” Response Company Benchmark – Strongly disagree 1.7% 2.3% – Somewhat disagree 0.9% 4.9% – Disagree 3.0% 3.1% – Neither 5.3% 6.2% – Agree 28.1% 41.7% – Somewhat agree 11.7% 12.5% – Strongly agree 49.3% 29.4%
  23. 23. 1/1/2015 5:01 PM 23 Employee Integrity Surveys “I can report unethical behavior or practices without fear of retaliation at Company X” By Business Unit Insufficient Insufficient 89% 100% 91% 84% 100% 100% 83% 90% 89% 89% 88% 83% 85% 0% 20% 40% 60% 80% 100% 120% Overall BU A BU B BU C BU D Finance Sales Engineering Facilities APAC LATAM Government EMEA North America IT % Favorable
  24. 24. 24 Employee Integrity Surveys “Senior leaders at my company take appropriate action upon unethical or inappropriate behaviors and practices” Response Company Benchmark – Strongly disagree 0.9% 1.7% – Somewhat disagree 1.2% 3.5% – Disagree 3.9% 2.3% – Neither 20.9% 18.8% – Agree 28.1% 39.0% – Somewhat agree 14.0% 12.8% – Strongly agree 30.9% 21.9%
  25. 25. 25 Employee Integrity Surveys “Senior leaders at my company take appropriate action upon unethical or inappropriate behaviors and practices” Insufficient Insufficient 73% 63% 82% 59% 85% 63% 66% 71% 81% 89% 60% 78% 31% 0% 20% 40% 60% 80% 100% Total Compliance Distribution Finance HR Legal Marketing R&D Procurement Sales Ops Bus Dev. Research Facilities IT Security % Favorable
  26. 26. 26 Measuring Across Different Levels “When was the last time [your manager spoke with you] [you spoke with your direct reports] about the Company Code of Conduct or the importance of business ethics?” Direct Report Response W/in past week 2.78% W/in past month 12.17% W/in past 3 months 16.69% w/in past 6 months 6.84% W/in past year 28.62% Never 32.91% Manager Response W/in past week 6.68% W/in past month 17.87% W/in past 3 months 24.73% W/in past 6 months 29.60% W/in past year 4.51% Never 16.61%
  27. 27. 27 “You have just learned from a co-worker that our company is about to acquire another company. There have been recent media accounts describing the target as an “up and coming” company and you were already thinking of buying their stock. Which of the following courses of action is appropriate? • Personally purchasing stock in the vendor company 4.62% • Telling family and friends so they may purchase stock in the vendor company 0.46% • Neither of the above 79.21% • I don’t know 15.70% Measuring Employee Understanding
  28. 28. Potential Key Effectiveness Indictors Indicators for Likelihood of Non-Compliance • Number/rate of allegations reported (Hotline/case mgmt system) • Number/rate of instances of misconduct observed (surveys/focus groups) • Fear of retaliation (surveys/focus groups/allegations) • Willingness to seek assistance (helpline/survey/focus groups) • Understanding of policies (surveys/direct testing/focus groups) • Effectiveness of process controls to prevent/detect (risk assessment) Indicators of Commitment to Ethics and Compliance • Number/rate of deviations from disciplinary stds • Exceptions/non-completion rate with key procedures/controls – Third party on-boarding process – Training completion – Gifts, Travel & Entertainment processes and satndards – Conflict of Interest disclosures & recusal process • Manager achievement of E&C performance criteria (performance appraisals/employee surveys)
  29. 29. Summary • Think before you report – What am I measuring and why? – Is the data/information reliable? – Who is the audience and what do they need to know? – What is the take away or implication? What should be done? – Keep the main thing the main thing. • Use charts (but not pie charts!) – Benchmarks (internal and external) – Time series – Internal comparisons (reporting units and geographies) • Activity-based v. results-based metrics • Trailing v. leading indicators • Don’t hoard information – Use effectiveness indicators to motivate and engage stakeholders. – Business people are competitive.
  30. 30. Resources • • • Metrics Reporting and Display – Stephen Few, Information Dashboard Design – Stephen Few, Show Me the Numbers – Gene Zelasny, Say It With Charts • Compliance Metrics – OCEG, Measurement & Metrics Guide • Compliance Risk Assessments – Jeff Kaplan, Compliance & Ethics Risk Assessment: Concepts, Methods and New Directions (e-book) • Employee Diagnostic Surveys – CELC, Risk Clarity (Leslie Altizer) – Ethics Resource Center • Program and Hotline Benchmarking Data – NAVEX Global – The Network – LRN – CELC