Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Keeping it cool in a crisis (Relate Live London)

Dave Dyson, Sr. Customer Service Evangelist, Zendesk
Keeping a cool head in a crisis isn’t just a good trait, it’s absolutely essential when things go wrong and your customers are expecting you to be the hero. Learn how to act fast, say the right things on the right channels, and deliver a solution quickly. In this session, you’ll learn best practices for managing customer satisfaction in a crisis— including the need to be responsive and transparent.

  • Be the first to comment

  • Be the first to like this

Keeping it cool in a crisis (Relate Live London)

  1. 1. #RelateLive
  2. 2. #RelateLive Keeping it Cool in a Crisis
  3. 3. Dave Dyson Zendesk Senior Customer Service Evangelist @dave_dyson
  4. 4. Crisis Communication What we’ll cover • Definitions • Goals • Ingredients for Success • Our Red Alert Process • Metrics
  5. 5. Definitions
  6. 6. What is this “Red Alert” that you speak of? Categories of Bad • Service Disruption • Security Incident • Legal Entanglement • Public Relations Nightmare • Physical Emergency
  7. 7. Goals
  8. 8. The “Prime Directive” Your Mission • Repair Trust with Customers • Restore Service ASAP • Consistent Response • Efficiency
  11. 11. Ingredients for Success
  12. 12. Planning Ahead A Haiku A Red Alert is not the time to figure out how to handle one.
  14. 14. Stay Calm Emotions are Contagious As a leader, your emotions carry extra weight. If you’re nervous, your team will be too.
  15. 15. Empathize Show You Care
  16. 16. Active Listening Helps Defuse Anxiety • Leave your ego behind • Frustration is normal • Let them speak • Verbal “nods” • Reflect what they say • Validate their emotions
  17. 17. “If you fail to plan, you are planning to fail” - Benjamin Franklin
  18. 18. “vaj nab luj SoH, luj nab SoH!” - Benjamin Franklin (original Klingon)
  19. 19. Red Alert Plans What to Include • Process • Roles & Duties • Staffing • Tools • Communications • Special Cases • Metrics
  20. 20. Stakeholders This isn’t just about you. • Support Team • Engineering • Operations • Security • Marketing / PR • Sales • Account Management • Customer Success • Executives
  21. 21. Your Plan is Not Set in Stone Hone & Refine
  22. 22. Documentation If it’s not written, it won’t be remembered • Complete • Process & People • Clear • Accessible • Up to Date
  23. 23. Training Practice Makes Perfect • Onboarding • Shadowing • Video • Drills
  24. 24. Take Ownership • Empathize • Apologize • Don’t Shift Blame
  25. 25. Communication “All decks, this is the bridge…” • Internal • External • Timely • Accurate • Compassionate • Honest • Transparent • One Voice
  26. 26. Red Alerts at Zendesk The Plan in Action
  27. 27. Red Alert Process Documentation Keep it secure, keep it safe • Process Documentation • Checklist • Twitter & Flowdock • How to get Notified • Training Materials • On-Call Schedules
  28. 28. Zendesk Red Alert Process Overview 1. Assess the Situation 2. Alert the Incident Team 3. Communicate to Stakeholders 4. Public Acknowledgement 5. Status Updates 6. Resolve the Issue 7. Wrap Up
  29. 29. Assessment Is This a Red Alert? • Is there a current, ongoing, threat to the security of customer or Zendesk data? • Are customers prevented from performing a critical Zendesk function? • Is the event impacting multiple customers?
  30. 30. Assessment Specific Criteria Critical Functionality: • Account Access • Performance Degradation • Channels • Core Partner Services • Business Rules / Routing • Agent Collision • Reporting • Apps Framework Security Threat: • Compromised data • Spoofing • Stolen passwords • DDoS Attack
  31. 31. Internal Alerts Tools and recipients PagerDuty: Incident Response Team (Support, Operations) Email: Internal Stakeholders (Engineering, Operations, Marketing) Flowdock:
 Wider Team Visibility (Support, Operations) Company-Wide Visibility
  32. 32. The Incident Response Team The Who’s Who Incident Lead (Support) • Owns the problem ticket for Support • Gathers scope and impact info to share with Incident team • Updates support team on status Support Duty Manager (Support) • Manages Support resources during the incident • Manages all customer-facing messaging Operations Manager (Operations) • Manages Operations resources during the incident • Confirms facts about the nature of the incident • Makes decisions necessary to restore service Incident Manager (Support Operations) • Assist with large incidents • Craft public post-mortem from internal version
  33. 33. Communication Flow Customers Customer Advocates Incident Lead Support Duty Manager Operations Manager Engineering & Operations Staff
  34. 34. Staffing On-Call Duty Rotation Weekly on-call shifts for each role • 8 hours x 7 days – based in AMER, APAC, EMEA Avoid consecutive duty shifts • Standard: one shift every 6 weeks Support Duty Manager has on-call Backup as well • Allows for escalation when primary duty manager is unavailable PagerDuty used for scheduling & alerts • Dashboard shows who is on call & upcoming schedule • Individuals can customizable alerts for incidents and upcoming shifts
  35. 35. The Zencident Room
  36. 36. Red Alert - Problem & Incident Ticket Problem Ticket • Created internally (manually or by our Red Alert App) • Uses Service Disruption ticket form • Technical updates & discussions recorded here • No risk of inadvertent customer communication • Solves attached incidents automatically Incident Tickets • Customer reports - attached to the Problem ticket • Proactive tickets attached as well
  37. 37. Service Disruption Ticket Form Additional Fields Added to the Problem Ticket: • Impacted Pods • Alert Duration • Number of Incidents • Link to public Help Center article • Link to internal Operations incident record • Checkbox: Post-mortem published
  38. 38. Communicating to Customers Key Points • Acknowledge Quickly • Take Ownership • Provide Scope and Impact • Regular Progress Updates • Be Transparent
  39. 39. Communication Cadence What to Share, and When Event Time After Alert Called Notes Acknowledgement of Incident ASAP but within 15 minutes The sooner we acknowledge an incident publicly, the less anxious customers become. Description of incident scope and impact ASAP but within 30 minutes Incident scope should be specific enough for customers to self-identify if they are impacted. Status updates on investigation/resolution Every 30 minutes thereafter When possible, status updates should provide new information to demonstrate progress is being made toward resolution. "All clear" ASAP when reached As soon as Operations Manager and Support Duty Manager agree. Pointer to Post-Mortem summary Include with "All Clear" Post-mortems should normally be posted within 3 business days of each incident.
  40. 40. Customer Communications Additional Details Tweet Acknowledgement to @ZendeskOps • Use list of sample tweets • Marketing Suspends Tweets to @Zendesk Respond to Customer Tickets • As received • Attach to Problem ticket Publish public Help Center article • Use article template • Timeline and eventual post-mortem • Location: Service Disruptions section of public KB Proactive Communication to Top Customers • Tickets & Phone Calls to affected customers Timed Status Update Tweets to @ZendeskOps • Default: every 15 minutes • Set expectations if it will be longer • Update Help Center article with tweets
  41. 41. “Tell them what you would want to know” - Susan Griffin-Black, EO Products
  42. 42. Resolution & Wrap-Up Once service is deemed restored by the incident team: 1. Ticket Resolution • Solving the Problem ticket solves all attached Incidents • Include link to Help Center article for post-mortem 2. Send “All Clear” Tweet • Include link to Help Center article for post-mortem 3. Post-Mortem (within 3 business days) • Operations team writes internal version • Support Operations Incident Manager edits for public consumption, and publishes to Help Center article
  43. 43. Service Disruption Post-Mortem What should it include? • Scope and Customer Impact • Incident Duration • Communication Timeline • Process / Training Gaps • Recommendations for Improvement
  44. 44. Special Situations Exceptions to the Rule Security Incidents Involve the Security Team Zopim Outage Communicate via Zopim Twitter & Facebook Partner Outage (Twilio, GoodData) Report incident to Partner Refer to Partner’s system status page Zendesk Support Instance is Down Use backup voice service (IfByPhone) Outgoing communication via Tweets only Shift Handoff Live handoffs only - no email Overlapping Incidents Separate Red Alerts & Problem Tickets
  45. 45. The Red Alert App Custom App in our Zendesk Instance: 1. Displays links to active Red Alerts 2. Turn a ticket into a Red Alert ticket • PagerDuty, Ticket Form & fields 3. Send Social Media Communications • Twitter, Facebook • Composition, templates, character counts • Salutation and closing • “All Clear” option adds link to Help Center article • Creates or updates Help Center article 4. Proactive Ticket Communications • Lists top customers, with Pod & feature info • Composition, templates, all-clear option • Creates or updates Incident tickets • Updates Yammer 5. All actions add Internal Notes to Problem ticket
  46. 46. Metrics Red Alert Impact Report (Insights) • Number of Red Alerts • Number of attached Incidents • Total Support Handle Time • Estimated Support Cost ($/ticket) • Number Proactive Tickets • Top Red Alerts (by # of incidents) • Red Alerts by “About” field • Customer Satisfaction over time • Process Scorecard
  47. 47. The Future! Where No One Has Gone Before System Status Page (in Beta) • Loads more detail! Improved Red Alert App • More capabilities Wider Proactive Communication • More channels • More customers Incident Management as a Job • Not just a role
  48. 48. Conclusion
  51. 51. Smooth Sailing “Second star to the right, and straight on ‘till morning…”
  52. 52. #RelateLive Q & A

    Be the first to comment

    Login to see the comments

Dave Dyson, Sr. Customer Service Evangelist, Zendesk Keeping a cool head in a crisis isn’t just a good trait, it’s absolutely essential when things go wrong and your customers are expecting you to be the hero. Learn how to act fast, say the right things on the right channels, and deliver a solution quickly. In this session, you’ll learn best practices for managing customer satisfaction in a crisis— including the need to be responsive and transparent.


Total views


On Slideshare


From embeds


Number of embeds