Assessing the Usability of Credit Card Disclosures

1,253 views

Published on

In 2006, I conducted a usability evaluation of printed credit card disclosure materials for the U.S. Government Accountability Office. This presentation covers the three methods used -- readability analysis, expert review using plain language guidelines and usability test -- to assess usability.

I gave the presentation at the Plain Language Association International Conference in Sydney, Australia, in October of 2009.

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,253
On SlideShare
0
From Embeds
0
Number of Embeds
103
Actions
Shares
0
Downloads
9
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Assessing the Usability of Credit Card Disclosures

  1. 1. Using multiple methods to assess the usability of credit card disclosures Angela Colter [email_address]
  2. 2. What are disclosures? Using Multiple Methods | Angela Colter card member agreements solicitation letters
  3. 3. Why was the U.S. Congress interested? <ul><li>Assess the need to revise disclosures so they better inform consumers of now-common penalty rates and fees </li></ul><ul><li>Find out how effectively pricing practices are disclosed to cardholders </li></ul><ul><li>Determine whether penalty charges contribute to cardholder bankruptcies </li></ul>Using Multiple Methods | Angela Colter
  4. 4. Three evaluation methods <ul><li>Readability analysis using formulas </li></ul><ul><li>Expert review using plain language guidelines </li></ul><ul><li>Usability test </li></ul>Using Multiple Methods | Angela Colter
  5. 5. Why those methods? Using Multiple Methods | Angela Colter Strengths Weaknesses Readability formulas Quantitative data Quick & cheap Only counts what can be counted Doesn’t tell you what to do about it Plain language review Predicts problems and gives guidelines for improvement Also relatively cheap Doesn’t mean users will have problems Usability test Shows how people actually use an interface Provides quantitative and qualitative data Not quick and not cheap Small sample size can be a hard sell
  6. 6. Readability analysis <ul><li>Used FOG, SMOG, and Flesch Grade Level </li></ul><ul><li>Took multiple samples </li></ul><ul><li>Carefully prepped the text (removed URLs, trailing periods and hard returns) </li></ul><ul><li>Used Readability Analyzer software </li></ul><ul><li>Explained strengths and weaknesses of readability formulas in the final report </li></ul>Using Multiple Methods | Angela Colter
  7. 7. Readability analysis findings Using Multiple Methods | Angela Colter Flesch Grade Level SMOG FOG Average grade level of cardmember agreements 12
  8. 8. Readability analysis findings Using Multiple Methods | Angela Colter Flesch Grade Level SMOG FOG Average grade level of cardmember agreements 12 Average grade level of annual percentage rate info 17
  9. 9. Readability analysis findings Using Multiple Methods | Angela Colter Flesch Grade Level SMOG FOG Average grade level of cardmember agreements 12 Median reading level of U.S. adults 8 Average grade level of annual percentage rate info 17
  10. 10. Expert review <ul><li>Used a plain language guidelines document developed by the U.S. Securities and Exchange Commission in 1998 </li></ul><ul><li>Deals with organization, content and design </li></ul>Using Multiple Methods | Angela Colter http://www.sec.gov/news/extra/ handbook.htm
  11. 11. Expert review findings Using Multiple Methods | Angela Colter The disclosures violated many of the guidelines. The report listed each guideline and gave examples where disclosures followed or violated them. Expressing a complicated, multistep process in paragraph form makes it difficult to understand the relationships between steps. By using bullet points, it’s much easier to see what the steps are and when they are applied. Guideline : Use bullet points
  12. 12. Expert review findings Using Multiple Methods | Angela Colter At times, gave suggestions for improvements Original (101 words) Rewrite (48 words) If at any time during any rolling consecutive twelve billing cycle period you fail to make two Minimum Payments on a timely basis or exceed your Credit Limit twice we may elect to increase your Purchase, Cash Advance and/or Balance Transfer APRs to the Penalty APRs. All Penalty APRs will remain in effect until, in a subsequent rolling consecutive six billing cycle period, you do not exceed your Credit Limit at any time and you make all of your required Minimum Payments on a timely basis when, in your next billing cycle, all Penalty APRs will no longer apply. If you pay late or go over your credit limit twice in a year, the interest rate you pay on most things goes up to the default rate. It will go back down when you pay on time and don’t go over your credit limit for six months. Guidelines : Eliminate jargon Use short words Use short sentences
  13. 13. Usability test <ul><li>Tested the material with 12 users </li></ul><ul><li>Gave them two sets of tasks using two different documents: one solicitation letter, one card member agreement </li></ul><ul><li>Topics covered: </li></ul><ul><ul><ul><li>Annual percentage rates </li></ul></ul></ul><ul><ul><ul><li>Grace period </li></ul></ul></ul><ul><ul><ul><li>Minimum payment </li></ul></ul></ul><ul><ul><ul><li>Late fee </li></ul></ul></ul><ul><ul><ul><li>Over limit fee </li></ul></ul></ul>Using Multiple Methods | Angela Colter <ul><ul><li>Cash advance fee/APR </li></ul></ul><ul><ul><li>Balance transfer fee/APR </li></ul></ul><ul><ul><li>Finance charges/balance computation method </li></ul></ul><ul><ul><li>Changes to terms </li></ul></ul><ul><ul><li>Payment allocation </li></ul></ul>
  14. 14. Usability test plan <ul><li>Can they use disclosures to locate information? </li></ul><ul><li>Do they interpret the info correctly once found? </li></ul>Using Multiple Methods | Angela Colter Question Correct answer
  15. 15. Usability test findings <ul><li>Gave an overview citing task success rate </li></ul><ul><li>Followed with a full explanation of outcome, observations and participant quotes </li></ul>Using Multiple Methods | Angela Colter Task 8: Finance Charges / Balance Computation Method   Solicitation Letters: Only 1 of 12 located the information and seemed to know the significance of the term average daily balance. No one knew what two-cycle ADB meant.   Cardmember Agreements: All were able to locate the information, but only one was able to correctly interpret the information she read.   This was, by far, the task with the lowest success rate.   Only one out of the 12 participants located the name of the balance computation method used by the credit card in the solicitation letter. Two participants said things like “I know it’s based on the average daily balance, but that's not what it’s called.” Part of the problem may have been the label used on the Schumer box: Balance Calculation Method for Purchases didn’t seem to convey “this is how we calculate the interest you owe.”   Only one person gave a plausible explanation for the definition of what Average Daily Balance Computation Method means. No one who was asked what Two-Cycle Average Daily Balance Computation Method means even tried to guess.
  16. 16. Usability test findings Using Multiple Methods | Angela Colter Detailed information and performance metrics were included in the report’s appendix. Question P# Doc Time Correct? P’s Answer Analysis This card uses the “two-cycle average daily balance” method to figure out how much interest you owe. What does it mean? P6 SL #1 0:15 N -- You pay interest on this month’s and last month’s balance. It means if you made a partial payment last month, you are paying interest on principle you have already paid off. No one got it right. P8 SL #1 0:50 N “ I can’t understand it” P7 SL #2 1:35 N “ I really don’t know” P9 SL #2 -- N “ You got me” P1 SL #3 1:10 N Described averaging P3 SL #4 0:16 N “ No clue”
  17. 17. Complimentary approach Using Multiple Methods | Angela Colter Findings Weakness Readability formulas 12 th grade reading level Doesn’t tell you what to do (but P.L. guidelines do) Plain language review Docs don’t provide hierarchy, group related information, use markers (TOC, meaningful headings), or use simple words and sentences Doesn’t mean users will have problems (but usability testing shows where they do) Usability test Users had trouble locating info and often didn’t understand it once they had located it. Disincentives for reading terms. Small sample size can be a hard sell (but video puts a face on problems; findings from formulas and P.L. review found some of the same issues identified in testing)
  18. 18. Outcomes Using Multiple Methods | Angela Colter <ul><li>Communicating concepts wasn’t the main problem, unfair practices were. </li></ul><ul><li>The U.S. Congress and Federal Reserve passed laws and regulations that: </li></ul><ul><ul><li>ban universal default and “two-cycle” billing </li></ul></ul><ul><ul><li>require high interest balances be paid first </li></ul></ul><ul><ul><li>require opt-in to over limit fees </li></ul></ul><ul><ul><li>disclose consequences of making only the minimum payment on monthly bills , not just in disclosures. </li></ul></ul>
  19. 19. Further reading Using Multiple Methods | Angela Colter GAO Report: CREDIT CARDS Increased Complexity in Rates and Fees Heightens Need for More Effective Disclosures to Consumers http://www.gao.gov/new.items/d06929.pdf A Plain English Handbook: How to create clear SEC disclosure documents http://www.sec.gov/news/extra/handbook.htm
  20. 20. Questions? Angela Colter Email: acolter@gmail.com Twitter: @angelacolter

×