Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Field Testing Legal Documents

816 views

Published on

How to Field Test Forms, Instructions & Web Content Good field-testing requires a different way of thinking. Instead of explaining, you listen. Instead of helping, the group helps you understand what they see and don't see. Once you grasp this, your writing will never be the same. Join this seminar to learn how to do your own field tests. Requirements: Submit a 1 or 2 page document/content you would like to have tested. Highlight on that document the 3 most important things you want your reader to know. Then come to our workshop, and learn how to:

Draft an instrument designed to give voice to your readers,
Recruit FT participants,
Train your testing team, and
Conduct the test and report your data.

Speaker Maria Mindlin

By Transcend

Published in: Law
  • Be the first to comment

  • Be the first to like this

Field Testing Legal Documents

  1. 1. Field Testing Legal Documents TIG Grant 15007 Maria Mindlin, Transcend Plain Language, Field Testing & Cultural Competency
  2. 2. Maria’s Background – 25 years at Transcend, an agency that specializes in Plain English and certified translations. – Lead author on the first (and so far only) quantitative test of Comparative Readability Study of Plain Language Court Forms in the U.S. (Scribes Journal of Legal Writing, 2005) – Drafted and tested plain English documents for legal aid agencies and courts in 20+ states. – Readability instructor, UC Davis Law School, under Professor Richard Wydick (author of Plain English for Lawyers). – Edited Bryan Garner’s Redbook text on visual accessibility. TIG Grant 15007
  3. 3. Why field test? TIG Grant 15007
  4. 4. Why field test? You want as many people as possible to read & understand the legal information you provide. The feedback you get from FT helps you achieve this goal. TIG Grant 15007
  5. 5. TIG Grant 15007 FT / FG is your opportunity to collect data and hopefully…. “Data-Driven Lexicon” You are listening! What words do they find easy or hard? What do they prefer?
  6. 6. Findings are based on what you hear and understand Other options: Fairly quick, reasonably inexpensive, extremely instructive! Observations, interviews, chats, focus groups TIG Grant 15007 Focus Group Testing is one Kind of Qualitative Testing
  7. 7. Record & Share Your Readers’ Preferences TIG Grant 15007 Track writing styles, words, images they prefer. Let your writing be guided by your readers’ preferences and reading proficiency. “I used to know how to write; then I went to law school.”
  8. 8. Must-Have Resource TIG Grant 15007 Focus Group Kit: 1 – Guidebook 2 – Planning 3 – Developing Questions 4 – Moderating 5 – Recruiting 6 – Analyzing & Reporting Cost for all 6: $15-$50
  9. 9. Goals for Today’s Webinar You will learn: 1 How to Prepare Your Instrument (Script) 2 How to Recruit Participants 3 How to Train Your FG Team 4 How to Conduct Your FG Testing 5 How to Report Your Data TIG Grant 15007
  10. 10. Materials Needed: Instrument Template & Sample Test Document TIG Grant 15007 ★
  11. 11. TIG Grant 15007 Template as a Starting Point You don’t have to reinvent the wheel! You see why we ask certain questions When we all use similar formats, we can talk apples – to – apples. You won’t commit the sin of running LONG! You’ll see examples of open questions
  12. 12. Instrument Collects Data from basic Categories 1. An introduction 2. Content questions 3. Language questions 4. Likes and Dislikes 5. Title 6. Other suggestions
  13. 13. How many questions? Researchers say no more than 12. Reasons: - Time is limited. You don’t want to rush through the great questions you have! - Ideal time is 1 hour, maybe 1.25 hours. People get tired, restless. Having 12 questions also means you really have to focus on your main goals. TIG Grant 15007
  14. 14. Developing An Instrument Your task: Look at your text. Highlight the areas you most want the readers to learn. Target those areas for questions. Let’s see some examples. TIG Grant 15007
  15. 15. You have language rights! The law says most public offices must give you services in your language. Public offices must not send you away because of your language, or ask you to bring your own interpreter. You have the right to a free interpreter who will keep your information private. Ask for an interpreter at: · Schools · Courts · Hospitals and doctor’s offices · Police · Government services · Some housing authorities Show the card below to ask for: · a free interpreter, and · written information in your language What if they do not give me an interpreter? Call or text the Language Access Coalition: 617 xxx-xxxx, or Call your local legal aid office. – Cut here – Show this card to ask for an interpreter. I speak _________. I need an interpreter to talk to you. Title VI of the Civil Rights Act says I have the right to an interpreter and information in my language. Questions? Talk with your supervisor or look to your agency’s language access policy. If you cannot give me an interpreter, contact the Language Access Coalition. Call or text: 617-xxx-xxxx (for English ) or 617 xxx-xxxx (target language) Target language translation will print on this side of page (Use as flyer, or half sheet w/ translation on back) Bottom area to be cut off and used as a card. TIG Grant 15007
  16. 16. How to Start TIG Grant 15007 Introduction: My name is <name>, and I work for Transcend. We are a company that helps make information easier for people to use. I am here today with my co-worker, <Name>. She will listen to your answers and make notes. This helps us to remember all of your suggestions! Today we need your help with a flyer. We want to know what you think about it and learn how we can make it better. We really want your honest opinions. It’s OK to say anything, even if it’s negative. Telling us what you like and dislike will help us make the flyer better for all users. OK, does everybody feel comfortable about saying anything they want? Shall we begin? Make participants feel comfortable about saying anything they want. Explain why their contribution is so important. Facilitator does not correct answers or provide add’l info. S/he responds with a warm OK, good, great…. to all answers.
  17. 17. Content TIG Grant 15007 1. What is this flyer about? (Allow 30 seconds. Observer writes each answer given using each participant’s exact words.) ________________________________________ ____ ________________________________________ ____ ________________________________________ ____ If users cannot quickly identify the purpose, our text and design have failed. Noting the participants’ exact answers tells us what terms they typically use. [Reading Interest]
  18. 18. Audience TIG Grant 15007 2. Who is this flyer for? (Observer writes each answer given using each participant’s exact words.) ________________________________________ ____ ________________________________________ ____ ________________________________________ ____ If users cannot identify the audience, they are less likely to persist. [Reading Persistence]
  19. 19. Match / Mismatch TIG Grant 15007 3. Do you think this flyer would be helpful for people who don’t speak English well? With each “yes,” answer, ask: OK, so you think it could be helpful? In what way? ________________________________________ ____ ________________________________________ ____ With each “no,” answer, ask: OK, so you said it may not be helpful? Can you tell me more about that? ________________________________________ ____ ________________________________________ ____ Their overall impression tells us about level of reading interest this flyer generates.
  20. 20. Contextualize TIG Grant 15007 4. Let’s say you did not speak English really well, and you want to apply for public services, such as Food Stamps or ask the police for help with a problem in your building. What does this flyer tell you to do? (Observer checks each item participants can name. Observer also notes any incorrect answers provided.)  Go to the office where you want services  Let them know you have the right to get an interpreter  Let them know you have the right to get written translations of imp info  Show them this card  Call the Language Access Coalition to report it  Text the Language Access Coalition to report it  Go to the masslegalhelp.org website to get more info.  Other: ______________________________________ Their answers reveal how clear each item is.
  21. 21. Is this an “open” question? TIG Grant 15007 5. Is it clear from the flier that calling or texting the number provided is only to report having been denied an interpreter, and that calling/texting will not get you an interpreter? If not, how can that be made more clear? Their answers reveal how clear each item is.
  22. 22. Make it “open.” TIG Grant 15007 5. What is this telephone number for? Point to tel # and see what they say. (Accept all answers, do not correct or explain.) Their answers reveal how clear each item is.
  23. 23. 2014 Transcend - Instrument for You have language rights! Flyer – Draft #1 ! OK, Great! Language 7. OK. Looking at the flyer again, take your yellow highlighter and highlight any words that might be difficult for some people. So, even if it’s easy for you, mark any words you think could be difficult for some people. OK. Is there an easier way of saying [insert identified words here]? Their answers will let us know what words may prove difficult. We will learn what terms may be more culturally competent. TIG Grant 15007
  24. 24. 2014 Transcend - Instrument for You have language rights! Flyer – Draft #1 ! OK, Great! Language 7. OK. Looking at the flyer again, take your yellow highlighter and highlight any words that might be difficult for some people. So, even if it’s easy for you, mark any words you think could be difficult for some people. OK. Is there an easier way of saying [insert identified words here]? Their answers will let us know what words may prove difficult. We will learn what terms may be more culturally competent.
  25. 25. 2014 Transcend - Instrument for You have language rights! Flyer – Draft #1 ! ! Likes and Dislikes 8. OK, looking at the flyer again, please tell me anything that you like or dislike about it. (Observer writes each answer given. Facilitator does not say the categories below unless none of the participants mentions anything about that category.) Likes: ! Size ! Font ! Colors ! Images (specify): ! Other (specify): Dislikes: ! Size ! Font ! Colors ! Images (specify): ! Other (specify): Their answers give us feedback on the general look and feel of the flyer. Facilitator does not read possible answers. Their answers can guide us to make it more appealing to other readers. Title 9. Is ¡Usted tiene derechos lingüísticos! an OK title for this flyer? YES/NO ~ Do you think most people will understand it? ~ Can you suggest another title that might be better? Confirm word choice for title If they do not volunteer other titles, toss out a few, such as: Ud. tiene derechos de idioma Tiene derecho de entender Sus derechos de comunicación… Other Improvements 10. Now, let’s step back for a minute and look at the flyer again. If you could change ANYTHING on the flyer to make it better, what would you change? (Observer writes each answer.) Facilitator lets participants say anything they want. S/he may ask other if they also like an idea offered. Facilitator accepts all ideas warmly!
  26. 26. Focus Group Size Large: 9-12 participants Small: 4-8 participants TIG Grant 15007
  27. 27. TIG Grant 15007 Who to recruit: 1. Recruits should match the profile of your typical consumers 2. Recruits should also match profile of under-served populations, e.g., language / ethnic minorities 3. With 8 spots – shoot for a range! 4. Different ages, education & literacy levels, primary languages, ethnic / racial backgrounds, gender, income levels
  28. 28. How to recruit TIG Grant 15007 It takes time to build a database of participants. Get permission from people in charge first! Start by recruiting from places where typical consumers are “waiting.” Ex: health clinics, social services offices, jury pools, explore your local resources! If not available this date, ask if interested in the future. Collect demographic information now!
  29. 29. Your FG Team: TIG Grant 15007 Facilitator / Moderator – Smile, speak pleasantly, calmly. – Repeat participant’s input, ex, OK, great, so you think that part is not clear. – Be flexible! If someone arrives late, explain quickly what you are doing, and get right back on track. – Address participants by name. (Have name tags ready.) – Summarize key points, ask if others agree. – Keep good pace. Not too fast. Not too slow. – Keep people on track. “Let’s talk more about that later after the group so we can hear from everybody.”
  30. 30. TIG Gant 15007 And most importantly… – If you are “attached” to the document and cannot embrace negative comments… – Let someone else be the facilitator.
  31. 31. TIG Grant 15007 Note Taker & Observer – Read the document and instrument carefully before the pretest. – Stay in the background but sit close enough that you can hear what everyone says. – Participate in FG report writing/review. Observer’s job is to take it all in without the distraction of note-taking, facilitating. S/he can help recall information & contribute to report so it accurately reflects the FG.
  32. 32. TIG Grant 15007 Why Pretest? 1. Rehearsal for the FG team - get each person comfortable in his/her role 3. Trouble shoot problems, e.g. – typos, – clumsy or ineffective questions, – major flaws! 2. Confirm logistics: where to sit, materials needed, timing 4. Additional data to compare w/ FG
  33. 33. Demographic Info Name: Tel: Email: Age: First language: Last grade completed: TIG Grant 15007
  34. 34. Pay TIG Grant # 15007 Regular Group (1 to 1.5 hours) $30 - $50 Cash Or equal amount in Supermarket certificate Enough to encourage people to juggle schedules and come! Stipends also convey that you value their participation, opinions, and sets the right tone. Makes it easier for you!
  35. 35. TIG Grant 15007 Prep Work… Ready Your Space – Chairs, table – Mock-up of text to test (print) – Pens and yellow highlighters – Address labels and markers for each participant to write his/her name – Comfortable, cozy is OK. – Toys & extra chairs – if someone shows up with kids – you will be ready! – Make sure there is a clock. You and observer must keep track of time. – Clipboards for Facilitator, Note taker, Observer
  36. 36. TIG Grant 15007 How to Collect Your Data Note Taker & Observer – Make your instrument suitable to record comments – Also use test document to make marks Facilitator – Use document or instrument as needed for facilitating – Example of suitable instrument
  37. 37. TIG Grant # 15007 What is this flyer for? (Allow 30 seconds.) ☐☐☐☐☐ Getting translations ☐☐☐☐☐☐☐ Getting interpreter ☐☐☐☐☐☐☐ Getting someone to help you at a public office ☐☐☐☐☐☐☐ Complaining about not getting help ☐☐☐☐☐☐☐ Teaching you about your rights ☐☐☐☐☐☐☐ Helping you to understand a new law ☐☐☐☐ Other: ☐☐☐☐ Other: ☐☐☐☐ Other:
  38. 38. TIG Grant 15007 Quiet / Shy People – Call them by name. Ana, do you agree with Bob? Which parts did you like most? – Some questions are suited to quick polls: Who else thinks this word is difficult? Raise your hand. – Some questions are suited to going around the table: Who do you think this flyer is for? Let’s start with Florencia, and go around this way.
  39. 39. TIG Grant 15007 Difficult People – Very interesting. Let’s talk about it more right after the group. – Thank you for the comment. I don’t want to interrupt you, but I need to finish these questions first. Let’s talk more afterwards. – Wow! You have such interesting experiences. Let’s talk after we finish.
  40. 40. You have language rights! The law says most public offices must give you services in your language. Public offices must not send you away because of your language, or ask you to bring your own interpreter. You have the right to a free interpreter who will keep your information private. Ask for an interpreter at: · Schools · Courts · Hospitals and doctor’s offices · Police · Government services · Some housing authorities Show the card below to ask for: · a free interpreter, and · written information in your language What if they do not give me an interpreter? Call or text the Language Access Coalition: 617 xxx-xxxx, or Call your local legal aid office. – Cut here – Show this card to ask for an interpreter. I speak _________. I need an interpreter to talk to you. Title VI of the Civil Rights Act says I have the right to an interpreter and information in my language. Questions? Talk with your supervisor or look to your agency’s language access policy. If you cannot give me an interpreter, contact the Language Access Coalition. Call or text: 617-xxx-xxxx (for English ) or 617 xxx-xxxx (target language) Target language translation will print on this side of page (Use as flyer, or half sheet w/ translation on back) Bottom area to be cut off and used as a card. TIG Grant 15007
  41. 41. TIG Grant # 15007
  42. 42. TIG Grant 15007
  43. 43. TIG Grant 15007
  44. 44. Thank you! Maria Mindlin me@transcend.net TIG Grant 15007

×