This dialog was displayed after clicking the You Tube link to view a list of videos posted by a user. F What caused this message? Does an error condition really exist? What happens if I press OK? What happens if I press the close button? If I do the same thing again, will the error message be displayed again? A If something is known to not work, could the user have been prevented from doing whatever caused the error? I Does this mean I can’t do what I tried to do? Will I be able to use this feature with different data? Is there an outage? Can I try again later? L Should this error be logged on the server to assist in identifying error trends? Who should get/use the logged information? Is there any log information that may help me, a tester, understand what caused the error? U What page says? The URL in the error dialog is just the start of the URL for the page I was using. Is this error message’s style consistent with other error messages on the site? R Do I need to wait and try later? Is part of the system down? What happens if I press OK? What do I need to do to make this work? E I can’t see my friend’s videos. I’m unhappy. If I knew more about why it is not working, I might be more understanding.
Sometimes we expect error messages. Just because a test script says an error is to be expected does not mean that we do not need to test the message itself.
F Does Yes reconnect me? Does No not reconnect me? A What is a “peer”? What does 10054 mean? Does the user need this? What is a connection reset? If the connection was reset, doesn’t that mean I’m set back to my expected connected state? This error message appears to have been written for technical people. Are those the only users of this application? I What does it mean to reconnect? What if I don’t reconnect? What does it mean that my connection was reset by a peer? L Does this need to be logged as an error? Is the reason for the reset logged somewhere else? On the client? On the server? U Could the software automatically reconnect me and not bother me with reconnecting? Would that be better? R If looks like I can select the “Yes” button to recover, but its not clear from what I am recovering. E I might be happy that a connection loss is being detected and I am being given an option to reconnect with the click of a Yes button.
F What causes this dialog? A Why is a success being reported as an “Error”? I What’s the impact of the success, or error, or whatever it is? L If this should have been logged, I hope the log tells me more than this message. U The text is contradictory. R How do I fix a success error? E I’m confused. Is it an error or is it a success?
F What value? How is it not valid? What do the buttons do on the dialog? Is the value really not valid? Why is there a Retry option for something that’s not valid? A Could Excel have prevented me from entering something that’s not valid? I don’t know what “not valid” means. I What is the impact of my entering invalid data? Is it lost? Can I continue anyway? L This probably doesn’t need to be logged, but I’d like to have logged info about what user restricted the values. U The dialog does not tell me what was invalid. It does not tell me what is valid. Why am I offered an option to retry? Retry what? Enter an invalid value? Won’t it still be invalid? Or does Retry give me an option to enter a new value? What does cancel do? Cancel the restriction on the values or cancel my entry? R How do I remove the restriction? E I might be happy that I was stopped from entering bad data. I might be unhappy that I am not allowed to enter valid data. I might be confused about what is required to make my data valid. I might be confused about the meanings of the buttons.
F Something looks very wrong here. A file not being open should not prevent a file from being opened. What happens if I press OK? A I don’t see any context information about what file cannot be opened. I So what is the impact of not being able to open a file that is not open? L Does this need to be logged? U I see no context information about the error. The message is confusing. Is this accurate? R If I can’t open a file that is not open, then how do I open a file in the first place? Will it work if I try again? How do I open the file? Is the file that could not be opened my file or something needed by whatever application displayed the message? E I am now confused and frustrated. How can this message be improved to reduce confusion and frustration?
F Does OK restart the computer? A Shouldn’t I have been warned about needing to restart before making the change? What if I can’t restart now? I What if I don’t restart now? L Is this logged? Does it need to be logged? U Why must I restart? Does OK restart me or just confirm that I read the message? R Can I back out of the setting change? If so, how? E I don’t want to restart. I was attempting to fix a display problem and now I feel that I am being forced to restart. Is it possible to give the user an option to back out? Could rewording the message and/or changing the button to state what clicking OK does alleviate user confusion?
I saw a dialog like this in a product demo. The demonstrator didn’t understand it and clicked cancel to make it go away and then went on to demo a different feature. F Wow. I have no idea how this is supposed to function. Are there blank dialogs in other parts of the application? A A blank dialog does not seem to be appropriate. I What the impact? I can’t tell because I don’t know what this dialog means. L Did this generate a blank log entry? Is there a log that might explain what happened here? U Users should not be asked to OK something without telling them what it is. There is no title or text. What does this mean? This dialog is very broken. R How do I recover from not knowing anything? This needs to be fixed. E I’m very confused now. I thinking that someone somewhere made a huge mistake to generate such an error dialog? And I might be laughing.
F If my statement is valid, why am I getting an error dialog? Is this an informative dialog that is mislabeled as a “Task Error”? What if I try an invalid SQL statement? A It doesn’t seem to be appropriate to give an error for a valid SQL statement? Is there an error in the SQL statement? Is there an error in the message? Is a dialog box the best way to communicate whatever is being communicated? I What is the impact? I first need to know if there is a problem with the message or the code that triggers the message. Is my statement lost because it is valid or invalid? L Does this need to be logged? U This message is really confusing. Why am I being shown an error for a valid SQL statement? -- if that’s really the case R How do I fix my valid statement so that I can continue? E Now, I’m either really frustrated, or I’m laughing at this silly dialog.
F If I re-enter $20, what happens? What if I request $40? If I enter a multiple of $50, will it work? Did the machine run out of $20 bills? Is the machine ever able to issue $20? A Why was the user allowed to select $20 is the system is not able to issue $20? Wouldn’t it be better not offer $20 as an option if the system in unable to issue $20? I What if I only want $20? What if there is less than $50 in my account? Will the machine still encourage me to withdraw more than I have available? It can’t “issue” me $20, but did it first deduct $20 from my account? I want to know that my money is still safely in my account. L Is this condition logged? U Why is the user allowed to select an invalid option? Why is there a picture of a $20 and a $50 bill in the error screen if the system can’t give me $20? Why is there no quick selection options for re-entering an amount? Where’s the OK that I am supposed to press? R This tells me how to recover. It tells me I need to enter an amount that is a multiple of $50. E I am unhappy that I am being forced to take more money than I want out of my account. I might have been happier if I had not been given an option for $20 if that were not available.
You can practice using FAILURE on the error messages included in this presentation.
Now that I have your attention, prepare to read fast v1.2 FAILURE.QualityFrog.com
You’re testing… And an error message appears v1.2 FAILURE.QualityFrog.com