Software is complicated. I had analyzed the defects in a sandwich store, based in Singapore. You can try to analyze how you would have thought about these problems. When analyzing defects it is easy to point fingers and say, 'How could you?'. Instead, I recommend coming up with rules or heuristics on how you would have prevented the problems, if you were designing the product (in this case - sandwich). This requires a bit of mental gymnastics.

1. Sandwich testing challenge
Nilanjan Bhattacharya
Counterfactuals

2. Overview
• In a previous presentation I had listed problems,
reported by customers in a desktop software, as
an exercise in thinking about testing
• In this presentation I list problems faced by a
customers in a sandwich fast food restaurant
• This presentation can be used for a non-technical
audience to appreciate the challenges of testing
• Feel free to skip the introductory slides if you’ve
seen the previous presentation

3. Background
Software testing has been compared to games
like 20 questions or Pictionary
In Pictionary one person draws pictures to
describe a card
Others try to guess what was on the card

4. Practicing thinking
Suppose you want to get better at creating clues
in Pictionary, you could look at example
drawings, which weren’t succesful, and think
about what you could do differently
In this presentation I show you problems faced
by customers of a software product and ask
you to think about how you (as a tester)
would have thought about those problems

5. Instructions
User experience
I have listed problems that users have faced.
Background
In some cases I try to give background information
Your turn
I’ve left a blank slide for you to enter your analysis
Learning
I then give my analysis
There are no scores :-(

6. My definition of a defect
If a customer experiences even a hint of discomfort or irritation
when using the software, and I hadn’t considered that
possibility, that is a shortcoming/defect in my thinking.
I need to think about how to think about that aspect of the
software which resulted in the user discomfort. In my analysis
I try to create (generic) rules which will make me think about
these issues during testing.

7. Application under test
Software is complicated. In this example instead
of looking at software, I decided to look at the
defects customers face in a fast-food sandwich
restaurant. This is not a burger restaurant. In
this store submarine sandwiches are
assembled for each customer
Note: I had written a similar article which was
focused on software

8. Customer experience
• Customer wanted Egg mayo with 2 additional
scoops of tuna.
• She was charged for Tuna with 2 scoops of egg
mayo
• Staff argued that she would have to pay the
higher price

9. Your turn
• As a tester, write down how you would have
prevented this problem

10. Learning
• Did I think about the different variations of
add-ons?
• Did I think someone would combine tuna and
egg-mayo?
• Given that tuna is more expensive did I think
about the confusion that might cause?
• As a business did I think about training wait-
staff to offer the customer the better option
even if it means loosing money?

11. Customer experience
• Store has run out of bread at 9 pm. Customer
is resigned….

12. Your turn
• As a tester, write down how you would have
prevented this problem

13. Learning
• This seems like business as usual. I included
this purposely to stretch your tester
imagination/brain
• I should have thought about how the wait-
staff can respond to customers.
• Can we do anything less vapid than ‘I’m
sorry….buh-bye….’?
• Can we distribute free coupons? Or more…?

14. Learning (contd.)
• As a response to this problem business
owners/developers may be tempted to write
better algorithms to predict when the store
will run out. No harm in that. However, you
should, at the same time, think about risk and
mitigation, i.e., what if the unthinkable
happens.

15. Customer experience
• Store introduces a special promotion – a
designer take-away bag. The customer
purchases a 6-inch sandwich and while
commuting the bag handle breaks

16. Your turn
• As a tester, write down how you would have
prevented this problem

17. Learning
• This is one of the issues which could be
classified as a ‘defect’
• I can only guess that the people who designed
the bag didn’t try to commute with something
in the bag

18. Customer experience
• ‘I can’t submit the survey form at
www.*****.sg’

19. Your turn
• As a tester, write down how you would have
prevented this problem

20. Learning
• Are there any alternatives if the website fails
• Does it start with displaying a phone number
on the receipt to call as an alternative?
• Should a customer service website fail? Isn’t
that terrible?

21. Customer experience
• Customer: ‘I was told the promotion bags are
available only from 2 pm to 5 pm’

22. Your turn
• As a tester, write down how you would have
prevented this problem

23. Learning
• I should have anticipated that there would be
miscommunication with customers, e.g.,
terms and conditions are not known
• In this case, should the staff have given the
bag to the customer?
• Should I have modeled all the possible sources
of confusion?

24. Customer experience
• I asked for the bread to be toasted longer, but
I was refused…only in this outlet

25. Your turn
• As a tester, write down how you would have
prevented this problem

26. Learning
• Did I think about the customer asking for
custom cooking times? How should the staff
respond?
• This is a classic case of a problem which a
‘developer’ would likely dismiss – ‘No user
would ever do that’

27. Customer experience
• Customer: ‘What type of meat do you use in
Meatball Marinara’

28. Your turn
• As a tester, write down how you would have
prevented this problem

29. Learning
• Did I think about the customer wondering what is
in the meatball?
• Is this more of a problem because the store is in
Singapore with a large Muslim and Hindu
population?
• I should think about the variety of stores and how
they can have different types of customers and
environments
• Compare with McDonalds – there is no mystery
meat – a burger is a beef burger?

30. Customer experience
• Customer: ‘Can I get the grilled chicken
sandwich?’
• We don’t serve ‘green chicken’
• Customer complained that he was ridiculed by
staff

31. Your turn
• As a tester, write down how you would have
prevented this problem

32. Learning
• Should the staff have been trained to be more
tolerant of accents
• Is this a problem in Singapore with the large
number of non-English accents?
• Should I have provided a menu with numbers,
so customers can order #1?

33. Customer experience
• The store ran out of bread and added the
usual extra charge for flatbread or wrap

34. Your turn
• As a tester, write down how you would have
prevented this problem

35. Learning
• Did I think about this scenario – when the
store runs out, if there is a substitute, don’t
charge the customer the extra amount, or
maybe charge less?
• Are there other combinations when the store
runs out and the customer asks for a
substitute? How will they be charged?
• Should I have spent more time focusing on
problems like this?

36. Customer experience
• Customer: ‘Not familiar with submarine
sandwiches, staff stares while I struggle to
figure it out’

37. Your turn
• As a tester, write down how you would have
prevented this problem

38. Learning
• Given a fast food business model, minimum
wage, staff turnover, explicit menus, what if a
customer needs help?
• Most people (staff) like to help customers – let
them do so
• If a customer needs help or is not sure, do
volunteer to help

39. Customer experience
• Waiting for change for 10 minutes while staff
guides another staff member

40. Your turn
• As a tester, write down how you would have
prevented this problem

41. Learning
• In hindsight, this seems like a simple fix, ‘Train
staff not to keep the customer waiting.’
• Did I think of emergencies/distractions?
Should I have educated staff to always have a
laser focus on the customer waiting? What if
the customers ordering and paying are both
waiting?
• Will training staff work in a franchise model?

42. Customer experience
• Customer: ‘Is Spicy Italian beef or pork?’

43. Your turn
• As a tester, write down how you would have
prevented this problem

44. Learning
• This is similar to a previous issue
• This could be more of a problem in Singapore
or other Asian countries
• Given the fast food business model, i.e., staff
turnover, minimum wage staff, it may be
worth listing the meat composition

45. Customer experience
• Customer: Same meal at two outlets is at a
different price
• Customer service response: Different
franchise outlets may charge different prices

46. Your turn
• As a tester, write down how you would have
prevented this problem

47. Learning
• The learning here for a tester is, ‘was I aware
of this policy’.
• If I was aware, it would probably mean a long
conversation with the product owner
• As a tester I am more interested in how
customers would perceive this policy. I may
not be able to change the design

48. Customer experience
• Customer is told, ‘Promotional meal not
available.’
• Customer service: Meal should be available at
all outlets

49. Your turn
• As a tester, write down how you would have
prevented this problem

50. Learning
• Did I think about an outlet being unaware of a
promotion?
• It’s easy to think that the only action is - make
sure the outlet is aware the next time.
• The more important learning is to train staff to
listen to the customer and do some fact
checking

51. Customer experience
• Customer says, ‘I don’t want this.’
• Staff: ‘I’ve already keyed in the amount.’

52. Your turn
• As a tester, write down how you would have
prevented this problem

53. Learning
• Can the customer change his mind at any
stage? (Can they always undo without a
threat?)
• Can we display what they are purchasing and
the price like McDonalds, i.e., display a screen
with the purchased items and price?
• Would it be better to pay first? If selecting the
ingredients is part of the experience, can we
take special care to avoid sticker shock?

54. Customer experience
• Customer: ‘I want the chicken strips heated
separately.’
Background
• Customer says that chicken remains cold

55. Your turn
• As a tester, write down how you would have
prevented this problem

56. Learning
• This is similar to a previous issue
• What if a customer needs a custom order?
• Should staff ask customers why they want a
special order?

57. Customer experience
• Customer is shocked by the price of avocado
add-ons. ‘The staff assumed I want 4 scoops.
I couldn’t understand the accent.’

58. Background
• Another example of sticker shock
• Was this even more of a problem with
Singapore’s diversity, i.e. ,with many different
accents, taking orders can result in confusion?

59. Your turn
• As a tester, write down how you would have
prevented this problem

60. Learning
• Should I have made sure price is explicit when
ordering add-ons?
• Should I have thought about the diversity of
the population and the potential confusion
when informing customers?

61. Customer experience
• Customer is puzzled that a foot long costs
more than two $5 meals

62. Your turn
• As a tester, write down how you would have
prevented this problem

63. Learning
• In hindsight, this seems like an obvious
problem.
• Was it as easy when I looked at the promotion
to think about a customer comparing two 6-
inch meals to a foot long?