This includes a series of defects customers found in software. I analyze how I would have found those defects, instead of complaining about the problems.

1. Photoapp - software
testing challenge
Nilanjan Bhattacharya
Counterfactuals

2. Background
Software testing has been compared to games
like 20 questions or Pictionary
In Pictionary one person draws pictures to
describe a card
Others try to guess what was on the card

3. Practicing thinking
Suppose you want to get better at creating
clues in Pictionary, you could look at example
drawings, which weren’t succesful, and think
about what you could do differently
In this presentation I show you problems faced
by customers of a software product and ask
you to think about how you (as a tester) would
have thought about those problems

4. Instructions
User experience
I have listed problems that users have faced.
Background
In some cases I try to give background information
Your turn
I’ve left a blank slide for you to enter your analysis
Learning
I then give my analysis
There are no scores :-(

5. My definition of a defect
If a customer experiences even a hint of discomfort or
irritation when using the software, and I hadn’t considered
that possibility, that is a shortcoming/defect in my
thinking.
I need to think about how to think about that aspect of the
software which resulted in the user discomfort. In my
analysis I try to create (generic) rules which will make me
think about these issues during testing.

6. Application under test
The application I use here is a desktop
application for organizing photos, like Google’s
Picasa.

7. User experience
User asked to sign in to Picasa every 6/12 hrs
on phone

8. Background
Android phones have a gallery app
Android syncs Picasa web albums with the
gallery app
Note that Picasa does not run on phones
More information : Link

9. Support experience
The initial reaction from support was that the
phone has nothing to do with Picasa
Another user gives them a earful about them
being the same company

10. Your turn
As a tester, write down how you would have
prevented this problem

11. Learning
● Always check who is consuming or might
consume the data I created
● If someone else is consuming the data how
do they authenticate
● Do they have any means of informing the
user that they are consuming the data

12. Learning (contd.)
● As a corporation is there any mechanism for
being notified when another group consumes
my data

13. User experience
How do I assign a date when I scan photos into
Picasa
Additional problem: Picasa insists on yyyy-mm-
dd when adding a date to photos

14. Your turn
As a tester, write down how you would have
prevented this problem

15. Learning
● Always have a paranoid focus on how data
gets into the system (in this case scanning)
● Think about metadata - suppose data gets in
by some other means can you add metadata
o Compare data entered manually with this other
method

16. Learning (contd.)
● When adding time stamps can you allow
only the year or year and month
● When following complex data formats
(including time/date) can we make it easy on
the user, instead of DD-MM-YYYY
● Remember that the users of this software
can be very non-technical

17. Learning (contd.)
● When importing data it’s possible to import
large amounts of data
o Always ask what happens if I use large amounts of
data
 When can that happen

18. User experience
User wants to backup data
Picasa directories contain two folders ‘temp’
and ‘db3’. These contain 100s of MB of data
compared to a few MB of the actual photos
User has a slow DSL line
Backing up the entire DB can be slow

20. Support experience
Forum tries to explain the concept of rebuilding
the database
User mentions he doesn’t care about rebuilding

21. Your turn
As a tester, write down how you would have
prevented this problem

22. Learning
● When my app is creating data, I should
consider how it will be backed up
● Backing up will require the user to know the
directory structure
● If I am creating working data, can I save it in
another location than the user data

23. Learning (contd.)
● I should characterize the value of the data
used - in this case pictures are extremely
valuable - this probably has a higher priority
than how I manipulate the pictures, e.g., face
recognition

24. Learning (contd.)
● Will anyone in support ever commit to
deleting the system folders? No! Make sure
there are clear instructions internally on the
data; whether it can be deleted and the
implication

25. User experience
Trying to open Picasa get a meaningless
message

26. Background
Database might be corrupt; try rebuilding the
database

27. Your turn
As a tester, write down how you would have
prevented this problem

28. Learning
● Always corrupt the database and check how
the program behaves
● In this case the database is probably quite
complex and there is room for error
● Note that the end user is probably non-
technical and will be confused by not being
able to use the program
o Always characterize your user

29. Learning
● Think about how often the user uses the
program. If used daily, any downtime can
make users irate.
● Is the database more susceptible since it is
on the client machine

30. Learning
I included this issue because people
inexperienced in testing are always bashful
about corrupting data/database

31. User experience
User wants to edit the photo but there are no
options to do so

32. Background
The buttons to make the the toolbar visible are
very small
troubleshooting

33. Your turn
As a tester, write down how you would have
prevented this problem

34. Learning
In hindsight this seems almost silly
No matter how modern the UI do think about
the obstacles in using the software

35. User experience
Picasa gives an error message when it boots
up

36. Background
An update to iTunes caused a conflict with
Picasa
r6034

37. Your turn
As a tester, write down how you would have
prevented this problem

38. Learning
What are the common libraries used by my
software?
What if the library misbehaves?
What if the library is updated?
Can I simulate library misbehavior?

39. Learning
Developers (and testers) often take operating
system behavior for granted. This represents a
weakness, i.e., it will not be well tested.

40. User experience
Scan a document but import button doesn’t
highlight

41. Background
Picasa uses the TWAIN drivers to connect to
the scanner
This may not always be reliable

42. Your turn
As a tester, write down how you would have
prevented this problem

43. Learning
•When choosing an open platform vs. specific
drivers there is a compromise
•As a tester/developer you may not have influenced that
choice
•You should primarily articulate how users are affected

44. Learning
•When the drivers don’t respond, how do users
help themselves
•Working with hardware that doesn’t respond
can be very frustrating
•Can we provide a support channel for such
users – not just the regular support

45. Learning
•Can we prompt the user about the support
channel?
•Can the software be intelligent enough to
figure out that the device is not responding?
(Don’t let the developer/designer saying ‘No’,
deter you)

46. User experience
The pictures in Picasa don’t show up in ‘My
Pictures’
Background
•The user sees the pictures in Picasa, but the
effects don’t show up when he sees the
pictures in Windows Explorer/My Pictures

47. Background
•What you see on the filesystem may not be
what you see in Picasa
•What you see in Picasa may not be what you
see in the filesystem
•Background on saving
•Original pictures are saved in a hidden folder
in the file system and are not altered
•This is done only when the user does a File->Save

48. Your turn
As a tester, write down how you would have
prevented this problem

49. Learning
•The filesystem is a strong mental model. How
would the software inform the user that we are
now changing that model? Is that a good idea?
•Did I consider the various activities during
which users might be confused?
•Was I able to convince developers to provide
some proactive help to the user?

50. User experience
The user reinstalled Windows
Changed username
Copied Picasa files to the same location
Now Picasa doesn’t recognize photos

51. Your turn
As a tester, write down how you would have
prevented this problem

52. Learning
•What can users do with the application data?
•In this case the data is created by users
(photos) and not the application and users may
try to move it around
•Does the system store hard coded paths
•Are there other applications which allow users
to move their data/profiles? Does Firefox allow
you to do this?

53. Learning
•What are the usecases when users may want
to move data to another location?

54. User experience
Picasa is compressing files for a few hours

55. Background
Compressing shouldn’t take too long. If it does
it could be a problem with the database. In that
case you should rebuild the database.
Rebuilding the database

56. Your turn
As a tester, write down how you would have
prevented this problem

57. Learning
•Another example of a corrupt database
•Can the software self heal (Don’t say ‘No’)

58. User experience
After upgrading Windows Picasa crashes
frequently

59. Background
When one of the devices being used was
disconnected, crashes stopped
The device had an updated driver

60. Your turn
As a tester, write down how you would have
prevented this problem

61. Learning
•When working with devices try upgrading the
driver
•What if the driver misbehaves? How does the
software respond?
•When are drivers updated, is it only during
upgrade?
•Can you simulate updating the driver? (not so
easy)

62. Learning
Software development teams are often not
comfortable with hardware (hardware teams
may not be comfortable with software). That is
an opportunity for test

63. User experience
Picasa is generally slow; tried everything -
reinstall, etc.

64. Your turn
As a tester, write down how you would have
prevented this problem

65. Learning
This is similar to the previous issue - the
database is corrupt

66. User experience
I keep trying to install the English Picasa on a
French OS without success

67. Background
Multiple languages

68. Your turn
As a tester, write down how you would have
prevented this problem

69. Learning
•How will the user know that the capability of
multiple languages is built in?
•Sometimes ‘smart’ can cause confusion (not
trying to be snipey).
•Can we inform the user when he is
downloading?
•Can we detect the user’s locale?

70. User experience
Why are my photos deleted from my camera
after I import them?

71. Background
When Picasa imports pictures there is an
option ‘Leave card alone’
However, user is still frustrated that they didn’t
have time to respond

72. Your turn
As a tester, write down how you would have
prevented this problem

73. Learning
•Deleting photos is heinous
•When testing import I should have been
paranoid about identifying cases when photos
could be deleted
•Did I list down every possible case when the
import might fail – for any possible reason,
including earthquakes
•Was I able to observe real beta users to see if
they had any uncertainty about the photos
being deleted

74. Learning
•Even when users agree to delete photos
should the software delete photos (did I ask the
Product Manager; did I question his response)
•I agree it is still my fault not the Product Manager’s
•Did I highlight the analogy between deleting
photos during import and Picasa not altering
photos on the hard disk? (see related issue and
the explanation)

75. User experience
Deleted photos from Picasa, but in the phone
gallery they show blanks
Background on gallery in Android

76. Background
This is similar to a previous issue. Take a look
at that
Also look at:
How to remove picasaweb pictures from phone
Also look at the comments in that web page
The phone automatically syncs with Picasa -
the user may not want to do that

77. Your turn
As a tester, write down how you would have
prevented this problem
Hint: this is related to a previous problem

78. Learning
Same as the previous issue

79. User experience
I downloaded pictures in RAW, when I click
they turn blue

80. Background
Picasa supports RAW formats. However, there
are hundreds with new ones being introduced.
Google can’t keep up with them.
RAW format

81. Your turn
As a tester, write down how you would have
prevented this problem

82. Learning
•Did I make sure I had a sufficient mix of RAW
format pictures to understand the issue?
•Did I realize that the user may not understand
the background of RAW format?
•Did I think how we could inform the user when
importing RAW?

83. Learning
Could the software have detected that the
picture was not looking right?

84. User experience
Tried to import photos from phone and get
message:
'An error has occurred while attempting to
import. Either the source is unavailable or the
destination is full or read only. (1)'
After struggling I got the photos, but videos
don’t come in

85. Background
Not sure of the final root cause
Users try fiddling around with files on the phone
and on Picasa
They also call tech support for the phone

86. Your turn
As a tester, write down how you would have
prevented this problem

87. Learning
•Import is always a challenge
•For this product import is particularly important
since it is a consumer product
•When the product was created did I anticipate
the proliferation of mobile devices
•Testing all devices is not a strategy!!
•Did I consider the various software which
might create data on the device
•Did I think about how the user can get help if
something does not work?

88. Learning
•Did I question if there is a point in giving a
generic message. Doesn’t the message in
effect mean, ‘….please self soothe…’

89. User experience
Picasa crashes (blue screen) after I use it for
some time

90. Your turn
As a tester, write down how you would have
prevented this problem

91. Learning
•Can we do anything?
•I included this since it seems like we can’t do
anything – ‘it’s a blue screen, obviously it’s your
problem??’
•Nowadays, some softwares keep track of
multiple crashes and report it
•At the very least you can keep track of
crashes, i.e., send a crash error report

92. User experience
When I export photos the description doesn’t
appear

93. Your turn
As a tester, write down how you would have
prevented this problem

94. Learning
•Did I put myself in the non-technical user’s
frame of mind when using the software?
•Does the user know that the description is
native to Picasa?
•Can we let her know what happens during
export, especially in this context, i.e.,
descriptions of photos aren’t part of the photos?

95. User experience
I updated Picasa. Now everytime I open Picasa
I get a message that I need to update.

96. Background
This is an example of a defect, i.e., something
is broken.
The root cause is not clear
Here is a link on how to fix the problem

97. Your turn
As a tester, write down how you would have
prevented this problem

98. Learning
•Although this is difficult to analyze, one
learning is that upgrade of a new release is
difficult to test
•I should make sure I have a dummy update
release to test update
•This is a good candidate for test automation
•Can the software detect repeated failures

99. User experience
Scanning old photos date taken is blank.
Picasa then uses 1/1/4501

100. Your turn
As a tester, write down how you would have
prevented this problem

101. Learning
•Did I completely understand the meta data
associated with a photo
•Did I consider the differences in various paths
that data might enter the system, e.g., import,
API and the related limitations
•In the grand scheme of things, this issue is
trivial; but symptomatic of shortcomings in
testing

102. User experience
Flip a photo and apply red-eye undoes flip

103. Your turn
As a tester, write down how you would have
prevented this problem

104. Learning
•Did I try different combinations of effects?
•Did I try the combinations in a systematic
manner?
•Could I have used a tool to try different
combinations? Could that have been verified
automatically?

105. User experience
I was moving photos around suddenly I lost all
my photos
The user was trying to delete some pictures
and moving others around. He may have hit a
key by mistake.
Not sure of the final root cause.

106. Background
Missing photos
Recover photos
Reorganize photos

107. Your turn
As a tester, write down how you would have
prevented this problem

108. Learning
This is a good example of users not
understanding that the software is not changing
the photos on the operating system.

109. User experience
How do I use my accounts on multiple
computers
When multiple users use the same computer,
the applications can get confused
Note that in this case the user was generally
irate

110. Your turn
As a tester, write down how you would have
prevented this problem

111. Learning
•Did I think about the challenges that users may
face with multiple applications?
•As a tester on a single application could I have
done something to mitigate this problem
•Did I think about what happens if multiple
users use the application on the same
computer?

112. User experience
It takes too long to copy pictures from SD card
Not sure of the root cause in this case.

113. Background
Pause autobackup
Use explorer to copy

114. Your turn
As a tester, write down how you would have
prevented this problem

115. Learning
•Did I consider the potential problems of using
SD cards on a variety of devices?
•I should not blindly test different SD cards
•Could the software have recognized that there
was a problem with the time taken to copy?
•Did I consider potential conflicts with the
backup software?

116. User experience
Image isn’t cropped, when I send it to someone
on Facebook it appears uncropped
Note that Picasa does not change the original
file, so the original is not cropped.

117. Your turn
As a tester, write down how you would have
prevented this problem

118. Learning
At every step I should have thought about how
users may misunderstand that the photo in
Picasa is the same as that on the disk.

119. User experience
I search for files and I am shown results, but
clicking on the results shows nothing
Missing photos

120. Background
Picasa has an option to ‘Scan once’
Folders are only scanned once and results are
not updated with changes in the folder

121. Your turn
As a tester, write down how you would have
prevented this problem

122. Learning
•Did I wonder if the settings might confuse non-
technical users?
•Can they comprehend what the software is
doing to the files on the operating system?