This study analyzed interviews conducted by 110 student analysts to develop a list of common mistakes made during requirements elicitation interviews. The researchers identified 34 unique mistakes classified into 7 categories: formulating questions, question omission, order of questions, communication skills, analyst behavior, customer interaction, and teamwork/planning. By analyzing interview recordings and observations, the researchers found that while experience did not impact performance, preparation, domain knowledge, addressing ambiguity, active listening, and rapport were important for effective elicitation. The goal was to provide guidance to help students learn from mistakes.

1. Learning from Mistakes
An Empirical Study of Elicitation Interviews performed by Novices
Didar Zowghi
Didar.Zowghi@uts.edu.au
Muneera Bano
mbano@swin.edu.au
Alessio Ferrari
alessio.ferrari@isti.cnr.it
Paola Spoletini
pspoleti@kennesaw.edu
Beatrice Donati
beatrice.donati@unifi.it
This work was partially supported by NSF under grant CCF-1718377

2. Interviews
• A communicative event in which an interviewer asks
questions to reach to the reality of a phenomenon
conceived inside the mind of the interviewee

3. Interviews
• Why challenging?
• Consideration for
• Human oriented factors
• Process oriented factors
• Context oriented factors

4. Teaching the art of interviewing!

5. Teaching the art of interviewing!
Guiding Educationists in Requirements Engineering
“how to teach art of requirements elicitation interviews?”

6. Motivation
Zowghi, D. and S. Paryani. Teaching
requirements engineering through role playing:
Lessons learnt. in 11th IEEE International
Requirements Engineering Conference. 2003.
IEEE.
Yusop, N., Z. Mehboob, and D. Zowghi. The
Role of Conducting Stakeholder Meeting in
Requirements Engineering Techniques. in
International Workshop on the Requirements
Engineering Education and Training. 2007.
IEEE Computer Society.
Donati, B., et al. Common Mistakes of Student
Analysts in Requirements Elicitation Interviews.
in International Working Conference on
Requirements Engineering: Foundation for
Software Quality. 2017. Springer.

7. Objective
• Therefore, the overall aim of this observational study is to
develop an empirically validated list of mistakes and related
examples to assist students in learning the skills of effective
elicitation interviews.
• Improving and building upon previous knowledge of REET
• Not just about students’ learning
• For teachers to do retrospective analysis of teaching
methodologies based on students’ performance during elicitation
interviews

8. Pedagogical Approach
• Corrective Feedback Learning
• Using failures, mistakes or bad decisions as learning opportunities to
improve in the future
• Role Playing
• rehearsing a real world environment for practicing certain skills
• Authentic Assessment
• replicating real-world challenges and standards of performance that
experts or professionals typically face in the field
• Collaborative Learning
• Involving groups of learners working together to solve a problem,
complete a task, or create a product while being challenged both
socially

9. Study Settings
28 groups (110 students) of
Masters’ course of Enterprise
Business Requirements
Case Study 1
Online
shopping and
elivery
System
Case Study 2
Appliance
repair service
management
system
1. Lynda.com video training
2. Lectures and Tutorials on
Requirements Elicitation
Course Designer and Instructor

10. Pedagogical and Research Design
First Interview Second Interview Third Interview
Minutes of
meeting
Minutes of
meeting
Minutes of
meeting
Feedback Feedback Feedback
Use Cases
SRS
Document
Analysis presented
in current study

11. Customer
Observer
Student Analysts
Customer
Think Aloud
Audio
Recordings
Observations
Minutes of
meeting
Interview Data Collection
Audio
Recordings

12. Data Analysis
Audio Recordings (28
files, 15 minutes each)
Observations
Reviewer 1
Reviewer 2
Review 1
Review 2
Customer
Think
Aloud
Synthesis
and
Thematic
Analysis
Minutes of
meeting
(28 files)
List of
mistakes
Feedback

13. Results
• We found 34 unique mistakes classified into following seven
categories:
• Formulating Questions
• Question Omission
• Order of interview questions
• Communication skills
• Analyst Behaviour
• Customer Interaction
• Team work and planning

15. Experience vs planning
Davis, A., et al. Effectiveness of
requirements elicitation techniques:
Empirical results derived from a
systematic review. in 14th International
Conference on Requirements
Engineering,. 2006. IEEE.
“Analyst experience does not appear to be a relevant
factor during information acquisition, at least using
interviews as an elicitation technique. ”

16. Domain knowledge

17. Ambiguity as a resource
Spoletini, P., Ferrari, A., Bano, M., Zowghi, D., &
Gnesi, S. (2018). Interview Review: An Empirical
Study on Detecting Ambiguities in Requirements
Elicitation Interviews. In International Working
Conference on Requirements Engineering:
Foundation for Software Quality (pp. 101-118).
Springer, Cham.
“Ambiguities identified during requirements
elicitation interviews can be used by the
requirements analyst as triggers for additional
questions and, consequently, for disclosing further,
possibly tacit knowledge. Therefore, every
unidentified ambiguity may be a missed opportunity
to collect additional information.”

18. It is not just enough to ask
the right questions in the
interview, but it is also
equally important to listen
carefully to the responses
given by the customer and
accurately record the
understanding developed.
The content of the minutes
gives us a good indication
of the level of students’
understanding of the
application domain and the
initial requirements
developed in their
interview.

19. Rapport with the customer

20. Future Directions
• Analysis of all three rounds of interview recordings
and MOM
• Tracing the mistakes for observing improvements in all
three interviews based on ‘corrective feedback
learning’ philosophy
• Tracing the mistakes in all three interviews to all the
way to SRS and quality of requirements
• Video recording for body language analysis

21. References
• Davis, A., et al. Effectiveness of requirements elicitation techniques: Empirical results derived
from a systematic review. in Requirements Engineering, 14th IEEE International Conference.
2006. IEEE.
• Donati, B., et al. Common Mistakes of Student Analysts in Requirements Elicitation Interviews.
in International Working Conference on Requirements Engineering: Foundation for Software
Quality. 2017. Springer.
• Spoletini, P., Ferrari, A., Bano, M., Zowghi, D., & Gnesi, S Interview Review: An Empirical Study
on Detecting Ambiguities in Requirements Elicitation Interviews. in International Working
Conference on Requirements Engineering: Foundation for Software Quality. 2018. Springer.
• Yusop, N., Z. Mehboob, and D. Zowghi. The Role of Conducting Stakeholder Meeting in
Requirements Engineering Techniques. in International Workshop on the Requirements
Engineering Education and Training. 2007. IEEE Computer Society.
• Zowghi, D. and S. Paryani. Teaching requirements engineering through role playing: Lessons
learnt. in Requirements Engineering Conference, 2003. Proceedings. 11th IEEE International.
2003. IEEE.

22. Learning from Mistakes
An Empirical Study of Elicitation Interviews performed by Novices