The importance of user-developer interactions during the development of an information system has been a long-running theme in information systems research. This research seeks to highlight a gap in the current literature: the contribution of the developer’s formal educational background to the relationship between developers and users. Using an interpretivist epistemology, the researchers employed qualitative interviews to examine how far developers’ perception of the importance of interacting with the user was influenced by their formal education, or the lack thereof. Interviewing both formally and informally trained developers, eleven categories of interest were identified as pertinent to determining the developers’ beliefs about the importance of user interaction. Three of these categories were explored as promising for future research: academic background, work experience, and developer’s access to user knowledge. This research has implications for education of information systems developers as well as for industry interested in hiring software developers.

1. Effects of Developers’ Training on User-Developer
Interactions in Information Systems Development
Jennifer McCauley, Sharoda A. Paul, Andrey Soares
April 20, 2006
IST541 – Qualitative Research in IST
College of Information Sciences and Technology
The Pennsylvania State University
{jmccauley, spaul, asoares}@ist.psu.edu
Abstract: The importance of user-developer interactions during the development of an
information system has been a long-running theme in information systems research. This
research seeks to highlight a gap in the current literature: the contribution of the developer’s
formal educational background to the relationship between developers and users. Using an
interpretivist epistemology, the researchers employed qualitative interviews to examine how far
developers’ perception of the importance of interacting with the user was influenced by their
formal education, or the lack thereof. Interviewing both formally and informally trained
developers, eleven categories of interest were identified as pertinent to determining the
developers’ beliefs about the importance of user interaction. Three of these categories were
explored as promising for future research: academic background, work experience, and
developer’s access to user knowledge. This research has implications for education of
information systems developers as well as for industry interested in hiring software developers.
Keywords: information systems development, user interaction, developers training
1 INTRODUCTION
User participation and involvement has been a long running theme in information systems (IS)
1

2. research, specifically with respect to the development of information systems. Research on the
correlation between user participation/involvement and success of information systems
development (ISD) has produced contradictory results. While some researchers have found that
greater user participation leads to more user satisfaction, there has been no agreement on whether
greater user participation guarantees successful ISD.
In examining the effect of user participation, much attention has been paid to the relationship
between users and developers, especially on the subject of how they share and transfer
knowledge about the system. However, there has been very little research into how the users’ or
the developers’ experiences and training affect this relationship. In our research we are hoping to
focus on the developer’s contribution to the user-developer relationship in ISD. By contributions
we mean when, how and why they seek user interaction during the ISD process. In this respect,
we are hoping to bring to light a little-studied aspect of developers’ contribution – how their
formal training, or the lack thereof, affects their perception and commitment to involving the
user in the development of an information system.
1.1 Relevance to IST
The research explores how technical knowledge and training affects developers’ relationship
with users. We hope to capture that information through exploring academic background,
experience, information system definitions, and user role beliefs. Our approach focuses on all
three constructs of the IST research philosophy – information, technology, and people. The ISD
process, though concerned with the developed of technological systems, is a social process
shaped by human actors. Developing an information system not only requires developers and
users to have knowledge about the domains of design and use of the system, but it also requires
effective information exchange between them. Our research attempts to shed light on the
2

3. dynamics of the developer’s role in user/developer interactions and points of influence for
improving those interactions. These findings will also highlight areas for future educational
opportunities and research collaborations between social and technical research within the realm
of Information Sciences and Technology.
The individual research interests of research team members cover a broad spectrum of issues
related to information systems development. Other than interest in user involvement in ISD, we
are interested in studying implementation of large-scale information systems, ontologies used to
capture users knowledge in ISD, and re-skilling of IT professionals into the information sector.
1.2 Roadmap
The following research proposal will clarify the contributions of the developer-user interactions
within the area of Information System Development. Section two will highlight the current
literature in the area of developer-user interaction. Section three will clarify the research
questions pursued, identify the methodological approach taken, reasoning behind the methods
employed, data collection implemented, issues that arose during the research process and roles of
the researchers. Section four examines the data gathered though coding, analysis of that data and
the findings. Section five summarizes the findings, implications and contributions the research
adds to the developer-user research area. Section six summarizes the research in its entirety.
2 LITERATURE REVIEW
User participation and involvement in the information systems development process has been a
long running theme in information systems research. User participation refers to the activities
performed by the user during systems development while user involvement is the importance and
personal relevance of the system to the user (Barki & Hartwick, 1989), (Barki & Hartwick,
1994), (Hartwick & Barki, 1994). Grudin (1991) argued the importance of users’ participation in
3

4. design, and suggested a classification for the term user (i.e. end-user, customer, client, etc.).
Even though users’ involvement and participation in the ISD process has been studied for over
four decades, it continues to present challenges for future research. Much attention has been paid
to issues related to users’ participation and involvement in the process of developing systems.
Several researchers (Bailey & Pearson, 1983), (Doll & Torkzadeh, 1989), (Hwang & Thorn,
1999), (Robey & Farrow, 1982), (Tait & Vessey, 1988) have concluded that user participation
significantly increases system success and end-user satisfaction. Other researchers, (Ives, 1984),
(Barki & Hartwick, 1994), (Butler & Fitzgerald, 1997), (Cavaye, 1995), have found that user
participation may not contribute much to improving the quality of the system being developed.
Most literature in this area refers to the relationship between users and developers as user
participation or user involvement. For the purposes of this research, we use the term interaction,
referring to a broader idea that covers both participation and involvement issues (Grudin, 1991),
(Rosove, 1969).
Empirical studies have also failed to reach consensus on whether user involvement ensures
success of the ISD process. Ives et. al. found no results supporting the correlation while
Pettingell et al. found a fair to moderate correlation (Emam, 1996). Wong’s case study analysis
of ETHICS versus a successful IS developed without structured theory found correlation
between users and success of ISD’s (Wong & Tate, 1994). Iivari’s Case Study analysis (2004)
suggested that users’ involvement could be influenced by the organization’s culture.
User participation should focus on users’ knowledge to achieve more effective design (He,
2004), (Wilson ET. Al., 1997). Users accumulate rich application domain knowledge due to long
period of exposure to their job context and “the knowledge perspective of user participation
captures the quantity (how much) or the frequencies (how often) of various knowledge activities
4

5. user participants perform during an ISD process” (He, 2004, p. 2). While He’s research addresses
the knowledge interaction between users and developers, our research will focus on the
developers’ perception of when the users’ knowledge is needed during the ISD phases and how
their formal training, or the lack thereof affects this perception.
Kautz & Pries-Heje (1999) and Nosek (2001) highlight that little research has focused on
education and training of systems development methodologies. Nosek suggests “education
involves understanding abstract theory while training involves the accretion of skills necessary to
perform a task” (p. 205). Roberts (2000) points out the importance of education for developers.
However, he states that some capable developers have achieved huge accomplishment with no
formal education. Roberts (2000) also suggests that “formal education in computing is neither a
necessary nor a sufficient predictor of productivity in the software development industry” (p.
85).
This issue with developers’ lack of training has been stressed by Rosove (1969), who
acknowledges the need for improving developers training, and updating computer sciences
programs. He argues that recent graduate students are weedy in software development techniques
because computer science programs emphasize operating systems, programming languages, and
some applications skills. Rosove also recommends that developers should be educated to work
and to interact with users, as well as to design systems for them. However, training has little
influence in adopting a system development methodology (Kautz & Pries-Heje, 1999). A recent
study (Latzina, 2002) has shown that developers’ training in usability still poses an important
issue on how developers involve users in ISD.
2.1 Preconceptions
The preconceptions applied to this research are based on the authors’ professional and academic
5

6. experiences. In terms of academic experience, two researchers have a Computer Science
background, and one researcher has a Management Information Systems background. Thus, we
have all been educated and trained in software engineering, and have been exposed to the ideas
and concepts related to ISD as part of our formal education. These preconceptions guided our
research approach with respect to defining the research questions, sampling selection,
elaborating the interview questions, performing the interviews, and analyzing the data. Based on
our own academic backgrounds and the common ground we shared, we initially started with the
assumption that developers who had courses in software engineering or ISD methodologies
qualified as “formally trained”. Using this criterion, we decided to consider undergraduate
degrees in computer science (CS) and information systems as “formal” training for ISD.
Developers with undergraduate degrees in all other fields were considered “informally” trained
or self-taught. The term “formal training” as used by us relates to formal education as defined by
Roberts (2000) to mean any completed undergraduate degree.
The interview questions, similarly, reflect our understanding of the IS development phases, and
the importance of users participation and involvement in systems development. We prescribe to
He’s (2004) view that the user has valuable contributions to make in the ISD process and hence
it is important for developers to involve the user in ISD.
6

7. 3 METHODOLOGY
3.1 Research Questions
Developer’s styles of user interaction differ based on how they acquire technological skills
related to ISD. Formally trained developers’ tend to have very structured views of user
interaction based on development methodologies defined in IS literature between 1967 to 1977
(Fitzgerald, 2000). Informally trained developers have possibly been formally trained in an
entirely different subject area and then migrated to an ISD role. Thus, we were interested in
answering the following research question:
How does developers’ style of user interaction differ based on whether the developer is
formally or informally trained?
Two related sub-questions that help in answering the above research question are:
During which phases of the ISD processes do developers seek users’ knowledge?
During which phases of the ISD processes do developers feel that the users’ knowledge is
highly needed?
In order to answer our main research question, we chose to interview two different groups of
information systems developers, formally trained and informally trained developers as per our
assumptions above. As we conducted the research, certain issues and concepts emerged which
forced us to change our initial definitions of “formal” and “informal”. We discuss these issues
and the evolving nature of our understanding of the terms formal and informal later in Section
3.4.4.
We expected the outcome of this open-ended interview process would help us answer several
specific questions related to developers’ beliefs of user interaction in the ISD process. From the
7

8. first group of interviewees, we expected to learn if there were differences between what is taught
in school regarding the importance of user interaction and what is practiced by developers on a
daily basis regarding capturing and applying users’ knowledge. From the second group of
interviewees, we expected to learn how and why they became developers, where they received
their ISD training, and how they capture and apply users’ knowledge in an ISD process. In
addition, we also expected to learn what both groups considered as ISD phases, and how
important they considered user interaction in ISD.
3.2 Rationale and Epistemology
We analyzed both qualitative and quantitative research approaches to decide which methods
were better suited to answer our research questions and the overall nature of the phenomenon we
were interested in studying.
Using quantitative research methods would have required us to quantify the phenomenon we
were interested in studying by representing numerically the levels of our theoretical constructs
and concepts (Straub, Gefen, & Boudreau, 2004). Qualitative research methods, on the other
hand would enable us to examine phenomenon by understanding people and the social and
cultural contexts within which they exist though interviews, participant observation or document
analysis (Myers, 1997). Since there is no theory of how user involvement is viewed by formally
and informally trained developers, we could not perform theory-testing or quantify the constructs
that we thought were significant in this area. Thus, we decided to approach this research by
examining the social actors’ (systems developers, in this case) perceptions and ideas about the
significance of user interaction.
The most important factors that influence the choice of qualitative methods include the nature of
the research problem, the researcher’s theoretical lens, the degree of uncertainty surrounding the
8

9. phenomenon, the researcher’s skills, and academic politics (Trauth, 2001). To ensure that our
research was well suited for qualitative analysis we explored those factors.
In examining user interaction in information systems research, we did not have any a priori
hypothesis about how formally and non-formally trained developers differ in how they perceive
and practice user interaction. We were interested in examining and understanding a socially
based phenomenon. Thus, the ontological perspective we adopted was concerned with social
relations, processes and practices and was well matched to a qualitative research methodology
(Mason, 2002).
We could have chosen a positivist, interpretivist or a critical theoretical lens. The positivist
ontological assumption is that there is an objective physical and social world that exists
independent of the actions of humans. Positivist studies assume the existence of “a priori fixed
relations within phenomenon which are typically investigated with structured instrumentation”
(Trauth, 2001; p. 6). In the domain of information systems research, such studies have attempted
to test theory, hypothesis, and formal propositions in order to draw inferences about phenomenon
from the sample to the stated population (Orlikowski & Baroudi, 1991). Interpretivism is based
on the ontological assumption that reality and our knowledge of it are socially constructed.
Interpretive studies assume that “people create and associate their own subjective and
intersubjective meanings as they interact with the world around them” (Trauth, 2001; p. 6). In IS
research, interpretive methods are aimed at understanding the context within which an
information system exists and how it influences and is influenced by this context (Walsham,
1993). Critical studies aim to critique the status quo of deep-seated structural contradictions
within social systems with an aim to transform alienating and restrictive social conditions
(Trauth, 2001). Critical studies in ISD consider information systems in their wider social context,
9

10. focusing on issues such as power, domination, conflict and contradiction (Howcroft & Trauth,
2004).
The users’ minor role in IS development has been due to an underlying assumption that there is
an objective truth in organizations that can be mapped to develop successful information systems
(R. M. Wong & Tiainen). This positivist perspective has ignored the view that information
systems development is a social process shaped by people, both users and developers. We
embraced this interpretivist perspective in order to understand the importance of user interaction
in the process of ISD. The investigation of social phenomena should be interpretivist in its
orientation so that contextualized meaning can be revealed to enable socially-based phenomenon
to be understood (Butler & Fitzgerald, 1997 ). The interactions between users and developers are
socially-based phenomenon. We were interested in understanding what “user involvement and
participation” means for systems developers and how their understanding of the term differs
depending on their level of formal training, therefore we used an interpretive lens.
In this case, there was a high degree of uncertainty surrounding the research problem as very
little research explored the relationship between developers’ formal education and their
perception of the importance of user interaction. Past research in this area has stressed how user
interaction results in increased levels of user satisfaction and perceived usefulness of the system
(Terry & Standing, 2004). However, research regarding the relationship between user interaction
and success of information systems development is not grounded in theory nor substantiated by
research data (Butler & Fitzgerald, 1997 ).
The last factor pertinent to this research is the researchers themselves. Researchers familiar with
qualitative research methods are more likely to use them. Members of the research team had past
experience with qualitative interviewing, focus groups, and content analysis and felt comfortable
10

11. to use qualitative approaches for this study.
3.3 Research Design
A methodological strategy is the logic by which one goes about answering research questions
(Mason, 2002). Having decided on qualitative methods for investigating the phenomenon of
interest, we attempted to link our research questions and rationale to research methods and
practicalities. We adapted the table from (Mason, 2002) and linked her criteria with the research
rationale described above to help us decide on a research method.
How does developers’ Interviews of developers Attempt to discover Need two groups
style of user difference of opinions of ISD
interaction differ based Other Possibilities: between developers developers:
on whether the observation, interviews based on their formally and
developer is formally with users acquisition of informally
or informally trained? technology skills. trained.
During which phases Interviews of developers Attempt to elicit Need access to
of the ISD processes personal standards of diverse pool of
do developers seek Other Possibilities: user interactions from ISD developers
users’ knowledge? observation, document developers.
analysis
During which phases Interviews of developers Attempt to elicit Need ISD
of the ISD processes when developers developers who
do developers feel that Other Possibilities: believe that users’ have worked on
the users’ knowledge observation, document knowledge is needed. ISD portions with
is highly needed? analysis users involved.
Table 1: Developing a research design (Mason, 2002)
Our ontological position suggested that developer’s knowledge, formal training, views, and
experiences shape their perception of how important user involvement is to the process of ISD.
Our interpretivist epistemology lead us to believe that via interviews we could construct the
social reality of the phenomenon under study by getting first-hand accounts of how developers
feel their training in formal or informal fields affects how they treat users in the process of
developing systems. Talking to developers and observing their attitude towards user involvement
11

12. enabled us, as researchers, to be active and reflexive in the research process (Mason, 2002).
The inclusion criterion for subjects in this study was that they should have been involved in the
development of at least one information system, either during the course of their education (say
as a graduate student research project) or in the course of working in industry. An information
system, in general, consists of software, hardware, databases, servers, online services and
integration tools. While this definition of information system definitely qualifies for the purposes
of our research, we would also consider a software system or tool as an information system. An
information system development life cycle for this study is composed of planning, analysis,
design, implementation, and testing phases (Dennis et al., 2005), (Maciaszek, 2001).
We were interested in interviewing a total of twelve subjects, six of whom were formally trained
in ISD practices and techniques and another six informally trained. “Formal” training meant
undergraduate degree in Computer Science (CS). Subjects with undergraduate degrees in all
other fields were considered “informally” trained. These developers were self-taught in that they
never had courses in software engineering or development techniques but had picked up ISD
skills through books or online resources. Due to ease of accessibility, the subject included four
current graduate students at the College of Information Science and Technology, three Penn
State University Libraries staff and one Penn State Information Technology Services staff.
Using a semi-structured interview method, we asked subjects open-ended questions related to the
above mentioned themes. Talking to developers about how their training, formal or informal, has
shaped their practice of user involvement in the ISD helped us to explore these issues in their
social context.
12

13. 3.4 Data Collection
This research followed Mason’s suggestions for planning and conducting qualitative
interviewing (Mason, 2002). She suggests we employ qualitative interviewing when we “want to
give maximum opportunity for the construction of contextual knowledge” (Mason, 2002, p. 64)
based on the participants’ experience. In this research, we were constructing knowledge about
participants’ perception of user interaction by examining how developers’ training has
influenced their ISD experience. An example of “constructing knowledge” is that instead of
asking the interviewees to sort a given list of ISD phases, we would rather expect them to
provide a list of ISD phases.
3.4.1 Subjects
We designed the study to include twelve participants, six of whom were formally trained in ISD
and the other six are informally trained. They were contacted via email (see email script in
Appendix A), and asked to participate in semi-structured interview lasting no more than forty-
five minutes. All participants contacted to participate in our study were Pennsylvania State
University staff or graduate students, and colleagues of the authors. In addition, we sought
people with technical skills and at least a year of professional experience in ISD.
From the initial twelve emails sent out inviting people to participate in our study, ten replied
confirming their interest in the study, one decided not to participate, and one did not respond the
email. From these ten confirmed participants, we were able to schedule only eight interviews
because of some time constraints. Fortunately, based on the eight interviewees’ academic
background and professional experience, we were able to form even groups of interest. Taking
into consideration the academic background, we had four participants formally trained and four
participants informally trained. Considering professional experience, all participants had
13

14. experience with industry while four participants moved from industry into academia.
Apart from the specific requirements above, this research considered that any other variable,
such as age, gender, race, and nationality, would not be important for the scope of the research.
Nevertheless, we had a well-balanced number of participants with different characteristics (men/
women, young/old, and different nationalities).
3.4.2 Interview Questions
The interviews we designed for this research (see Appendix B) consisted of open-ended
questions related to three main themes – participants’ education and skills, participants’
perception of what the ISD process is, and participants’ perception of user interaction in ISD.
The first theme, regarding educational and skills, helped us to learn and identify the
interviewee’s background in systems development. Although we had a broad idea about what a
‘formally’ trained developer would be, we relied on our findings to identify what kind of formal
education actually influences the way developers perceive the user’s interaction in ISD. The
development process theme covered information about the interviewee’s knowledge about ISD
phases and sub-phases, as well as their relations with user’s participation and involvement in
ISD. We expected that people with different educational backgrounds would have different
opinions about the ISD phases. Lastly, the user participation and involvement theme would
indicate how important the interviewee feels user interaction is within the ISD phases. For this
theme, we were expecting to learn about the interviewee’s experience involving the user in
different ISD phases. Although, it is widely known that users’ participation in ISD should be
carried throughout all the ISD phases, we suspected that it would not happen in practice.
Each interview question aims to understand and to cover a specific category within the themes.
These categories are the initial source for our coding schema (see Appendix C).
14

15. 3.4.3 Interview Process
Each interview lasted approximately forty-five minutes, and was performed by at least two
interviewers. One interviewer was responsible for conducting the interview and asking the open-
ended questions. The other interviewer was responsible for taking notes, although, the interviews
were also audio recorded. The second interviewer could also contribute with questions,
comments, or clarifications.
After each interview, the interviewers got together to discuss possible issues or suggest
corrections for the next interview. Sub-section 3.4.4 presents a summary of the main issues
found during the interview process.
The environment, where the interviews took place, performed an important role for the
accomplishment of the interviews. The collaborative space room was considered by the group as
the best place to conduct interviews. It provided us with a quite place, and an excellent layout
for talking to the interviewees. The interviewees’ office was relatively quite, except for some
interruption of their officemates. Lastly, the interviewers’ laboratory was the worst place to
interview people because it provided a significant level of distraction to the interviewee. The two
laboratories used to do the interviews are shared with other graduate students, and consequently
caused too much noise.
3.4.4 Issues
The first interviews helped us to identify issues such as missing questions and topics that are
relevant to the information we are seeking in this research. Instead of asking participants about
their proficiency in computer languages and development methods/tools, it was more useful to
ask them about their experience with the same. Based on the first interviews answers, we noticed
that the term “proficiency” implied the participant’s current ability with a technology. On the
15

16. other hand, the term “experience” implied all abilities (present and past) with a technology. By
switching the terms, we were able to generate more data about the participant’s experience. The
sequence in which the interviewer asked questions was also changed to allow the conversation to
flow smoothly, as well as to help the interviewees to better remember their experiences. For
instance, we noticed that first asking the interviewees to list their experience with programming
languages and development methodologies would help them to trigger their memory to
remember how they learnt their skills. Asking the other way around was making the interviewees
to spend extra time retrieving their memory in order to provide an accurate answer.
During the interview process, we also came across some issues resulting from the authors’ lack
of experience with conducting interviews. First, because some interviewees provided vague
answers to the questions asked, we noticed a tendency for interviewers to lead the participants’
answers. A possible explanation for that is because the questions were not well formulated and
led to multiple interpretations, or the interviewee was not elaborating enough and the answers
were not producing the adequate information. Second, some interviewees had the tendency to
speak a lot, and consequently dominated the interview. In these cases, we witnessed the tendency
to add extra information to illustrate points, usually extending the conversation, and introducing
many facts (relevant and irrelevant). Guided by our curiosity and the expectation of some useful
data, we usually let the interviewee talk until we felt that we were not getting anymore relevant
information. It was a challenge for the interviewers to interrupt the conversation and get it back
on track. Third, some interviewees suggested us to hand out the questions in advance, so they
would be better prepared to answer them. Having the questions in advance would help the
interviewees to go through their past professional and academic experiences, and to remember
relevant facts to answer the interview questions. Prior knowledge of the questions could also
16

17. sway the position of the interviewee in the realm of user interaction and its importance. Finally,
some interviewees got easily distracted, and started to give us irrelevant information. For
instance, an interviewee became interested for one of the author’s MAC laptop, and started to
ask about its features.
Another significant issue was defining where to draw the line between ‘formally’ and
‘informally’ trained developers. Our initial idea was to define people with degrees in computer
science and information systems as formally trained and all others as informally trained. This
presented a philosophical difficulty because although CS and IS programs may have some
overlap in terms of the courses taught, the fields themselves have different underlying
philosophies with respect to the importance of the user in the development process. This could
mean that ISD training imparted in these fields is very different and hence we need to redefine
our notion of ‘formally’ trained developer. Therefore the number and the type of such courses
typically taken by participants’ from CS and IS helped us to define ‘formal’ training for the
purposes of this research.
3.5 Role of Researchers
All members of the research group took an active interest in all aspects of this study and worked
well as a team. Through regular meetings members of the team discussed the research and the
findings as they emerged, as well as contributed equally to writing reports, recruiting
participants, and conducting interviews. Each researcher recruited four participants, conducted at
least two interviews, took notes of at least two interviews, and coded four interviews. The
approaches provided us with valuable opportunities to learn and perform different tasks in a
qualitative research process. More information about the role of researchers can be found in sub-
sections Preconceptions, Interview Process, and Data Analysis.
17

18. 4 RESULTS
4.1 Data Analysis
Data analysis began during the first interview and continued to refine itself throughout the entire
data collection process. Since qualitative methods were employed, much of the analysis
included the researchers internally processing the information and then elaborating their thoughts
during meetings and across email. The analysis process included physical processing of data,
data categorizing and coding, and analysis.
4.1.1 Processing of Data
Physical processing of data included two main tasks: reviewing and transcribing interview audio
and integrating additional researcher interview notes. During interviews one researcher was
appointed interviewer and responsible for ensuring adequate coverage of all the main themes of
our interview guide (see Appendix B). That researcher was also responsible for the interview
transcription. This ensured that the interviewer gained a full working knowledge of the interview
since they were typically slightly distracted by formulating questions and exploring additional
opportunities the interviewee may have mentioned.
The other researchers present for the interviews were responsible for note taking and then
integrating those notes into the appropriate location within the transcripts. This was useful for the
coding and analysis because it highlighted when and where areas of researcher enlightenment
became apparent within the process. The revisiting of interviews and notes was an important
factor in developing the researchers’ intimacy with the data.
4.1.2 Categorizing and Coding
We were interested in understanding what “user involvement and participation” means for
systems developers and how their understanding of the term differs depending on their level of
18

19. formal training, therefore we were using an interpretive lens to understand the data. Mason
cautions that the main issue with using an interpretive lens is representing research participants’
views rather than imparting your own views onto the data they provide (Mason 2002, p.76). In
order to counteract that tendency, we individually formulated our thoughts on emerging concepts
and then held group meetings to check our approaches.
During the research group meetings specific categories of interest became evident. Some of the
categories were driven by literal data such as work experience, years, type of training while
others were more interpretive in nature; user importance. Eleven high-level categories emerged
from the group meetings and were utilized to create data codes. Those categories are described in
detail in Appendix C. The allocations of interview participants in the identified categories are
presented in Table 2.
19

20. Category Codes Participant Results
Academic The participants included four formally trained with Computer Science
Background degrees (two recent graduates, two experienced graduates) and four
informally trained with degrees in Business, Engineering, Geology and Hotel
Restaurant & Management.
Acquired Skills All participants cited self-teaching as main approach to skill acquisition.
Half of the formally and informally participants gained skills through
coursework. One formally trained participant utilized all three venues for
skill acquisition.
Languages Only two of the formally trained participants had both types of experience
while two of the formally and one informally trained participant had low-
level programming experience and three informally trained participants had
script/web experience.
Information Unfortunately it was not until the researchers were half way through the
System interview participants that it emerged as potentially important. For the
Definition participants captured (two formally and three informally) only one of the
formally could defined an IS and all of the informally had working
definitions.
Information The participants typically cited a waterfall or cyclical approach with only one
System Phases formally trained participant unable to identify any phases to the ISD process.
Career Path Four of the five graduate experience developers also had experience in the
software industry while two of the four information technology experienced
developers started their career in industry other than technology.
Knowledge Three of the four direct type developers were also informally trained while
Elicitation three developers were indirect and one developer had no interest in
knowledge elicitation from users.
Past Participation Two of the four formally trained and two of the four informally trained
in ISD developers participated in all phases they identified.
Past User In one extreme case, a developer admitted, without a bit of remorse, that
Interaction when lacking appropriate details about requirements they would simply role
play scenarios of the system in their head and then use their best guess to fill
in the requirement gaps.
Participant’s A majority of the participants did not see a difference between ideal and
Ideal User current ISD, the few that would change the interaction were typically
Interaction involved in low-level programming and would only change if the project
necessitated user interaction.
User Importance The participants were fairly dispersed along the levels of user importance:
one None, three Low, one Medium, and three High level.
Table 2: Applying Code Schema to interviews
After we identified the eleven categories, two researchers applied the categories to the results at a
20

21. high level (Figure 1).
Figure 1: High-Level Interview Coding
Independently, the third researcher mapped the categories and codes to the transcripts and
integrated notes (Figure 2). The qualitative software package Atlas was employed to perform
content analysis of the interview transcripts. The low-level codes assigned during the content
analysis were later mapped to the high-level codes developed by the other two researchers. This
approach ensured corroboration of results by two independent coding techniques and helped
decrease individual researcher biases that may have influenced data interpretation.
21

22. Figure 2: Atlas Coding
22

23. 4.1.3 Analysis
Through categorizing and coding of the interview data, three main factors emerged as promising
determinants of developer’s user interaction beliefs: Academic background (formal and informal
training), experiences and access to users. To analyze these factors we looked at the interpreted
user importance levels versus the individual factors with the assumption that user importance
levels are key to determining developer-user interaction during the ISD process.
Examining formal and informal training versus our interpreted user importance illustrates that
formal training (Computer Science) spread evenly across the importance levels while informally
trained users had a higher level of user importance.
High
Medium
Other
Low
Computer Science
None
0 1 2
# of Participants
Figure 3: Academic Background vs. User Importance
This comparison leads us to believe that academic experience plays a smaller role than we
originally assumed. This does not mean that academic experience plays no role in the developer-
user interaction. There is also notable increase in user importance with participants that received
an informal training in Information System Development. This may be due to their own
experiences prior to becoming a developer.
The second factor, developer’s experience, refers to the languages the developer has experienced
23

24. and the career path they took to get there. Two of the formally trained developers’ experience
consisted of low-level programming and graduate work while the other two had experience in
both low-level and script/web programming. All four of the informally trained developers had
strictly script/web programming experience.
High
Medium
Both
Low Script/Web
Low-Level
None
0 1 2
# of Participants
Figure 4: Experience vs. User Importance
Low-level developer’s tended to have a lower level of user importance while the others had a
high level of user importance. We believe this tendency of Low-level developer’s was a product
of their work environment. The Low-level developers focused on more system based
programming while the Script/Web developers focused on high level applications with a higher
level of usability for gathering and displaying information. While users utilize system based
programs they do not typically interact and extract information from those programs. System
based programs are more task-driven than information distribution driven.
24

25. The final factor, developer’s access to user, refers to the level of insulation between the
developer and the user during the ISD process. Throughout our interview and data collection
process it became apparent that user interaction beliefs hinged on the role of the developer within
the ISD process. If the developer obtained the user information indirectly (through a third party)
then the level of importance was typically lower. We believe this was due to the insulation
between the developer and user. Since the user is less prominent in the developer’s work, they
are not as visible and therefore not deemed as important.
High
Medium
Direct
Low
Indirect
None
0 1 2
# of Participants
Figure 5: User Knowledge Access vs. User Importance
4.2 Findings
Our main research question was how developers’ style of user interaction differs based on
whether the developer is formally or informally trained. We found that developers’ academic
background played a small role in how important they perceived the user to be. In shaping
developers’ perception of the importance of user involvement on-the-job experience and work
context were seen to play a more important role than their academic background and education.
Specifically, we found that developers with undergraduate degrees in computer science did not
25

26. perceive user involvement to be as important as did self-taught developers. This shows that
formal education in software development does not emphasize the importance of user
involvement in ISD. Self-taught developers perceived user involvement to be highly important
mainly because their organizational context and work experience had taught them the importance
of user involvement.
Developers’ perception of users and the importance of interacting with them during the
development of an information system were highly influenced by their work context. The
organization’s work culture, development philosophy, freedom and constraints placed on
developers with respect to resources and access to users were found to influence the way
developers actually involved the user to a much larger extent than what they may or may not
have learned through their formal education.
We also found that developers working with Web-based or scripting applications perceived the
user to be more important than programmers who write low-level code. Butler (2003) makes a
distinction between Web-based information systems and “traditional” information systems with
respect to the social and organizational issues associated with their development and
implementation. Along those lines, we found that developers who write Web-based or scripting
applications are more likely to perceive user involvement as important. It could be due to the fact
that Web-based applications are more interactive and user satisfaction is much higher when the
user is highly involved iteratively in the development process.
Finally, the access to the user or “level of insulation” between the developer and the user affects
developer-user interaction. The higher the “level of insulation” the developer has from the user
the less important user interaction is for the developer. By “level of insulation” we mean
organizational hierarchy which prevents the system developer from coming in direct contact with
26

27. the user for knowledge elicitation purposes. We found that low-level programmers were
“insulated” from the user and obtained user requirements and needs via other members of the
development team who actually interacted with the user. The organizational hierarchy prevented
them from directly talking to users. Therefore, when they perceived insufficient or incomplete
information from the user, they were unable to go back to the users and get more information. In
this case, one formally trained developer said he would “make assumptions about how the
common user is going to use” the system. Even then, being an expert, it was hard for him to
imagine how a user without technical-knowledge would use the system he was designing. This
example highlights that work context and organizational constraints influence how far
developers involve the user much more than formal training does.
5 DISCUSSION
In summary, developer’s training, whether it be formal or informal, plays only a small role in
determining developer’s beliefs in user interaction within the ISD process. Work experience and
context play a much larger role than academic experience because developers must conform to
their current working environment. Developer’s in script and web based ISDs tend to interact
with the user more which we believe increases their likelihood to deem the user as important to
the system development. Lastly, the level of access to the user also plays a factor in user
importance and therefore user interaction seeking because of the nature of low-level
programming.
5.1 Implications
The findings of our study have important implications for education and industry. They make us
question the role of education in making systems developers aware of the importance of user
involvement. Our research illustrates that developers with undergraduate degrees in computer
27

28. science did not receive any formal courses that stressed the importance of involving users. These
formally trained developers were typically recruited in coding jobs. Among them, the low level
programmers typically did not learn the importance of the user because of their insulation from
them. This raises concerns about the inadequacy of traditional computer science curricula to
address the need for teaching developers about the importance of user involvement. Computer
science curriculum studies are still slow in embracing user-centered development methods
(Kaasinen & Clarke 1998). User-centered design techniques should be taught as part of
traditional CS courses as they stress on involving users throughout the development phases.
According to Rosson & Carrol (2001), usability has been studied for the last three decades, and
become a focal point in software development, especially in requirements analysis and interfaces
design. Seffah & Andreevskaia (2003) found that most of systems analysts have little or no
training in usability techniques and suggested that developers should be made aware of user-
centered techniques.
Also, our study raises concerns that organizations hiring computer science graduates are hiring
systems developers who do not necessarily perceive users to be important to the process of ISD.
Consequently, much of the usability training and education for developers is left to corporate
training organizations (Kaasinen & Clarke 1998). This may lead to products being developed
which have low usability and lead to low level of user satisfaction unless the organization has a
culture of promoting user interaction in ISD. Thus, if organizations are interested in hiring
developers who perceive users as important, they will have to either hire developers who have in
their past job-experience treated users as important or train them to consider the same.
5.2 Contributions
Past research into importance of user involvement in ISD has looked mostly at end-user training
28

29. as important for the success of information systems. However, very little research has looked at
how developers’ formal educational background and training influence their perception of user
interaction. This research highlights potential factors for future research within the realm of
developer examination. It also illustrates that current educational efforts are not determining the
ways developers choose to interact with users.
5.3 Limitations
Due to time and resource constraints, our study has several areas for improvement. Even though
we were interested in studying the differences between formally and informally trained
developers, we struggled with the issue of defining “formally trained” developers throughout our
research. We started out considering that developers who had courses in software engineering or
ISD methodologies qualified as “formally trained”. Using this criterion, we decided to consider
undergraduate degrees in computer science and information systems as “formal” training.
However, the underlying educational philosophies of computer science and information systems
fields with respect to the perception of the user differ considerably. Also, as we spoke to
computer science graduates, we realized that most of them had not had any courses in software
development methodologies or software engineering. Thus we changed our initial definition of
formally trained to include only computer science graduates as they had formally learnt ISD
skills as opposed to all other degree-holders who had learnt ISD skills on their own.
Also, our study used a small sample of eight developers and the sample could be considered to
be biased as most of the developers we talked to belonged to the same organization. The
graduate students were mostly from the College of Information Sciences and Technology and the
developers in industry were Penn State University Libraries or Information Technology Services
staff. Since we found that work context plays a large role in perception of the user, using a more
29

30. diverse sample of developers would have lent more credibility to our research.
5.4 Future Research
Our research highlights a gap in IS literature with respect to the developer’s role in user-
developer interactions during the ISD process. In the future this research can be extended by
using a larger sample size and talking to developers from diverse organizations. The distinction
between Web-based and traditional developers with respect to their difference in perception of
the user needs to be studied in more detail. Some unexpected issues emerged from our research
other than the ones we were interested in addressing. We found that the age of developers
affected their perception of user interaction. Older formally trained developers said that their
education was so long ago that it had ceased to have much of an influence on their career today.
Also, computer science degrees in the 70s and 80s did not include software methodology
courses. Thus, older developers tend to be more influenced by their on-the-job experience than
younger developers whose education is still fresh in their memory to be applied in their work.
6 CONCLUSION
Our research preliminarily highlights a gap in the area of developer-user interactions. Although
our sample group was small, we believe it was a significant source of evidence towards the
determining factors in developer’s user interaction beliefs. The three main factors that emerged
in this research were developers’ academic background, experience and level of insulation. With
further exploration of these factors, we believe that valuable insights can be revealed to aid with
future educational, employment, and research efforts.
7 REFERENCES
Bailey, J. E., & Pearson, S. W. (1983). Development of a tool for measuring and analyzing
computer user satisfaction. Management Science, 29(5), 530-545.
30

31. Barki, H., & Hartwick, J. (1989). Rethinking the Concept of user Involvement. MIS Quarterly,
13(1), 53-63
Barki, H., & Hartwick, J. (1994). Measuring User Participation, User Involvement, and User
Attitude. MIS Quarterly, 18(1), 59-82.
Butler, T., & Fitzgerald, B. (1997 ). A case study of user participation in the information systems
development process. Proceedings of the eighteenth international conference on
Information systems, Atlanta, Georgia, United States
Cavaye, A. L. M. (1995). User Participation in Systems Development Revisited. Information and
Management, 28, 311-323.
Doll, W. J., & Torkzadeh, G. (1989). A Discrepancy Model of End-User Computing Involvment.
Management Sciences, 35(10), 1151-1171.
Emam, K., Quintin, S., Madhavji N. (1996). User participation in the requirements engineering
process: An empirical study. Requirements Engineering, 1(1), 4.
Fitzgerald, B. (2000). Systems Development Methodologies: the Problem of Tenses. Information
Technology & People, 13(3), 174 -185.
Grudin, J. (1991). Interactive Systems: Bridging the gaps between Developers and Users. IEEE
Computer, 24(4), 59-69.
Hartwick, J., & Barki, H. (1994). Explaning the role of user participation in Information System
Use. Management Science, 40(4), 440-465.
He, J. (2004). Knowledge Impacts of User Participation: A Cognitive Perspective. Paper
presented at the SIGMIS, Tucson, AZ, USA.
Howcroft, D. and E. M. Trauth (2004). The Choice of Critical Information Systems Research.
Relevant Theory and Informed Practice - Looking Forward from a 20 Year Perspective
on IS Research. Boston, MA, USA, Kluwer Academic Publishers
Hwang, M. I., & Thorn, R. G. (1999). The Effect of User Engagement on System Success: A
Meta-analytical Integration of Research Findings. Information and Management, 35,
229-236.
Iivari, N. (2004). Enculturation of User Involvement in Software Development Organizations:
An interpretative Case Study in the Product Development Context. ACM NordCHI,
Finland, 287-296.
Ives, B., Olson, M. (1984). User Involvement and MIS Success: A Review of Research.
Management Science, 30(5), 586-603.
Kautz, K., & Pries-Heje, J. (1999). Systems Development Education and Methodology
Adoption. ACM SIGCPR Computer Personnel, 20(3), 6-26.
Latzina, M., & Rummel, B. (2002). Collaboration-Based Usability training for Developers.
Mensch & Computer 2002: Vom interaktiven Werkzeug zu kooperativen Arbeits- und
Lernwelten. Stuttgart: B. G. Teubner, S. 285-291.
Mason, J. (2002). Qualitative Researching (2nd Edition ed.). London, UK: Sage Publications.
Myers, M. D. (1997). "Qualitative Research in Information Systems." MIS Quarterly 21(2):
31

32. 241-242.
Nosek, J. T. (2001). Justifying the Business Value of Information Systems Education: A Report
on Multicultural Field Experiments. SIGCPR, San Diego, 205-211.
Orlikowski, W. J. and J. J. Baroudi (1991). "Studying information technology in organizations:
Research Approaches and Assumptions." Information Systems Research 2(1): 1-28.
Roberts, E. (2000). Computing Education and the Information Technology Workforce. SIGCSE
Bulletin, 32(2), 83-90.
Robey, D., & Farrow, D. (1982). User Involvement in Information Systems Development: A
Conflict Model and Empirical Test. Management Science, 28(1), 73-85.
Rosove, P. E. (1969). Training Requirements for computer programmers in relation to system
development phases. SIGCPR, Illinois, 65-76.
Rosson, M. B., & Carroll, J. M. (2001). Usability Engineering: Scenario-Based Development of
Human-Computer Interaction. Morgan Kaufmann.
Seffah, A., & Andreevskaia, A. (2003). Empowering Software Engineers in Human-Centered
Design. IEEE International Conference in Software Engineering, 653-658.
Straub, D., D. Gefen, et al. (2004, January 7, 2005). "The ISWorld Quantitative, Positivist
Research Methods Website." Retrieved February 5, 2006, from
http://www.dstraub.cis.qsu.edu:88/quant/.
Tait, P., & Vessey, I. (1988). The Effect of User Involvement on System Success: A
Contingency Approach. MIS Quarterly, 91-108.
Terry, J. and C. Standing (2004). "The Value of User Participation in E-Commerce Systems
Development." Informing Science Journal 7
Trauth, E. M. (2001). Qualitative Research in IS: Issues and Trends. Hershey, PA, USA, Idea
Group Publishing.
Walsham, G. (1993). Interpreting Information Systems in Organizations. New York, NY, USA,
John Wiley & Sons, Inc.
Wilson, S., Bekker, M., Johnson, P., & Johnson, H. (1997). Helping and Hindering User
Involvement – A Tale of Everyday Design. ACM-CHI’97, Atlanta, 178-185.
Wong, E., & Tate, G. (1994). A Study of User Participation in Information Systems
Development. Journal of Information Technology 9(1), 51.
32

33. 8 APPENDIX A
Email script for recruiting participants
Dear <Participant>,
We would like to request your voluntary participation in our study “Effects of
Developers’ Training on User-Developer Interactions in Information Systems Development”
(IRB# 22846). We believe that you are currently, or have in the past, worked on designing,
developing, implementing or maintaining an information system. We would like to interview
you about your experiences during this process, specifically about your involvement of the user.
The questions in the interview will be related to the following main themes: your
educational background, how you practice information systems development, where you learnt
the skills you use, and when and how you involve the user. The interview will be audio-recorded
with your permission.
The interview will take place at a time and location convenient to you and will not exceed
45 minutes.
This research may shed light on how you, as a developer, perceive user involvement as
an important part of the information systems development process. Also, this could lead you to a
better understanding of how your formal training in information systems development, or the
lack thereof, leads you to involve the user.
If you are interested in participating in this research, please reply to this email with
possible times when you would be available to participate in the interview. If you have further
questions about the research, please contact Andrey Soares at asoares@ist.psu.edu.
This research is associated with the Spring Semester IST541: Qualitative Research in IST
at The Pennsylvania State University.
Thank you for your consideration,
Jennifer McCauley, Sharoda Paul, and Andrey Soares
Sharoda A. Paul, IST PhD Graduate Student
323 Information Sciences and Technology Building
University Park, PA 16802
(814) 865-6846
spaul@ist.psu.edu
Jennifer McCauley, IST Master Graduate Student
102 Paterno Library
University Park, PA 16802
(814) 863-7098
jmccauley@ist.psu.edu
Andrey Soares, IST PhD Graduate Student
309 Information Sciences and Technology Building
University Park, PA 16802
(814) 865-6177
asoares@ist.psu.edu
33

34. 9 APPENDIX B
Interview Questions Guideline
The following interview questions will cover three categories that we believe can help us
to identify and to reveal how developers: learn their skills, understand systems development
process, and perceive the user participation in ISD phases.
Education and Skills
• What is your academic background (i.e. Computer Science, Geography, etc)?
• How did you learn the skills to become a system developer (i.e. Associate Degree,
Bachelor Degree, Master Degree, PhD Degree, Technical/Professional Courses,
or Self)?
• Which programming languages and methodologies you have experience with (i.e.
Java, C, Perl, Visual Basic, O-O, E-R, etc)?
• What is Information Systems?
• How long have you worked developing systems for?
• List all job positions related to Information Systems Development that you have
already performed (i.e. Programmer, systems analyst, Project Manager, etc)?
Please indicate how long you spent working in each position.
Development process
• What are the phases and sub-phases of an Information Systems Development life
cycle (i.e. analysis, implementation, etc)?
• How do you learn about new suggestions and new implementations?
• Who is responsible for designing interfaces and reports? Do you ask for user’s
approval?
• How much time developers usually spent testing the system?
User Participation
• What is the importance of users’ involvement in IS development? Explain.
• Which activities of an ISD do users get involved in (i.e. requirements, etc)?
• How important user suggestions are to an ISD process (i.e. very important,
reasonable, and unimportant)? Which ISD phases?
• Are users usually involved in quality assurance?
• Have you worked with users for capturing their knowledge about the systems?
o If yes:
 How did you collect the user’s knowledge? Which ISD phases?
 How did you save their knowledge?
o If no:
 Who was the information source?
 How is the information captured?
 How was the information passed to you? In what format?
 Was the information given to you enough to understand the
system?
 Would you be able to explain how the system works?
34

35. 10 APPENDIX C
Coding Schema
− Academic background - Undergraduate or graduate work within formal or informal
areas relevant to ISD.
o CS
o IS
o Business
o Engineering
o Other
− Acquired Skills - Acquisition of skills used to build ISD’s were acquired: coursework,
self-teaching, or training (sponsored by industry employer).
o Courses
o Self-training includes books, tutorials, web resources
o Required training
− Languages - Type of programming languages developer has utilized while participating
in ISD: Low-level programming (Drivers, kernel, operating system based) or
Scripting/Web based (Javascript, ColdFusion, etc) or both.
o Programming
o Scripting/web
o Both
− Definition of IS - This category was created to capture whether or not the participant
could define an information system as an entity.
o Can and what they are
o Can’t
− Career path - Capturing the path the developers took towards ISD experience.
o Information Technology (Primary & Secondary)
o Non-Technology
o Undergraduate and Graduate work
− IS phases – Meant to capture whether or not the developer could define specific phases
of development,
o Can and what they are
o Can’t
− Knowledge Elicitation - Within user interactions, vital information for ISD is elicited
through varying approaches, we categorized these into two types: direct and indirect.
The direct approaches involved the developer interacting with the user while the indirect
approach includes involvement of a third party to gather and then translate/communicate
requirements to the developer.
− Past Participation in ISD - This category sought to identify the level of experience the
developers had with ISD.
o Match to part of the definition of IS
− Past User Involvement - Designed to identify if participants actually practice what they
“preach”.
− Participant’s Ideal User Interaction - Politics motivate many areas of our lives and
35

36. development of ISD is no exception. This category was an opportunity for the
participants to identify their ideal user interaction level within ISD without regard to their
political situation.
− User suggestion importance - The most interpretive category in our research, user
importance was determined based on the researcher’s impression of the participant and
not the participant’s declaration of user importance.
o none (“I am the user”)
o low (“I know the user exists, but are not necessary”)
o medium (“When possible it is a bonus to involve the user”)
o high (“I start with the user and end with the user”).
36