Research Design 1Research Design, Practice, Logistics and Ethics Edgardo Donovan RES 603 – Dr. Alan B. Flaschner Module 1 – Case Analysis Monday, October 18, 2010
Research Design 2 Research Design, Practice, Logistics and Ethics Eveland and Bikson utilize an experimental research design in conducting their studytitles “Work Group Structures and Computer Support: A Field Experiment” as they attempted toscientifically measure the propensity that people provided with technological support wouldgravitate towards the use of the latter in a field environment measured over a long period of time. It is frequently suggested that work groups that have computer technology to supportactivities such as text editing, data manipulation, and communication develop systematicallydifferent structures and working processes from groups that rely on more conventionaltechnologies such as memos, phone calls, and meetings (Eveland 354). This field experiment created two task forces, each composed equally of recently retiredemployees and employees still at work but eligible to retire. They were given the identical tasksof preparing reports for their company on retirement planning issues, but they were randomlyassigned to different technology conditions. Interviews were conducted four times during theyear-long project; in addition, electronic mail activity was logged in the on-line group (Eveland354). Figure 1. (Research Design)
Research Design 3 Eveland’s research can be defined as utilizing a quantitative, field, and experimentaldesign. We can classify designs into a simple threefold classification by asking some keyquestions. First, does the design use random assignment to groups? If random assignment isused, we call the design a randomized experiment or true experiment. If random assignment isnot used, then we have to ask a second question: Does the design use either multiple groups ormultiple waves of measurement? If the answer is yes, we would label it a quasi-experimentaldesign (Trochim 2). The main strength of the research design comes from random assignment toexperimental conditions plus control over other potentially interfering variables such as type oftask, type of technology, prior experience with electronic mail, preexisting group structures, andthe like (Eveland 358). Figure 2. Questions about the Real World (McGrath 181) The research process can be viewed as a series of interlocking choices, in which we trysimultaneously to maximize several conflicting desiderata. There is not one true method that will
Research Design 4guarantee success (McGrath 179). A researcher begins by noticing a real world phenomenon andattempts to create new knowledge by inquiring about its dynamics. The research problem is thenincorporated into a specific design which then in turn gives birth to an operational plan. Thatplan may take on different approaches. In most cases the end-result from the execution of thatplan involves acquiring data that proves correlations or associations among a well defined set ofvariables that support a new way of understanding. Although there are many different ways tosequence research, typically the series of choices is locally directional: plan must come beforeexecution; data collection must come before data analysis (McGrath 180). Figure 3. The Research Process (Gabaney)
Research Design 5 Eveland probably chose the experimental design based on field gathered data probablybecause cross-sectional or retrospective research designs do not allow this hypothesis to be testedwith much power (Eveland 354). A qualitative research design based on the conventionalsubjective wisdom of the literary tradition related to the topic at hand would undoubtedly havethe potential of being very interesting for the field. However, it would have been very difficult toextrapolate universal truths related to how employees utilize technology when given anopportunity to do so. Another advantage of this type of design was that it was conducted in thefield and not in a lab setting. Further research perhaps in different geographical areas as well asorganizational cultures would be required to compound the validity of Eveland’s results.Regardless, Eveland credibly demonstrates that groups that are given the resources to utilize newtechnology over time gain an edge over similar groups not as well resourced while expendinggreater time and effort in learning new ways of doing things. This knowledge is very applicablein the private sector. Technological savvy acquired after technology tools are resourcedextensively over time will give a company an edge over its competition. However, this comes atnot only a financial cost but by also focusing employee effort away from productive endeavorstowards learning and mastering new ways of doing things. This in the short-term does not benefitan organization. This phenomenon would have been more difficult to prove in a lab wheremediating variables and control mechanisms would be required in an attempt to increase thepotential for validity. Any research project has limitations related to time, money, and connections which forcesits creators to become adept at the art of making trade-offs during the design process. Thesesevere constraints must be taken into consideration early if one wishes to avoid failure in theearly to mid stages of a project. Dr. Eveland was fortunate enough to participate in a research
Research Design 6project commissioned by RAND which typically is very experienced in resourcing large projectsfor the government and a variety of private sector organizations. Even though capital, time,personnel, and access to data constraints were probably less of a problem with RAND’s backingthey were potential showstoppers nonetheless. It was necessary to find an organization thatwould be willing to make the financial and time sacrifices necessary to commit its people toundergo a long and intensive research participation project. Dr. Eveland had to ensure that thesubjects agreed to take part and to continue their participation in randomly assigned groups.They were to be selected from a common “community”; that is, they should come from acommon culture, share some concerns, and have some reason to think they might want to workwith one another (Eveland 358). Dr. Eveland needed to select a sample group that conveyed enough validity but alsogeneralizability to the project. The randomly assigned sample size selected was small. Resourceconstraints limited them to only two task forces of about 40 members each. All the memberswere older men whose careers led to midlevel management or professional positions; we do notknow how the inclusion of younger employees, women members, or representatives of the top orbottom of the organizational hierarchy might have affected the results. Third, participation in thetask force was voluntary; results might not be the same for collaborative activities that are part ofregular job assignments (Eveland 358). It is necessary to strike the right balance when attempting to incentivize a group toparticipate in a study for conflicts of interest have a strong potential for rendering the results ofotherwise well designed research to be invalid. Typically, it is important for researchers to behonest about their research goals so as to not mislead and disgruntle participants later on whowill respond with a lack of seriousness. Sometimes financial incentives may succeed in swaying
Research Design 7people to participate but will create unnecessary conflicts of interest. The longer a researchproject is the more fragile it becomes for it is more difficult to ensure that participants participatewholeheartedly over a long period of time. Eveland and Bikson utilize an experimental research design in conducting their studytitles “Work Group Structures and Computer Support: A Field Experiment” as they attempted toscientifically measure the propensity that people provided with technological support wouldgravitate towards the use of the latter in a field environment measured over a long period of time.
Research Design 8 BibliographyEveland, JD & Bikson, TK (1988). Work group structures and computer support: a fieldexperiment. ACM (Association for Computing Machinery). 6 (4). 354-379.Gabaney, Steve (2007). Flow chart describing the research process. Indiana StateUniversity.McGrath, Joseph E & Brinberg, David (1983). External validity and the researchprocess: a comment on the calder/lynch dialogue. The Journal of Consumer Research.10(1). 115-124.McGrath, Joseph E. (1981) Dilemmatics: The study of research choices and dilemmas.American Behavioral Scientist. 25(2). 179-211.Research Design. PDF of powerpoint presentation. University of Western Ontario.Available athttp://www.ssc.uwo.ca/psychology/undergraduate/psych266a/lectureslides/Psych%20266%20Research%20Methods%20x3-%20web%20version.pdfSchlichter, J. & Brüggemann-Klein (1997). A CSCW (Computer Supported CooperativeWork). Technische Universität München, Germany.Trochim, William (2007). Introduction to research design. Cornell University.Sabherwal, R., Jeyaraj, A., & Chowa, C. (2006). Information system success: individualand organizational determinants. Management Science, 52(12), 1849.