Information Systems 1Information Systems Success Model: Difficulty in Quantifying User Frame of Reference Edgardo Donovan ITM 604 – Dr. Indira Guzman Module 1 – Case Analysis Monday, April 19, 2010
Information Systems 2 Information Systems Success Model: Difficulty in Quantifying User Frame of Reference An information systems success model is a theoretical tool that arguably has thepotential to measure the scope and effectiveness of an organization’s overall information systemsutilization and overall return on investment. The first and most important article written byDeLone and McLean in 1992 and then the revised model updated a decade later (2003) providedresearchers and practitioners with a way to quantify the effectiveness of information systems.This model centered around the variables of “systems quality” and “systems characteristics” andutilized information drawn from survey on the organization’s information systems consumersand practitioners. Although it is debatable whether this tool can be utilized by executivemanagement to effectively gauge the success of their information systems, it constitutes one ofthe most important pieces of work that attempts to define and measure information systemssuccess in a universal fashion. A large number of studies have been conducted to identify thosefactors that contribute to information systems success. However, the dependent variable of I/Ssuccess has been hard to define. This taxonomy posits six major dimensions of categories of I/Ssuccess: system quality, information quality, use, use satisfaction, individual impact, andorganizational impact (McLean 60). Although the wide popularity of the model is strong evidence of the need for acomprehensive framework in order to integrate IS research findings (McLean 10) there is littleevidence establishing that there is a broad consensus that DeLone and McLean have indeedsucceeded in filling such a demand. In every organization there is a demand for easy solutionsbut not a functional need for uniform glossed over methodologies leading to broadinterpretations. DeLone and McLean have provided a methodology to survey in an attempt tocull data that can then be used to classify practitioner and end-user opinions of information
Information Systems 3systems effectiveness. However, their framework does not differentiate between the wide varietyof frame of references within its survey base. For example, a technical support specialist atMicrosoft Corporation may have degrees in Computer Science along with 20 years experience asa software engineer, whereas the CIO at a small trucking company may be many years his or herjunior in terms of academic credentials and actual information systems management experience.McLean and DeLone do not offer a way to differentiate the two. The potential misleading resultby using their study is that a self-critical organizational culture at an organization like Microsoftwhen compared to an enthusiastic culture in a small trucking company may lead the latter toscore more effectively than the former. Unfortunately, until someone will be able to propose amodel that is able to accomplish the seemingly impossible task of quantifying the strategic andinformation systems related frame of reference of its survey base, theoretical models involvingassessing information systems effectiveness will continue to be potentially misleading. The role of information systems has changed and progressed during the last decade.Similarly, academic inquiry into the measurement of IS effectiveness has progressed over thesame period. McLean and DeLone reviewed more than 100 articles, including all the articles inInformation Systems Research, Journal of Management Information Systems, and MIS Quarterlysince 1993 in order to inform this review of IS success measurement. The purpose of theirrevised paper, therefore, was to update the D&M IS Success Model and evaluate its usefulness inlight of the dramatic changes in IS practice, especially the advent and explosive growth of e-commerce (McLean 10). The greater portion of their updated paper deals with confronting the various critics totheir model over the years. Many of their critics outlined what they thought were weaknesses inthe D&M IS Success model but were not able to propose what the authors believed to be
Information Systems 4stronger alternatives. None of the critics outlined in their updated paper took a position similar tomine which casts doubt on the viability of any universal information systems effectivenessassessment tool. DeLone and McLean did make some changes to their model and added a thirddimension, “service quality,” to the two original system characteristics, “systems quality” and“information quality.” Conversely, they considered it more parsimonious to combine“individual” and “organizational impacts” into a single variable, “net benefits” (McLean 22).Although expanding the frame of study into the arena involving how people perceive theinformation systems impact on service quality makes their study more relevant, sophisticated,and interesting, it does not provide a way to effectively quantify survey respondents informationsystems frame of reference. This only exacerbates the original problem by adding additionallayers of user opinion regarding service quality. Despite the confident pose of DeLone and McLean who continue to uphold the validityof their model, in their updated work they seem to echo my contention that they fail to quantify auser’s or organization’s frame of reference describing these as mere minor deficiencies that canbe improved upon in time: “The second issue of concern is: benefits for whom—the designer, the sponsor, the user, or others? Different actors, players, or stakeholders may have different opinions as to what constitutes a benefit to them . Thus, it is impossible to define these “net benefits” without first defining the context or frame of reference. The fact that the D&M Model does not define this context is a matter of detail, not of oversight. The focus of any proposed study must be defined. Our model may be useful to both Microsoft and the user community, but each may have a very different definition of what constitutes net benefits and thus IS success (McLean 22).”
Information Systems 5 The DeLone and McLean success model is useful and an overall successful piece oftheoretical work due to the finding that it produced involving system usage. It substantiates thatsystems use continues to be a popular success measure. However, Straub et al. studied 458 usersof a voice mail system and found that self-reported systems usage and computer-recorded usagewere not correlated. Their findings suggest that self-reported system usage and computer-recorded usage should both be measured in empirical studies because the two do not necessarilycorrelate with one another (McLean 20). The DeLone and McLean success model is also useful due to the finding that it producedinvolving the variable of systems quality. The study successfully correlates the latter with otherrelated variables involving the information systems apparatus such as: accuracy, reliability,human factors, content of the database information quality, output timeliness, reliability,completeness, relevance, precision, and accuracy. Not enough MIS field study research attempts to measure the influence of the MIS efforton organizational performance (McLean 81). It would be important to conduct such studiesbecause there is a huge organizational demand to better quantify the overall effectiveness of theirinformation systems investment. Worldwide expenditure on IT probably exceeds one trillion USdollars per year and growing at 10% annually. The success of such investments and the qualityof the systems developed is of the utmost importance both for research and in practice (Livari8). A large number of studies have been conducted to identify those factors that contribute toinformation systems success. However, the dependent variable of I/S success has been hard todefine (McLean 60). Most organizations build processes and systems around achieving a particular goal. Manyminds come and go and constantly revise the organizational agenda over time. Mostly in larger
Information Systems 6organizations, statistics are run to see if particular goals are being met or not. Consequently,policies are changed in an attempt to generate different numbers until the assessments fromfuture statistics are available to validate or invalidate po policy modifications. Information qualityand system quality are not the primary drivers but merely one of the many variables thatorganizational executives are forced to manage manage. Figure 1. Systems Development Cycle I believe that organizational managers have a better chance at success if they have ahistorical knowledge of all the strategic decisions that have contributed to current informationsystems apparatus. If tasked with developing an information systems success model I would model,structure it to take into account the totality of past organizational information systemsdevelopment cycles. The surveys I would devise would attempt to track organizational short short-term and long-term goals for each fiscal quarter over time. In essence, the methodology would term .ossify the process that takes place frequently in executive meetings assigning numbered success sifyscores to each objective. Ideally, the study would enable to managers to correlate the pursuedobjectives, whether successful or not over time with the organizations information systems not, ionsdevelopment. I believe that this would better solve the frame of reference issue given that the
Information Systems 7organizational executive staff surveyed would be those with the highest degree of vested interestin information systems success achieving organizational goals. Rather than being a universal tool achievingfor information systems success it could be relied upon as a universal tool for self self-assessinginformation systems success vis- vis helping to achieve organizational objectives. By tallying -à-visthe survey data compiled over many quarters the organization would be able to objectivelyvisually represent the presence or absence of a correlation between goal success and informationsystems expenditures over time. The weakness of a model like the one I would propose is that it wouldwould take many years of use to prove or disprove its worth to an organization and ultimatevalidity. Figure 2. Multiple Sequential Overlapping IT Systems Development Cycles Over a 10-30 Year Period A collaborative technology such as Web 2.0 would be interesting be extremely difficultto study due to infinite range of user interaction among multiple disparate systems andapplications. Designing a research model that would attempt to apply variable constraints on aphenomenon of that magnitude would be incredibly challenging. In the past perceived system erceived
Information Systems 8quality was also a significant predictor of system use. User satisfaction was found to be a strongpredictor of individual impact, whereas the influence of system use on individual impact wasinsignificant (Livari 8). However, as use of services provided by information systems becomesever more ubiquitous in our society, the reliability of end-user perceptions as valid opinions toevaluate the effectiveness of information systems models will decrease in importance. In the1970s, if you were a computer user there was a probably a 90% chance that you were either anengineer or an avid technology hobbyist with a solid understanding of how hardware andsoftware operated together. Today, 95% of computer users have little to no knowledge of thelatter. In similar fashion, the importance of the opinion of Web 2.0 users regarding technicalmatters will decrease as the user base expands and the transparent use of Web 2.0 informationsystems become interwoven within the fabric of our user daily experience.
Information Systems 9 BibliographyDeLone, W. H., & McLean, E. R. (1992). Information systems success: The quest for thedependent variable. Information Systems Research, 3(1), 60-95.DeLone, W. H., & McLean, E. R. (2003). The DeLone and McLean model ofinformation systems success: A ten-Year Update. Journal of Management InformationSystems, 19(4), 9-30.Ivari, J. (2005). An empirical test of the DeLone-McLean model of information systemsuccess. The Data Base of Advances in Information Systems, 36(2), 8-27.Sabherwal, R., Jeyaraj, A., & Chowa, C. (2006). Information system success: individualand organizational determinants. Management Science, 52(12), 1849.