Assessing Quality 1Assessing Quality in Research Edgardo Donovan RES 620 – Dr. Wenli Wang Module 1 – Case Analysis Monday, October 24, 2011
Assessing Quality 2 Assessing Quality in Research Delivering original, high quality writing, and compelling consequences is necessary inorder to get published in peer reviewed journals, gain notoriety in ones field, and be successfulas an academic. The PhD Dissertation is a challenging research assignment taken at the end of aseries of courses designed to impart its students with the ability read, analyze, critique, andultimately, produce qualitative/quantitative academic research that brings new theoreticalknowledge to one’s chosen field. The PhD dissertation is the ultimate educational product. It reflects the training of itsauthor and the technical, analytical, and writing skills he or she has developed in a doctoralprogram. Successful completion of the dissertation and the ward of the PhD certify that thedegree recipient can do independent scholarly work (Lovitts 2005, p. 18). Figure 1. The Dissertation’s Progression (Eveland 2011, p. 8)
Assessing Quality 3 There is a wide range of criteria that a PhD dissertation committee may consider whenevaluating whether a dissertation is good enough for publishing. What is the intellectual merit ofthe proposed activity? How important is the proposed activity to advancing knowledge andunderstanding within its own field or across different fields? How well qualified is the proposerto conduct the project? To what extent does the proposed activity suggest and explore creative,original, or potentially transformative concepts? How well conceived and organized is theproposed activity? What are the broader impacts of the proposed activity? How well does theactivity advance discovery and understanding while promoting teaching, training, and learning?To what extent will it enhance the infrastructure for research and education, such as facilities,instrumentation, networks and partnerships? Will the results be disseminated broadly to enhancescientific and technological understanding? What may be the benefits of the proposed activity tosociety (National Science Board 2007, p. 29)? Outstanding dissertations are characterized by original, high quality writing, andcompelling consequences. The dissertations display a richness of thought and insight and manean important breakthrough. The students who produce outstanding dissertation are described bytheir faculty as very creative and intellectually adventurous. The love and are passionate aboutwhat they are doing and the display intense curiosity and drive. They typically workindependently. At the same time a dissertation may also be a function of luck. The student maysimply be in the right place at the right time (Lovitts 2005, p. 19). Most dissertations are very good which is the level of quality the faculty members saidthey expect of most graduate students. Very good dissertations are solid and well written but theyare distinguished by being less original, less significant, less ambitious, less exciting, and less
Assessing Quality 4interesting than outstanding dissertation. They display mastery of the field address the nextquestion or problem, in a research program and are executed competently and confidently(Lovitts 2005, p. 20). Acceptable dissertation a adequately meet the criteria for the award of the PhD, whereasmarginal acceptable ones are just barely over the threshold of acceptability by being somewhatpedestrian and distinguished by being not very original, significant, exciting , or interesting.Students who produce acceptable dissertation were said to be functioning close to theircapabilities. Although most are bright, they are missing a certain quality of mind they lackintellectual power and the ability to think like a researcher. They also lack independence andinitiative and thus require coaching and hand holding. Their advisors often give them their topicsof problems, feed them ideas, and spend much time writing and copy editing their work (Lovitts2005, p. 21). Writing a successful dissertation is just one of the many challenges facing researchers asthey attempt to produce new and outstanding research over the course of a career. In order togain notoriety in ones field and be successful as an academic one must get published in peerreviewed journals. This is not an easy task. Manuscript rejection rates for the top IS academicjournals average 85-90%. An undesirable consequence of this level of rejection is that thecommunity becomes discouraged and disaffected with the review process. Part of the reason somany manuscripts are not ready for publication may lie in the lack of agreement andunderstanding among researchers on the key criteria for evaluating research (Straub 1994, p. 1).Given the wide array of specialties in which research studies are conducted it is difficult todelineate a step-by-step formula for successful research adaptation for publication. Generally, allproposals are evaluated according to intellectual merit and broader impacts. Consideration is also
Assessing Quality 5given to how well the proposed activity fosters the integration of research and education(National Science Board 2007, p. 6). Beyond getting research approved as a dissertation or getting published in peer-reviewedjournals, it can be a great challenge just to acquire the resources such as grants required forconducting certain kinds of projects. On average the number of proposals an investigator submitsbefore receiving a grant has stayed constant at 2.2 proposals for the past five years (NationalScience Board 2007, p. 18). Although young researchers are told that they need to produceresearch results, sometimes no exact requirements are specified. Instead, researchers are usuallygiven vague encouragement to achieve something ``significant or have ``high qualitypublications without any precise explanation of what it means. The reason a junior staff membercannot obtain a more precise explanation of how to measure research is that no singleexplanation exists. Instead, there are a variety of measures -- each group tends to use a measurethat maximizes their goals. Indeed, someone wants to make a point in favor or against a person,they choose a measure that helps. (Comer 1994, p. 18). Delivering original, high quality writing, and compelling consequences is necessary inorder to get published in peer reviewed journals, gain notoriety in ones field, and be successfulas an academic. The PhD Dissertation is a challenging research assignment taken at the end of aseries of courses designed to impart its students with the ability read, analyze, critique, andultimately, produce qualitative/quantitative academic research that brings new theoreticalknowledge to one’s chosen field.
Assessing Quality 6 BibliographyComer, Douglas (2005) Ways To Measure Research. Working Paper, PurdueUniversity. Retrieved May 19, 2008 from:http://www.cs.purdue.edu/homes/dec/essay.research.measure.htmlEveland, JD (2011). RES620: What, why, and how? TUI UniversityEveland, JD, Wang, Wenli, Shackman, Fitzgerald, Guzman (2010) Review of criteria forqualifying examination papers. TUI UniversityLovetts, Barbara (2005) How to Grade a Dissertation. Academe. Nov-Dec.National Science Board and National Science Foundation Staff (2007) . National ScienceFoundations Merit Review Process Fiscal Year 2007. Retrieved October 31, 2011 from:http://www.nsf.gov/nsb/publications/2008/nsb0847_merit_review_2007.pdfStraub, Detmar, Soon Ang, Roberto Evaristo (1994). Normative standards for ISresearch. TUI UniversityU.S. Department of Health and Human Services (2003). NIH grants policy statement.Retrieved October 31, 2011 from:http://grants.nih.gov/grants/policy/nihgps_2003/nihgps_2003.pdf