• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
CIES 2012
 

CIES 2012

on

  • 921 views

 

Statistics

Views

Total Views
921
Views on SlideShare
921
Embed Views
0

Actions

Likes
0
Downloads
3
Comments
0

0 Embeds 0

No embeds

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike LicenseCC Attribution-NonCommercial-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    CIES 2012 CIES 2012 Presentation Transcript

    • ASSESSMENT CULTURE AND INFORMATIONAL SELF-DETERMINATION:A COMPARATIVE STUDY OF ELECTRONIC PERFORMANCE MONITORING INONLINE LEARNING ENVIRONMENTS IN THE U.S. AND GERMANYDaniel Knox: The University at Albany, State University of New York, U.S.Markus Deimann: Der FernUniversität in Hagen, Germany
    • Roadmap Conceptual frame  Cultureof Assessment  Informational Self-Determination Data & Methods Preliminary Results Implications for Research & Practice
    • Issue and Purpose
    • Culture of Assessment Proponents  Institutions should commit to assess student learning outcomes (Michaud, 2010).  (Typically) quantitative methods over qualitative methods (Turgeon, 2010).  Formative assessments over summative assessments (Weiner, 2009).
    • Culture of Assessment Critics
    • Assessment as EPM Online assessment is electronic performance monitoring (EPM):  The use of computer and communication technologies to collect and store information about individual or group performance (Aiello & Douthitt, 2001) Two distinct aspects:  Automated  Totalizing “Blackboard records every click by a user within a course and allows instructors to generate graphical reports on course usage and actvity” (UWE Staff Guide, 2009).
    • Informational Self-Determination Legal concept:  The degree to which a subjects maintain control of their own data (Hornung, 2009).  Distinct from the U.S. concept of privacy. Legal Origins:  Volkszählungsurteil (census verdict), December 15, 1983  Personal data in Germany are constitutionally protected.  Individuals have the power to decide when and to what extent personal information is viewed.
    • Informational Self-Determination Implications for online learning:  Data Protection Commissioner at every institution.  Place strict limitations in the collection and storage of personal information.  Every automated act of data processing that can be directly linked to a certain person must be regulated by a specific law.  Otherwise data processing must be authorized in each instance by the individual.
    • Informational Self-Determination In practice:  Inonline, asynchronous courses, German faculty may not:  Require participation in discussions.  Grade discussions posts.  Creates uncertainty on the part of instructors.
    • The Culture of Assessment vs. InformationalSelf-Determination Not universal ‘best practices,’ but culturally embedded. Both encroach upon academic freedom. So…what do faculty do to promote engagement in online discussions?
    • SLN Data & Methods Data:  Stratified random sample of 160 asynchronous, fully-online courses drawn from the State University of New York Learning Network’s (SLN) archives. Methods:  Course documents (syllabi) analyzed for references for online discussion policies (1141 of 2058 documents) and compiled.  Discussion policy documents analyzed using grounded theory methodology (Strauss & Corbin, 1998).  Each sentence analyzed using the message as the unit of analysis in Atlas.ti.
    • The Culture of Assessment Goes Online Absence of a critical stance. Universality of practices.  National Research Council’s Commission on Behavioral and Social Sciences and Education  How People Learn (Bransford et. al. 2000)  “Good learning environments are knowledge centered, learner centered, and assessment centered.”  “Effective teachers … do a great deal of on-line monitoring of both group work and individual performances…” (pg. 140).  Community of Inquiry (Anderson, Rourke, Garrison, & Archer, 2001 )
    • FernUniversität Data & Methods Internet survey  20 item questionnaire  154 faculty from cultural and social sciences  Delivered via LimeSurvey  25 responses (16.2%)
    • Results SLN Quantitative Properties (9) Quantitative Dimensions (5) General participation Hard count Initial posting Minimum Lead discussion Maximum Lines Optimum Logins Range Logins with post Respond to initial posts Respond to peers Words Qualitative Properties (12) Qualitative Dimensions (4) Accuracy Undefined Creativity General Critical Thinking Specified Etiquette Highly Specified General Interactivity Integrity Objectivity Reference Peers Relevance Subjectivity Supporting Evidence Writing
    • SLN ExampleMonitored properties and dimensions in a graduate education course: Property Presence Dimension Number Low Range High Range Unit Quantitative Yes General participation Property Not Found Initial posting Property Not Found Lead discussion Property Not Found Lines Property Not Found Logins Property Found Minimum 2 Week Logins with post Property Not Found Respond to initial posts Property Found Minimum 2 Module Respond to peers Property Found Hard Count 1 Discussion Words Property Not Found Presence Specification Qualitative Yes Accuracy Property Not Found Creativity Property Not Found Critical Thinking Property Found General Etiquette Property Found Specified General Interactivity Property Not Found Integrity Property Not Found Objectivity Property Not Found Reference Peers Property Not Found Relevance Property Not Found Subjectivity Property Found General Supporting Evidence Property Found Specified Writing Property Not Found
    • FernUniversität Results Content  Introduce provocative topics/material  Pose challenging questions  Refer to previous questions  Assignments with a practical orientation for the students Structure  Small group discussion for low participating students.  Use the LMS environment to present discussion as a ‘normal’ part of online learning. Modeling  Posting by instructor to begin the course.  Explain interactions and advantages of participation.  Explain relevance.  Use peer tutoring and mentoring to model online discourse.
    • Conclusion Implications for research  ‘Universal’ assessment best practices should be tested for validity (and appropriateness) across different cultures.  Develop and test alternate models of assessment. Implications for practice  Develop culturally sensitive monitoring policies and incorporate into faculty training.  Instructors should consider opening a dialogue with students about the topic.
    • ReferencesAnderson, T., Rourke, L., Garrison, D. R., & Archer, W. (2001). Assessing teaching presence in a computerconferencing context. Journal of Asynchronous Learning Networks, 5(2) Retrieved December 10, 2004, fromhttp://www.aln.org/publications/jaln/v5n2/Aiello, J. R., & Douthitt, E. A. (2001). Social Facilitation from Triplett to Electronic Performance Monitoring. GroupDynamics: Theory, Research, and Practice, 5(3), 163-180.Bransford, J.D., A. L. Brown, and R.R. Cocking, eds. (2000). How people learn: brain, mind, experience, andschool. Washington, D.C.: National Academy Press, 206-230.Michaud, Olivier (2010). The effects of assessment: A reflection from within the economic worldview in education.Analytic Teaching and Philosophical Praxis. 30 (1): 20-30.Strauss, A., & Corbin, J. (1998). Basics of Qualitative Research: Techniques and Procedures for DevelopingGrounded Theory (2nd ed.). Thousand Oaks, CA: Sage Publications, Inc.Turgeon, Wendy C. (2010). Confessions of a department chair of assessment. Analytic Teaching and PhilosophicalPraxis. 30 (1): 12-19.University of Western England (2009). Blackboard Staff Guide: Tracking and Statistics. University of WesternEngland, BristolWeiner, Wendy F. (2009). Establishing a culture of assessment: Fifteen elements of assessment success – how manydoes your campus have? Academe Online (July-August).http://www.aaup.org/AAUP/pubsres/academe/2009/JA/Feat/wein.htm