Published on

The learning outcomes of an online reflective journal in engineering

Published in: Education, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide


  1. 1. The learning outcomes of an onlinereflective journal in engineering Stuart Palmer, Dale Holt and Sharyn Bray Institute of Teaching and Learning Deakin University 1
  2. 2. IntroductionThis presentation: explores the value of reflective journaling in engineering education presents a case study of the introduction of an online reflective journal summarises the results of the students‟ initial perceptions of the online reflective journal, & investigates the contribution of the online reflective journal to unit learning outcomes 2
  3. 3. Reflection and reflective journalsIn the context of learning, reflection refers tothe active intellectual monitoring and evaluationof one‟s own formal learning and professionalpractice activities, to examine them for newunderstandings that add to the individual‟saccumulated knowledge and experienceReflective thinking based on experientiallearning is a key skill required for the lifelonglearner and the socially mature professional 3
  4. 4. Reflection and reflective journalsThe use of a reflective learning journal (dueto the requirement to transfer thoughtprocesses into words) is thought to be avaluable tool in developing self- and criticalreflection in studentsFor the practicing professional, the use of awork journal offers additional benefits 4
  5. 5. Source: http://legal.european-patent-office.org/dg3/biblio/t930039ex1.htm 5
  6. 6. Ex-Air Force Official Gets Prison Time Boeing Received Special Treatment in Procurement By Renae Merle and Jerry Markon Washington Post Staff Writers Saturday, October 2, 2004; Page A01 A former high-ranking Air Force procurement official was sentenced to nine months in federal prison yesterday after admitting that she approved excessive prices on contracts awarded to Boeing Co. to enhance her job prospects with the company. Conceding that she lied to prosecutors, Darleen A. Druyun, 56, revealed that she committed the Air Force to buy 100 airplanes from Boeing at an inflated price of about $20 billion as a "parting gift" before her Pentagon retirement to ingratiate herself with her future employer. She also slipped to Boeing proprietary pricing information from a rival European bidder on the aircraft contract. Druyun awarded Boeing an unrelated $4 billion contract because she felt in debt to the company for hiring her daughter and future son-in-law, according to court documents. An "objective selection" process, she said, may not have picked Boeing from the four competitors. In a quivering voice, Druyun apologized before U.S. District Judge T.S. Ellis III in Alexandria, telling the court she felt "shame and remorse" that her 30-year tenure as a government employee "has been tarnished. . . . I understand that this was wrong and I regret any damage my conduct may have caused to the Air Force." Druyuns case is the highest-profile defense procurement scandal since the Operation Ill Wind investigation, which resulted in more than 60 convictions starting in the late 1980s. It is expected to ripple throughout the industry, renewing concern about the potential pitfalls of the revolving door between government and the defense industry. Chicago-based Boeing, the Pentagons second-largest contractor, will likely face fresh questions about several of its contracts, and the procurement system that allowed Druyun to favor one company over another will come under sharper scrutiny. Druyun, a civilian, was at the grade of a lieutenant general when she retired and became vice president in charge of Boeings missile defense systems in January 2003. Druyuns Boeing salary -- $250,000 plus a $50,000 signing bonus -- was nearly double the top Pentagon pay for her position. Officials said Druyun admitted the extent of her deceptions only after being subjected to a polygraph test. She acknowledged altering her personal journal before turning it over to prosecutors. "She did great harm to the government, and that harm is continuing now," said Assistant U.S. Attorney Robert W. Wiechering.Source: http://www.washingtonpost.com/ac2/wp-dyn/A64968-2004Oct1 6
  7. 7. Reflection and reflective journalsIt is recognised that reflection is not only anindividual activity, but may containsocial/learner-learner aspects as wellIt is reported that collaborative reflective activityand the ability to compare one‟s own thinkingwith that of other learners yields positive resultsand better facilitated learning than individualreflection 7
  8. 8. The contextSEB421 Strategic Issues in EngineeringThis unit consisted of three modules:1. Technological Forecasting and Assessment;2. Policy Design in Engineering Organisations;3. Issues in Productivity Improvement. 8
  9. 9. SEB421 student locationsSource: http://commons.wikimedia.org/wiki/Image:Welt_Mercator_Atlantik.png 9
  10. 10. A reflective journal exerciseAt the completion of the weekly class, studentswere asked to respond in writing in their journalto the following two questions: “What did I learn today?”, and “How will this be of use to me in the future?”Off-campus students completed their journalacross the semester and submitted their writtenjournal at the end of the semester 10
  11. 11. A reflective journal exerciseIssuesOn-campus: The journal took up a significantamount of class timeOff-campus: Some journals were obviouslyproduced at the end of semester, rather thanacross the semester 11
  12. 12. An online reflective journalStudents were asked to make their weeklyjournal entry in an online discussion areaA separate discussion sub-area was created foreach nominal class week to provide somestructure and direction to students, and to breakthe large number of student postings intomanageable sections 12
  13. 13. An online reflective journal - issuesWhile it is recognised that criterion referencedmarking of student reflective journals maypotentially constrain student expressionIt can also encourage greater studentengagement with the intended pedagogicalaims of the journal 13
  14. 14. An online reflective journalThe online nature of the system meant that allstudents could post their journal entries weekly,regardless of their study location or mode ofstudyBecause the journal entries were posted to adiscussion area with open read access to allclass members, all journal postings werepotentially available to all students to read 14
  15. 15. Evaluation of initial trialQuestionnaire: demographic information – age; gender; course of study; location of study; reflective journal – did you understand its purpose?; rate its value; use of journal – frequency of access; did you read submissions of other students?; online system – rate the system‟s ease of use; general – what aspects of the online reflective journal were most useful?; least useful? 15
  16. 16. Response rate and demographicsNumber of valid responses Total class enrolment Response rate 43 83 51.8 percent Mean age Standard deviation Age range Median age 26.1 years 7.7 years 19 to 47 years 22 years Characteristic Respondent sample Class population Chi-square testFemale 9.3 percent 8.4 percent χ21 = 0.027, p > 0.869Male 90.7 percent 91.6 percentEngineering 88.4 percent 95.2 percentTechnology 7.0 percent 2.4 percent χ22 = 2.079, p > 0.353Other† 4.6 percent 2.4 percentOn-campus 55.8 percent 41.0 percentOff-campus (Australia) 30.2 percent 28.9 percent χ23 = 4.555, p > 0.207Off-campus (Singapore) 0.0 percent 1.2 percentOff-campus (Malaysia) 14.0 percent 28.9 percent†- International exchange students and non-engineering students enrolled in a single unit only 16
  17. 17. Value and use of reflective journalQuestion ResponseClearly understood the purpose of the reflective journal? Yes = 92.9 percent No = 7.1 percentValue of the reflective journal in their learning (1=low;5=hi)? Mean = 3.6 Std Dev. = 0.87 Daily 2.3 percent 2-6 per week 14.0 percentFrequency of accessing the reflective journal? Weekly 51.2 percent Less than weekly 32.6 percentRead the reflective journal entries of other students? Yes = 97.7 percent No = 2.3 percentIf yes, did reading entries of others assist your learning? Yes = 76.2 percent No = 23.8 percentEase of use of the online journaling system (1=low;5=hi)? Mean = 3.8 Std Dev. = 1.12Value of the general SEB421 discussion area (1=low;5=hi)? Mean = 3.2 Std Dev. = 0.85 17
  18. 18. Most useful aspects of journalMost useful aspect of online reflective journal Frequency of reportingContinuous revision of study materials 16Ability to compare my own thoughts with others 8Electronic submission of entries was convenient 6Chance to critically evaluate the study materials 4Weekly entries helped to pace my study 1 18
  19. 19. Least useful aspects of journalLeast useful aspect of online reflective journal Frequency of reportingUser interface difficult to use 7Problems with operation 2Having to think critically 1Operation slow 1No immediate feedback on weekly entries 1 19
  20. 20. Contribution to learning outcomes• gender;• age;• mode of study (on- or off-campus);• weighted average mark (WAM – a proxy for general prior academic ability);• semester of offer cohort grouping;• number of reflective journal entries posted (a proxy for ‘quantity’ of engagement);• average mark per journal entry (a proxy for ‘quality’ of engagement);• number of reflective journal entries read; and• final unit mark (out of 100 – a proxy for unit learning outcome) 20
  21. 21. Pooled data characteristics summaryGender Male 88.9 percent Female 11.1 percent Mean Standard deviation Minimum MaximumAge 25.07 5.67 20 49Mode of study On-campus 69.8 percent Off-campus 30.2 percentWAM Mean (of 100) 66.31 Standard deviation 9.91Number of posts Mean 9.12 Standard deviation 2.08Average mark Mean (of 1.0) 0.95 Standard deviation 0.71Number of post read Mean 92.86 Standard deviation 95.88Final unit mark Mean (of 100) 69.74 Standard deviation 12.77 21
  22. 22. Regression model for final unit markANOVA test: F386 = 144.18, p < 2×10-62R2 = 0.53Std Err of model = 8.79Variable Coefficient Standard error Beta SignificanceWeighted average mark 0.805 0.049 0.624 p < 1.2×10-46Number of journal posts 1.273 0.236 0.208 p < 1.4×10-7Mode of study -3.418 0.999 -0.124 p < 0.0007Constant 7.127 3.204 - p < 0.027 22
  23. 23. Contribution to learning outcomes• prior academic performance was an important predictor of final unit mark• the number of journal posts that a student made was an important predictor, and this was separable from WAM, and contributed at somewhat above the „face value‟ of up to 1 mark per posting• even though mode was not significantly correlated to final unit mark on a direct pair-wise basis, after the effects of WAM and number of posts, mode became significant in accounting for some of the remaining variation in final unit mark, this being a negative influence for off-campus students 23
  24. 24. Conclusions / take-home messagesIt appears that it is the activity of formallyreflecting on learning that makes a contributionto student learning outcomes,rather than the act of simply reading the postsof other students 24
  25. 25. Conclusions / take-home messagesAlthough, student prior academic performancewas a significant predictor of unit learningoutcomes, the additional contribution made byeach new reflective journal posting suggeststhat introducing an element of „compulsion‟ forstudents to make journal postings, such asthrough attaching marks to journal postings,would be beneficial 25
  26. 26. Thank you for your time 26