Download to read offline
Uploaded byEdina Renfro-Michel
The Ultimate Course Search Learning Tool
The document discusses the development of the Ultimate Course Search (UCS) tool aimed at enhancing learning for millennials in computer science courses. It presents the objectives of UCS, findings from research comparing control and experimental groups in terms of student performance, and feedback on the tool's usability. Key implications for higher education include reduced attrition, improved course organization, and the need for individualized learning opportunities.
More Related Content
Similar to The Ultimate Course Search Learning Tool
The Ultimate Course Search Learning Tool
- 1. The Ultimate CourseSearch Learning Tool Edina Renfro-Michel, Ph.D., LPC, ACS renfromichee@mail.montclair.edu Sailume Walo-Roberts, MA, ABD, LPC waloroberts1@mail.montclair.edu Montclair State University
- 2. Agenda • Millennials! • OurNSF Grant • UCS and Learning Preferences • Demonstration of UCS • Implementation • Preliminary Data • Implications for Higher Education
- 3. We are TeachingMillennials! • Multitask • Have Short Attention Spans • Tend to be Visual Learners • Bore Easily • Want Instant Gratification • Want Control Over Their Learning • Have an Expectation to Achieve • Lack Self-Reflection Skills • Need Individualized Educational Opportunities
- 4. Our NSF Grant- iSECURE • To Reduce Attrition in Computer Science Security Courses • Increase availability to materials • Focus Studying Time • Access to Multiple Learning Materials • Ultimate Course Search (UCS)
- 5. Our Objectives forUCS • Create a program that will accurately search all electronic course materials • Integrate UCS into Courses • Help students understand learning preferences as connected to UCS • Create a user friendly, clean interface • Determine the effectiveness of the tool
- 6. Learning Preferences • Indexof Learning Preferences (Felder & Soloman, 1993) Four Types of Learners • Active – Reflective • Sensing – Intuitive • Visual – Verbal • Sequential - Global
- 7. Your Results • ACT_______________________________________REF 11a9a 7a 5a 3a 1a 1b 3b 5b 7b 9b 11b • SEN________________________________________INT 11a 9a 7a 5a 3a 1a 1b 3b 5b 7b 9b 11b • VIS________________________________________VRB 11a 9a 7a 5a 3a 1a 1b 3b 5b 7b 9b 11b • SEQ________________________________________GLO 11a 9a 7a 5a 3a 1a 1b 3b 5b 7b 9b 11b
- 8. What UCS Does •Indexes PowerPoint Slides - The set of slides belonging to a presentation file are mapped relationally to that presentation along with the values of presentation title and presentation filename • Segments Videos - In order to find where the slide exists in a video, the lecture video transitions are determined, and segmented. Then we determine the transition of videos. • Indexes Textbook – The Textbook’s Index was used to determine the ontology to form our index (Apache Lucene) • Creates Search Terms - The materials are searched for matches in keywords, and a presentation’s relevancy is calculated
- 9.
- 10. The Research Collected Datain a Security Course • Control and Experimental • Face-to-Face and Hybrid • Same teacher, same book, same lectures
- 11. Research Questions • Isthere a statistically significant difference in post-test and final exam outcomes between the control and experimental groups? • Is there a difference in attrition between the control and experimental classes? • How did the students utilize the tool? • How did the students utilize the learning preferences information?
- 12. Student Learning Preferences Face-toFace Control • Active = 6 • Reflective = 21 • Sensing = 20 • Intuitive = 7 • Visual = 21 • Verbal = 6 • Sequential = 14 • Global = 13 Experimental • Active = 10 • Reflective = 9 • Sensing = 15 • Intuitive = 4 • Visual = 17 • Verbal = 2 • Sequential = 12 • Global = 7
- 13. Student Learning Preferences Hybrid Control •Active = 10 • Reflective = 7 • Sensing 13 • Intuitive = 4 • Visual = 13 • Verbal = 4 • Sequential = 13 • Global = 4 Experimental • Active = 18 • Reflective = 12 • Sensing = 22 • Intuitive = 8 • Visual = 28 • Verbal = 2 • Sequential = 19 • Global = 11
- 14. Student Demographics F2F Control •N = 28 (66 enrolled in course) • Mean Age = 23.8 • Year in School = 3.54 • Gender • Female = 4 • Male = 24 • Racial/Ethnic Identifiers • African American/Black = 5 • American Indian or Alaska = 0 • Asian = 3 • Caucasian/White = 12 • Hispanic/Latino = 9 • Pacific Isl/Native Hawaiian = 1 • Other = 4 • No Answer = 3 Experimental • N = 21 (30 enrolled in course) • Mean Age = 23.19 • Year in School = 3.52 • Gender • Female = 1 • Male = 20 • Racial/Ethnic Identifiers • African American/Black = 2 • American Indian or Alaska = 1 • Asian = 6 • Caucasian/White = 6 • Hispanic/Latino = 8 • Pacific Isl/Native Hawaiian = 1 • Other = 5 • No Answer = 0
- 15. Student Demographics Hybrid Control •N = 19 ( 27 enrolled in course) • Mean Age = 22.89 • Year in School = 3.16 • Gender • Female = 1 • Male = 18 • Racial/Ethnic Identifiers • African American/Black = 2 • American Indian or Alaska = 0 • Asian = 9 • Caucasian/White = 4 • Hispanic/Latino = 5 • Pacific Isl/Native Hawaiian = 0 • Other = 2 • No Answer = 2 Experimental • N = 30 (36 enrolled in course) • Mean Age = 21.97 • Year in School = 3.40 • Gender • Female = 6 • Male = 24 • Racial/Ethnic Identifiers • African American/Black = 2 • American Indian or Alaska = 0 • Asian = 11 • Caucasian/White = 11 • Hispanic/Latino = 9 • Pacific Isl/Native Hawaiian = 1 • Other = 5 • No Answer = 0
- 16. Pre and PostTest Results F2F Control • Pre Test Mean = 9.39 • Standard Dev = 2.25 • Post Test Mean = 12.18 • Standard Dev = 2.29 • Change in Scores = 2.79 Experimental • Pre Test Mean = 9.10 • Standard Dev = 2.16 • Post Test Mean = 11.70 • Standard Dev = 3.09 • Change in Scores = 2.60
- 17. Pre and PostTest Results Hybrid Control • Pre Test Mean = 8.89 • Standard Dev = 2.424 • Post Test Mean = 12.59 • Standard Dev = 2.647 • Change in Scores = 3.7 Experimental • Pre Test Mean = 10.13 • Standard Dev = 2.569 • Post Test Mean = 11.69 • Standard Dev = 3.253 • Change in Scores = 1.56
- 18. Final Exam Results- F2F Control • Score = 144.57 (out of 200) • Standard Dev = 47.60 Experimental • Score = 150.86 (out of 200) • Standard Dev = 17.59 An independent T-test showed no between statistical significance in the final exam scores: t(47) = 6.286, p=.568.
- 19. Final Exam ResultsHybrid Control • Score = 116.68 (out of 200) • Standard Dev = 24.347 Experimental • Score = 123.97(out of 200) • Standard Dev = 23.576
- 20. Attrition Findings -F2F Control • 66 students enrolled • 39 students completed the semester • 41% attrition rate Experimental • 30 students enrolled • 26 students completed the semester • 13% attrition rate
- 21. Attrition Findings Hybrid Control •27 students enrolled • 26 students completed the semester • 4% attrition rate Experimental • 36 students enrolled • 36 students completed the semester • 0% attrition rate
- 22. Survey Feedback: Howdid the students use UCS? • Study for the exam • Review lecture videos – past and present • Search for Information/specific words & terms • Review video podcast lectures • As a reference and to take notes • To help complete homework assignment/class projects • To ‘test the tool’
- 23. Survey Feedback: Whatdid the students like about UCS? • User friendly • Freeware • Search engine • Fast and accurate • Search exact words • Tabs and specific information • Search Videos • Searches lead to a lot of information • Helped Students Understand Concepts • Made studying easier • Able to better understand material covered in class
- 24. Survey Feedback: Comments AboutUCS • “I didn’t feel overwhelmed cause I had all the information in tools.” • “…it was like having the professor actually explaining & answering the questions I had.” • effectiveness of the search when looking for a topic to study about” • “All needed information in one place.” • “it was excellent reference on slides where the prof. talked about how to do something like spinning tree” • “fast search engine.” • “taught me tricks I didn’t know.” • “it saves me the work of actually taking notes.” • “maybe have most viewed notes, or what topic most students have problems maybe put as the 1st thing.”
- 25. Implications for HigherEducation • Reduce attrition • Increase clarity of course organization • Increase accessibility of materials – One stop shop • Increase student interaction with materials • Individualize learning • Create connections within and between courses
- 26. Questions? • Our YouTubeChannel: http://bit.ly/1imcF8o • This Presentation on Slideshare:
Editor's Notes
- #3 Sailume
- #4 Sailume Millennials need an interactive environment, with short lectures, clear goals and connections between what they know and need to learn, reinforcement often (so short quizzes, often), lots of technology but clear reasons for using it and connections to their current use of technology, and teach them how to learn – create connections for them
- #5 Sailume Develop, implement and determine effectiveness of UCS for security courses Ontology – creates connections of the material – what items relate to the search terms Integrate Learning Preferences? – Millennials have lack of self reflection
- #6 Sailume
- #7 Sailume Why ILS? Normed on Engineering students, has a continuum, easy to understand, quick to administer and free for educational/research use
- #8 Sailume Active learners like to interact with new material Reflective learners like to think about applying material Sensing learners like material that applies to the real world Intuitive learners enjoy learning new material and dislike repetition Visual learners remember material presented through pictures or demonstrations Verbal learners tend to remember written and spoken explanations Sequential learners like to link information in logical steps Global learners like looking at the big picture and how it related to larger themes before understanding all of the details
- #9 Edina Searches The score of each slide that belong to the same presentation is summed up The presentations are ranked according to their total score of relevancy (higher to lower) The book’s index words and page numbers of the word’s occurrences are organized as the following XML <Textbook title = ‘textbookTitle’ > <Keyword> <Word> index word </Word> <PageNumber>1,2,3 </PageNumber> <Keyword> </Textbook>
- #10 Edina
- #11 Edina Same course – same teacher, same book, same lectures The experimental class is our beta testing class this semester, they are going to get their ILS information, links to videos as well as printed version of Learning Preferences information with hints for studying and asking questions in class. The students will be trained on the tool, we will track usage – time, searches, etc. at midterms we will ask for usage feedback, then post test for both classes
- #12 Edina
- #13 Edina
- #14 Edina
- #15 Edina For the control Group, Thirty-nine students completed the course with 21 students completing the post-test. For the experimental, Twenty-six students completed the course and all 21 students completed the final research packet
- #16 Edina
- #17 Edina A T-test was run to determine within group differences. The pre/post test scores were statistically significantly different for both the control class, t(27)= 9.39, p<.0005, and the experimental class, t(21)=12.182, p<.0005. A one-way ANOVA was run to determine statistical significance of learning outcomes between the groups as measured by the pre/post test. There were no statistical differences, F(1,47)=.567, p=.456.
- #18 Edina
- #19 Edina It’s important to note that for the control group, their scores had very highs and lows (scoring 199 or 80). Whereas, for the experimental group, their scores were evenly distributed. Two students in the control group failed the final exam, whereas all the students in the experimental group passed the exam.
- #20 Edina It’s important to note that for the control group, their scores had very highs and lows (scoring 199 or 80). Whereas, for the experimental group, their scores were evenly distributed. Two students in the control group failed the final exam, whereas all the students in the experimental group passed the exam.
- #21 Edina both classes had a high attrition rate. However, it is important to note that none of the students participating in the experimental research group dropped the class.. Whereas, the high attrition of the participants in the control group can be attributed to the high attrition of the class itself. Additionally, it can be deduced that, because the students in the experimental group had access to the tools, and were invested in giving feedback on their usage, they were less likely to drop the course.
- #22 Edina both classes had a very high retention rate. This is in sharp contrast to the face to face class which has a much higher attrition rate. The difference in retention can be attributed to numerous factors, including this being a hybrid class, course is a requirement for Information Security majors and many in the class were seniors, which means this was their last semester to take the course.
- #23 Sailume
- #24 Sailume
- #25 Sailume – one student requested to continue access to the tool this summer so he could use the tool to continue to learn the material
- #26 Edina If students are able to access any course information – more individualized learning and less course confusion, which are the reasons students drop online courses – increase retention. UCS, when integrated into a CMS and with Learning preferences, can help students to individualize learning and fill ‘learning gaps’ simply – when they don’t remember a key component of learning, maybe something from a lecture or a previous course – they can look it up immediately – read or listen to the information – then go back to their original studying.