What Students Know and Don’t Know
Jennifer Day, Cheryl Reindl-Johnson, and Jennifer Romero
Sinclair Community College
The Issues
• Belief that students come to college knowing MS
Office from high school classes and/or work
experience.
• BIS...
BIS 1120
• Three semester-hour, 16-week course covers
computer concepts and introductory
Word, Excel, Access, PPT.
• Custo...
Student Enrollment
4%
58%
3%
4%
2%
29%
Location
CVCC
Dayton
ELC
HHLC
PCLC
WWW
Exam Breakdown
• 70 Items in 75 Minutes
– 6 tasks – Windows 7
– 13 tasks – Word
– 14 tasks – Excel
– 14 tasks – Access
– 1...
A computer interprets every signal as either “on”
or “off” using numbers known as _____________
1 2 3 4
25% 25%25%25%
1. m...
The rules that establish an orderly transfer of data
between sender and receive are called
1 2 3 4
25% 25%25%25%
1. protoc...
________ changes every time you add or remove
hardware; therefore, it is often referred to as
semipermanent memory.
1 2 3 ...
If a Web page provides an RSS feed, the orange
Feeds button appears on the _______________.
1 2 3 4
25% 25%25%25%
1. Comma...
Online (29%) vs. Face-to-Face (71%)
Population # Students Average Min Max
Online (Full Term) – 225 80% 11% 99%
Online (12-...
The RESULTS
Exam Questions
• Common questions missed among high scorers
– Multiple choice questions
• CMOS question
• A URL’s ___ is a...
13Fall Pretest Frequency Analysis
• Multiple choice questions missed on the pre-exam
were also missed on the post-exam.
• ...
14Spring Pretest Analysis
• Multiple choice questions missed on pretest are the
same three missed on fall pretest.
• Task ...
Scores – Fall 2013 – Post-Exam
– 835 registered for sections did not take exam
– 108 scores removed because of irregularit...
Average by Application
Subject # Tasks
Pre-test
SP14
Pre-Test
F13
Post-Test
F13
Improvement
MS Access 14 24% 24% 86% 62%
M...
High Scorers
Students Pretest
Average
Post Test
Average
Women 10 37.2% 98%
Men 15 46.4% 98%
Total Students 25 41.8% 98%
Day vs. Evening Sections
Students Average Min Max
Day 511 79% 6% 99%
Evening 117 76% 14% 99%
Time
Shortest time = highest scores Highest time = lowest scores
• 41 student took 75 mins (max)
– 10 (24%) scored 50% or
...
Full-time v. Adjunct
• 59% students taught by adjuncts, 41% by Full-time
• Overall Average = 79%
– Adjunct Average = 78%
–...
Highest Average by Faculty
• Highest average 86%
– Day (FT - online) - 21 students with only 1 student not
passing post-ex...
Post-Exam Statistics
• 70.4% overall passed (75% or higher)
• 73% male passed
• 68% female passed
Post-Exam by Gender
Male – 423 (42.7%)
A - 57 (13.5%)
B – 142 (33.6%)
C - 109 (25.8%)
D – 66 (15.6%)
F – 49 (11.6%)
13%
33...
Post-Exam by Gender
Female – 567 (57.3%)
A - 65 (11.5%)
B - 141 (24.9%)
C - 182 (32.1%)
D – 102 (18%)
F – 77 (13.6%)
11%
2...
Lessons Learned
• More faculty involvement in test creation
• Better communication about significance of exam
with adjunct...
Next Steps
• Involved faculty reporting other data (not SAM data)
• Improving the pre/post exam to focus on course
objecti...
Upcoming SlideShare
Loading in …5
×

What Students Know and What They Don't Know - Course Technology Computing Conference

455 views
321 views

Published on

What Students Know and What They Don't Know - Course Technology Computing Conference

Presenter: Cheryl Reindl-Johnson, Jennifer Day & Jennifer Romero, Sinclair Community College

We have been hearing for years that our jobs will be obsolete as students enter college knowing more about computers than their college faculty members know. At Sinclair Community College we have NOT found that to be the case. In fact, we have struggled with success and retention rates in our Computer Concepts and Applications course. We offer roughly 80 sections of our BIS 1120 Computer Concepts and Applications course each semester, enrolling approximately 4,000 students per year. We realize that there are myriad reasons why students are not successful in a course, but to begin to find a solution to address low success and retention rates in this high enrollment course, we felt we needed to do some analysis of our student population. We are using SAM to administer a pre-test to all students enrolled in a section of the course during the second week of the semester, so that we can better analyze student knowledge of the material when they arrive in our classes, and collect information on student mastery of the material at the end of the semester using the same content in a post-test. SAM's reporting features allow us to analyze section level results to compare: Day classes that tend to include more “traditional” students who are straight out of high school, to evening classes which tend to include more non-traditional students; Face-to-face to online sections; Full semester classes to 12-week or 8-week sections; Sections taught by full-time faculty versus adjunct faculty. When the pre-test and post-test are scheduled by our SAM faculty administrator, we can also analyze the data at the question level (frequency analysis) across all sections to examine: What content most students (70% or more) come to the class knowing (specific skills that are used across applications and groups of skills by application); What content most students do not know when they begin the class; What content most students struggle with at the end of the class.

Published in: Education
1 Comment
0 Likes
Statistics
Notes
  • Be the first to like this

No Downloads
Views
Total views
455
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
9
Comments
1
Likes
0
Embeds 0
No embeds

No notes for slide

What Students Know and What They Don't Know - Course Technology Computing Conference

  1. 1. What Students Know and Don’t Know Jennifer Day, Cheryl Reindl-Johnson, and Jennifer Romero Sinclair Community College
  2. 2. The Issues • Belief that students come to college knowing MS Office from high school classes and/or work experience. • BIS 1120 Computer Concepts and Applications - high enrollment course (4275 per year), but low success rates (53.7%).
  3. 3. BIS 1120 • Three semester-hour, 16-week course covers computer concepts and introductory Word, Excel, Access, PPT. • Custom New Perspectives text and SAM. • A variety of approaches to explore low success rate over the years. • Pre-exam/Post-exam approach
  4. 4. Student Enrollment 4% 58% 3% 4% 2% 29% Location CVCC Dayton ELC HHLC PCLC WWW
  5. 5. Exam Breakdown • 70 Items in 75 Minutes – 6 tasks – Windows 7 – 13 tasks – Word – 14 tasks – Excel – 14 tasks – Access – 14 tasks – PowerPoint – 5 MC – Browser & Email Basics – 4 MC – Computer Concepts
  6. 6. A computer interprets every signal as either “on” or “off” using numbers known as _____________ 1 2 3 4 25% 25%25%25% 1. millibytes 2. binary digits 3. ASCIIs 4. hexadecimals
  7. 7. The rules that establish an orderly transfer of data between sender and receive are called 1 2 3 4 25% 25%25%25% 1. protocols 2. channels 3. NICs 4. priorities
  8. 8. ________ changes every time you add or remove hardware; therefore, it is often referred to as semipermanent memory. 1 2 3 4 25% 25%25%25% 1. BIOS 2. CMOS 3. ROM 4. The CPU cache
  9. 9. If a Web page provides an RSS feed, the orange Feeds button appears on the _______________. 1 2 3 4 25% 25%25%25% 1. Command bar 2. Status bar 3. Address bar 4. Favorites Feeds tab
  10. 10. Online (29%) vs. Face-to-Face (71%) Population # Students Average Min Max Online (Full Term) – 225 80% 11% 99% Online (12-Week) 13 84% 67% 94% Online (8-Week Combined) 74 79% 19% 99% Online (A-Term) 21 84% 49% 99% Online (B-Term) 53 78% 19% 99% F2F (On Campus) 532 79% 6% 99% F2F (Off Campus) 137 76% 20% 97% F2F (8-Week Term) 7 85% 70% 96%
  11. 11. The RESULTS
  12. 12. Exam Questions • Common questions missed among high scorers – Multiple choice questions • CMOS question • A URL’s ___ is a standardized procedure the computers use to exchange data (protocol) • RSS Feed question – Task Modify the style of the selected table to create a new table style option named “MODEL” with the Header Row using Italic Font and apply it to the table • click Format as Table drop down arrow, right click selected style (modify is dim) – select “Duplicate…” – that opens dialog box which then you select Header Row and click Format, select Italics, save, click OK, then select Table drop down arrow and apply formatting
  13. 13. 13Fall Pretest Frequency Analysis • Multiple choice questions missed on the pre-exam were also missed on the post-exam. • Task EX2870 no students answered correctly on pre- exam. Frequently missed (87%) on post-exam. • Exam average – 34%
  14. 14. 14Spring Pretest Analysis • Multiple choice questions missed on pretest are the same three missed on fall pretest. • Task EX2870 (Modify table) most missed question on exam (3 students). • Exam average 33%
  15. 15. Scores – Fall 2013 – Post-Exam – 835 registered for sections did not take exam – 108 scores removed because of irregularities • 990 Scores used from Post exam • 55 sections – 21 taught by FT Faculty (5 faculty members) – 34 taught by Adjunct faculty (23 faculty members)
  16. 16. Average by Application Subject # Tasks Pre-test SP14 Pre-Test F13 Post-Test F13 Improvement MS Access 14 24% 24% 86% 62% MS Excel 14 19% 19% 68% 49% MS PowerPoint 14 39% 40% 82% 42% MS Windows 7 6 60% 59% 88% 29% MS Word 13 33% 34% 83% 49% Overall Average 62 32% 32% 80% 48%
  17. 17. High Scorers Students Pretest Average Post Test Average Women 10 37.2% 98% Men 15 46.4% 98% Total Students 25 41.8% 98%
  18. 18. Day vs. Evening Sections Students Average Min Max Day 511 79% 6% 99% Evening 117 76% 14% 99%
  19. 19. Time Shortest time = highest scores Highest time = lowest scores • 41 student took 75 mins (max) – 10 (24%) scored 50% or – 28 (68%) did not pass Score Time 94% 0:12:56 97% 0:15:38 96% 0:15:58 97% 0:16:16 94% 0:16:34
  20. 20. Full-time v. Adjunct • 59% students taught by adjuncts, 41% by Full-time • Overall Average = 79% – Adjunct Average = 78% – Full-time Average = 80% • 9 of 56 sections average below 75% - 7 were taught by adjuncts – 2 of those brand new faculty – • Highest class average =86%, lowest 65%
  21. 21. Highest Average by Faculty • Highest average 86% – Day (FT - online) - 21 students with only 1 student not passing post-exam (scored 64%)- 10 of 21 scored 90% or higher. – Paranjpe (Adjunct - 8 week A term) - 2 of 15 did not pass (57% and 70%) – 9 of 15 scored 90% or higher – Aungst – (FT – F2F) 2 of 10 students did not pass (57% and 74%) – 6 of 10 scored 90% or higher
  22. 22. Post-Exam Statistics • 70.4% overall passed (75% or higher) • 73% male passed • 68% female passed
  23. 23. Post-Exam by Gender Male – 423 (42.7%) A - 57 (13.5%) B – 142 (33.6%) C - 109 (25.8%) D – 66 (15.6%) F – 49 (11.6%) 13% 33% 26% 16% 12% A B C D F
  24. 24. Post-Exam by Gender Female – 567 (57.3%) A - 65 (11.5%) B - 141 (24.9%) C - 182 (32.1%) D – 102 (18%) F – 77 (13.6%) 11% 25% 32% 18% 14% A B C D F
  25. 25. Lessons Learned • More faculty involvement in test creation • Better communication about significance of exam with adjunct faculty • Naming/scheduling of exam by Superuser • SAM paths allow students to focus on what students don’t know
  26. 26. Next Steps • Involved faculty reporting other data (not SAM data) • Improving the pre/post exam to focus on course objectives • Department changing course content to focus only on apps

×