This document summarizes a webinar for classroom pilot testing of educational materials developed by the Geoscience Education Through Solving Issues Together (GETSI) project. The webinar covered the goals of collecting student data during the pilot testing, an overview of the timeline and process for data collection, and important resources on the GETSI website. Key aspects of the data collection include administering pre- and post-surveys on student geoscience literacy and attitudes, collecting student work samples, and having instructors complete a post-pilot reflection. The goal is to gather feedback that can be used to improve the educational modules before broader implementation.

1. This work is supported by the National Science Foundation’s Transforming Undergraduate Education in STEM program within the
Directorate for Education and Human Resources (DUE-1245025).
CLASSROOM PILOT-TESTING BY GETSI AUTHORS
November 30, 2018
12 pm PT | 1 pm MT | 2 pm CT | 3 pm ET
Beth Pratt-Sitaula
You may need to turn on your mic in the
lower left of the Zoom screen.

2. WEBINAR GOALS
By the end of the webinar you will be able to:
• State what data will be collected, why, and
how (GLE, attitudinal surveys, student work)
• Find more information and resources as
needed on the GETSI website to support pilot
testing

3. CLASSROOM PILOT & DATA COLLECTION
Development Timeline
http://serc.carleton.edu/getsi/info_team_members/timeline.html

5. CLASSROOM PILOT – DATA CHECKPOINT A
Data Check A: Before your course begins
• Inform (if you haven’t already) your Institutional
Review Board of participation in the GETSI project
• Decide how you will provide data to GETSI/SERC
(digital or paper)
• Decide on a student ID method
• Complete the GETSI Course Setup Form
v

6. GETSI COURSE SET UP FORM
https://serc.carleton.edu/getsi/info_team_members/forms_docs/course_form.html

7. COURSE STATUS PAGE
• Once your course set-up form is completed, it will
create your Course Status Page, this will be your go-to
place to see your progress in the assessment process
• You can always reach it through your SERC account
page - https://serc.carleton.edu/account/

8. For overview of what the different elements of the Course Status Page are see:
http://serc.carleton.edu/integrate/info_team_members/currdev/status_page.html

9. CLASSROOM PILOT – DATA CHECKPOINT B
Data Check B: As your course begins
• Tell GETSI Facilitator (Beth) the student IDs of any
students who withdraw consent. She will inform SERC.
• Submit initial roster of student ID's (using your Course
Status Page)
• Direct students to initial attitudinal survey
• Administer the initial GLE multiple choice questions
v

10. IRB & STUDENT CONSENT
• It is no longer necessary to have “signed consent”
from the students, but you still need to “inform”
them of the research and they must have an
opportunity to withdraw use of their work.
• Main points to make to the class on Day 1.
1. You are developing (super cool cutting edge)
curriculum to be published online and the initial
version is being tested in this class
2. Anonymous copies of their work and survey results
will be reviewed by the project assessment team.
3. They can opt to not be part of this review by telling
you at any time. If they withdraw, tell me their ID#.
4. However, being part of the research or not will not
change the required course workload in any way.

11. Data Checkpoint B
--includes 2 pre-surveys

12. • Pre- & Post-course Geoscience Literacy
Assessment (GLE 8 MC questions)
• 2 from each literacy document
• 4 Single answer and 4 “select all that apply”
http://serc.carleton.edu/integrate/info_team_members/currdev/documents/gle/index.html
v

13. Link to GLE survey files
page:https://serc.carleton.edu/integrate/info_team_members/currdev/documents/gle/index.
html

14. ATTITUDINAL PRE-SURVEY
• Links to give students available in several places
– Confirmation page
– Confirmation email
– Course Status Page

15. ATTITUDINAL PRE-SURVEY
Questions relate to opinions on science, geoscience, careers, etc.

16. Data Checkpoint C
--students do the work
that is the summative
assessment for the module

17. • Graded by faculty of the pilot testing classroom
based upon a rubric
• Summative and conclusive measuring the level
of student accomplishment of module goals
• Used in the same fashion in all materials pilot-
testing classrooms
http://serc.carleton.edu/integrate/info_team_members/currdev/assessment.html#sum

18. CLASSROOM PILOT – DATA CHECKPOINT D
Data Check D: End of course
• Administer the final GLE multiple choice questions
• Direct students to complete the final attitudinal survey
• Check Course Status Page for student ID mis-matches
• Complete the post-pilot reflection survey on your
individual reporting forms
• All summative student work is submitted to SERC and
will be recorded in the author's course status page
v

19. Data Checkpoint D
--2 post-surveys
--self reflection
--upload or mail
student work

20. GETTING SUMMATIVE STUDENT WORK TO SERC
• Option 1: digital upload using Course Status Page
• Option 2: paper mail to SERC
– Attn: Ellen Iverson
Science Education Resource Center
Carleton College
One North College St
Northfield, MN 55057
– Note tracking numbers and be sure to have additional paper
copy in case there is a mail loss situation
Make sure student work:
• has ID numbers
• does not have student names (although SERC students
workers will do an additional sweep and redact any names
that slip through).

21. AUTHOR REFLECTION DURING AND POST TESTING

22. Data Collection
http://serc.carleton.edu/integrate/info_team_members/currdev/course_status.html
Where can
you find
help?

23. IMPORTANT LINKS
• This webinar – within “Team Meetings - Webinars”
http://serc.carleton.edu/getsi/meetings/index.html
• Material Testers information
http://serc.carleton.edu/getsi/info_team_members/index.html#testers
– GETSI Course Set-up Form – do it today or Monday
http://serc.carleton.edu/getsi/info_team_members/forms_docs/course_form.
html
– Materials Testing Overview
http://serc.carleton.edu/integrate/info_team_members/currdev/course_statu
s.html
– Links to GLE files
https://serc.carleton.edu/integrate/info_team_members/currdev/documents/
gle/index.html
– Data Collection
Detailshttp://serc.carleton.edu/integrate/info_team_members/currdev/cours
e_status.html
– Course Status Page
http://serc.carleton.edu/integrate/info_team_members/currdev/status_page.
html
• Materials in Progress – access to your modules
http://serc.carleton.edu/getsi/info_team_members/materials.html

24. FOLLOW UP WORK
• Complete Course Set-up Form
• Review website information
• FINISH YOUR MODULES