Discussion of methods and tools to use for data analysis in SoTL projects, including available tools like Qualtrics, BlackBoard, and resources for qualitative analysis options.

1. SoTL: Scholarship of
Teaching and Learning
Data Collection and Analysis
Dr. Staci Trekles, atrekles@pnw.edu

2. Review: SoTL Basics
• Reflective process, similar to research in any disciplinary field
• Goals typically relate to improving student learning and experiences
• Five primary steps (Bishop-Clark & Dietz-Uhler, 2012):
1. Generate the research question and do literature review
2. Design the study
3. Collect data
4. Analyze data
5. Present and publish

3. Types of Data to Collect
• Mainly you will have either qualitative
or quantitative data, or both (in mixed
methods)
• Quantitative data usually involves
some statistical work (such as results
from surveys)
• Qualitative data will involve looking
through and finding commonalities or
themes

4. Tools for Data Collection
• Qualtrics: Great choice as it gathers data anonymously and does common statistics on surveys
• Cameras/microphones/iPad/smartphone: Great for interviews or focus groups, recordings are
easy to transfer or listen to again and transcribe
• BlackBoard: good place to get assessment data from within your courses, can do basic statistics on
test scores, rubrics
• SPSS: University has a site license available to use SPSS to run statistical analysis on quantitative data
• NVIVO and HyperRESEARCH: Common tools in qualitative analysis (not free) but there are
others – see https://en.wikipedia.org/wiki/Computer-assisted_qualitative_data_analysis_software
• University Institutional Research Office: http://www.pnw.edu/institutional-research/

5. Qualtrics Overview
• http://purdue.qualtrics.com
• Log in with your Purdue career account
• They just updated their interface! You now have a choice between classic and
new views
• Detailed and helpful tutorials:
• New “Insight Platform”: http://support.qualtrics.com/explore-the-new-qualtrics
• Classic platform: http://www.qualtrics.com/university/

6. Differences in Interfaces

7. Let’s Explore Qualtrics
• Great for anonymous surveys, research, feedback, and voting
• A link is provided that you can send to anyone to complete the survey
• Reports are detailed and can be analyzed in a number of ways
• More help and manuals: http://centers.pnw.edu/teaching/qualtrics/

8. The Basics
• Use the Create Project button at the
top right to create a new survey
• Once you’re editing a survey you have
a toolbar of options
• Survey: Content and design
• Distribution: Get your link and other
settings
• Data & Analysis: Analyze your data
collected
• Reports: Download reports of results

9. Qualtrics Survey Questions
• Qualtrics offers many different
types of questions, everything from
basic multiple choice to matrixes,
ranking, and more
• Additional options are available for
timing, custom validation, and skip
logic

10. Coding and Categorizing
• Your data will start to line up into
categories based on repetition,
comparisons
• Some people like tools like NVIVO which
help with coding digitally
• Others will use colors (markers or digital)
to show where themes intersect visually, or
make a table
• Use the method that makes sense for you,
and realize that it may take several passes
through the data
• Resources for coding methods:
• https://researchrundowns.com/qual/
qualitative-coding-analysis/
• http://programeval.ucdavis.edu/docu
ments/Tips_Tools_18_2012.pdf
• http://onlineqda.hud.ac.uk/Intro_Q
DA/how_what_to_code.php

11. Analyzing Data and Making Conclusions
• You’ll never be able to say with 100%
certainty that you proved something in
research
• But, you can usually say that there is good
evidence for a conclusion or
recommendation based on what you see so
far
• If it’s still unclear, get more data from
other sources and triangulate the findings

12. Digging Deeper
• Remember that it’s difficult to
generalize most SoTL research
conclusions
• Recommendations, though, can lead to
further research or to new literature
out there to review
• Often the process is cyclical and you’ll
find yourself going back, adding new
data sets, and confirming or altering
your conclusions over and over again

13. Your Turn!
• Where are you at so far with your project?
• Where do you need more help (more advice, more data, more questions)?
• What’s confounding you so far in the process?
• What questions do you have about analyzing data, getting tools to help, and
so on?

14. Places to Publish
• Annual SoTL Conferences: http://www.washington.edu/teaching/sotl-
annual-conferences/
• Great list of interdisciplinary and teaching journals:
http://www.fctl.ucf.edu/researchandscholarship/sotl/journals/
• Directory of Teaching Journals: http://cetl.kennesaw.edu/teaching-journals-
directory
• Directory of Teaching Conferences: http://cetl.kennesaw.edu/teaching-
conferences-directory

15. More Resources
• Vanderbilt SOTL “getting started” guide:
https://my.vanderbilt.edu/sotl/doing-sotl/getting-started/
• Guidebook to SOTL – thinking of a problem and the questions:
https://my.vanderbilt.edu/sotl/files/2013/09/1SoTLProblem4.pdf
• Faculty Center for Teaching and Learning at Univ. of Central Florida:
http://www.fctl.ucf.edu/ResearchAndScholarship/SoTL/
• Center for Engaged Learning at Elon University videos:
https://www.youtube.com/channel/UCNIm8Apo1feU73SPyxEXXgg

16. References
• Bishop-Clark, C, & Dietz-Uhler, B (2012). Engaging in the scholarship of teaching and learning.
Sterling, VA: Stylus.
• Boyer, E.L. (1990). Scholarship reconsidered: Priorities of the professorate. San Francisco, CA:
Jossey-Bass.
• Felten, P. (2013). Principles of good practice in SoTL. Teaching and Learning Inquiry: The
ISSOTL Journal, 1(1), 121-125.
• Hutchings, P. (2000). Opening lines: Approaches to the scholarship of teaching and learning. Menlo
Park, CA: Carnegie.
• Nelson, C. (2003). Doing it: Examples of several of the different genres of the scholarship
of teaching and learning. Journal of Excellence in College Teaching, 14(2), 85-94.