Generative AI in Higher Education Workshop Crash Course
1.
‘
1
1
Generative AI inHigher Education:
A Crash Course
for Faculty and Staff
Amie Cohen amie.cohen@stonybrook.edu
Luis Colón luis.colon@stonybrook.edu
Jenny Zhang yi.zhang.13@stonybrook.edu
Center for Excellence in Learning and Teaching (CELT)
2.
‘
2
Topics
1
2
3
4
Introduction of GenerativeAI tools
How to communicate with students about the pros and cons of
ChatGPT and other Generative AI
Use AI as a pedagogical tool for teaching, learning, and
engagement
How to update syllabus with AI statements
5
How to modify assessments to be AI-proof, and the potential risk
and opportunities of assessments
3.
‘
3
What is ChatGPT?
(accordingto ChatGPT)
• ChatGPT is an AI large language model (LLM) developed by OpenAI
for generating human-like text in conversations. It's based on the GPT
(Generative Pre-trained Transformer) architecture and is fine-tuned to
understand and respond in a dialogue format.
• Generative AI is a type of artificial intelligence that can create new
content, like text, images, video, or music, that looks and sounds
similar to what humans would produce. It learns patterns from
existing data and uses that knowledge to generate new output.
‘
6
What is anappropriate use of AI?
(Jamboard Activity)
7.
‘
7
How Should WeTalk to Students
about Generative AI?
• Including an AI statement in your course syllabus and reviewing the statement
with your students at the start of the course can be an effective way to begin the AI
discussion.
• When talking to students about AI, it should be in a clear, unambiguous way in order
to properly communicate the ethical and intellectual implications of using AI
technologies.
• Consult with your TAs and discuss the use of generative AI and consider where
students may attempt to use generative AI in their coursework
• AI is here to stay. It is important that students develop critical thinking skills and are
reflective in how they use AI technologies.
8.
‘
8
What is anAI syllabus
statement?
• In addition to an academic integrity statement in your syllabi, it is valuable to also
have a syllabus statement regarding AI
• An AI syllabus statement addresses the expectations set for the course regarding the
use and/or misuse of Generative AI
• All courses will be different and as such there may not be one syllabus statement that
is appropriate for every course depending on the context and intended use of
generative AI.
• In addition to this statement, it is also important to note within specific assignment
descriptions whether the use of generative AI is permitted or not permitted.
9.
‘
9
Sample AI SyllabusStatement
Generative AI: Generative AI (such as ChatGPT) can be a valuable resource when used
properly and responsibly. Accordingly, you should research/know its limitations before
using it in coursework. For instance, it can sometimes help you phrase ideas but it is very
bad at generating ideas or solving problems. For [course number], you are generally allowed
to use generative AI, and may even be asked to do so on occasion. However, you may not
simply copy and paste content from generative AI into an assignment unless instructed
otherwise; if used, it must be quoted and attributed. Failure to follow this policy will result in
referral to the academic judiciary.
- Sample Classroom Policy from Matthew Reuter, Department of Applied Mathematics
and Statistics, Stony Brook University
10.
‘
10
Sample AI SyllabusStatement
The use of generative AI tools (e.g. ChatGPT, Dall-e, etc.) is permitted in this course for the following activities:
● Brainstorming and refining your ideas;
● Fine tuning your research questions;
● Finding information on your topic;
● Drafting an outline to organize your thoughts; and
● Checking grammar and style
The use of generative AI is not permitted in this course for the following activities:
● Impersonating you in classroom contexts, such as by using the tool to compose discussion board
prompts assigned to you or content that you put into a Zoom chat
● Completing group work that your group has assigned to you, unless it is mutually agreed upon that you
may utilize the tool
● Writing a draft of a writing assignment
● Writing entire sentences, paragraphs, or papers to complete class assignments.
You are responsible for the information you submit. Your use of AI tools must be properly
documented and cited in order to stay within university policies on academic honesty.
- Center for the Advancement of Teaching, Temple University
11.
‘
11
AI Detectors:
Effective orIneffective?
As the interest in generative AI has grown, so has the interest in effective AI detection solutions especially
in academia. While there are AI detection resources out there, they might not be the best option in some
cases:
• Muhammad Abdul-Mageed, a professor who supervises research in natural-language processing
and machine learning at the University of British Columbia expressed that, “It’s particularly difficult
to detect AI-generated text because the whole point of AI language models is to generate fluent
and human-seeming text, and the model is mimicking text created by humans”
• Upon release, OpenAI (who produced ChatGPT) released a detector that could only correctly identify
26% of AI-written text as “likely AI-written.”
• Vanderbilt University has disabled Turnitin AI detection until further notice after the company
reported a 4% false positivity rate. In 2022, they received 75,000 papers so a 4% false positive would
mean that around 3,000 papers would be unintentionally marked as AI generated when they were
not.
Heikkilä, M. (2023, February 7). Why detecting AI-generated text is so difficult (and what to do about it). MIT Technology Review; MIT Technology Review.
https://www.technologyreview.com/2023/02/07/1067928/why-detecting-ai-generated-text-is-so-difficult-and-what-to-do-about-it/
12.
‘
12
AI Detectors:
Effective orIneffective?
In the article GPT Detectors are Biased Against Non-Native English Writers, Weixin Liang et al. (2023)
explore different generative language model detectors and explore how they respond to writing samples
from both native and non-native English writers.
• The detectors demonstrated near-perfect accuracy for US 8th grade essays but misclassified over
half of the TOEFL essays as "AI-generated" (average false positive rate: 61.22%). All seven detectors
unanimously identified 18 of the 91 TOEFL essays (19.78%) as AI-authored, while 89 of the 91 TOEFL
essays (97.80%) are flagged as AI-generated by at least one detector.
• They found that many generative language model detectors misclassified non-native English writing
to be AI-generated, while finding native English writing samples to be more accurately identified.
• Ultimately, it was determined that while employing strategies to effectively develop prompts can
help to mitigate the bias and bypass the detectors. This means that these detectors may
unintentionally penalize writers with constrained linguistic expression such as non-native English
writers.
Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., & Zou, J. (2023). GPT detectors are biased against non-native English writers. ArXiv.org.
https://arxiv.org/abs/2304.02819
13.
‘
13
What kind ofAI guidance
can we provide?
Add an AI/ChatGPT
statement in your
syllabus
Take an AI writing
bot/ChatGPT for a
test drive
Teach students the
benefits and
limitations of using
AI writing
bots/ChatGPT
Think about AI
writing
bots/ChatGPT as a
tool
14.
‘
14
How can weuse AI in the classroom?
Rubric Calibration
with AI Writing
Drafting
using AI
AI Thesis
Revision
AI Feedback
The instructor can use AI to generate either an essay, thesis, or other type of written
work. Groups of students can then use a rubric to evaluate the AI’s work and suggest
changes or improvements to the rubric. This can help students think about how they
define high-quality work and how helpful rubrics can be in identifying AI-generated work.
Students can use AI to draft a writing assignment. Students can then analyze the AI’s
writing, focusing on accuracy, bias, or other characteristics important to the style of
writing appropriate for your course. Students can also be asked to improve on the draft
and complete a second draft.
Students can use AI to develop thesis statements on the topic of their study and can
revise these statements as a way to “get started” with their own writing process. They can
present their findings as well as the prompt(s) they used with their peers for feedback.
Students can ask AI to provide feedback on an initial draft of a writing assignment they
are working on focusing on a specific element or stylistic device. Students can use this
feedback to improve their writing and receive another perspective to work off of.
Schmidli, L. (2022). Considerations for Using AI in the Classroom. L&S Instructional Design Collaborative. https://idc.ls.wisc.
edu/guides/using-artificial-intelligence-in-the-classroom/#:~:text=Integrating%20AI%20into%20the%20Writing,or%20improve
ment%20to%20the%20rubric.
15.
‘
15
Assessment and GenerativeAI
Challenge: Do our assessment tasks measure students abilities as they relate to
our course learning outcomes, or reflect their ability to use generative AI?
Options for student use of AI:
1. assessment tasks that prohibit the use of AI
2. in-person assessment that does not use AI
3. authentic assessment tasks that may use AI as outlined by the instructor
Recommendation: limit to quick formative
assessments (e.g. minute paper)
16.
‘
16
Test your assessmenttasks:
1. Enter a prompt for one of your written
assessment tasks into ChatGPT:
a. What is produced?
b. Would the work pass?
2. Regenerate the response at least once
3. Add prompts you might provide as
feedback to a student:
a. Cite the experts in the field
b. Include only peer-reviewed
resources
c. Explain your process step by step
d. Add more information about xxx
https://chat.openai.com
Include the strengths and weaknesses of each type of assessment and cite at least 5 scholarly references
us assessment methods, formative and summative assessments stand out as fundamental tools for gauging student progress and achievem
and instructors to improve learning outcomes…
17.
‘
17
Ways to address(or not address)
AI through assessment redesign
Ignore Ban Invigilate
Embrace
Design Around Rethink
18.
‘
18
As always, leveragePractices that
decrease the motivation to cheat
• Lower stakes assessments, incorporate multiple sections/components of a paper/project worth
fewer points
• Multiple drafts/opportunities for feedback and improvement
• Open discussion about academic integrity and AI; reiterate your policy for each assignment
19.
‘
19
Questions Raised aboutAI
What is a appropriate use of AI by students?
How should educators speak with students about AI?
How can AI be used in the classroom?
How will learning be assessed in the age of AI?
What equity issues may arise with access to AI?
How can instructors leverage AI to optimize their workflow?
20.
‘
20
Upcoming Sessions withCELT
AI Ethics in Teaching and Learning - A Panel Discussion
Feb 20, 2024 at 2:00 PM on Zoom
Faculty Conversations: Sharing Ideas through Discussion
Feb 27, 2024 at 11:00 AM on Zoom
Cohosted by CELT & OEE
Generative AI in Higher Education: A Deep Dive into Assessment Redesign
Feb 29, 2024 at 1:00 PM on Zoom
Inclusive Pedagogy Panel Discussion (with FITA): Storytelling and
Positionality as Inclusive Pedagogy
Friday, March 1, 2024 at 11:30 AM on Zoom
Formative Assessment as a Means to Student Engagement & Active Learning
March 20, 2024 at 1:00 PM on Zoom
Generative AI Boot Camp: How can AI make your life as a professor easier?
March 28, 2024 at 1:00 PM on Zoom
Scan for CELT’s
Full Calendar of
Events!
21.
‘
21
Generative AI inHigher Education: Faculty Showcase
This will be an in-person event! Date/Time is TBD
Interested in participating or attending?
Complete our interest form!
https://bit.ly/CELTAIshow
‘
23
Resources
● CELT: AIBots Teaching Resources
● AI Tool List
● LibGuide: AI Writing
● Classroom Syllabi Policies for AI Generative Tools
● Artificial Intelligence - US Office of Educational Technology
● Library AI workshops for students
#1 Good afternoon,
Welcome everyone! Thank you for joining us today for CELT AI workshop. Let’s introduce ourselves: Amie, Luis, Jenny
with lots of opportunities and challenges in the generative AI field, as educators, we might have a few burning questions: how do I deal with this generative ai thing in my class? what are the best practices? how to talk to my students about that? hopefully we can help addressing some of them.
#2 as we said, this is a crash course, if you want to learn more, feel free to email us at celt@stonybrook.edu, make appointment with us for consultations. we can totally work with you individually or in a small group.
#3 It can understand and generate text in multiple languages and can assist users in tasks like answering questions, summarizing documents, providing explanations, offering suggestions, writing articles, and telling jokes. ChatGPT is one of the most popular one, there are other generative Ai tools. I will demonstrate some later.
Language models like ChatGPT are trained on a vast amount of text data from the internet, allowing them to learn patterns, grammar, and contextual information. By processing input text and generating coherent and contextually appropriate responses, ChatGPT can engage in conversations on a wide range of topics.
#5 Elicit
https://elicit.org/
Grading
https://help.packback.co/hc/en-us
AI detector
#6 Luis
which of the uses would you consider cheating?
which uses are relevant to our students’ future?
which uses would you use in your work as a professional?
https://itali.uq.edu.au/teaching-guidance/teaching-learning-and-assessment-generative-ai?p=3#3
#7 It is always best to speak about AI in a manner that is clear and leaves little room for interpretation.
Acknowledgement of AI from the beginning through language dedicated to AI in your syllabus and discussions with students regarding AI early in the course are suggested.
#8 Having an AI syllabus statement and communicating this policy with students is an effective way to have a conversation regarding AI early
Courses are different and you may find that a standard AI syllabus statement may not apply to all courses. They may need to be modified and adjusted to meet the expectations of other courses.
If there are assignments throughout the semester that you anticipate students may try to use AI for, specifically note within the assignment description of those assignments the policy regarding AI usage.
#11 While AI detectors may seem to be the appropriate response to student’s usage of AI, prior experiences show that they may not be the answer (at least not yet)
Since AI is designed to create writing that mimics fluent and human-seeming language, this makes it very difficult to detect.
OpenAI, the company that produced ChatGPT, created a detector that could only identify 26% of AI-written text as “likely AI-written”
Vanderbilt University disabled their Turnitin AI detection as they realized that a 4% positivity rate is a large number of students that can likely be accused without reason. Since 2022, if this AI detector had been used around 3,000 papers would be unintentionally marked as AI generated when they were not which is a large number.
#12 There is a bias when it comes to using AI detectors against learners where English is not their first language
A large majority of the papers were flagged as AI-generated when they were not
#16 My account example:
https://chat.openai.com/c/ccaa4d53-ff1b-4f6a-878d-69328385042a
#17 Assessment redesign for generative AI: A taxonomy of options and their viability
Strategies for Creating “Generative AI-Resistant” Assessments
active learning, collaborative learning
#19 JZ
It is always best to speak about AI in a manner that is clear and leaves little room for interpretation.
Acknowledgement of AI from the beginning through language dedicated to AI in your syllabus and discussions with students regarding AI early in the course are suggested.
Most read
![‘
9
Sample AI Syllabus Statement
Generative AI: Generative AI (such as ChatGPT) can be a valuable resource when used
properly and responsibly. Accordingly, you should research/know its limitations before
using it in coursework. For instance, it can sometimes help you phrase ideas but it is very
bad at generating ideas or solving problems. For [course number], you are generally allowed
to use generative AI, and may even be asked to do so on occasion. However, you may not
simply copy and paste content from generative AI into an assignment unless instructed
otherwise; if used, it must be quoted and attributed. Failure to follow this policy will result in
referral to the academic judiciary.
- Sample Classroom Policy from Matthew Reuter, Department of Applied Mathematics
and Statistics, Stony Brook University](https://image.slidesharecdn.com/aiworkshop1crashcourse-250624140032-5061d302/85/Generative-AI-in-Higher-Education-Workshop-Crash-Course-9-320.jpg)
Most read
