1) The document discusses an introductory course on social informatics that uses generative AI to engage students' critical thinking skills through assignments that have them evaluate AI outputs. 2) One assignment had students apply prompts to ChatGPT 3.5 and evaluate the usefulness and validity of the information provided. Students assessed outputs for credibility using methods like CRAAP. 3) The document concludes that while generative AI can help with understanding and brainstorming, it has limitations and students must evaluate outputs, as AI cannot replace critical thinking or application of knowledge. More work is still needed to make students aware of AI limitations and biases.

1. Dr Andiswa Mfengu
Andiswa.Mfengu@uct.ac.za
Department of Knowledge & Information Stewardship
Generative AI based assessment to
engage student’s critical skills in the
Humanities
22 November 2023

2. Intro to Social Informatics (LIS1000S)
• Servicing UG course – elective course
• First offered in 2023 – blended course
• Part of Khanyisa Courses in the Humanities
• Khanyisa courses - enriched pedagogy that focuses on the development of
critical Humanities skills; and innovative teaching and delivery and
assessment methods
• Focuses on equipping 1st year students with literacies to support their
performance in other courses
2

3. AI literacy
• Information literacy or competency is an essential skill, that students
need to find, evaluate and use information
• Misinformation, fake news, and disinformation - information literacy
has become a vital skill for students
• AI literacy, are important skills for students in today’s connected society
– to navigate abundance and proliferation of digital technologies
• Navigate the information and related spaces ethically and effectively
• Assessment as learning
3

4. AI literacy practical
Practical assignment which gave students prompts to apply then evaluate the
outputs (ChatGPT 3.5)
• Prompt – What is the importance and implications of Social Relief of Distress
Grant (SRD) Grant
• Is the information provided useful, in what way, and if not, explain?
• Is the information valid and credible
• Prompt - What is your view on SRD grant?
• Prompt – Design a presentation for SRD grant
• What do you think of the presentation?
• What are your thoughts with using generative AI in your personal and academic
space?
4

5. Usefulness of AI output
5
Importance and implications of Social Relief of Distress Grant
(SDR) Grant

6. 6
The information is valuable because it provides a complete
breakdown of the implications.
It provides a broad understanding of SRD grant.
The information provided is useful as it clearly lays out the
importance and implications of the social distress grant. It
formats it in a way that is easy to understand with clear
headings.
Usefulness of AI output

7. 7
The information provided was useful as it
clearly highlights both the importance as
well as the implications of the SRD.
The information is useful as it provides a good
summary of the topic given through finding
patterns within multiple different sources.
Useful as it provide many different definitions
and ways to understand what SRD is.
Usefulness of AI output

8. Validity and credibility
• “Yes, the information can be valid and credible
to some extent.”
• “The information is not valid nor is it credible
as this was generated by an AI Language model.
AI does not have access to reputable sources
such as government publications, academic
journals”.
• “The information is based on internet documents
dating back from before 2021.”
• “The information is credible; however, the South
African context is missing in the response
generated by ChatGPT.”
8

9. Validity and credibility
Evaluating the information provided using CRAAP method:
• Currency: This information is sourced from data up until September 2021.
This could negatively affect the usefulness of the information
• Relevance: this may be more relevant than information gained through
colloquial conversation but it is not an academic piece of work that has gone
through intensive academic practice of research, referencing, etc.
• Authority: not clear, as ChatGPT is not an expert in any discipline.
• Accuracy: The response was succinct, non-emotional and written in a
factual manner, however no use of citations and references which leads us
to be dubious of the information provided.
• Purpose: This information was generated to inform me, the user, as a
response to my prompt.
9

10. 10
SRD grant presentation

11. SRD grant presentation
Thought that the presentation outline was thorough, well organised, in a
logical order and succinct.
I believe that this is a good suggestion for a presentation.
The outline is well-structured and follows a good logical sequence of steps
It gives a well-planned outline of the presentation. It gives you a focus point for
each slide. I think its best that it does not fully do your presentation for you but
instead acts as a guideline that helps you get stared if you had no idea how.
11

12. Thoughts with AI
use
12

13. 13
For personal use, I would use ChatGPT to broaden my understandingof
certain concepts and topics. For academic use, I would not use ChatGPT
to write essays or to do assignments because it does not have access to
peer-reviewedarticles or real-time data. This would result in inaccurate
information and plagiarism in the academic tasks. I would however
consult ChatGPT if unsure of how to approach a certain task or require
an explanationof the task to help me betterunderstandwhat is
requiredof me.
ChatGPT can be used as an effectiveacademic tool in terms of structure
and planning. However, it is not effective to generate entire essays, as
the information provided can be out of date, lacking context,and
ethicallywrong, as the source of ChatGPT information is unknown.
Opportunities with AI use

14. 14
It has a wide range of uses if applied correctly from helping
formulate recipes, to summarising long pieces of text to
helping planning your day and weekly schedule
Very helpful, can reduce the amount of time required to
complete tasks, especially technical tasks like programming. I
recently computed a chrome extension to alert me of Jammie
Bus schedule without the help of ChatGPT I would not have
been able to do this.
I believe that the use of generative AI can be incredibly
helpful. Specifically, as a starting point.
Opportunities with AI use

15. 15
If you are struggling with understanding certain topics or
struggling to understand an assignment question, ChatGPT can
provide you with a well-structured breakdown and outline to
help you better understand the work.
Generative AI can help lead you in the right direction and
brainstorm, however this is as far as I would use Generative AI
for academia. It is limited for any academic use beyond the
above, this is due to the accuracy, currency and most
importantly relevance of information provided. Without proper
references and citations one cannot rely on generative AIs.
Opportunities with AI use

16. 16
I think while it is an easy way to do things, it could take away the
learning experience both personally and academically. I have
noticed that it makes me be dependent on its tools instead of using
my own intelligence and abilities to formulate new ideas that are
innovative.
The tool is helpful; but it does have its flaws.
I do however fear the accuracy as it not guaranteed that the
information generated by the AI tool is always 100% accurate and
authentic.
When in an academic position, I think it is best to acknowledge that
the AI tool does have limitations.
Overreliance on generative AI for academic work can lead to issues
of plagiarism and raise ethical concerns.
Concerns with AI use

17. 17
Its great yet feels like cheating as it will not help with developing
one’s academic voice but rather creates dependence.
In the case of Grammarly another AI text generative tool, that is a
writing assistant, it can improve one’s experience in personal and
academic as it acts as a proof-reader and helps one notice things
they may have overlooked. However, not all AI are completely
free, as they have a limited free version for everyone to use and
have access to.
This also raises the issue of privacy as one must create an
account to store their work and generated content but may be
unaware of third parties using the data in the created accounts.
These tools are not only services but businesses as well.
Concerns with AI use

18. How would you describe generative AI
18

19. 19
How would you describe generative AI

20. Conclusion
• Useful to assist students with “know what”
• Brainstorming
• Understanding
• Summarising
• But is not able to “what how”
• Critical thinking
• Application
• Evaluating the outputs is important
• Some students seem to be aware of the limitations and biases – more
work still to be done
20