Invited presented for the 2015 Australasian Pharmaceutical Science Association (APSA) and Australian Society for Clinical and Experimental Pharmacology and Toxicology (ASCEPT) Joint Scientific Meeting (Hobart, Tasmania). Abstract: All graduates need the skills and habits of mind to solve the complex, unstructured problems they will face in the 21st Century workforce (Bybee & Fuchs, 2006). In science, analysing technical literature, identifying conflicts and gaps, developing relevant, testable hypotheses, collecting and analysing the evidence to these hypotheses, and putting forward reasonable, specific and qualified conclusions, is our bread and butter – the basis of scientific reasoning (Kuhn & Pease 2008). Research experiences and inquiry-based curricula aim to help undergraduate students develop these habits of mind and cognitive skills (Zimbardi & Myatt, 2012). In our inquiry-based curricula we have documented the development of students’ scientific reasoning skills (Zimbardi et al., 2013) and their understanding of the contestable nature of scientific knowledge (Zimbardi et al., in press). We have also developed a series of meta-cognitive assessment items which have reveal students’ ability to translate these learning outcomes into employability skills. Specifically, undergraduate biomedical science students in their final semester are provided with a job interview scenario and asked behavioural questions (e.g “Tell me about a time when you successfully used your scientific problem skills”) and hypothetical questions (e.g “Suggest a potential approach for investigating this issue…”). Students’ responses to these open-ended questions have revealed the diverse skill levels amongst the cohort in translating educational experiences to workplace situations. Notably, we have found several underlying assumptions and misconceptions that hinder students’ articulation of their employability skills, as well as useful models of specific, evidence-based, and convincing, approaches to answering such questions. Bybee RW & Fuchs B (2006) J Res Sci Teach 43(4): 349–352. Kuhn D & Pease M (2008) Cogn Instruct 26: 512–559. Zimbardi K et al (2013) Adv Physiol Educ 37 (4): 303-15. Zimbardi K et al (in press) IJISME Zimbardi K & Myatt P (2012) SHE 39 (2): 233-250

1. ASPA-ASCEPT2015Hobart, Tasmania
Translating
research experiences
into employability skills:
using evidence to make a
convincing case.
Kirsten Zimbardi & Kay Colthorpe
Educational Research Unit, School of Biomedical Science,
University of Queensland, St Lucia, QLD

2. ASPA-ASCEPT2015Hobart, Tasmania
Science education
for the 21st century workplace
Students need to develop the skills to deal with
novel, complex, unstructured problems
The Boyer Commission (1998) Reinventing undergraduate
education: a blueprint for America’s research universities.
National Research Council (2003) BIO2010: Transforming
undergraduate education for future research biologists
President’s Council of Advisors on Science and Technology
(2012) Engage to Excel: Producing one million additional college
gradates with degrees in STEM
Students need to learn to
‘think like a scientist’

3. ASPA-ASCEPT2015Hobart, Tasmania
What is “scientific thinking”?
aka scientific reasoning?
1. Existing knowledge used to build a hypothesis
• Previous evidence from scientific literature
2. Hypothesis includes a specific cause and
specific measurable outcome
3. Methods appropriate to test hypothesis
4. Results analysed and presented in relation to
hypothesis
5. Findings interpreted in relation to existing
evidence and knowledge
• Dealing with unexpected findings
Dunbar and Fugelsang(2006) The CambridgeHandbook ofThinking and Reasoning,pp 705,708

4. ASPA-ASCEPT2015Hobart, Tasmania
Australian Learning Teaching Academic Standards
Threshold Learning Outcomes
Inquiry and problem solving (TLO3)
Critically analyse and solve scientific problems by:
• Gathering, synthesising and critically evaluating
information from a range of sources
• Designing and planning an investigation
• Selecting and applying practical and/or theoretical
techniques or tools in order to conduct and
investigation
• Collecting, accurately recording, interpreting and
drawing conclusions from scientific data
Jones,Yates & Kelder (2011) Learning & Teaching Academic Standards Project:Science

5. ASPA-ASCEPT2015Hobart, Tasmania
Nature of scientific knowledge
a threshold learning concept in biology
Understanding science3 (TLO1)
Demonstrate a coherentunderstanding ofscience by
• Articulating the methods of science and explaining why current scientific
knowledge is both contestable and testable by further inquiry
1. Taylor etal (2011) Threshold Concepts in Biology;2.Zimbardi,Meyer etal (2014) NAIRTL;3.Jones,Yates &
Kelder (2011) Learning & Teaching Academic Standards Project:Science;
Hypothesis formulation is a threshold concept
in the biology1,2
Essentialto developing disciplinary expertise &
habits of mind

6. ASPA-ASCEPT2015Hobart, Tasmania
Undergraduate research experiences &
inquiry-based curricula
Embedding research experience in undergraduate curricula1
• Apprenticeships& Industry placements
• Inquiry & Methods courses
Vertically-integrated, inquiry-based practical curricula2,3
1. Zimbardi & Myatt(2014) Studies in HEd;2.Zimbardi etal (2013) Adv Phys Ed;3.Kirkup & Johnson (2013)
Good practice guide:Threshold LearningOutcome 3 - Inquiry & Problem Solving
Year 1
Sem 2
Yaer 2
Sem 1
Year 2
Sem 2
Year 3
Sem 1
Year 3
Sem 2
Scaffolded Guided
choice
Extended
choice
Cutting edge
methods
Professionalisation

7. ASPA-ASCEPT2015Hobart, Tasmania
What does the development of scientific
thinking skills look like – in assessment?
Students’ scientific writing (reports)
Early writing tended to use definitive terminology.
Later reports more often contained more speculative language.
Student 2:
Yr 2 Sem 1: “The results of this study confirmed the hypothesis...”
Yr 3 Sem 2: “This result suggests that...is most likely.”
Student 5:
Yr 2 Sem 1: “This study has proven that...”
Yr 3 Sem 2: “...there is no certainty that an association is present, however, the trend
observed is of interest.”
Colthorpe, Zimbardi et al (in press) IJISME

8. ASPA-ASCEPT2015Hobart, Tasmania
Evidence of scientific thinking development
Dealing with unexpected results
Student 4:
Yr 2 Sem 2: “As such the lack of effect observed in the current study may be
a result of the relatively low intensity and short duration for which subjects
exercised.”
Yr 3 Sem 2: “No change in the expression of LEP-R in the kidneys of rats
exposed to either vehicle or STZ was found (Figure 1)...Alternatively LEP-R
regulation in hypoleptinemic conditions may be via changes in the localisation
of the receptor. Soluble LEP-R is able to bind circulating leptin and delay
degradation (Huang, et al., 2001) as well as inhibit leptin signalling (Schaab,
et al., 2012). It is thus possible, that in conditions of hypoleptinemia, there
may be a proportional decrease in soluble LEP-R without any change in total
receptor expression.”
Colthorpe, Zimbardi et al (in press) IJISME

9. ASPA-ASCEPT2015Hobart, Tasmania
The development of
scientific thinking skills – in action…
UQ T&L Fellowship 2012
• Videos of 4 groups of students in a series of 3
inquiry-practical classes
• 3 interviews with each student across semester
• Students annotated their videos highlighting
inquiry skills in action
Zimbardi et al (in press) IJISME

10. ASPA-ASCEPT2015Hobart, Tasmania
Critical points of variation
in students’ ‘lived experience’
5 students working together in the same group
same experiment, same results, very different interpretations &
learning outcomes è phenomenographical study
Curriculum interacts with students'
prior experience & expectations
Critical ‘stuck’ points
Scientific knowledge is:
• concrete & complete
• NOT tentative, contestable & ‘gappy’

11. ASPA-ASCEPT2015Hobart, Tasmania
The development of scientific thinking
skills – in students’ own words
Meta-cognitive awareness of learning strategies and difficulties
improves academic performance1
Meta-cognitive assessmenttasks that prompt students to:
• think about their learning gains and strategies2
• translate educationalexperiences into workplace scenarios
Behaviouralquestions
e.g. “Tell me about a time when you successfully used your scientific problem skills”
Hypotheticalquestions
e.g “Suggest a potential approach for investigating this issue…”
1. Hattie (2009) Visible learning;2.Colthorpe etal (in press) J.Learning Analytics

12. ASPA-ASCEPT2015Hobart, Tasmania
Hints in the questions
• Deal with complex,
unstructured, novel
problems
• Break the problem into
testable chunks
• Use an evidence-based
approach to the solution
• Arrive at the best
solution at a given time

13. ASPA-ASCEPT2015Hobart, Tasmania
Methods
3rd year (final semester) students from 2013 - 2015
Mix of science/professional career paths
1-2 Meta-cognitive tasks during semester
Behavioural question (2014: n = 78)
Hypothetical questions (2013-2015: n = 399)
2-3 scenarios – near transfer to far transfer
Inductive thematic analysis
Qualities of convincing answers
Critical stuck points

14. ASPA-ASCEPT2015Hobart, Tasmania
Evidencing scientific problem solving
- the devil is in the detail
No transfer: I think that most situations in a lab require scientific inquiry
Parroting back the question: The solution, I found, was to take science as it is and only in sizeable,
digestible chunks that I can deal with seperate from the rest of everything.
More detail in setup, but still parroting back the question: When I was doing work experience in
Tanzania, I was exposed to a myriad of medical mysteries, ranging from fascinating, to simply
disturbing. Each new patient I was introduced to brought with them a problem that required a number of
doctors, medical students and volunteers to put their heads together [complex, collaboration] and come
up with a solution. For me though, I have had minimal exposure to much of the pathologies of third world
medicine [novel], and my biggest problem was maintaining a vertical stance, and not offending the
patients by my fainting/vomiting/absence. However, I reminded myself of skills I have acquired through
practical attendance at UQ. My rather new ability to objectively observe a situation, ask the right
questions and cross-check facts with others and literature made me feel as though my presence in the
hospital was actually beneficial for all involved.
Still lacking specific details – what & how?

15. ASPA-ASCEPT2015Hobart, Tasmania
Too many details derail the relevance
In BIOM2012 systems physiology, a course at UQ, during the practical component, we conducted an
experiement to examine the effects of caffeine on skeletal muscle. Although we had reviewed prior
research, and it had found a significant effect, we did not find any significant results in our data.
Consequently we re-read the research we had found and had to go searching for further articles which
may tell us as to why we did not observe significant results. Through further research we discovered that
the dosage that we gave in the practical was insufficient to observe the positive effects of caffeine on
exercise performance. This information enabled us to effectively explain our results and suggest future
avenues of research.
Completing a 3rd year microbiology subject at the University of Queensland me and my research group
were ask to determine experimentally if similarities between the metabolomic profile of brewing yeasts
can be used to infer their genetic relatedness. We established a number of cultures of 4 separate yeast
strains in the same media [controls] which would be analysed via GC-MS to produce a metabolomic
profile. Rapid PCR [methods] was used to determine if there were any obvious genetic differences. The
genome of the yeast strains were sequenced and mapped to the laboratory yeast strain [standards] to
identify genetic differences. Also a gene implicated [?] in the production of a specific metabolic was
analysed in each yeast strain (both metabolomic profile and protien translation) to identifiy any
differences.
Lack of explicit links/transfer to workplace

16. ASPA-ASCEPT2015Hobart, Tasmania
Workplace transfer with specific hypothesis,
methods and tentative solution
Coming from a nursing background, I was once with an elderly male patient who was admitted for
repeated falls. These falls happened almost exclusively in the morning. This patient had multiple co-
morbidities, however did suffer from long standing hypertension [identify/narrow problem]. After
conversing with the patient for a space, and reviewing his medication chart [evidence], it came to my
attention that almost his entire regimen of anti-hypertensive medications (there were at least 4) were
prescribed in the morning [hypothesis]. I took this man's blood pressure in regular intervals (after these
morning medications were administered) standing up for approximately 5 minutes and noticed a
considerable [quantity] decrease in his systolic pressure [evidence]. Additionally, he became
symptomatic when standing [sufficient/necessary test]. It had become obvious that his symptoms were
related to this systolic decrease when standing (postural hypotension), and so I discussed this with the
medical registrar and suggested [tentative, collaborative] at least a portion of his anti-hypertensive
medications be changed to night time just before sleeping, to eliminate these symptoms in the morning.

17. ASPA-ASCEPT2015Hobart, Tasmania
Evidence-based predictive modeling
- in the workplace, linked to education
An example of such a situation was during my time as acting store manager when I had to compile an
ordering sheet to order stock for the next week. I understand this may seem like a situation that
really doesn't use science inquiry skills but I believe that my studies in science had provided the
thinking skills necessary to overcome this problem. Initially I began with some very background
thinking, akin to beginning a scientific investigation, it was going to be school holidays and it was
predicted to rain, meaning it was going to be busier than usual and thus I needed to order more stock
than we had ordered in the previous week, but I didn't know how much [narrowing problem]. I decided
that if I was going to know how much to order, I had to know how much stock we currently had in the
first place. From there, I compared my count of stock to my manager's from the week before and put a
baseline order that matched up with what she had put based on the amount of stock. From there I
looked at product sales from previous Saturdays, as I knew that school holidays usually do the same
sales per day as a Saturday [equivalent model] and found which products we had been selling the
most of and made a note to order more of those than usual [specific]. From there I also took in to
consideration the fact that it was going to rain so I decided to order 2 extra of everything to ensure we
didn't run out because we could just freeze escess stock. In the end my ordering was sufficient [final
check on success] and I believe it was my knowledge in scientific process and analysing of data to
make conclusions that allowed me to overcome this problem.

18. ASPA-ASCEPT2015Hobart, Tasmania
Conclusions & Implications
Meta-cognitive tasks are an easy/efficient way to prompt
students to:
• Practice articulating their answers in a low stakes situation
• Reflect on their learning gains and strategies
• Think abouthow to evidence ‘soft skills’
Good articulation of answers to interview questions
• Use specific, detailed, relevant evidence
• Include explicit links showing relevance to question/context
• Probably requires practice for many students
• may benefit from seeing analyses like these

19. ASPA-ASCEPT2015Hobart, Tasmania
Thank you
Funding
UQ T&L Fellowship
UQ SoTL Early Career Grant
Thank you to all of our students
Kirsten Zimbardi
k.zimbardi@uq.edu.au