The Gentle Art of Questioning
                     WRITING GREAT CLICKER
                           QUESTIONS




      Dr. Stephanie V. Chasteen
           Physics Department
                    &
       Science Education Initiative
      Univ. of Colorado at Boulder

       http://colorado.edu/sei



Web and blog: http://sciencegeekgirl.com
Email: stephanie.chasteen@colorado.edu
Who are you?

What is your primary role?
                                  Show of hands

A. STEM faculty
B. Administrator
C. Teacher educator
D. Education researcher
E. Graduate student or post-doc
F. Other
Have you used response systems (clickers) in
                your teaching?

                                Take a clicker & turn it on
                                If the green light flashes, your
                             vote has been counted


A. Not at all, and I haven’t seen them used
B. Not at all, but I’ve observed their use somewhat
C. I’ve used them a little
D. I’ve used them a lot
E. I could be (should be?) giving this workshop
How familiar are you with Mazur’s “Peer
                  Instruction”

                                                Colored cards




A. Fairly familiar, and I like it
B. Fairly familiar, but I’m not sure that I like it
C. I’ve heard of it but only have a vague idea what it is
D. Not familiar at all
E. Not sure
Introducing Me
                                    5


Science Education Initiative
    http://colorado.edu/SEI
    Applying scientific principles to improve science
    education – What are students learning, and which
    instructional approaches improve learning?

Physics Education Research Group
    http://PER.colorado.edu
    One of largest PER groups in nation, studying
    technology, attitudes, classroom practice, & institutional
    change.
 Blogger
     http://blog.sciencegeekgirl.com
Why question?
                             7




Credit: Rosie Piller                   whiteboard
Agenda
                                        9

1. When and how we can ask questions
2. About clickers as a way to ask
   questions, including some common challenges
3. Writing good questions. Example
   questions, writing our own.
4. Action plan

 Learning goals: Participants will be able to….
 A. Explain several benefits of questioning and of using clickers to question
 B. Defend the use of best practices in questioning to overcome common
    challenges
 C. Formulate and revise clicker questions to target student learning goals
Exercise #1: Question brainstorm
                           10

 What questions could you ask to help students
  achieve your assigned learning goal -- to test mastery
  and stimulate learning?
 Brainstorm as a group



                                      5 minutes




                                                whiteboard
When can we ask questions?
                                    11

       BEFORE
Setting up instruction                                     DURING
     Motivate                                             Developing
                                                          knowledge
     Discover
     Predict outcome                                Check knowledge
     Provoke thinking                               Application
     Assess prior knowledge                         Analysis
                                                    Evaluation
                                                    Synthesis
     AFTER         Relate to big picture            Exercise skill
    Assessing      Demonstrate success              Elicit misconception
    learning
                   Review or recap
                   Exit poll
                   Credit: Rosie Piller and Ian Beatty.
We can ask questions in lots of ways
                             12

  Ask rhetorically
  Target the class (how?)
  Target someone in particular (in what order?)
  Answer your own question
  Leave the question unanswered


 Or ask out of class
  Blogs
  Discussion boards
  Homework…
Credit: Rosie Piller
Why use clickers to target the class?
 An outline of Peer Instruction.
                  13
Clickers are a tool for questioning
                  14




       But not a magic bullet!
Anatomy of Peer Instruction
                           15




                   Ask Question
…Lecture…                               (Maybe vote)


Class Discussion                      Peer Discussion


                        Vote

         * See also: Peer Instruction, A User’s Manual. E. Mazur.
How is a clicker question the same or different?*
                              16
                                   * From other types of in-class questions
 Similar in terms of goals
 Multiple choice
 Anonymous (to peers)
 Every student has a voice – the
  loud ones and the shy ones
 Forced wait time
 You can withhold the answer
  until everyone has had time to
  think (choose when to show the
  histogram)
                                              What does this tool help
                                                    us to do?
U. Colorado clicker resources…
                               17


Videos of effective use of clickers          2-5 mins long
           http://STEMvideos.colorado.edu




Clicker resource page
          http://STEMclickers.colorado.edu

         • Instructor’s Guide
         • Question banks
         • Workshops
         • Literature / Articles
Which of these could be clicker questions?
                                    18

       BEFORE
Setting up instruction                                     DURING
     Motivate                                             Developing
                                                          knowledge
     Discover
     Predict outcome                                Check knowledge
     Provoke thinking                               Application
     Assess prior knowledge                         Analysis
                                                    Evaluation
                                                    Synthesis
     AFTER         Relate to big picture            Exercise skill
    Assessing      Demonstrate success              Elicit misconception
    learning
                   Review or recap
                   Exit poll
                   Credit: Rosie Piller and Ian Beatty.
Let’s try it
                                             19



Which superpower would you
rather have? The ability to…

  A. Change the mass of things
  B. Change the charge of things
  C. Change the magnetization of things
  D. Change the boiling point of things

                                                  19
      Question: Ian Beatty, UNC Greensboro             Image: Thibaultfr on Wikimedia
Example question: Math

Your sister in law calls to say that she’s having twins. Which of
  the following is the most likely? (Assume she’s having
  fraternal, not identical, twins)

A.    Twin boys
B.    Twin girls
C.    One girl and one boy
D.    All are equally likely




                                                         20
Derek Bruff, Vanderbilt
Example Question: Survey
                                 21

Which of the following are you least comfortable using to solve
 problems?

A. Kinematics
B. Newton’s Laws
C. Work-Energy Theorem
D. Momentum-Impulse Theorem
E. Angular Momentum-Angular Impulse Theorem




                                               Ian Beatty, UMass Amherst
Let’s try it

   I think the toughest thing about using clickers
         and peer instruction in class will be:
A. Writing good questions
B. Getting students to really think about them
C. Getting students to discuss the questions
D. Getting students to share their ideas in the class
   discussion
E. It takes too long / I have a lot of content to cover
              Or something else? Write it on your handout!
           What are some *solutions* to some of these challenges?
Some “best practices” and solutions


            We’ll do these quickly
            These are repeated in handouts



 Underlying philosophy to most of these “best
  practices” are:
“It is important for students to discuss their
  ideas, to feel safe doing so, and for the
  teacher to be aware of those ideas”
1. Ask Question
                        26

What are some challenges/ things to consider
when posing a clicker question?




               •Ask several times during lecture
               •Ask challenging, meaningful questions
               •Don’t post until ready
               •Give time to read (read silently)
               •Don’t read question out loud
                                                          26
                               Handout/worksheet / whiteboard
2. Peer Discussion
                         27

                           • Students learn more deeply by
                             teaching each other
                           • Makes them articulate answer
                           • Lets you see inside their heads
Why is peer discussion
important?       What are challenges /
                  how can you help make it work?


                  •Make it clear why you’re doing this
                  • Circulate and ask questions / model
                  •Use questions they want to discuss
                  •Allow enough time (2-5 mins)
                  •Focus on reasoning in wrap-up
Student buy-in is key!
3. Wrap-Up Discussion
                                29

                  Challenges?
                  What might you
                  do to facilitate an
                  effective wrap-up
                  discussion?
•Establish culture of respect
•Consider whether to show the
  histogram immediately
• Ask multiple students to defend their
  answers
• Why are wrong answers wrong and
  why right answer is right
Giving the answer stops student thinking!




                   31
Preparing to Write Questions

 Read briefly over the “tips for writing clicker
  questions” handout. Which is going to be most
  challenging for you?
 Then, in groups of 2-3, choose one of the questions
  that you brainstormed at the beginning of the
  workshop. You will write a multiple choice version
  of this question (but not yet!)



                                        3 minutes
Gallery Walk

 As a table, look at the “example questions” trio that I
  have given you. What’s a common theme(s)?
 Write the themes you find down on the sheet so that
  other groups will be able to read it.


 After 5 minutes, circulate to see the themes of
 questions on other tables.
 Shop for ideas for your own questions!
 See handouts for a place to jot your notes.

                                           10 minutes
Question brainstorm
Exercise #3: Writing Questions
                           35


 Using ideas you’ve learned, write a multiple choice
  version of your question in groups of 2-3.
 Look on the “Sample Questions” handout for some
  examples.
 Show your question to another group (and to me) for
  suggestions on revising it.
 If you have time, write another question from
  another part of the questioning cycle.

                                      10 minutes
Action Plan
                          36

 Take a few minutes to write down your action plan to
 implement ideas you heard about in the workshop
References & Resources

Web and blog: http://sciencegeekgirl.com
Email: stephanie.chasteen@colorado.edu                                     Thanks!
 Clicker Resource Page from the Science Education Initiative:
    http://STEMclickers.colorado.edu. Has clicker question banks (in the sciences), an instructors’ guide, and
    videos of classroom use. Useful books (such as Eric Mazur’s Peer Instruction are cited there.

 Workshop handouts will be uploaded to the above website, or email me.

   Many materials in this workshop (particularly the questioning cycle and the participant exercises) were
    adapted fromRosie Piller, Making Students Think: The Art of Questioning. Short papers published in:
    Computer Training & Support Conference, 1995; ISPI International Conferences, 1991 and 1996; ASTD
    National Conference on Technical & Skills Training, 1990. Related workshop description at
    http://www.educationexperts.net/nstworkshop.html

   Other materials (particularly sample clicker questions and goals of clicker questions) adapted from Ian
    Beatty’s Technology Enhanced Formative Assessment (TEFA) program. http://ianbeatty.com/crs

 My upcoming travel: Chicago, West Virginia, Pittsburgh. Need a speaker?

 Upcoming free webinar for K12. See flyer.
Learning Goals

 Biology: Recognize the components of a cell and
    describe why each is necessary for the function of a cell
   Physics: Identify the different ways that light can
    interact with an object
    (i.e., transmitted, absorbed, reflected).
   Chemistry: Explain trends in boiling points in terms of
    intermolecular interactions
   Earth science: Understand the formation of the three
    major types of rocks (igneous, sedimentary, and
    metamorphic) and the processes by which they
    form, relating them by the rock cycle.
   Math: Solve a system of linear equations in two
    variables using algebra or graphing.
What Do I do if…?
                              39

What can you do if you ask questions and..
 There is no response
 The same people keep raising their hands
 The answers are called out before everyone has a
  chance to think
                                            We’ll discuss in
 The answers take too long                 Workshop #2.
                                        For now: Many of
 Someone gives a wrong answer          these challenges are
 Only some students are prepared       addressed by clickers

?

Phystec Conference: The Gentle Art of Questioning. Writing Great Clicker Questions

  • 1.
    The Gentle Artof Questioning WRITING GREAT CLICKER QUESTIONS Dr. Stephanie V. Chasteen Physics Department & Science Education Initiative Univ. of Colorado at Boulder http://colorado.edu/sei Web and blog: http://sciencegeekgirl.com Email: stephanie.chasteen@colorado.edu
  • 2.
    Who are you? Whatis your primary role? Show of hands A. STEM faculty B. Administrator C. Teacher educator D. Education researcher E. Graduate student or post-doc F. Other
  • 3.
    Have you usedresponse systems (clickers) in your teaching? Take a clicker & turn it on If the green light flashes, your vote has been counted A. Not at all, and I haven’t seen them used B. Not at all, but I’ve observed their use somewhat C. I’ve used them a little D. I’ve used them a lot E. I could be (should be?) giving this workshop
  • 4.
    How familiar areyou with Mazur’s “Peer Instruction” Colored cards A. Fairly familiar, and I like it B. Fairly familiar, but I’m not sure that I like it C. I’ve heard of it but only have a vague idea what it is D. Not familiar at all E. Not sure
  • 5.
    Introducing Me 5 Science Education Initiative http://colorado.edu/SEI Applying scientific principles to improve science education – What are students learning, and which instructional approaches improve learning? Physics Education Research Group http://PER.colorado.edu One of largest PER groups in nation, studying technology, attitudes, classroom practice, & institutional change. Blogger http://blog.sciencegeekgirl.com
  • 6.
    Why question? 7 Credit: Rosie Piller whiteboard
  • 8.
    Agenda 9 1. When and how we can ask questions 2. About clickers as a way to ask questions, including some common challenges 3. Writing good questions. Example questions, writing our own. 4. Action plan Learning goals: Participants will be able to…. A. Explain several benefits of questioning and of using clickers to question B. Defend the use of best practices in questioning to overcome common challenges C. Formulate and revise clicker questions to target student learning goals
  • 9.
    Exercise #1: Questionbrainstorm 10  What questions could you ask to help students achieve your assigned learning goal -- to test mastery and stimulate learning?  Brainstorm as a group 5 minutes whiteboard
  • 10.
    When can weask questions? 11 BEFORE Setting up instruction DURING Motivate Developing knowledge Discover Predict outcome Check knowledge Provoke thinking Application Assess prior knowledge Analysis Evaluation Synthesis AFTER Relate to big picture Exercise skill Assessing Demonstrate success Elicit misconception learning Review or recap Exit poll Credit: Rosie Piller and Ian Beatty.
  • 11.
    We can askquestions in lots of ways 12  Ask rhetorically  Target the class (how?)  Target someone in particular (in what order?)  Answer your own question  Leave the question unanswered Or ask out of class  Blogs  Discussion boards  Homework… Credit: Rosie Piller
  • 12.
    Why use clickersto target the class? An outline of Peer Instruction. 13
  • 13.
    Clickers are atool for questioning 14 But not a magic bullet!
  • 14.
    Anatomy of PeerInstruction 15 Ask Question …Lecture… (Maybe vote) Class Discussion Peer Discussion Vote * See also: Peer Instruction, A User’s Manual. E. Mazur.
  • 15.
    How is aclicker question the same or different?* 16 * From other types of in-class questions  Similar in terms of goals  Multiple choice  Anonymous (to peers)  Every student has a voice – the loud ones and the shy ones  Forced wait time  You can withhold the answer until everyone has had time to think (choose when to show the histogram) What does this tool help us to do?
  • 16.
    U. Colorado clickerresources… 17 Videos of effective use of clickers 2-5 mins long http://STEMvideos.colorado.edu Clicker resource page http://STEMclickers.colorado.edu • Instructor’s Guide • Question banks • Workshops • Literature / Articles
  • 17.
    Which of thesecould be clicker questions? 18 BEFORE Setting up instruction DURING Motivate Developing knowledge Discover Predict outcome Check knowledge Provoke thinking Application Assess prior knowledge Analysis Evaluation Synthesis AFTER Relate to big picture Exercise skill Assessing Demonstrate success Elicit misconception learning Review or recap Exit poll Credit: Rosie Piller and Ian Beatty.
  • 18.
    Let’s try it 19 Which superpower would you rather have? The ability to… A. Change the mass of things B. Change the charge of things C. Change the magnetization of things D. Change the boiling point of things 19 Question: Ian Beatty, UNC Greensboro Image: Thibaultfr on Wikimedia
  • 19.
    Example question: Math Yoursister in law calls to say that she’s having twins. Which of the following is the most likely? (Assume she’s having fraternal, not identical, twins) A. Twin boys B. Twin girls C. One girl and one boy D. All are equally likely 20 Derek Bruff, Vanderbilt
  • 20.
    Example Question: Survey 21 Which of the following are you least comfortable using to solve problems? A. Kinematics B. Newton’s Laws C. Work-Energy Theorem D. Momentum-Impulse Theorem E. Angular Momentum-Angular Impulse Theorem Ian Beatty, UMass Amherst
  • 21.
    Let’s try it I think the toughest thing about using clickers and peer instruction in class will be: A. Writing good questions B. Getting students to really think about them C. Getting students to discuss the questions D. Getting students to share their ideas in the class discussion E. It takes too long / I have a lot of content to cover Or something else? Write it on your handout! What are some *solutions* to some of these challenges?
  • 22.
    Some “best practices”and solutions We’ll do these quickly These are repeated in handouts  Underlying philosophy to most of these “best practices” are: “It is important for students to discuss their ideas, to feel safe doing so, and for the teacher to be aware of those ideas”
  • 23.
    1. Ask Question 26 What are some challenges/ things to consider when posing a clicker question? •Ask several times during lecture •Ask challenging, meaningful questions •Don’t post until ready •Give time to read (read silently) •Don’t read question out loud 26 Handout/worksheet / whiteboard
  • 24.
    2. Peer Discussion 27 • Students learn more deeply by teaching each other • Makes them articulate answer • Lets you see inside their heads Why is peer discussion important? What are challenges / how can you help make it work? •Make it clear why you’re doing this • Circulate and ask questions / model •Use questions they want to discuss •Allow enough time (2-5 mins) •Focus on reasoning in wrap-up
  • 25.
  • 26.
    3. Wrap-Up Discussion 29 Challenges? What might you do to facilitate an effective wrap-up discussion? •Establish culture of respect •Consider whether to show the histogram immediately • Ask multiple students to defend their answers • Why are wrong answers wrong and why right answer is right
  • 27.
    Giving the answerstops student thinking! 31
  • 28.
    Preparing to WriteQuestions  Read briefly over the “tips for writing clicker questions” handout. Which is going to be most challenging for you?  Then, in groups of 2-3, choose one of the questions that you brainstormed at the beginning of the workshop. You will write a multiple choice version of this question (but not yet!) 3 minutes
  • 29.
    Gallery Walk  Asa table, look at the “example questions” trio that I have given you. What’s a common theme(s)?  Write the themes you find down on the sheet so that other groups will be able to read it. After 5 minutes, circulate to see the themes of questions on other tables. Shop for ideas for your own questions! See handouts for a place to jot your notes. 10 minutes
  • 30.
  • 31.
    Exercise #3: WritingQuestions 35  Using ideas you’ve learned, write a multiple choice version of your question in groups of 2-3.  Look on the “Sample Questions” handout for some examples.  Show your question to another group (and to me) for suggestions on revising it.  If you have time, write another question from another part of the questioning cycle. 10 minutes
  • 32.
    Action Plan 36  Take a few minutes to write down your action plan to implement ideas you heard about in the workshop
  • 33.
    References & Resources Weband blog: http://sciencegeekgirl.com Email: stephanie.chasteen@colorado.edu Thanks!  Clicker Resource Page from the Science Education Initiative: http://STEMclickers.colorado.edu. Has clicker question banks (in the sciences), an instructors’ guide, and videos of classroom use. Useful books (such as Eric Mazur’s Peer Instruction are cited there.  Workshop handouts will be uploaded to the above website, or email me.  Many materials in this workshop (particularly the questioning cycle and the participant exercises) were adapted fromRosie Piller, Making Students Think: The Art of Questioning. Short papers published in: Computer Training & Support Conference, 1995; ISPI International Conferences, 1991 and 1996; ASTD National Conference on Technical & Skills Training, 1990. Related workshop description at http://www.educationexperts.net/nstworkshop.html  Other materials (particularly sample clicker questions and goals of clicker questions) adapted from Ian Beatty’s Technology Enhanced Formative Assessment (TEFA) program. http://ianbeatty.com/crs  My upcoming travel: Chicago, West Virginia, Pittsburgh. Need a speaker?  Upcoming free webinar for K12. See flyer.
  • 34.
    Learning Goals  Biology:Recognize the components of a cell and describe why each is necessary for the function of a cell  Physics: Identify the different ways that light can interact with an object (i.e., transmitted, absorbed, reflected).  Chemistry: Explain trends in boiling points in terms of intermolecular interactions  Earth science: Understand the formation of the three major types of rocks (igneous, sedimentary, and metamorphic) and the processes by which they form, relating them by the rock cycle.  Math: Solve a system of linear equations in two variables using algebra or graphing.
  • 35.
    What Do Ido if…? 39 What can you do if you ask questions and..  There is no response  The same people keep raising their hands  The answers are called out before everyone has a chance to think We’ll discuss in  The answers take too long Workshop #2. For now: Many of  Someone gives a wrong answer these challenges are  Only some students are prepared addressed by clickers ?

Editor's Notes

  • #2 HAVE PEOPLE SIT BY DISCIPLINE
  • #8 How do you feel about asking students questions in class?How many times have you given a lecture and found that students hadn’t followed you?Can you rely on students to ask questions if they don’t understand something?Can you rely on students to know if they don’t understand something?So, what are the benefits of questioning?Why do you think people don’t question more?
  • #12 During each section, ask people for examples of questions that they wrote that fall into this category. Give clicker booklet for responding.Point out the handout where each one is detailed more.
  • #13 Model each one of these. What are some ways to ask questions? One is to ask rhetorically.Class, what’s another way to ask a question? Target the whole class.John, what’s another way? Target someone else.Are there other ways to ask a question? Let’s think about it. Target class: verbally, clickers, other waysTarget someone in particular: randomly, in seating order, call on particular personWait and then…. Call on volunteers, call on someone who hasn’t volunteered, answer own question
  • #19 During each section, ask people for examples of questions that they wrote that fall into this category. Give clicker booklet for responding.Point out the handout where each one is detailed more.
  • #26 Questions threaten studentsI get no volunteers to answerStudents don’t talk to each otherAnswers take me off trackTakes too much time
  • #27 What comes first? Learning goals.
  • #28 Instructor circulates, may need to show that you’re serious
  • #31 Usually one second. Trained to wait 3-5 seconds. Students tend to speak in bursts with 3-5 seconds between bursts: Wait time of 1 second interrupts these bursts.
  • #33 Shop for ideas
  • #40 Weigh advantages of covering more material against checking comprehension and actively involving students. It’s challenging. How a teacher does this determines how well it works. NO RESPONSE: Wait longer. Rephrase the question. Give a hint. Have students discuss. Call on someone. Leave unanswered. SAME PEOPLE: Someone other than X. Ask an easier question and call on new volunteer. Be alert to non-verbal cues. Make it clear that participation required. IF ANSWERS CALLED OUT: Ask it others agree. Ask for other answers. Ask students to think for a minute. Turn away to signal time for thought. Ask to write answers down. IF TAKE TOO LONG: Interrupt and summarize. Set boundaries and expectations. WRONG ANSWER: Break down question so others can see error. Ask for comments. Ask for other answers. Find merit in answer and explain why common mistake.