Using Coding Queries and Survey Tools to Illuminate Relationships
•Download as PPTX, PDF•
1 like•163 views
See how NVivo’s coding feature, matrix queries, and survey tools helped a research team locate relationships among three types of data, including verbal interactions from teacher-student writing conferences, written data from student writing assignments, and surveys.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Using Coding Queries and Survey Tools to Illuminate Relationships
Similar to Using Coding Queries and Survey Tools to Illuminate Relationships (20)
More from QSR International
More from QSR International (17)
Recently uploaded
Recently uploaded (20)
Using Coding Queries and Survey Tools to Illuminate Relationships
- 1. Using Coding Queries and Survey Tools to Illuminate Relationships Angie McKinnon Carter Lecturer, Utah Valley University March 22, 2017
- 2. Conferencing is something we do, but unexamined, it remains something we do not understand and thus cannot improve. Laurel Johnson Black
- 3. Value of NVivo Offers ways to test assumptions Sometimes the coding results surprise us LimarieCabrera
- 4. Examining Writing Conferences Knowledge Types PernillaRydmark BrianTeutsch
- 5. Questions Which speaker uses which knowledge types most in the writing conferences? Does the amount of discussion of a certain knowledge type connect to the amount or types of changes the writer makes? How does the draft illustrate how well the writer knew how to make changes? Do the writers understand how to make changes? To what extent do the changes improve the quality of the paper?
- 7. NVivo for Transcripts Transcribed manually—full transcription with pauses and time stamps and intonation units Codes Segmenting Thematic (for Knowledge Type) How What Other Queries
- 8. Page 16 Page 1 Page 7 Page 11 Page 4 Page 13 Drafts Pre-NVivo Preparation Compare Drafts using Word Create PDF Import into NVivo
- 9. NVivo for Drafts Coding (Coding Stripes and highlighting) No Change Change Type of Change Level of Change Queries Draft change Comparison Change Draft Full Comparison--Michael and --Tina (shows # of types of changes and levels of change)
- 10. Importing Surveys Direct from Qualtrics Surveys need to be set up to handle this Participants (either ID only or put in pseudonyms instead of real names) Set up questions to facilitate importing
- 15. Import Survey as Excel doc When downloading from Qualtrics, select export as Choice Text instead of numeric values Cleaning up the data Combine Columns Rename Columns Benefits of cleaning up during import to NVivo
- 16. Survey Dataset Show columns “I will make revisions because that’s what the instructor wants.” “I will make revisions because I want to produce a more quality paper.” “I actually understood what revisions to make.”
- 17. Surveys I actually understand what revisions to make. Michael: Strongly agree Tina: Neither Agree nor Disagree Brian: Strongly Agree Gwen: Strongly Agree I will make revisions because I want to produce a more quality paper. Michael: Strongly Agree Tina: Strongly Agree Brian: Neutral Gwen: Strongly Agree
- 18. Illuminating Relationships— Key Features Undock Column and Row Settings Double-click to drill down
- 19. References Black, L. J. (1998). Between talk and teaching: Reconsidering the writing conference. Logan, UT: Utah State University Press. Cabrera, Limarie. Drilling 2. CC by 2.0. Retrieved from Flickr on 29 Nov. 2016. https://creativecommons.org/licenses/by/2.0/ Rydmark, Pernilla. Jump rope. CC BY-SA 2.0. Retrieved from Flickr on 28 Nov. 2016. https://www.flickr.com/photos/63158617 @N07/7190171420/ Sommers, N. (1980). Revision strategies of student writers and experienced adult writers. College Composition and Communication, 31(4), 378–388. Teutsch, Brian. The Tina Turners. CC BY 2.0. Retrieved from Flickr on 28 Nov. 2016. https://www.flickr.com/photos/ brianteutsch/203265779/
- 20. Using Coding Queries and Survey Tools to Illuminate Relationships Angie McKinnon Carter Lecturer, Utah Valley University March 22, 2017
Editor's Notes
- Have NVivo already loaded so that when I switch to it, it is already working. Turn off notifications on the computer. Thank Kevin and QSR International for the opportunity to present. Thank you for attending. I look forward to showing how NVivo is helping me and my research team code and analyze data to understand more about teacher-student writing conferences.
- This quote from Laurel Johnson Black was the beginning of my team’s journey to understanding writing conferences. I teach writing at an open-enrollment university in the Intermountain West. In an effort to improve my teaching practice, I decided to examine writing conferences which I use frequently when teaching writing, and I asked one of my colleagues, Chris Lee, to help me. Over the years, we have had six student interns work with us on the project. Initially, we wanted to examine if students having writing conferences with their teachers leads student to produce higher quality final papers.
- We came to NVivo after the study was designed. I heard of it and went to a training workshop, and I felt that it would help us keep track of our data and explore it more efficiently than what we could do with pen and paper alone. Our process has been a high-bred one where we do a lot of initial coding with pen and paper as a team. When we have reached consensus, we transfer that coding into NVivo and run descriptive statistics and look for patterns there. Test assumptions: Does more change lead to a better quality paper? Coding results sometimes surprise us. We thought that more change would indicate better quality. But drilling into the data helped us make sense about why this is not necessarily the case.
- In the project that I’m going to share with you today, we focused on knowledge types. WHAT-- Specifically we focused on instances of declarative knowledge, or “what to do,” which we named as “What” for our coding schema; and procedural knowledge, or “how to make changes,” which we named as “How” in our coding schema. As shown here, the difference in being able to name something and being able to do something can be dramatic. For instance, I can name a “jump rope” and I can even name “double-dutch” and describe how it is done. But doing “double Dutch” has remained elusive. Students can have similar experiences when writing papers. From our review of our data, our research team sensed a difference between the problems that teachers and students could identify in a paper and the students’ ability to make changes later. NVivo helped us start figuring out the significance of that difference.
- The following research questions guided this study’s exploration.
- We collected data throughout a semester. For two papers, we collected rough drafts and final drafts, recorded conferences, and administered surveys before, immediately following the conference, and after the paper had been graded. We collected data in a way that would triangulate information and provide ways to understand data from different angles. Because we collected several pieces of data for each person, we put this data into Cases in NVivo. Developing tools Surveys—Used students to help develop the surveys to test language and concepts. Surveys are designed to have multiple questions that ask about different aspects of an issue. For instance, one question asks students what the ideal ratio of teacher and student talk is in a conference. Another question asks whether the teacher talked more than the student. I am now going to explain how we prepared each of type of data for NVivo and how NVivo helped us code the information in each data type.
- Audio recordings of writing conferences Transcribed ourselves Put those into intonation units Uploaded to NVivo Coding in Nvivo allows us to look at different patterns within the same conferences. For instance, in this project, we focused on knowledge types. Specifically we focused on instances of declarative knowledge, or “what to do,” which we named as “What” for our coding schema; and procedural knowledge, or “how to make changes,” which we named as “How” in our coding schema. Some talk was not related to the paper, so we created an “Other” category for that talk. SHOW CODES for Segmenting and What/How SHOW Coding Stripes that show what has been coded. SHOW Queries: How-What by speaker (Brian, Tina, Michael, Gwen). Show how to create the query (Matrix Coding).
- Comparing the rough draft to the final draft provided another way to verify our findings from coding the transcripts. We need to know the types of changes and the total amount of change to answer the following questions: Does the amount of discussion of a certain knowledge type connect to the amount or types of changes the writer makes? How does the draft illustrate how well the writer knew how to make changes? To what extent do the changes improve the quality of the paper?
- Draft change coding schema Show the levels of change and types of changes, adapted from Nancy Sommers’ article “Revision strategies of student writers and experienced adult writers.” Show Tina’s draft at the thesis level. Tina—Never really understands the troubles with her thesis, but she tries to address those concerns anyway. Look at the coding. Turn on coding highlighting for addition. Some changes are shown as additions, but they are Rating rubric—The rating of the rough draft and the final draft become attributes in for each student writer.
- Importing surveys was the last piece of the puzzle for us. We would have organized our surveys differently had we known about NVivo earlier. I will illustrate some of those concerns as I show what would happen if I tried to import our surveys into Nvivo directly from Qualtrics. SHOW WHERE TO FIND THE SURVEY DATA IMPORT IN NVIVO. Then switch back to the slides.
- Show the problems with importing questions with multiple correct answers or fill-in-the blank with a Likert.
- SHOW: Import Immediate Feedback Conference Semester short questions. One of the first things we learned when exploring the data is that students need more time in the conference. Originally, the instructors were scheduling conferences at 10 minute intervals. Since evaluating the data, they are scheduling them at 15 minute intervals and allowing students to schedule back-to-back sessions for more time.
- Helps answer the question: What information do the surveys provide about how the writers understood how to make changes? Then look at Gwen’s in Immediate Feedback Conference Semester answers to these questions.
- Test assumptions: Does more revision lead to better revision? No. Examine the differences in quantity between Michael’s and Tina’s draft changes
- Thank you for listening to my presentation. I am happy to answer any questions that you may have.