When should we use qualitative research versus other methods? What are the major challenges in design and analysis ? What should we consider for an AMR Project with qualitative research versus what we might do for a published research project or for our own professional work?
Messy Research: How to Make Qualitative Data Quantifiable and Make Messy Data Understandable
1MESSY RESEARCHHow to Make Qualitative DataQuantifiable and Make Messy DataUnderstandableDr. Gigi Johnson
Core • When to chose itQualitative • Major challenges in designIssues and analysis • How to tell messy stories for real decisions and action by business
3Why • Make a decisionBusiness • Deploy resources as anResearch organization? • Convince others in organization • Rule out alternatives • Influence certain people in the company • Understand change ahead of company
4How Much Do We Need?• How much data and what data?• From where?• What analysis do we need to do?• What narrative/presentation is enough? Often, in business research, we tend to focus on volumes . . . . . . missing focus on analysis and presentation for decision-making
5What is Truth in Business?• What is enough information to make a decision—or real “Truth”?• How much information of what type is “enough”?• How messy will this be and still be “normal”?• Who can best find the data?• Which people know what part of the answers we need?• Is the core data we need reachable?• What is our role as researcher with perspective?
64 Types of Qualitative Data Leech, N. L., & Onwuegbuzie, A. J. , 2005
7Focus on Quantitative Issues Design characteristic• Validity • Sample sizes• External Validity: • Statistical significance generalize findings across • Sampling bias populations, tasks, and • Coding consistency environments (Campbell • Control groups & Stanley, 1966) • Pre- and post-testing• Internal Validity: Design rule out other factors • Instrument design (tested other than the validity) Independent Variable
10Qualitative vs. QuantitativeQuantitative Qualitative•Helpful when “answering •Looking at single case orquestions of who, where, small number of caseshow many, how much, and •Looking at in-contextwhat is the relationship situation, framed by wordsbetween specific variables” and narratives(Adler, 1996, p. 5) •Looking for in-context•Striving for causation or for relationships andgeneralizing to larger connectionspopulations •Creating hypotheses or instruments for quantitative
11Qualitative Can Enrich QuantitativeExamples:“Prebriefing” (Collins et al., 2006), checkingpotential quantitative survey participants forwillingness and suitabilityPilot study to assess the appropriateness of aninstrument like a questionnaire or surveyRuling out hypotheses
12Challenge of Qualitative • Difficulty in capturing lived experiences via text • Creating a “bricolage” – an assemblage of representations that fit a complex situation (Denzin & Lincoln, 2005) Use of Qualitative Analytical tools helps connect this complex in-context environment into a way that others can understand.
13DESIGNING THE RESEARCHGoalsNarrativesPopulations & SamplesDataInstruments
15Populations and “Level”• Populations: Total target group • AMR: Group could be a regional or business population, or could be members at a level in the organization• Sample: Group in study
16“Who” has the Data?• Thinking in terms of Five Forces • Vendors, Customers, Competitors• Reasons to share• “Knowing” • Belief, research, or connections • Expert does not mean “knows” real information• Similar question: Secondary Research and connecting Primary to it
18Literature Review• Check out what research has been done on the research methods that you are considering, e.g., focus groups, narrative research, document analysis• Google Scholar: Good launching pad
19Sampling Methods and Size• Quantitative: Concern with probabilities and similarities to overall population• Qualitative? • Snowball sampling: uses social networks and connections to identify unknown populations • Convenience sampling • Judgment Sample: based on framework of variables from researchers • Maximum Sampling, Extreme• How much is enough? • Saturation (repeated patterns) (Rubin & Rubin, 1995).
20Instruments• Creating a Questionnaire• Focus Group – Outline, Objectives• Surveys – may be instruments already tested for validity• Interviews • Open Ended • Semi-Structured • Test and plan coding methods upfront; what will you input the answers into? • Grounded Theory: Grand Tour Question(s)
21DOING THE DARNEDRESEARCHRecording and measurementTranscriptionField Notes
23Transcription as Friend and Foe01: Exactly. And, as far as doing it, the other, I think the biggest obstacle, is training. Is getting=G Is opportunity.01 It is an opportunity. But . . .Group ((chuckles))01 . . . it is an obstacle as far as the [district is concerned.]=G =[It is hard to not say it.]=01 Because they will not give that time to really teach and train. Even, you know, Im gonna walk in as the . . . the real Luddite. And be able to walk out and feel like I can go out and use the equipment. Not just say it.11 Yeah.
24Undercurrents from Field Notes Individual impressions Notes before and after sessions Bring your own biases, context, and observations to the table
25ANALYZING THE DARNEDRESEARCHUsually NOT in qualitative business researchplans
26Designing the Analysis• Not just casually connecting• Causality vs. Correlation• Two analysis directions • Old-fashioned and robust • Excel worksheets or written on documents • Hand coding and counting • Alternatives • Computer-assisted data qualitative data analysis software (CAQDAS)
27Recursive Abstraction• Fancy phrase for summarizing, then summarizing the summaries • Usual accidental business research method• Helps to have consistent methods for summarizing between coders/team members, or a coding worksheet
28Coding• Chunking text data, then adding a code • You can code and iteratively recode/emergent (Tesch, 1990).• Method: aimed to continue to narratively code while bridging to new ideas and surfacing new categories until you began to find pattern codes and themes (Miles & Huberman, 1994).
29Key Phrase Frequency• Word counts are based on the belief that all people have distinctive vocabulary and word usage patterns.• “Linguistic fingerprints” (Pennebaker, Mehl, & Niederhoffer, 2003, p. 568).• Gives context to words like “many,” “frequently,” etc. terms are fundamentally quantitative.
30KWICKeywords-in-context (KWIC; Fielding & Lee, 1998)•Data analysis method that reveals how respondents usewords in context•Compares words that appear before and after “key words”
31Narrative Analysis (NA)• (Nearly) all qualitative research is filtered by contexts, beliefs, and methods of communication• NA evaluates patterns, threads, tensions, and themes within the transcripts and field notes (Clandinin & Connelly, 1994, 2000; Ryan & Bernard, 2000).• Can pull out portions of text where themes are mentioned (Ryan & Bernard, 2000)
32Triangulation• Assesses the integrity of the inferences that one draws from more than one vantage point (Lincoln & Guba, 1985)• Use of multiple data sources, multiple researchers, perspectives, tools, and/or methods (Denzin, 1989; Schwandt, 2001)• Adds confirmability, dependability, and credibility to data collection
342 Reasons for Tools• Help the team gather, sort, visualize, and engage messy and abundant qualitative data• Explain and convince client of validity of research done • Ability to walk through analytical process and explain the patterns in the data
39DATA VISUALIZATION ANDINFOGRAPHICSFinding the StoryTelling the StoryPersuading Change with the Story
40Using Data to Tell and Be the Story• Abundant Data (“Big Data”) from in-context data collection in our connected world• Social Network Analysis (SNA) – how we are all connected• Big company problem – large volumes of data to digest and act upon • “Investigating relationships” – not just for presentation, but for research teams to visualize emerging patterns
41Concept Mapping:One Method of Data Visualization
42 • Great list:Data http://dailytekk.com/2012/02/27/over-100-incredible-infographic-tools-aVisualizatio • Piktochart – Transforms your information into memorable presentations. • Infogr.am - Create interactive charts and infographics.nInfographics for • Gephi – Like Photoshop for data. Graph visualization and manipulation software.Decisions • Tableau Public - Free data visualization software. • Free Vector Infographic Kit – Vector infographic elements from(others in MediaLoot.Appendix slides) • Weave – Web-based analysis and visualization environment. • iCharts – Charts made easy. • ChartsBin – A web-based data visualization tool. • GeoCommons – See your data on a map. • VIDI – A suite of powerful Drupal visualization modules. • Prefuse – Information visualization software. • StatSilk – Desktop and online software for mapping and visualization. • Gliffy – Online diagram and flowchart software. • Hohli – Online charts builder. • Many Eyes – Lets you upload data and create visualizations. • Google Chart Tools – Display live data on your site.
43Questions? Dr. Gigi Johnson @maremel Maremel Institute
44Playing • Wordle – http://www.wordle.comwith – fun tool to turn words fromWords documents into word maps • Tagxedo -- http://www.tagxedo.com – similar to Wordle, Tagxedo lets you create word clouds and sculptures from URLs, Tweets, and other social media documents, as well as export them into a variety of formats.
45 We can tinker with maps, both as pre-Playing made images as well as data-drivenwith Maps tools. •Web Resources Depot -- http://www.webresourcesdepot.com/free-vector-w -- shares a variety of world map images for use •Free PSD Files -- http://freepsdfiles.net/3d-renders/3-isolated-3d-us -- This site has easy images for further editing for presentations •GunnMap -- http://www.gunn.co.nz/map -- creates world maps with your data •StatPlanet Map Maker -- http://www.statsilk.com/software/statplanet -- also creates interactive maps
46 • Several tools let you expand how you lay outPlaying concept maps and linked ideas: • FreeMind -- http://freemind.sourceforge.net – I enjoywith this free tool. Graphically simple, it lets you play with a free tool for mind mapping that can be adaptedConcept into all sorts of other applications. • Webspiration – http://www.webspirationpro.com – IMaps miss its freemium mode; it now has a trial period and then costs $6/month. I found Inspiration and Webspiration wonderful for group presentations and immediate work. • MindManager -- http://www.mindjet.com – this concept management tool starts at $20/month for one and discounts for group collaboration. • MindNode -- http://www.mindnode.com – This tools for Mac computers comes at a moderate price -- $20 for the mac and $10 as an iOS Apps. • VUE by Tufts -- http://vue.tufts.edu -- I really enjoy this “Visual Understanding Environment” tool, which combines concept maps with search and graphics. •
47 • Prezi -- http://www.prezi.com -- My recentPresentations undergraduate class spent half of their projects in Prezi, which has a zooming camera metaphor across a vast digital white board. They enjoyed putting in music, video, and other embedded content. I got a bit dizzy, but enjoyed the creativity. • Sliderocket -- http://www.sliderocket.com -- Several of my students enjoyed using Sliderocket for class presentations. It gave them a robust and elegant toolset to work with. • Brainshark -- http://www.brainshark.com -- Friends who are professional business development executives heartily recommend Brainshark as a way to pre- package and present content at a distance.
48 • Google Charts API - http://code.google.com/apis/chart/ -- you can useGraphs Google Charts to create animations in charts, dashboards, and lots of other goodies • Gliffy -- http://www.gliffy.com/ -- I just found Gliffy, a great diverse creatorand of charts and graphs. Different versions of it work with different social workspace/sharing software: • Hohli -- http://charts.hohli.com – free online chart builderCharts • Creately -- http://creately.com -- (paid but cheap at $5/month/person) is a online tool to build charts, and collaborate around them • Many Eyes -- http://www-958.ibm.com/software/data/cognos/manyeyes/ -- an experiment by IBM Research and the IBM Cognos software group let users create and evaluate data visualizations. • GGobi -- http://www.ggobi.org/ -- free data visualization tool for your datasets • Mondrian -- http://www.rosuda.org/Mondrian/ -- open source toolset for charting and graphing data plots and more complex graphs and data- driven visuals • OpenDX -- http://www.opendx.org -- Older open source software, based on IBM’s visualization data explorer. • Spotfire -- https://silverspotfire.tibco.com – a whole visualization suite, free for individuals for the first year, then $99/year thereafter. • Visualizefree -- http://www.visualizefree.com/ -- Sampler of more complex system; shows real-time images from the FAA of flights as a sample • Mycrocosm -- http://mycro.media.mit.edu/ -- quirky tool to create displays of your own personal data that you can input by cell or email and track
49 • Hans Rosling’s GapminderPlaying Foundation worked with Trendalyzer, which then was sold to Google in 2007, thenwith folded away when Google Labs.Motion • VIDI -- http://www.dataviz.org/ -- VIDI Data, run by the Jefferson Institute, provides aCharts visualization module for Drupal CMS to show motion charts, timelines, geodata, and comparative data. • TrendCompass -- http://epicsyst.com/trendcompass -- lets you add your own data to their data visualization tool if you register • Eurostat Explorer -- http://www.ncomva.se/flash/explorer/eur o/ -- sample with EU data that can be played with using a motion graphic.
50Playing • Tweakersoft’s Vector Designer -- http://www.tweakersoft.com/vectwith ordesigner.html -- This $20 MacImages App helps users create simple vector designs. • GIMP -- http://www.gimp.org -- For those who would want to tinker with Photoshop, but wince at the pricetag, GIMP (“GNU Image Manipulation Program”) is an open source alternative. • Inkscape -- http://inkscape.org/ -- open source vector graphics •
51Playing There are lots of extensive tools to work with large public databases.with Data •Google Public dataResources -- http://www.google.com/publicdata -- From the creators of abundant and specialized search comes search just for public data sources •Visualizing.org -- http://www.visualizing.org/data/br owse and http://www.visualizing.org/ data/channels -- Visualizing provides links to all sorts of sample and interesting data sets
52 • KDnuggets News newsletter on Data MiningAdditiona and Knowledge Discoveryl Tools -- http://www.kdnuggets.com/software/visualiza tion.html -- longer list of free and paid data visualization tools
53Select References• Leech, N. L., & Onwuegbuzie, A. J. (2007). An array of qualitative data analysis tools: A call for qualitative data analysis triangulation. School Psychology Quarterly, 22, 557-584.• Lewins, A., & Silver, C. (2007). Using Software in Qualitative Research: A Step-by-Step Guide, Sage.• Lewins, A. (2008). CAQDAS: Computer Assisted Qualitative Data Analysis in (ed) N. Gilbert, Researching Social Life (ed.)(3rd ed). London: Sage.• Lewis, R.B., & Maas, S.M. (2007). QDA Miner 2.0: Mixed-Model Qualitative Data Analysis Software, Field Methods 19: 87-108• Lincoln, Y. S., & Guba, E. G. (1985). Naturalistic inquiry. Beverly Hills, CA: Sage.• Silver, C., & Fielding, N. (2008). Using computer packages in qualitative research. In C. Willig & W. Stainton-Rogers (eds.), The Sage Handbook of Qualitative Research in Psychology. London: Sage.