How might we marry quantitative and qualitative methods to see the big picture?
Generating knowledge through big and deep data and why and how you can use mixed methods.
Many people don't like their jobs, and many organizations fail to survive in changing environments.
Here's a story of what happened before, and what should (or could) happen now, to try and make things better.
UX STRAT Online 2021 Presentation by Rina Tambo JensenUX STRAT
These slides are for the following session presented at the UX STRAT Online 2021 Conference:
"How to Incorporate Mixed Methods Research"
Rina Tambo Jensen
Hypothesis driven design is a powerful method to ensure features you build in your product are valuable and well evidenced. This quick presentation, delivered internally to DWP Digital colleagues, gives tips on writing high quality hypotheses to work from.
December 2017 presentation covering: What is design thinking? What does it look like in practice? What are some case stories of design thinking being used in the real world? How can we use design thinking in our organization? Where can I learn more?
Many people don't like their jobs, and many organizations fail to survive in changing environments.
Here's a story of what happened before, and what should (or could) happen now, to try and make things better.
UX STRAT Online 2021 Presentation by Rina Tambo JensenUX STRAT
These slides are for the following session presented at the UX STRAT Online 2021 Conference:
"How to Incorporate Mixed Methods Research"
Rina Tambo Jensen
Hypothesis driven design is a powerful method to ensure features you build in your product are valuable and well evidenced. This quick presentation, delivered internally to DWP Digital colleagues, gives tips on writing high quality hypotheses to work from.
December 2017 presentation covering: What is design thinking? What does it look like in practice? What are some case stories of design thinking being used in the real world? How can we use design thinking in our organization? Where can I learn more?
Pragmatic Product Strategy - Ways of thinking and doing that bring people tog...Jonny Schneider
Presented at XConf Tech Manchester in 2014 - Video at http://thght.works/1xdSvqK
This talk explores new ways of framing the work we do in order to create effective software products. A super-pragmatic model of thinking and doing that promises to bring together technologists, designers and business folks alike, across the entire software delivery lifecycle.
Experience mapping serves as a perfect activity to bring into sprints. Diagrams allow you to pull together a wealth of information in a compact and compelling format that is efficient to use. They are well-suited for agile teams.
The key is to focus on engaging others in dialog. It’s not about the map (noun), it’s about mapping (verb). Turn customer insight in to action within the context of a sprint.
This talk will show you how to visualize the user experience quickly and leverage mapping in sprints. I’ll debunk the myth the mapping is a heavy, upfront activity. In fact, when done rapidly, mapping experiences becomes a springboard into creativity and solving real customer problems quickly.
User centred design (UCD) and the connected homeCyber-Duck
This presentation is a summary of a workshop that was conducted at UX London and Mozfest by Cyber-Duck, an agency that merges lean and agile deliver with user centred design (UCD). The workshop was aimed at those wanting to apply UCD to futuristic technologies. The workshop explored the concepts and thinking of ‘how to design an Internet Connected Dishwasher app’ while considering a wider eco system. The workshop started by introducing IoT (and ‘nearables’), why its relevant now and how the UCD process can adapt to it. The workshop frames UCD in a wider product delivery context and is aimed at those wanting to learn on how UX tactics can be applied to successfully design IoT products and systems.
Design Thinking explained with project experiences.
- What is Design Thinking
- What are the steps
- What is SAP Apphaus
- The Next View Design Experience Center Amsterdam
UX Research within an Agile Design and Development Sprint CycleUXPA International
Want to know how to deliver high-value, strategic research insights within a lean sprint process? Learn a quick, useful, and inexpensive process for incorporating user research & usability into Agile Design & Development sprint cycles. We will share a case study that demonstrates how it works and how we work together (research + UX design + dev).
Some of the topics we'll cover:
User Research on a slim budget & tight timeline
Planning research while still designing (what, when, how)
Rapid prototyping to support usability testing
The Post-Testing debrief (meeting with core team to discuss observations & agree on next steps for design and development)
Design iteration based on testing observations (not based on a lengthy expensive report)
Evolving your Design System: People, Product, and Processuxpin
You'll learn:
How to create and maintain a design system over several years
How people, process, and product change alongside a design system
Lessons learned from growing the Linkedin design system
Tips for better surveys: better questions in your questionnaire, better overall survey process. From UPA2012 in Las Vegas.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Introduction for Design thinking :
What is Design thinking?
Why to use Design thinking?
What is Design thinking mindset?
Balance for Analytical and Intuitive thinking.
Traditional thinking vs Design thinking.
Combination of Divergent and Convergent thinking.
Pragmatic Product Strategy - Ways of thinking and doing that bring people tog...Jonny Schneider
Presented at XConf Tech Manchester in 2014 - Video at http://thght.works/1xdSvqK
This talk explores new ways of framing the work we do in order to create effective software products. A super-pragmatic model of thinking and doing that promises to bring together technologists, designers and business folks alike, across the entire software delivery lifecycle.
Experience mapping serves as a perfect activity to bring into sprints. Diagrams allow you to pull together a wealth of information in a compact and compelling format that is efficient to use. They are well-suited for agile teams.
The key is to focus on engaging others in dialog. It’s not about the map (noun), it’s about mapping (verb). Turn customer insight in to action within the context of a sprint.
This talk will show you how to visualize the user experience quickly and leverage mapping in sprints. I’ll debunk the myth the mapping is a heavy, upfront activity. In fact, when done rapidly, mapping experiences becomes a springboard into creativity and solving real customer problems quickly.
User centred design (UCD) and the connected homeCyber-Duck
This presentation is a summary of a workshop that was conducted at UX London and Mozfest by Cyber-Duck, an agency that merges lean and agile deliver with user centred design (UCD). The workshop was aimed at those wanting to apply UCD to futuristic technologies. The workshop explored the concepts and thinking of ‘how to design an Internet Connected Dishwasher app’ while considering a wider eco system. The workshop started by introducing IoT (and ‘nearables’), why its relevant now and how the UCD process can adapt to it. The workshop frames UCD in a wider product delivery context and is aimed at those wanting to learn on how UX tactics can be applied to successfully design IoT products and systems.
Design Thinking explained with project experiences.
- What is Design Thinking
- What are the steps
- What is SAP Apphaus
- The Next View Design Experience Center Amsterdam
UX Research within an Agile Design and Development Sprint CycleUXPA International
Want to know how to deliver high-value, strategic research insights within a lean sprint process? Learn a quick, useful, and inexpensive process for incorporating user research & usability into Agile Design & Development sprint cycles. We will share a case study that demonstrates how it works and how we work together (research + UX design + dev).
Some of the topics we'll cover:
User Research on a slim budget & tight timeline
Planning research while still designing (what, when, how)
Rapid prototyping to support usability testing
The Post-Testing debrief (meeting with core team to discuss observations & agree on next steps for design and development)
Design iteration based on testing observations (not based on a lengthy expensive report)
Evolving your Design System: People, Product, and Processuxpin
You'll learn:
How to create and maintain a design system over several years
How people, process, and product change alongside a design system
Lessons learned from growing the Linkedin design system
Tips for better surveys: better questions in your questionnaire, better overall survey process. From UPA2012 in Las Vegas.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Introduction for Design thinking :
What is Design thinking?
Why to use Design thinking?
What is Design thinking mindset?
Balance for Analytical and Intuitive thinking.
Traditional thinking vs Design thinking.
Combination of Divergent and Convergent thinking.
I will give a talk titled ‘The role of social media in research and career building’ at the forthcoming Seminar on Theories & Concepts and Skills Training organised by the University of Luxembourg in the framework of INCOOP. This multi-disciplinary Initial Training Network (ITN) on Inter-institutional Cooperation in the EU (INCOOP) brings together Universities, professional organisations and high-level officials that all share a long-term interest in a better understanding of the functioning of institutions in the European system of multi-level governance.
This multi-disciplinary Initial Training Network (ITN) on Inter-institutional Cooperation in the EU (INCOOP) brings together Universities, professional organisations and high-level officials that all share a long-term interest in a better understanding of the functioning of institutions in the European system of multi-level governance.
www.albertoalemanno.eu
Social Media and Archaeology: Where Does it Fit and Why Should We Participate?Terry Brock
Presentation on using social media for archaeology and building strategy for archaeological projects given at The Institute for Digital Archaeology by Terry P. Brock and Lynne Goldstein
Researching Social Media – Big Data and Social Media AnalysisFarida Vis
Researching Social Media – Big Data and Social Media Analysis, presentation for the Social Media for Researchers: A Sheffield Universities Social Media Symposium, 23 September 2014
Working with Social Media Data: Ethics & good practice around collecting, usi...Nicola Osborne
Slides from a workshop delivered for the University of Edinburgh Digital Scholarship programme, on 18th October 2017. For further information on the programme see: http://www.digital.cahss.ed.ac.uk/ or #DigScholEd. If you are interested in hosting a similar workshop, or adapting these slides please contact me: nicola.osborne@ed.ac.uk.
Information experience design: improving library customers' experiences of in...Kate Davis
This is the slide deck for the information experience design workshop my colleague Elham Sayyad Abdi and I facilitated at Information Online 2017 on 13 February 2017. The content behind this workshop was developed in collaboration with our colleague Kathleen Smeaton.
Confetti background images are digital paper available on Etsy from DanaGarsonDesign at https://www.etsy.com/listing/160330622/confetti-polka-dots-digital-paper-set
Icons are from The Noun Project and used without citation in line with their terms of service for premium account holders.
AAPOR - comparing found data from social media and made data from surveysCliff Lampe
This presentation was for the 2014 AAPOR conference, and deals with specific components of how "big data" from social media is different from data acquired through surveys.
Choosing the Right Research Methods for Your Project (webinar)Susan Mercer
It’s very easy for User Experience researchers to get stuck in the rut of using your favorite research methods for gathering information and getting user feedback. But, are you really gathering the best information that you can? Or are there other methods that are better suited for your project’s specific needs?
Or, if you’re just starting out – how do you know whether you should conduct interviews, run a survey or a card sort, or something different all together?
Don’t stress – in this webinar, we’ll cover the most popular user research methods and discuss their strengths and weaknesses. Each method shines in different circumstances, and we’ll highlight the factors that will make each successful. We will also present a structured approach to helping you choose the best method or methods for a particular situation.
Data Science: Origins, Methods, Challenges and the future?Cagatay Turkay
Slides for my talk at City Unrulyversity on 18.03.15 in London. Discuss the term Data Science, touch upon the origins and the data scientist types. A longer discussion on the Data Science process and challenges analysts face.
And here is the abstract of the talk:
Data Science ... the term is everywhere now, on the news, recruitment sites, technology boards. "Data scientist" is even named to be sexiest job title of the century. But what is it, really? Is it just a hype or a term that will be with us for some time?
This session will investigate where the term is originating from and how it relates to decades of research in established fields such as statistics, data mining, visualisation and machine learning. We will investigate how the field is evolving with the emergence of large, heterogeneous data resources. We will discuss the objectives, tools and challenges of data science as a practice, and look at examples from research and industrial applications.
CODATA International Training Workshop in Big Data for Science for Researcher...Johann van Wyk
Presentation at NeDICC Meeting on 16 July 2014. Feedback from CODATA International Training Workshop in Big Data for Science for Researchers from Emerging and Developing Countries, Beijing, China, 5-20 June 2014
Picturing the Social: Talk for Transforming Digital Methods Winter SchoolFarida Vis
This talk highlights the work of the Visual Social Media Lab and the Picturing the Social project. It summarises the key research questions and aims of the project. It highlights the value of interdisciplinarity and working closely with industry in this area. It also focuses on the way in which me might study different types of structures involved in the circulation and the scopic regimes that make social media images more or less visible. It also tries to unpack how we can start to think about APIs as 'method' and looks at the different ways in which we can get access to different kinds of social media image data. Both through public ('free') APIs and ('pay for') firehose data.
Similar to Mixed methods and the unknown unknowns of ux uxce15 (20)
As Europe's leading economic powerhouse and the fourth-largest hashtag#economy globally, Germany stands at the forefront of innovation and industrial might. Renowned for its precision engineering and high-tech sectors, Germany's economic structure is heavily supported by a robust service industry, accounting for approximately 68% of its GDP. This economic clout and strategic geopolitical stance position Germany as a focal point in the global cyber threat landscape.
In the face of escalating global tensions, particularly those emanating from geopolitical disputes with nations like hashtag#Russia and hashtag#China, hashtag#Germany has witnessed a significant uptick in targeted cyber operations. Our analysis indicates a marked increase in hashtag#cyberattack sophistication aimed at critical infrastructure and key industrial sectors. These attacks range from ransomware campaigns to hashtag#AdvancedPersistentThreats (hashtag#APTs), threatening national security and business integrity.
🔑 Key findings include:
🔍 Increased frequency and complexity of cyber threats.
🔍 Escalation of state-sponsored and criminally motivated cyber operations.
🔍 Active dark web exchanges of malicious tools and tactics.
Our comprehensive report delves into these challenges, using a blend of open-source and proprietary data collection techniques. By monitoring activity on critical networks and analyzing attack patterns, our team provides a detailed overview of the threats facing German entities.
This report aims to equip stakeholders across public and private sectors with the knowledge to enhance their defensive strategies, reduce exposure to cyber risks, and reinforce Germany's resilience against cyber threats.
Levelwise PageRank with Loop-Based Dead End Handling Strategy : SHORT REPORT ...Subhajit Sahu
Abstract — Levelwise PageRank is an alternative method of PageRank computation which decomposes the input graph into a directed acyclic block-graph of strongly connected components, and processes them in topological order, one level at a time. This enables calculation for ranks in a distributed fashion without per-iteration communication, unlike the standard method where all vertices are processed in each iteration. It however comes with a precondition of the absence of dead ends in the input graph. Here, the native non-distributed performance of Levelwise PageRank was compared against Monolithic PageRank on a CPU as well as a GPU. To ensure a fair comparison, Monolithic PageRank was also performed on a graph where vertices were split by components. Results indicate that Levelwise PageRank is about as fast as Monolithic PageRank on the CPU, but quite a bit slower on the GPU. Slowdown on the GPU is likely caused by a large submission of small workloads, and expected to be non-issue when the computation is performed on massive graphs.
Mixed methods and the unknown unknowns of ux uxce15
1. MIXED METHODS AND THE
UNKNOWN UNKNOWNS OF UX
JUDITH MÜHLENHOFF
@judithmp
HOW MIGHT WE MARRY QUANT AND QUAL METHODS
TO SEE THE BIG PICTURE?
UXCE15, BERLIN
2. User and Innovation Researcher
Sociologist
PhD candidate, “Culture-Driven Innovation”
Owner of the User Research Lab: enhancing audience engagement with qualitative
insights
@judithmp
Judith
HELLO, MY NAME IS…
3. “There are known knowns; there are things we know
we know.
We also know there are known unknowns; that is to
say we know there are some things we do not know.
But there are also unknown unknowns – the ones we
don't know we don't know.”
Donald Rumsfeld, 2002
THE UNKNOWN UNKNOWNS
R. D. Ward [Public domain], via Wikimedia Commons
https://commons.wikimedia.org/wiki/File%3ADefense.gov_News_Photo_020304-D-9880W-013.jpg
Donald Rumsfeld
4. 1. GENERATING KNOWLEDGE THROUGH DIFFERENT DATA
2. MIXED METHODS
2.1 WHY & HOW
2.2 CASES
3. TAKE AWAYS
AGENDA
5. Type of Knowns and Unknowns
GENERATING KNOWLEDGE THROUGH DIFFERENT DATA
Knowns Unknowns
known
Known Knowns
Things we are aware of and
understand
Known Unknowns
Things we are aware of but
don‘t understand
unknown
Unknown Knowns
Things we understand but are
not aware of
Unknown Unknowns
Things we are neither aware
of nor understand
6. “With enough data, the numbers
speak for themselves.”
Chris Anderson, 2008
Big Data – The Holy Grail of Customer Experience?
GENERATING KNOWLEDGE THROUGH DIFFERENT DATA
Anderson pic by James Duncan (Flickr)
https://commons.wikimedia.org/wiki/File%3AEtech05_Chris.jpg
7. Numbers do not speak for themselves, they have to be interpreted within their context.
Big Data Needs Deep Data
GENERATING KNOWLEDGE THROUGH DIFFERENT DATA
8. Big Data are not objective, they inhibit high risks of hidden biases and faulty data.
Big Data Needs Deep Data
GENERATING KNOWLEDGE THROUGH DIFFERENT DATA
9. Big Data tell us about “how many/how often”, but not about the “why” and “how”.
Big Data Needs Deep Data
GENERATING KNOWLEDGE THROUGH DIFFERENT DATA
10. Qualitative methods, like ethnography, provide us with “Deep Data”
Tell us about the why and how of human behaviour
Put data into the context of the messy social world
Explore neglected research fields in depth and breadth
Big Data Needs Deep Data
GENERATING KNOWLEDGE THROUGH DIFFERENT DATA
11. Types of Knowns and Unknowns and Data
GENERATING KNOWLEDGE THROUGH DIFFERENT DATA
Knowns Unknowns
known
Known Knowns
Things we are aware of and
understand
• Known Data
• To be updated and refined
Known Unknowns
Things we are aware of but
don‘t understand
• Insufficient Big Data
Deep Data
• To be explored in depth
unknown
Unknown Knowns
Things we understand but are
not aware of
• Esp. Big Data
• To be made aware of
Unknown Unknowns
Things we are neither aware
of nor understand
• Deep Data + Big Data
• To be explored in depth
and breadth
13. Qualitative and quantitative methods each have its strengths and weaknesses.
With mixed methods, the strength of a single method outweighs the weakness of the
other.
Why Mixed Methods?
MIXED METHODS – WHY & HOW
https://www.flickr.com/photos/archeon/
14. Combining methods provide a bigger picture, like different angles in a movie.
These can lead to corroboration, verification…
… or might spur complete new thinking and theories. Unknown Unkowns
Why Mixed Methods?
MIXED METHODS – WHY & HOW
15. At which stage do you combine methods?
Do you choose an identical or non-identical sample?
What is your overall goal of mixing methods (e.g. corroboration, exploring)?
Will you focus on one method?
How to Conduct Mixed Methods?
MIXED METHODS – WHY & HOW
16. How to Conduct Mixed Methods?
MIXED METHODS – WHY & HOW
Sequential Design
Embedded Design
Parallel Design
Fully Integrated Design
Conversion Design
17. Help newsrooms to measure the quantitative and qualitative impact of their stories.
Two tasks:
Understanding what it means for a story to “do well” and what happened to cause
that.
Managing an event-tracking workflow while juggling other responsibilities
MIXED METHODS – CASES
NewsLynx (Columbia Journalism School/Tow Center)
18. Proposal of a taxonomy to measure impact beyond metrics
MIXED METHODS – CASES
NewsLynx Columbia Journalism School/Tow Center)
20. MIXED METHODS – CASES
Self-Observation: Diary Studies and Experience Sampling Method (ESM)
www.user-research-lab.org
Digital Media Diary (User Research Lab)
Self-observation of news consumption
21. Experience Sampling Method (e.g. Google)
Track experiences in the moment
Qualitative data at larger scale
MIXED METHODS – CASES
Self-Observation: Diary Studies and Experience Sampling Method (ESM)
www.pacoapp.com
22. Exploration of computer usage and temporality through tracking…
MIXED METHODS – CASES
Ethno-Mining (Intel)
Anderson et al. 2009: Numbers Have Qualities Too: Experiences with Ethno-Mining
23. …and participatory data interpretation
MIXED METHODS – CASES
Ethno-Mining (Intel)
Anderson et al. 2009: Numbers Have Qualities Too: Experiences with Ethno-Mining
25. 1. Leave your method comfort zone.
2. Aim for highly integrated research designs with different angles.
3. Take advantage of participatory data interpretation.
4. Plan and decide deliberately.
5. It takes a lot of effort, but may lead you to your unknown
unknowns.
TAKE AWAYS
27. References
Newslynx: http://newslynx.org/
Big Data:
Kate Crawford: The hidden biases in data
https://hbr.org/2013/04/the-hidden-biases-in-big-data/
Mikkel Krenchel and Christian Madsbjerg: Your big data is worthless if you don’t bring it into the real world
http://www.wired.com/2014/04/your-big-data-is-worthless-if-you-dont-bring-it-into-the-real-world/
Experience Sampling:
Kathy Baxter, You too can collect big data!
https://www.epicpeople.org/pblog10/
Overview of programmes and apps: Experience Sampling and Ecological Momentary Assessment with Mobile Phones
http://www.otago.ac.nz/psychology/otago047475.pdf
Ethno-Mining:
Anderson et al. 2009: Numbers Have Qualities Too: Experiences with Ethno-Mining
Mixed Methods Research Design:
Miles MB, Huberman AM (1994) Qualitative Data Analysis: An Expanded Sourcebook. Sage Publications, Inc, Thousand
Oaks
Tashakkori A, Teddlie C (2009) Integrating qualitative and quantitative approaches to research. In: Bickman L, Rog DJ (eds)
The SAGE handbook of social research methods, 2d ed. Sage, Los Angeles, pp 283 – 317
Teddlie C, Tashakkori A (2006) A general typology of research designs featuring mixed methods. Res Sch 13:12–28.