Today, billions of activities and interactions happen online. The level of interactions with online applications are getting more complex as well. Within the UX, we have the opportunity to understand collective behavior and various experiences through big data.
Specifically, large scale and strategical directions to products can be determined and evaluated through big data behavioral analysis. In this talk, I will go through various types of research objectives, appropriate methodologies and explain how we can use quantitative methodologies to solve UX and user behavior problems and drive product strategy. In this presentation, I will go through a couple of example case studies and topics where behavioral data can help us better understand users and inform strategic product development.
Presented by Saide Bakhshi
Improving Your Surveys and Questionnaires with Cognitive InterviewingUXPA International
Do you use surveys or questionnaires in your work? If so, it is important to be sure you are collecting the data you need. Too often, we write surveys without much attention to how participants will understand and respond to our questions. Just like forms, instructions, websites, or any other product, users often have their own interpretations of our content.
We can improve our survey questions with a method called cognitive interviewing. This method was developed to improve questionnaires for large survey companies and government organizations. The method is similar to usability testing, as it evaluates how a user experiences a survey, but there are also some critical differences between the methods. In this session, we will describe cognitive interviewing, and show how it is similar to and different from usability testing. We will discuss how to conduct cognitive interviews, what data to collect, and how to analyze the results.
Presented by Jean Fox, Jennifer Edgar and Scott S. Fricker
Less is More: An Empirical Investigation of the Relationship Between Amount o...UXPA International
Does more information elicit users’ compliance and engagement, or the other way around?
This paper explores the relationship between content strategy and user experience (UX). Specifically, we examine how the amount of information provided on marketing web pages, often called “landing pages,” impact users’ willingness to provide their e-mail address (a behavior called “conversion” in marketing terms). We describe the results of two large-scale online experiments (n= 535 and n= 27,900) conducted in real-world commercial settings. The observed results indicate a negative correlation between the amount of information on a web page and users’ decision-making and engagement.
Presented by Nim Dvir
Privacy, Ethics, and Big (Smartphone) Data, at Mobisys 2014Jason Hong
Keynote talk I gave at the Mobile and Cloud Workshop at Mobisys 2014. I talk about my experiences and reflections on privacy, focusing on (1) Urban Analytics, (2) Google Glass, and (3) PrivacyGrade.
How to Analyze the Privacy of 1 Million Smartphone AppsJason Hong
These slides are from a briefing to Congressional staffers about privacy, October 30 2014. It talks about our ongoing work with PrivacyGrade.org, which uses crowdsourcing techniques plus static analysis techniques to infer the privacy-related behaviors of apps.
Buttons on forms and surveys: a look at some research 2012Caroline Jarrett
Does 'Submit' or 'Send' or 'OK' go to the left or right of 'Cancel'? Does 'Next' go to the left or right of 'Previous'? This talk at the Information Design Conference 2012 discusses three research studies on forms and surveys.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Improving Your Surveys and Questionnaires with Cognitive InterviewingUXPA International
Do you use surveys or questionnaires in your work? If so, it is important to be sure you are collecting the data you need. Too often, we write surveys without much attention to how participants will understand and respond to our questions. Just like forms, instructions, websites, or any other product, users often have their own interpretations of our content.
We can improve our survey questions with a method called cognitive interviewing. This method was developed to improve questionnaires for large survey companies and government organizations. The method is similar to usability testing, as it evaluates how a user experiences a survey, but there are also some critical differences between the methods. In this session, we will describe cognitive interviewing, and show how it is similar to and different from usability testing. We will discuss how to conduct cognitive interviews, what data to collect, and how to analyze the results.
Presented by Jean Fox, Jennifer Edgar and Scott S. Fricker
Less is More: An Empirical Investigation of the Relationship Between Amount o...UXPA International
Does more information elicit users’ compliance and engagement, or the other way around?
This paper explores the relationship between content strategy and user experience (UX). Specifically, we examine how the amount of information provided on marketing web pages, often called “landing pages,” impact users’ willingness to provide their e-mail address (a behavior called “conversion” in marketing terms). We describe the results of two large-scale online experiments (n= 535 and n= 27,900) conducted in real-world commercial settings. The observed results indicate a negative correlation between the amount of information on a web page and users’ decision-making and engagement.
Presented by Nim Dvir
Privacy, Ethics, and Big (Smartphone) Data, at Mobisys 2014Jason Hong
Keynote talk I gave at the Mobile and Cloud Workshop at Mobisys 2014. I talk about my experiences and reflections on privacy, focusing on (1) Urban Analytics, (2) Google Glass, and (3) PrivacyGrade.
How to Analyze the Privacy of 1 Million Smartphone AppsJason Hong
These slides are from a briefing to Congressional staffers about privacy, October 30 2014. It talks about our ongoing work with PrivacyGrade.org, which uses crowdsourcing techniques plus static analysis techniques to infer the privacy-related behaviors of apps.
Buttons on forms and surveys: a look at some research 2012Caroline Jarrett
Does 'Submit' or 'Send' or 'OK' go to the left or right of 'Cancel'? Does 'Next' go to the left or right of 'Previous'? This talk at the Information Design Conference 2012 discusses three research studies on forms and surveys.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Researching Social Media – Big Data and Social Media AnalysisFarida Vis
Researching Social Media – Big Data and Social Media Analysis, presentation for the Social Media for Researchers: A Sheffield Universities Social Media Symposium, 23 September 2014
Dark Data and Improving Human Rights in Fulton CountyAnidata
In this talk, Dr Baxley discusses the application of data science techniques to fight human trafficking. He gives a brief overview of Anidata, then dives into the technical details of the implementation of the graph-based entity resolution algorithm developed and implemented using Python, Luigi, and NetworkX.
Using Data Science for Social Good: Fighting Human TraffickingAnidata
In this talk, Vincent Emanuele tells the story of how Anidata has applied data science techniques in collaboration with law enforcement to fight human trafficking.
Lecture on ethical issues taught as part of Heriot-Watt's course on Conversational Agents (2021). Topics covered:
- General Research Ethics with Human Subjects
- Bias and fairness in Machine Learning
- Specific Issues for ConvAI
Presentation for: Masterclass 19: Using social media in public engagement for the Public Engagement & Impact Team at The University of Sheffield, 26 November 2014.
Cloud of Knowing MRS 2010 conference slides - the award winning paperJohn Griffiths
The actual presentation I gave at the Market Research Society conference 2010. There is an earlier preview uploaded here on slideshare from I think the Cloud 2 meetup when I did a dummy run in front of the group. The presentation was shortlisted for best presentation and won best new thinking the last time the award was given funnily enough
Leveraging Human Factors for Effective Security Training, for ISSA 2013 CISO ...Jason Hong
Talk I gave at ISSA 2013 CISO forum, looking at some human factors issues in cybersecurity. I discuss some of our research in anti-phishing, user interfaces, mental models of cybersecurity, and ways of motivating people.
Brief tutorial on Influence and Homophily in social networks. Key concepts. How to distinguish influence from correlation. Information diffusion processes. Influence Maximization Problem
and viral marketing.
Picturing the Social: Talk for Transforming Digital Methods Winter SchoolFarida Vis
This talk highlights the work of the Visual Social Media Lab and the Picturing the Social project. It summarises the key research questions and aims of the project. It highlights the value of interdisciplinarity and working closely with industry in this area. It also focuses on the way in which me might study different types of structures involved in the circulation and the scopic regimes that make social media images more or less visible. It also tries to unpack how we can start to think about APIs as 'method' and looks at the different ways in which we can get access to different kinds of social media image data. Both through public ('free') APIs and ('pay for') firehose data.
It is essential for a business organization to get the customer feedback in order to grow as a company. Business organizations are collecting customer feedback using various methods. But the question is ‘are they efficient and effective?’ In the current context, there is more of a customer oriented market and all the business organizations are competing to achieve customer delight through their products and services. Social Media plays a huge role in one’s life. Customers tend to reveal their true opinion about certain brands on social media rather than giving routine feedback to the producers or sellers. Because of this reason, it is identified that social media can be used as a tool to analyze customer behavior. If relevant data can be gathered from the customers’ social media feeds and if these data are analyzed properly, a clear idea to the companies what customers really think about their brand can be provided.
Researching Social Media – Big Data and Social Media AnalysisFarida Vis
Researching Social Media – Big Data and Social Media Analysis, presentation for the Social Media for Researchers: A Sheffield Universities Social Media Symposium, 23 September 2014
Dark Data and Improving Human Rights in Fulton CountyAnidata
In this talk, Dr Baxley discusses the application of data science techniques to fight human trafficking. He gives a brief overview of Anidata, then dives into the technical details of the implementation of the graph-based entity resolution algorithm developed and implemented using Python, Luigi, and NetworkX.
Using Data Science for Social Good: Fighting Human TraffickingAnidata
In this talk, Vincent Emanuele tells the story of how Anidata has applied data science techniques in collaboration with law enforcement to fight human trafficking.
Lecture on ethical issues taught as part of Heriot-Watt's course on Conversational Agents (2021). Topics covered:
- General Research Ethics with Human Subjects
- Bias and fairness in Machine Learning
- Specific Issues for ConvAI
Presentation for: Masterclass 19: Using social media in public engagement for the Public Engagement & Impact Team at The University of Sheffield, 26 November 2014.
Cloud of Knowing MRS 2010 conference slides - the award winning paperJohn Griffiths
The actual presentation I gave at the Market Research Society conference 2010. There is an earlier preview uploaded here on slideshare from I think the Cloud 2 meetup when I did a dummy run in front of the group. The presentation was shortlisted for best presentation and won best new thinking the last time the award was given funnily enough
Leveraging Human Factors for Effective Security Training, for ISSA 2013 CISO ...Jason Hong
Talk I gave at ISSA 2013 CISO forum, looking at some human factors issues in cybersecurity. I discuss some of our research in anti-phishing, user interfaces, mental models of cybersecurity, and ways of motivating people.
Brief tutorial on Influence and Homophily in social networks. Key concepts. How to distinguish influence from correlation. Information diffusion processes. Influence Maximization Problem
and viral marketing.
Picturing the Social: Talk for Transforming Digital Methods Winter SchoolFarida Vis
This talk highlights the work of the Visual Social Media Lab and the Picturing the Social project. It summarises the key research questions and aims of the project. It highlights the value of interdisciplinarity and working closely with industry in this area. It also focuses on the way in which me might study different types of structures involved in the circulation and the scopic regimes that make social media images more or less visible. It also tries to unpack how we can start to think about APIs as 'method' and looks at the different ways in which we can get access to different kinds of social media image data. Both through public ('free') APIs and ('pay for') firehose data.
It is essential for a business organization to get the customer feedback in order to grow as a company. Business organizations are collecting customer feedback using various methods. But the question is ‘are they efficient and effective?’ In the current context, there is more of a customer oriented market and all the business organizations are competing to achieve customer delight through their products and services. Social Media plays a huge role in one’s life. Customers tend to reveal their true opinion about certain brands on social media rather than giving routine feedback to the producers or sellers. Because of this reason, it is identified that social media can be used as a tool to analyze customer behavior. If relevant data can be gathered from the customers’ social media feeds and if these data are analyzed properly, a clear idea to the companies what customers really think about their brand can be provided.
Slides for class session I taught at USC Annenberg on approaching big data for a non-technical audience so that they can learn the project planning skills to work with technical teams. The goal is to teach students the mindset they should when taking in mixed methods and applying to large datasets prior to selecting software packages and methodology. The slides take us through a previous use case and guidance moving forward from a process and cross-functional team perspective.
Practical Applications for Social Network Analysis in Public Sector Marketing...Mike Kujawski
Over the past decade there has been a growing public fascination with the complex connectedness of modern society. This has been driven in large part by the wide availability of public digital data produced through our daily interactions on the modern social web. This data can now easily be mined and analyzed to produce valuable and actionable business insights leading to better decision making in nearly every field of practice, especially marketing and communications. In this presentation, Joshua Gillmore and Mike Kujawski introduce the basics of social network analysis and some of the privacy related challenges that this rapidly growing space brings with it. Focus of this deck is on public sector organizations.
By: @mikekujawski and @joshuagillmore
Social Media Mining - Chapter 10 (Behavior Analytics)SocialMediaMining
R. Zafarani, M. A. Abbasi, and H. Liu, Social Media Mining: An Introduction, Cambridge University Press, 2014.
Free book and slides at http://socialmediamining.info/
UXPA 2023: Start Strong - Lessons learned from associate programs to platform...UXPA International
Imagine creating experiences for your rookie designers’ first couple years that are rewarding, enriching, and full of learning — without taking all your time or energy to manage. We’ll share techniques any team leader can put into practice using real-life examples from associate programs, apprenticeships, and internships.
Topics include onboarding, varied work challenges, developing multiple capabilities, buddy systems, group sharing, guest speakers, time with executives, and mentorship. We’ll also share how to operationalize learning, soft skills like communication and collaboration, setting boundaries, time management, achieving deep work, and more skills we all wish we were explicitly taught early on.
We’ll focus on modern-day associate programs, but even if you can’t create a full-fledged program, you’ll leave this session with ideas to use with your fledgling professionals. The benefits go beyond efficiency; it’s a foundation for culture, camaraderie, autonomy, and mastery.
UXPA 2023: Disrupting Inaccessibility: Applying A11Y-Focused Discovery & Idea...UXPA International
Digital advances are being made at a rapid-fire pace, yet disability inclusivity continues to fall short of the digital revolution. As the number of people living with disabilities rises, the time to take digital accessibility to the next level is now. Let’s disrupt inaccessibility together! Come hear about a multi-part discovery research and ideation project informing foundational UX designs for our customers. You’ll get insights from our unique study, which are widely applicable across industries, and walk away with tips and inspiration to kick off your own accessibility-focused discovery and ideation. Only YOU can prevent inaccessibility – are you in?
User experience can be drastically elevated by combining data science insights with user-based insights from research. Data analytics on its own can make themes and correlations difficult to explain and to provide accurate recommendations. For example, themes identified via large global surveys and usage data can be better understood with UX insights from focused user research, such as user interviews and/or cognitive walkthroughs. This presentation will highlight the complimentary nature of data science and UX and will focus on the benefits of bringing the two disciplines together. This will be buttressed with practical examples of enterprise projects and applications that combined data and skills from the two disciplines, guidance on how the two disciplines can better work together, and the skills needed to improve as a UX professional when working with data science teams.
UXPA 2023: UX Fracking: Using Mixed Methods to Extract Hidden InsightsUXPA International
Users do not always accurately describe what they mean or feel. There are many reasons for this, ranging from politeness to poor introspection, to lack of sufficient technical vocabulary. Fortunately, UX researchers have tools in their trade to deduce what was really meant. We call this UX Fracking, a mixed methods approach that is optimized for extracting hidden user insights. We will illustrate the dangers of inadequate, superficial research, and how this may lead to outcomes incapable of addressing the users’ core issues. We will explore ways to avoid these pitfalls by leveraging mixed research methods to test hypotheses about the users’ intent and needs. This starts with a thorough understanding of who the user is, their goals, and how they work today, to an approach that combines surveys, interviews, and comment analysis with behavioral observation, and finally, validating the newly discovered user insights with the users themselves.
UXPA 2023: Learn how to get over personas by swiping right on user rolesUXPA International
This session walks through the concept of user roles as an alternative to personas as a means to generate and disseminate user insights for product development teams. We will describe the tools and methods used to create a research database organized by user roles, along with examples and short exercises to help attendees think through user roles within their own context.
By the end of the session, attendees should be aware of tools and approaches for:
Organizing user research information in a database
Disseminating user role information to product and design teams
Managing a user roles database as part of a long term UX Research program
If you’re ready to ditch personas but don’t know how, this session is for you!
We will present a case study that details our approach for replacing user personas with user roles for a multi-national SAAS company. We will take the audience on a journey that starts with an executive request for personas, travels through the tribulations of realizing personas suck, and concludes with convincing others to accept a new and innovative way to understand the people who use the product. Our key message is that personas lack real value for organizations that already understand the importance of empathizing with users. Building user-centered products requires easily accessible and well organized user insights. We will discuss defining users through a process of stakeholder consultation and content review, and structuring data around Jobs to Be Done and product interactions. We will also discuss the dissemination of user roles in our organization using relational databases, interactive dashboards and online wikis. Spoiler alert, our stakeholders loved user roles!
UXPA 2023: Experience Maps - A designer's framework for working in Agile team...UXPA International
Agile Methodology refers to software design and development methodologies centered around the idea of iterative design and development, where requirements and concepts evolve through collaboration between self-organizing cross-functional teams. Thus, Agile enables teams to deliver value faster, with greater quality and predictability, and greater aptitude to respond to change. With evolving product features every design sprint, designers & researchers find it difficult to follow the design process. This sometimes leads to designs delivered in haste or sub-par design artifacts which result in UX debt. UX debt is accumulated when design teams take actions or shortcuts to expedite the delivery of a piece of functionality or a project which later needs to be refactored. It is the result of prioritizing speedy delivery of design to the development team over a perfect experience journey. Experience Maps is a great tool to practice UX in Agile as well as manage UX Debt.
UXPA 2023: UX Enterprise Story: How to apply a UX process to a company withou...UXPA International
How to build a UX Department from scratch, in an environment they think UX people do social media posters and posts! An agile implementation just started, and people are moving from a waterfall and ad-hoc mindset to agility. In this session, I will talk about my Journey to establish a UX Department for a company that is part of a global brand, but this local branch just started the digital transformation movement. Challenges like: spreading awareness and educating people about UX, hiring the right team, defining the right team structure, establishing workflow and day-to-day operations, and applying localization (non-western culture).
UXPA 2023: High-Fives over Zoom: Creating a Remote-First Creative TeamUXPA International
I started my current job in March of 2020. Many of us remember something clearly about the month that COVID started to shut things down. I remember being surprised to hear that my new on-site-only job would be starting in my living room over zoom. How do you lead a design team when none of the team members live near each other and creativity is highly collaborative? Taking from over a decade of working in HR software, I knew whatever I did needed to put people first. That what employees love about a job is often deeper than the work, it’s the culture, the relationships and people they work with. It’s the feeling that their work has value, and their contribution matters. In this talk I will walk though some of the rituals and best practices I have learned over the last two years building a remote-first creative team.
UXPA 2023: Behind the Bias: Dissecting human shortcuts for better research & ...UXPA International
As humans, we are biased by design. Our intricate and fascinating brains have developed shortcuts through centuries of human evolution. They reduce an unimaginable load of paralyzing decisions, keep us alive, and help us navigate this complex world. Now, these life saving biases affect how we behave with modern technology. Understanding some of the theories and reasons why these biases exist is the key to unlocking their power. In this workshop we will cover some theories around how the brain works. We will review some of our mental shortcuts, take a look at some common biases, and learn how they affect our users, our research, and our designs. Lastly we will review some advantages of biases, and ways to identify and reduce bias. This workshop is targeted for designers who do their own research, and researchers looking to learn more about removing bias from their studies.
UXPA 2023 Poster: Improving the Internal and External User Experience of a Fe...UXPA International
UXPA 2023 Poster: Improving the Internal and External User Experience of a Federal Government Legacy Application Using User Experience and Agile Principles
Are you new to UX management, or thinking of getting into management? Then this talk is for you. After reading countless books, attending countless trainings, mentoring and being menteed, nothing quite prepared me for management like my first year. I’ll share with you what I wish they’d told me. I’ll also share my process for generating team research roadmaps, establishing team values, keeping employees motivated, and not burning out.
UXPA 2023: Redesigning An Automotive Feature from Gasoline to Electric Vehicl...UXPA International
Join us for an interaction design case study from the automotive industry. We created a Human-Machine Interface (HMI) for a vehicle feature that provides household-levels of power in electrical outlets for our customers to use at work and play. This case study will reveal: · Our debate of re-using version 1.0’s HMI vs designing a new user interface for the electric vehicle—when to break with consistency and why? · User research we conducted to guide our early design concept. · Paper prototypes we created to support our usability testing of the concept with vehicle owners. · How we solved internal debate over the interaction design in moving from internal combustion vehicles to electric vehicles. * Advice to help you evangelize user-centered design that is also brand-centered for a new product.
Have you ever wondered how search works while visiting an e-commerce site, internal website, or searching through other types of online resources? Look no further than this informative session on the ways that taxonomies help end-users navigate the internet! Hear from taxonomists and other information professionals who have first-hand experience creating and working with taxonomies that aid in navigation, search, and discovery across a range of disciplines.
Acorn Recovery: Restore IT infra within minutesIP ServerOne
Introducing Acorn Recovery as a Service, a simple, fast, and secure managed disaster recovery (DRaaS) by IP ServerOne. A DR solution that helps restore your IT infra within minutes.
This presentation, created by Syed Faiz ul Hassan, explores the profound influence of media on public perception and behavior. It delves into the evolution of media from oral traditions to modern digital and social media platforms. Key topics include the role of media in information propagation, socialization, crisis awareness, globalization, and education. The presentation also examines media influence through agenda setting, propaganda, and manipulative techniques used by advertisers and marketers. Furthermore, it highlights the impact of surveillance enabled by media technologies on personal behavior and preferences. Through this comprehensive overview, the presentation aims to shed light on how media shapes collective consciousness and public opinion.
0x01 - Newton's Third Law: Static vs. Dynamic AbusersOWASP Beja
f you offer a service on the web, odds are that someone will abuse it. Be it an API, a SaaS, a PaaS, or even a static website, someone somewhere will try to figure out a way to use it to their own needs. In this talk we'll compare measures that are effective against static attackers and how to battle a dynamic attacker who adapts to your counter-measures.
About the Speaker
===============
Diogo Sousa, Engineering Manager @ Canonical
An opinionated individual with an interest in cryptography and its intersection with secure software development.
Sharpen existing tools or get a new toolbox? Contemporary cluster initiatives...Orkestra
UIIN Conference, Madrid, 27-29 May 2024
James Wilson, Orkestra and Deusto Business School
Emily Wise, Lund University
Madeline Smith, The Glasgow School of Art
This presentation by Morris Kleiner (University of Minnesota), was made during the discussion “Competition and Regulation in Professions and Occupations” held at the Working Party No. 2 on Competition and Regulation on 10 June 2024. More papers and presentations on the topic can be found out at oe.cd/crps.
This presentation was uploaded with the author’s consent.
Competition and Regulation in Professional Services – KLEINER – June 2024 OEC...
How to use Big Data to drive product strategy and adoption
1. How to use big data to drive UX
strategy?
1
Saeideh Bakhshi
Quantitative UX Researcher, Facebook
bakhshi@fb.com
UXPA 2018
2. At Georgia Tech College of Computing
Started PhD in CS
2009 2013
Focused on behaviors that drove users'
engagement with photos
Started working at Y! Labs
As a Quantitative UX Researcher
Started working at Facebook
2016
2012
Got interested in HCI and started
working on user engagement
Changed thesis topic
2014
And started to work full time at Yahoo
Labs
Graduated with PhD
3. Agenda
3
1
How does big data fit into UX?
2
Which big data methods are
useful for UX?
3
What are some examples of
using big data methods?
4
How is this different from
what data scientists/analysts
do?
6. Concept
Qualitative
Concerned with
understanding human
behavior from end user's
perspective
Assumes dynamic and
negotiable reality
Used to gain an
understanding of
reasons, opinions, and
motivations
Less generalizable
Quantitative
Concerned with
understanding behavior
from researcher's
perspective
Assumes a fixed and
measurable reality
Uses measurable data to
formulate facts and
uncover patterns
More generalizable
Minichiello, 1995
Method
Qualitative
Data is collected
through participant
observation or
interviews
Data is presented
through the language
of the end user
More in-depth
information on a few
cases
primary inductive
process used to
formulate theory or
hypotheses
Quantitative
Data is collected
through measuring
things
Data is presented
through the language
of statistical analysis
Less in-depth but more
breadth of information
across a large number
of cases
Primary deductive
process used to test
pre-specified concepts,
constructs, and
hypotheses that make
There's been a lot of progress
made in quantitative research
methods since 1995
7. Advantages of using big data
7
1Actions may tell a different
story than words. 2We can find new patterns that
were previously hidden.
3
Technology use cases are
getting more complex and
harder to find patterns. 4A lot of this data is already
available.
8. Big data methods can be useful at various stages of user
research
StrategicFoundational
Predictive and Hypothesis
driven
Strategic research can specially
benefit from big data methods
where we need to test
hypotheses about behaviors or
predict trends.
Exploratory & Generative
Variety of big data methods (e.g.
unsupervised methods) can help
us explore behaviors and
attitudes associated with product.
Characterize use and
measure success
After a product is launched, big
data methods can help us
characterize behaviors, measure
success and connect the two.
Evaluative
8
9. 9
Theorize/Predict what's
next: We can use past
behavior to predict future
behavior and strategize
products around that.
Characterize
behaviors and
estimate success
Use unsupervised
methods to find
patterns and trends Conceptual model
around behavior: Based
on findings, we can come up
with theories on how/why it
is happening
Test
hypotheses
Analysis: We can analyze
large scale behavioral data to
find patterns around the product
or topic we are interested in
How can behavioral data help us with strategy?
10. What are some useful machine learning
methods we can use with large data?
2
10
11. 11
Problem type Big data methods Types of data
Exploratory
exploratory data analysis, unsupervised
methods (clustering, assoication rules),
text analysis
behavioral or user generated data or self
reported data
Deep dive in an existing behavior
supervised methods (regression,
classification)
behavioral + attitudinal
Hypothesis validation
Hypothesis testing, supervised methods
(regression, classification)
behavioral + attidunal
Trends forecasting
Timeserie analysis, Forecasting,
prediction
behavioral
12. Data Preparation,
exploration and
reduction: Collection,
Cleaning and preparing
data, bivariate analysis,
visualization,
dimensionality reduction
Prediction: Linear
regression, K-nearest
neighbors, Regression trees,
Neural networks, Ensembles
Time-series analysis:
Regression based,
Smoothing methods
Theorize/Predict
what's next: Sometimes,
we can use past behavior
to predict future behavior
and strategize around that
Classification: K-nearest
neighbor, Naive-bayes,
Classification trees, Logistic
regression, Neural networks,
Discriminant analysis, Ensembles
What goes together:
Association rules,
collaborative filtering
Segmentation:
Clustering
Model evaluation
and selection:
accuracy and
performance evaluations
Descriptive analysis:
bivariate and
multivariate analysis,
Data visualization
Insights: Gather insights
How to use machine learning in UX to drive strategy?
14. Are the social signals of a review indicative of the
polarity of the text?
How do users use various voting options on
Yelp reviews?
Are the social signals of a review indicative of how much
the rating differs from the mean rating for that business?
14
15. 15
We obtained data that was shared by Yelp.
11,537 businesses
43,873 reviewers
229,907 reviews
It contained, data about businesses in Phoenix,AZ area.
Yelp data
16. We created the response variable to indicate the difference between
review's rating and the average rating
Business's avg rating Review's rating
Funny, cool and useful
votes
Different set of variables we considered
16
18. Take-aways
18
1
Funny votes imply lower
ratings and higher negativity
in the tone. 2
Useful votes imply lower
ratings and lower positivity in
the tone.
3
Cool votes imply higher
ratings and higher positivity
in the tones. 4
Community signals carry
implied meaning beyond
their specific labels.
We used a big data model to better understand how our users use the features that are available to them on the site.
19. Connection: What is the structure of social connection
on Pinterest? For example, are women or men more
connected?
Comparison: How does behavior on Pinterest compare
to behavior on other social network sites? Letting Twitter
be our point of reference, do Pinterest and Twitter users
systematically differ in what they talk about?
What is Pinterest and what do people do
there?
Activity: What drives user activity on Pinterest: what
about a pin—and its pinner—“grabs the attention” of other
users? For example, what role does gender play? Do pins
from women receive more, less, or equal attention than
those from men?
19
21. Pins
21
Pin is a unit of analysis for content on Pinterest
• We collected data around pins:
• repins: how many times this pin was repinned
• likes: how many other users liked this pin
• comments: textual comments included by the pin’s
pinner
• text: the pin’s description and comments by others
22. Users on Pinterest are called pinners.They can create
boards and pin images to their boards.
Pinners
22
Aenean eu leo quam Vehicula Etiam sed diam eget risus varius
blandit sit amet non magna.
• We collected data around pinners:
• followers: how many pinners follow this pinner
• follows: how many users this pinner follows
• boards: how many boards this pinner has created
• pins: how many pins this pinner has created
23. we also collected additional data features that could
help us understand Pinterest activities better.
Additional data
23
• location: a free-text description of a pinner’s location,
via either Pinterest or Facebook; if they specified it in
both places, we used the Pinterest location
• gender: a pinner’s self-reported gender via their
Facebook profile page
• tweet text: via their linked Twitter handle, the text of
all tweets written by this pinner in the last year
24. Take-aways
24
1Being American or British
earns repins. 2Being female means fewer
followers.
3Pinterest’s verbs: use, look,
want, need. 4Twitter: lol, watching now.
26. We run a negative binomial regression with number
of followers as a dependent variable.
Understanding follows
26
Main findings:
Gender: being female is associated with lower number
of followers
Country: Being American or British earns repins.While
we were unable to analyze every country in our dataset
because of sparsity, we do find that pinners from the
United States and the United Kingdom attract more
repins than the rest of the world.
27. We plot the distributions of followers for each
gender:
Gender follow up analysis
27
while the median male follower
count is lower than the female median follower
count (67 vs. 86), the mean male follower count is
substantially higher than the mean female follower
count (1,063 vs.270).
28. What differentiates posts on Pinterest from the posts on Twitter?
28
Pinterest’s verbs: use, look, want, need.
31. How is this different from Data Science/
Analytics?
4
31
32. Basic unit is interactionBasic unit is task Basic unit is action
32
Qualitative methods DS big data methodsUX big data methods
Large dataBasic unit is task Large data
Data from replication of real
world usage
Data from real world usage Data from real world usage
Human centric Human centric Business centric
Answers why
Answers what, sometimes
why
Answers what
33. Footercontent20pts.Loremipsumdolorsitamet,erosverteremsalutatusnamte.Viscutamquamconclusionemque.
With Large scale behavioral
analysis we can be both
human centric and business
centric.
We can connect people's
needs and behaviors to
business metrics and propose
new metrics using behavior.
Small scale
Large scale
Business centric Human centric
Qualitative
UX methods
Data Science
Methods Large-Scale
Behavioral
Methods
Survey
Methods
Large scale machine learning methods can help us report business
metrics based on actual behavior
34. Accuracy
Explainability
UX models are often
intended to be
descriptive
Business models often
aim to be predictive
Explainability vs accuracy trade-off
35. 1 2
Make sure to learn and understand how data
is collected and what are the limitations of it.
3 4 Having access to big data and being able to
analyze that data will not do you a bit of good
if you are unable to translate those efforts into
successful actions.
Keep it simple: that any modeling exercise
has inherent risk.Although advanced
statistical methods indisputably make for
better models, complex models are often not
practical.
Big data can't answer all types of research questions
Start with a hypothesis
Complexity is not
always better
Understand your data
Turn insights into
actions
It's easy to get lost with big data. Often
projects are exploratory so there is always a
need to approach these projects with
established hypotheses.
Things to remember
35
36. Be ready to triangulate with
qualitative methods
36
Algorithms are not as good as Humans in telling
the full story
37. How to use big data to drive UX
strategy?
37
Saeideh Bakhshi
Quantitative UX Researcher, Facebook
bakhshi@fb.com
UXPA 2018