What Is User Research by Dropbox Product Manager
•Download as PPTX, PDF•
3 likes•359 views
Product Management Event Held at the Product Conference in San Francisco. What is user research from the PM prospective? Ketan shared what the value in user research is and why it's so important to product managers. He also gave insight on how to empower people, inspire empathy and uncover truths using user research. Ketan Nayak is a Product Manager at Dropbox. He has 4 years of Product Management experience in various software companies. Before Dropbox, he worked at Applied Predictive Technologies in Washington D.C. Before becoming a PM, he studied his Master's at MIT and Bachelor's at IIT, Madras.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to What Is User Research by Dropbox Product Manager
Similar to What Is User Research by Dropbox Product Manager (20)
More from Product School
More from Product School (20)
Recently uploaded
Recently uploaded (20)
What Is User Research by Dropbox Product Manager
- 1. What Is User Research by Dropbox Product Manager www.productschool.com
- 3. @KetanNayak
- 5. Image from Boston Burger Company
- 9. @KetanNayak
- 10. JTBD USER STORY I’m coming to [product] because I need [JTBD] When I [context], I want to [goal] so I can [success criteria] FOCUS; PRODUCT LEVEL SPECIFICITY; FEATURE LEVEL ble from Cristen Torrey @KetanNayak
- 11. INDIVIDU AL BUSINES S JTBD #1 JTBD #2 JTBD #1 JTBD #2 Feature Feature Feature Feature Feature Feature Feature Feature Feature Feature Feature Feature @KetanNayak
- 12. What’s your product hired for? EMPOWER
- 16. Phase 0 0 - Why? 1 - How? 2 - What? @KetanNayak
- 17. EMPATHIZE Start with the WHY
- 20. @ketannayak Image from TechCrunch Image from Facebook
- 22. @KetanNayak
- 24. Image from Boston Burger Company EMPOWER EMPATHIZE UNCOVER Jobs to be done Power of the WHY Quantified and unbiased
- 27. Part-time Product Management Courses in San Francisco, Silicon Valley, Los Angeles, New York, Austin, Boston, Seattle, Chicago, Denver, London, Toronto www.productschool.com
Editor's Notes
- Good Morning Thank being here Pleasure Josh Product insightful valuable Ketan excited User Research - stories , perspective ----------- Good morning everyone! Thank you all for being here It is a pleasure to hear Josh speak - he always has something insightful to share around product and it’s valuable to all of us! I’m Ketan and I’m excited to be here speaking in front of all of you Today talk about user research -> share my stories about user research from the perspective of a PM
- Before begin - myself building ecosystem and platform - strategic Growth - analytics DI Enterprise SaaS , prior unsuccessful MIT ------- A little bit about myself. I’m Ketan, currently a PM at Dropbox building out the Dropbox ecosystem and platform through our strategic integrations and partnerships Previously I was the PM there on growth related analytics and data infrastructure Prior to Dropbox I worked at an enterprise SaaS startup and prior to that unsuccessfully tried to build startups out of MIT
- So UR Worth? Value? Value obvious Ask to tell, recite mission EP IE UT EP - 2 groups Own org - various parts - eng, marketing, sales Users - needs and struggles IE - creators deeply understand UT - validate/invalidate -------- So User Research Why is user research worth talking about? What value does it provide? As PMs or aspiring PMs the value of user research is obvious to most of us If you were to ask me to tell you the value of user research in one line, I would then recite to you the mission of our user research team, which is Empower people by inspiring empathy and uncovering truths EP -> people is two groups First is within your own organization, great user research empowers various parts of your company, your engineers to build the right product, marketing to send out the right message, sales to sell to the right people The second group is the customers whom you can empower to overcome their needs and struggles IE ->Allows creators like us to deeply understand our users UT -> Minimizes risk by validating or invalidating assumptions while building a product
- Few weeks, break three down and talk Totally randomly, humongous dessert, boston burger company Famous, look at this Anyway, realized 3 things - MCC -> interesting lessons EP, IE, UT ---- A few weeks ago when I was preparing for this talk, I was thinking about to break these three values of user research down and talk about them Then Totally randomly, I ended up thinking about this humongous dessert that I had a while back at this restaurant called the boston burger company Now, Boston burger company is famous for their desserts. I mean look at this thing! Anyway I realized that the three things this dessert had, the milkshake, the cake and the cookies could actually point to some interesting lessons around empowering people, inspiring empathy and uncovering truths
- So today, talk share stories EP from M ID from C UT from C ----- So today, I want to talk about and share some stories around how we can learn empower people from milkshakes, how we can empathize deeply through cupcakes and how we can uncover the truths via cookies Not sure what I’m talking about? We’ll figure it out
- 1st Milkshake Product you’re managing Improve buy more how? Traditional, competitors, ask make it better Companies failed CC - fame - book CAL - diff approach FYI “What jobs hiring” - spawned Premise - “what jobs - struggles - circumstance” How customers - our product - struggles ------ First up, the milkshake! For a second, think that the product you’re managing is a milkshake If you want to improve the milkshake and get customers to buy more of it. How would you go about doing that? Traditional thought dictates that you look at competitors, do a market analysis, ask your customers how you can make it better That’s what milkshake companies were trying out and it was totally failing But Clayton Christensen of the Disruptive innovation fame in his book competing against luck recommended a totally different approach for this FYI, if you haven’t read the book Competing against Luck, I highly highly recommend it Clay instead asked a different question “What job are your users hiring your product to do?” that spawned off the “Jobs to be done” theory The primary premise of jobs theory is “What jobs are customers trying to accomplish and how can we help with their struggles in the circumstances in which these jobs arise. How can customers hire our product to resolve their struggles
- Applying theory - 2 groups Alone - early - themselves - commute - long time full longer finish Narrower straw Parents - say yes - kids - guilty Smaller size Two jobs - two groups - 1 product Deep insight - jobs circumstances - non intuitive Nothing container consumed ----- In applying the jobs theory then they found at least two groups of milkshake buyers The first group bought milkshakes alone early in the morning for their long commute and preferred milkshakes because it took a long time to finish and kept them full An improved product for them actually takes longer to finish So you might make the straw narrower The second group of buyers who were parents who wanted to say yes when their kids asked for a milkshake but didn’t want to feel guilty For them an improved product is a milkshake that’s of a smaller size that they can give to their kids Two totally different jobs being met by two groups of customers with the same product But this deep insight around jobs, and understanding circumstances in which jobs arise actually reveals totally non-intuitive ways to improve products In this case the improvements had nothing to do with the milkshake itself, it was improvements to the container and how it was consumed
- Want products, hiring job done Value theory - jobs stable - time Rather - variable - requests, ideas - stable Jobs same - solved diff time Job could be mood music Diff cassette - streaming Theodore levitt - quarter ----- Furthermore, customers don’t want products, they are hiring them to get a job done And the value of the jobs theory is that the jobs that arise in customers lives are stable and don’t change over time So rather than building products with variable product inputs like customer requests, sales requests or feature ideas, you are instead relying on something that’s is stable over time Jobs stay the same, they’re just solved in different ways over time So a customer’s job could be change their mood with music and over the time that has been solved by cassettes, CDs, ipods, phones and now streaming services Another way to think about this is the quote by HBS professor Theodore Levitt : Customers don’t want a quarter inch drill, they want a quarter inch hole
- Might ask - user stories Type of US - scoped diff - moment purchase Good - initial user expec - purchase motiv - fall short Dropbox - organized home US - ongoing value - satisfied long time Shared file - current version - duplicating Focus product vs. Specific feature level --- One might then ask, what about user stories then? JTBD are just a type of user story, but scoped differently -> scoped to the moment of purchase JTBD are good for understanding initial user expectations and purchase motivations, knowing where and when we fall short of initial user expectations “I’m coming to Dropbox because I need one organized home for my team’s work” User stories tell us how to ensure they realize ongoing value -> become satisfied long term user “When I’m working on a shared file, I want to know where to find the most current version, so I can avoid duplicating work” JTBD provides focus and operates at the product level, whereas User Stories are more specific and operate at the feature level
- DBX - framework extensively As mentioned - Lot of sense - product - hiring DFB? Same product diff jobs Theory - one job reason adopt One job - purchase - some subset ------ At Dropbox we use the Jobs to be done framework extensively across our products As mentioned above, the Jobs To be Done makes a lot of sense at the product level: What jobs is someone hiring Dropbox Business for? Note that the same product could be hired for a variety of jobs, But as per jobs theory, there is only one job that’s the reason a user adopts or buys the product That one job driving the purchase could be attained by some subset of features that your product has or provides
- Next time Shoes of manager Figure out jobs struggles What jobs? --- The next time you’re looking to build a product. Put yourself in the shoes of the manager of a milkshake line and figure out what the jobs to be done for your product to really empower them to overcome their struggles. What job is your customer hiring your product to do?
- Next up 1 of 5 values Delight experience Works great - solves elegant - cupcakey However - only delightful - deeply empathize ------ Next up cupcake One of Dropbox’s 5 company values is simply a cupcake The cupcake represents the delight that a user experiences when they use our product Whenever something works great, solves a problem for a user in an elegant manner, we call it a ‘cupcake experience internally’ However you can only build such a delightful experience for the user if you understand them and empathize with them deeply
- 1st job - mattered - adoption 1st project - better predictive model Assume better model preferred (of course) 6 months - 2x better Pushed prominent Couple weeks - no takers- low Turns out - sales, mkting - savvy In addition - knowledge + smart - bosses - explain well Complicated and blackboxy Struggled understand explain - 2x better Hadn’t “why” certain aspects - empathized - need to explain ------ In my first job as a PM analytics startup, I didn’t realize how much empathy mattered for product adoption one of my first projects was to build a better predictive model Without knowing too much, I inherently assumed that a better predictive model would actually be desired by customers (I mean of course right?) I spent about 6 months on the project, and we came up with a model that was 2x better we pushed this model out to the product in a fairly prominent spot But after a couple of weeks after release, I noticed that there were no takers and usage was really low Turns out our users were mostly sales people or marketing analysts who aren’t the most data savvy In addition to running analyses, these analysts wanted to appear knowledgeable and smart in front of their bosses and be able to explain well about the analysis they were presenting The new model was more complicated and black boxy These users struggled to understand it and explain it and hence simply didn’t use it even though it was 2 times better So, I hadn’t understood “why” our users cared about certain aspects of the model and hadn’t deeply empathized with their need to explain
- Ideally? DD - prod design - british design council PD 2 diamond phases Start - generic prob Expand- research, constrain -insight Specific prob definition Expand ideas- ideation and constrain - prototype Specific soln for general - set to solve Earlier - conflate gen prob with spec def ---- If you were to ask me, how I would do this ideally, I would use the following Some of you may have seen this diagram before that’s called the double diamond model for product design, which was first created by the british design council in 2005 The double diamond diagram says that product design really happens in two diamond phases First you start off with a general problem Then you expand your knowledge of the problem and it’s details through research And then with insight into the problem you constrain it you a more succinct problem definition Once you have the succinct problem definition, you ideate to expand to a set of solutions And then constrain back through prototyping to arrive at the solution for the general problem that you set out to solve In my earlier example, I ended up conflating the problem to the specific definition, didn’t spend enough time on the first diamond
- DBX - exact fallacy Everyone - Drew - cares why DBX - process - phases First diamond - spec prob def - Phase0 Users /segments? User prob/dropbox prob? Why now? Oppty space and hypotheses User research to arrive at crisp prob def All proj - internal/processes - rejected, don’t go thru Why? 1- How? 2- What? (explain) ------ At Dropbox however, we have a product process that’s designed to capture this exact fallacy Everyone going all the way up do Drew cares deeply about the Why? And why we’re doing something At Dropbox, we have a product process that goes through multiple phases, Phase 0, Phase 1 and so on That first diamond of clarifying the specific problem definition is captured within our process as the Phase 0 for the project Phase 0 focused on Who are our users and user segments? User problems/Dropbox problems? Why now? The opportunity space and your hypotheses on success At this stage, you drive toward user research look to arrive at crisp problem definitions We do this for all project, even do this for our internal tools or for processes and Phase 0s or even Phase 1s are rejected or don’t go through Phase 0 - P0 conflation Following the Why? We talk about Phase 1 - the How? And Then Phase 2 - The What
- Empathize - deep understand Before how and what - start why - cupcake exp Look - incorporate - formalize - why Align - team/leadership - why matters? Why solved ------ To empathize with the user, you have to deeply understand them So even before you figure out the How and the What? You have to understand and start with the why? To provide that cupcake experience Look to incorporate a P0 type process to formalize the process of asking the Why more and getting alignment across your team and leadership on why a problem matters and why it should be solved
- Finally - cookies (http) - capture understand data - user behavior Broadly quantified approach certain immensely Josh point - inflection points - change in behavior Twitter -30, Fb - 7 Ecosystem - app linkage - strong correlation - success engagement ---- Finally, I want to talk about using cookies (not these cookies but HTTP cookies) to capture and understand data around user behavior, broadly adopt quantified approaches to user research Certain types of quantitative data can be immensely helpful To Josh’s point earlier, some of the Inflection points in adoption, or change in behavior can be very powerful Twitter 30 followers, Facebook 7 friends At Dropbox, on the ecosystem team where I now work, we found that app linkage at certain levels has a strong correlation of user success and engagement
- Why data + anedoctes First - how many -groups Beyond - don’t know own behavior misrepresent UR - 2 flows - revamp Freq flow B - 30% Data and logs - flow B 6% vs. 30% told us about Changed UX and entry points totally ---- Why do you want to use data in addition to anecdotal information? Well first of all, you want to know how many users are there in each of your user persona types But even beyond that, users in many situations don’t know about their own behavior enough or misrepresent it A while back, I was running user research for a product which had two major flows in it -> we were looking to revamp the product I asked a question around how frequent flow B was used and got the answer that it was about 30% of the time But when I found some data and logs, it turned out that flow B was used only 6% of the time vs. the 30% that users had told us about This changed how we looked at the UX and the entry points into the flow completely
- Other - tell you don’t like - data otherwise TC 2006 - newsfeed Huge negative - numbers different - engagement up - know story FB ending explore feed User and anecdotal - looking for an explore feed Totally deceiving ---- In other situations, users may actually tell you that they don’t like a product or a feature but data can show otherwise The left hand image is a screenshot from techcrunch article from 2006 when the newsfeed was first introduced At that time, users gave a huge negative response to the Newsfeed. But the numbers after release said something different. The user engagement went up and we now know how successful the news feed was The image on the right hand side is a more recent one in which Facebook announced that they were ending their Explore feed Though in this case, user research and anecdotal information did say that users were looking for an explore feed type product Again this was a case in which user research was totally deceiving!
- No amount of prior Not worth it Paper - onboarding - get users to invite others on their teams Prompt to share - positive impact on team activation Few diff ways this prompt could look Instead research - AB test - hard data - won variant ----- Sometimes, no amount of prior research is actually going to help you in designing or figuring out Or the amount of research required is simply not worth it At Dropbox, a while ago on our Paper product, while onboarding new users, we were trying to get users to invite others on their teams to use Paper The hypothesis here was that, putting a prompt up to share during onboarding asking other users on the team to also use Paper with generate a net positive impact on team activation. And as you can see, there were a few different ways that this prompt could look Instead of figuring out which one was the best by research, we simply ran an AB test and got the hard data to prove which variant won
- Powerful informative - issues or gotchas Biggest pitfall - biased Expt - TC not random Surveys - leading Q, more active users PM - ensure - data has rigor and validity ------- As powerful and as informative quantitative are, the are not without issues or gotchas The biggest pitfall in running these analysis is that setups or analyses end up being biased In experiments, the test and control don’t end up being truly random Surveys can have leading questions, can be biased with responses from more active users and so on It’s up to you as the PM to ensure that the data you’re using to make decisions has the appropriate rigor and validity.
- The third and the final takeaway around uncovering the truth is to supplement your user research with unbiased data, user session behaviors and more
- Next time UR Think 3 elements Put shoes manager - JTBD for customer struggles. Job hiring Think DBX cupcake - delighting - deep empathize - integrate into your prod/company Start with why and formalize Phase 0 Finally think cookies - quantifying user session behaviors, inflection points, power up UR with quantitative info - uncover truths ----- So next time you’re conducting user research, Think about the the three elements we talked today Put yourself in the shoes of the manager of a milkshake line and figure out what the jobs to be done for your product to really empower them to overcome their struggles. What is your customer hiring your product for? Think about Dropbox’s cupcake value and the aspect of delighting your users by deeply empathizing with them, which is something you should work to integrate into your own products and company. To do so, always remember to start with the Why and formalize that through a process like the Phase 0 Finally, think about cookies when you are quantifying user session behaviours, find inflection points for your product and power up your user research with quantitative information to uncover truths
- Before I sign off, I wanted to quickly put in a plug for Dropbox We’re growing very quickly and also growing the PM team as well We’re looking for PMs across various seniority levels! We’re hiring across all areas but specifically more so in growth and analytics related areas And if you’re interested in working at Dropbox, please take a look at our careers page at dropbox.com/careers
- Thank you everyone! If you had any questions on what I covered here, please find me over lunch or at some point later. Happy to exchange war stories or talk about product And if you want to find me on the interwebs, you can find me on Twitter. My handle is @ketannayak With all this talk about food, I hope everyone is hungry, so please proceed for lunch over the break Thank you!