Diy user engagement, white paper


Published on

This is a guide for those with limited budget on how to engage end users in product development. For more information, see

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Diy user engagement, white paper

  1. 1. 1 DIY USER ENGAGEMENT A guide on how to engage users in product development This White Paper is based on a Product Doctor Drop in Surgery that I ran with Katrina Damianou during Over The Air 2011, 36 hours of Mobile Development. We offered 25minute free Product Health Checks to the developers in attendance. Some of the examples of products brought to us are referenced in this paper.___________________________________________________________________________Contents Agree this premise first! 1. It is never too early (or too late) to engage end users 2. What do you show users? 3. How to find your end users 4. Can you have the conversation with end users? 5. How to begin the conversation 6. Write a test script 7. “I can’t explain what my product does” 8. Showing a prototype 9. Testing for usability 10. Keep checking back with users as you develop and improve each new feature 11. Build and test your product before developing your brand 12. Be honest with yourself as to why you are developing the App ConclusionAgree this premise first!Before reading any further, we have to first accept that engaging users throughout thedevelopment cycle and from the concept phase (i.e. before you build anything) is goodpractice. I have proved both of these points over the past years, time and time again.A. Engaging users WILL give your product a far greater chance at success. We can all think ofgreat examples of products that may work beautifully, but fail as they do not meet a userneed or desire.B. Furthermore, products that are not tested properly with users often are too featureheavy -there may be a handful of features that are used while the effort spent to build therest has been wasted.When I asked patients whether they had engaged users at any stage of their development,those with live Apps did say proudly that they certainly studied their App Store feedback.October 2011 Product Doctor Insights from Over The Air 2011
  2. 2. 2App Store feedback is not enough! Not only is it once the App is out there, but throughoutuser tests I have carried out, I have found a bias on males giving written feedback on theApp Stores. Even then, it is only a certain type of personality that gives written feedback onthe App stores. They may give you suggestions on new features or how to improve theiruser experience, but this will often be on one or two actions / features they are carrying out.You are unlikely to get confirmation that the needs / desires you are trying to meet are valid– again, this can throw the whole validity of the App in to question.Remember that these people who give you feedback are likely to be early adopters. Wehave to remember that Apps are still in the “introduction phase” of the product lifecycle.This is the point where all the product and experience glitches/bugs will be highlighted andironed out. So by all means, you may well get good feedback on what is and isn’t workingbut not on the bigger picture.So here is my guide to DIY User Engagement1. It is never too early (or too late) to engage end usersI am a bit of a broken record on this one, but seriously, it is something that recurs time andtime again! Before you put your time in to developing, at any stage, please test out whatyou are intending that the product is going to do. It is crucial to check that the user needsor desires exist and that the product and features you are going to build are a good solutionto those needs or desires.2. What do you show users?One of our patients was very surprised to hear that he did not need a prototype to startconsulting with users and he could in fact use a statement of the Product Concept at a veryearly stage. They were part way through development of a home energy monitoring App.They could see how this would save time and energy and focus them on spending their timedeveloping the right features.So, you do not have to show people a prototype, you can just talk about your concept - tryto define it in one paragraph. It may help to write your concept up as a statement / userstory"...As a user I need something that does "x" so that I can "y"..."For example, "...As a user of the Boris Bikes scheme, I want to be able to easily find a freebay to dock back my bike so that I dont have to waste my time trying to find one..."You can also show functionality that you haven’t yet beautified on the front end (like theenergy App mentioned above). In fact, people are more comfortable to give youconstructive feedback if they see the product isn’t finished yet.October 2011 Product Doctor Insights from Over The Air 2011
  3. 3. 33. How to find your end usersOn questioning, every patient could identify what sort of people they thought would usetheir product, but very few have checked their ideas out with end users that are notrelations or friends. Note that those close to you will want to be encouraging andsupportive and so their feedback may be biased.Here are examples of how to find your users:  ...for the Boris Bikes scheme App - go stand at the bike docking bays and chat to users there;  ...for an App to learn to Chinese - go to International House and talk to students and teachers  ...for navigating around a festival - go to a festival!Once you talk to users you are likely to understand more about where your product sweetspot is. You will find people who you thought would be interested in using your product thatare not; and you are highly likely to find scenarios and other target groups that you wouldnot have thought of. In my 10 years of talking to users on a large range of digital products Ihave always found out something I had not imagined!With many products, it makes sense to test in pairs of friends. Often the conversationbetween the two respondents will be very open and you will gauge more open and honestfeedback from the discussion between them.In some cases, it may be appropriate to ask your friends / family if they know anyone thatfits your criteria. Importantly, unless your product is specifically aimed at the techcommunity, you really want someone outside of our industry. Sometimes, even tweeting orfacebooking who you are looking for can help.You may also find groups around particular interests that may be happy to test out yourproduct – you can find them in Meet Up; Facebook groups; Ning and so on. They may behappy to do this also without you have to pay them. Perhaps showing up with a tray ofdonuts will do the job! If you have a product that is targeted to the youth, talk to me abouthow to work with schools / colleges / universities as these sorts of exercises can double upas learning experiences for young people. Many teachers really appreciate real life productsand people to use as case studies not only for enterprise skills development but also tocompliment particular courses, such as Product Design, Technology, Marketing, BusinessStudies etc.You may well need to reward the respondents. You can offer cash or a voucher; the amountobviously depends on how long the interview is going to be. For the last set of user groups Iran, I offered £40 for a 90 minute interview, so around £15 for 30 minutes. Sometimes afree pint in the pub works – again – if that is where you can find your target group.October 2011 Product Doctor Insights from Over The Air 2011
  4. 4. 44. Can you have the conversation with end users?Some of our patients expressed concerns about their comfort levels with facilitating theconversation with end users. If you are not comfortable, you could ask someone else - afriend or family member who you think could help you. Of course, you can always askProduct Doctor as we may be able to do it for you on an affordable hourly rate!).Getting feedback from user is a practised skill; something I have been trained to do andhave practised over many years, but if you do not have any budget to hire someone in,please try to do this yourself. H0nestly, it is better than not doing it all. Take a recording ofthe interview – on your phone or on a dictaphone – that you can play back after the event.You will pick up on comments that you missed at the time and also be able to improve onyour interview techniques, for example you will hear where you might have “lead thewitness”. If you want to be good at this, you can be and I am always happy to give you anypointers you may want. You may even be able to provide a real life learning experience foryoung people who are still studying – they could carry out the surveys for you – again, get intouch if you would like to talk about this in more detail.5. How to begin the conversationStart by setting the scene; say (yes, lie if need be!!) that you did not develop this yourself,you are just helping out a friend – people will be less likely to be honest if you seem tooinvested in the product. Do not be passionate about the product. Do not appear to beselling. Try to be as neutral as you can.6. Write a test scriptIn addition to obviously being able to explain what the product is going to do, list out all theassumptions you have made. This will include items like: a. The purpose of the product / product concept b. The user needs / desires that you think your product is meeting c. Assumptions for financial modelling – e.g. how often the user is going to use your product d. The full features that you have / are intending to build These are examples. Of course, you should also test out pricing, naming and so on.So your questions will follow the above – example a. Can you put in your own words, what this product does? b. Which of the following needs can you relate to? Perhaps there are things we have not thought of / how could the product help you? If it is not for you, do you know anyone else that might use it? You can keep coming back to the user needs throughout the interview as you will find they may well change throughout the interview c. How often could you see yourself using this product (based on scenarios you have thought of above)October 2011 Product Doctor Insights from Over The Air 2011
  5. 5. 5 d. Rate these features in order of importance to you – please do not rate any features that you would not use at all. Are there any features you would like to see that are not listed here? How would you prioritise those in this list? Feature over-load is one of the most common mistakes I see being made. Sometimes the killer benefit of the product is a feature that you are thinking of as a side-product. This exercise will help you to find what is really going to make your product stand out. One of our patients had an App relating to finding a taxi, which had a number of different features. He had one feature that seemed particularly unique and instinctively, it felt as if this could form the basis of the entire App (patient confidentiality stops me from giving any detail).7. “I can’t explain what my product does”This was a wonderful quote from one of our patients. He had an idea that he felt wouldhelp with person to person transactions. Since the rise of E-Bay we have certainly seen anincrease in person to person transactions. More traditionally, they have always existed –direct purchases with no middle man - buying and selling cars and landlord/tenanttransactions. He talked us through his idea and then reflected, saying “I can’t really explainwhat the product is” but he knew he was on to something. The prescription in this case,was to go talk to people that have had experience of these person to person transactions.Find out whether the pain he felt he could address really existed. From there, he coulddevelop a one pager for each of these user scenarios that he could pull out when presentingwhat his product did. Stories are a great way to explain what your product does and how itmeets a user need. Products can be easily explained by exposing the solution to a user needin a real life scenario.8. Showing a prototypeAs above, it is not always necessary to show the user something – testing out the conceptbefore build is crucial in product development. However, if you do have a prototype,remember not to sell the product! Do not say things like “this bit is really cool”. Don’t lookexcited, you are just trying to demo the functionality.Once you have shared the prototype and worked through your questions, it is a good idea toask users how they would recommend you showed the product in the App Store. How theywould describe it and what screens they would show. You can also ask users how theywould expect to hear about the App. This will help you with your marketing plan.9. Testing for usabilityOnce you have a prototype / something to show, you will also need to check usability anddesign. The guide for usability testing is to list out a number of user tasks. Let’s take anexample: London Journey Planner App – so sample tasks might be i) Find the route from here to Oxford Circus ii) Find the current status of the Circle LineOctober 2011 Product Doctor Insights from Over The Air 2011
  6. 6. 6 iii) Find the W7 Bus RouteThese tasks should reflect all the different actions that users can take. Give them the task.Then stop talking. Watch and listen. Try to not jump in and show a user how to do it. After5 interviews (as per Jacob Neilson, the Godfather of Usability for his guidelines), you arelikely to see the same usability difficulties repeated. You will find the phrase “did thatbehave as you expected?” to be a very useful question to ask. Try to replay for clarity “Sowhen you pressed that you expected….but instead it took you to….” Confirming what youheard the user say is very important, you may find that you have made some interpretationsthat are not correct.Testing out design is part usability and part look and feel. Let your users know that theyshould feel free to make suggestions and improvements as they go through. Again, try tonot feedback to their suggestions as to whether you like them or not or whether they aretechnically possible. In some cases, user suggestions need further working through. Sohypothesise with them and walk through the change they suggest to make sure that it has alogical conclusion.In general, be very sparing with your words. State…Ask them what they think of that…Bequiet and listen! If you find yourself doing any interpreting in your head, replay that back –so “when I said that blah blah you said blee blee, does that mean bleugh bleugh?” Seekclarity and try not to get side tracked in to a conversation off topic!If users have difficulties with navigating through a particular feature, do ask them how theywould make it easier. Same goes for any icons that you are using. Just tips, and again, one ofmy mantras, do not reinvent the wheel. If there is a good and well known user interface outthere that you can adopt for your product then use it. As much as users should be able tonavigate through an intuitive user interface, they love it when the interface is familiar andwill often suggest you should build it just like they do on x-another App.10. Keep checking back with users as you develop and improve each new featureGathering user input is a necessary on-going exercise. If you are working with Scrum, youcan build it in to the sign off of the user stories for each iteration.11. Build and test your product before developing your brandI am a strong believer that you live and die by the strength of your product – product is king.Brand and design are super important, but is nothing if your product is not meeting userneeds and desires and providing a great user experience. One of our patients was adesigner, rather than a developer. He had developed a brand before his product was up toscratch. To me, this is the cart before the horse. I can empathise with developers that aredriving new product development as they have to fund developers to build their vision. Somany of our patients were developers hacking in their spare time – while they may need toengage designers down the line, at least they are not having to pay out at this stage.Recently I was asked to help a company come up with a name for their product, but theycould not explain in one paragraph what their product did. This illustrates the point well. Ido not see how you can get a product name or develop your branding without defining whatOctober 2011 Product Doctor Insights from Over The Air 2011
  7. 7. 7your product does. Being product-centric, this view should not surprise you. Naming aproduct and developing a brand is also an exercise where users need to be engaged.12. Be honest with yourself as to why you are developing the AppI often meet young developers who are working on their pet App in their own time. Theymay be doing this as their day jobs are not giving them the chance to work on particularcode or areas that they want to so they are supplementing their education and ability toshow their skill in their own time. Great. But in some cases, this is not about developingprofitable Apps. This is just about enhancing their CV – great to be able to say, “I developedan App that has had x downloads and is rated as 4/5”. Leave it at that! Don’t drive yourselfcrazy to find the revenues as that is not your core motivation. When I ask patients “what doyou see yourself doing in 5 years’ time?” very few can see themselves as running a companythat takes them away from their core love, which is developing! Many do not want to buildup the skills to market and PR their Apps for example. “Just leave me with the fun bit – thecoding”.On the other hand, some developers absolutely are looking to make the killer App and wantto make their million. In these cases, it is really important to think through a) The revenue model, examples  This can include partnership conversations with revenue share  Exit strategies where they may be able to sell their App  App stands on its own two feet and revenue models work independently b) The go to market strategy In my opinion, you cannot rely on just putting your App in the App stores and hoping for the best. You need to “DIY PR” to coin Lisa Devaney’s phrase – for example - how to build up an online profile for the product and generate online noise so that for example, industry bloggers write about your App. Where are your target customers going to find out about your App – we had some great examples of brainstorming with some of our patients of societies, online groups centred around particular interests, magazines and known bloggers and so on. There is no excuse; you can find groups on Meet Up, Facebook, Ning and so on Additional Notes: (i) Proving in the niches first If you have a product, like Audio Books, that is going to have pretty much mass market appeal, rather than try to tackle the market en mass, you can prove in the niches first and work your way out from there. There may be some obvious places to start; or perhaps places where the social media machine will work for you best; maybe you already have contacts that can help you in those niches. (ii) Nothing wrong with backing a few different horses A few of our patients had products that could be own brand or white label. I am sure we can all name many products that work in this way and we know the benefits of workingOctober 2011 Product Doctor Insights from Over The Air 2011
  8. 8. 8 with each tranche. Ultimately, the core functionality will be the same, with a different wrap. If you have the capacity to manage both, then best to run with the two horses and see what happens. c) Your product positioning & understanding your competition  How do you sit in relation to your competition – around half of the patients we saw did not have an up to date grip on what was out there and how their product was better / different.  What are the 3 key product benefits to using your AppAgain, no surprise here, but these entire elements can also be tested out with users. Forexample for “go to market”, you can ask the users that say they would use your product“How would we best reach you”. For Product Positioning “What do you think are the 3 keybenefits of using this product”?I hope that you find this Guide useful, if you want to contact me for any clarification oradvice please do contact me: T. 07956 376472 E. W. 2011 Product Doctor Insights from Over The Air 2011