Art of Conducting Visitor Research
•Download as PPTX, PDF•
1 like•462 views
Conversion optimization without research is like shooting in the dark. You can never write a measurable A/B testing hypotheses without research. Before building your optimization roadmap, it’s critical to learn the art and science of conducting visitor behavior research, an essential skill for every optimization expert. This session will help you in getting answers to “what” your visitors are doing, “how” they behave, and “why” they behave in a specific, repetitive way on your website.
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Art of Conducting Visitor Research
Similar to Art of Conducting Visitor Research (20)
More from VWO
More from VWO (20)
Recently uploaded
Recently uploaded (20)
Art of Conducting Visitor Research
- 1. Art of Conducting Research Rahul Jain Product Manager VWO Abhishek Mandloi Product Manager VWO Dinesh Kasliwal Customer Success Manager VWO
- 2. Ask Questions on Twitter #AskVWO
- 3. A step-by-step approach to increase conversions Conversion Rate Optimization Process Research Phase Tracking metrics and identifying what parts of conversion funnel need fixing Hypothesis Phase Constructing educated hypothesis, based on your research Prioritization Phase Planning and prioritizing your hypothesis Testing Phase Testing your hypothesis against the existing version of the website Learning Phase Deploying the winning hypothesis and/or gathering learning for subsequent tests
- 4. We know it can be very frustrating to figure out what visitors want… 1 in 8 • So how do you measure the impact of web testing on organizational learning? • How do you understand why tests fail? • How do you keep from constantly chasing short-term goals? What about the bigger picture? Only creates a major impact on conversions. A/B TESTS
- 5. Research Potential 1 million monthly traffic 30% bounce rate 700k Qualified Visitors 30% to 70% people are in market to buy 350k Potential buyers 7000 completed orders 343,000 visitors is our opportunity
- 6. What are different ways to research customers?
- 7. What are different ways to research customers? • Quantitative Methods • Qualitative Methods • Technical • Heuristic
- 8. • Web Analytics • Conversion Funnel Tracking • Form Analytics Quantitative Methods Collecting metrics about users on how they interact and engage with your web properties
- 9. Quantitative data shows you what is happening, but why it is happening is discovered by qualitative tools.
- 10. • Heatmaps and Clickmaps • Customer Interviews • Surveys • Session Replays • Click Testing • Customer Support Interactions Qualitative Methods and so on
- 11. Technical Research Heuristic Analysis Discovering broken functionality and user experience • Browsers • Devices • Page load speed Evaluation of a website by a usability expert against a set heuristics to identify potential conversion barriers.
- 12. Challenges with different research methodologies
- 13. Research Challenges • Keeping close eye on business metrics • Which methods to chose? • Understanding and documenting customer’s mind with data and evidence. • Connecting research tools to derive more value
- 14. How VWO overcomes common research challenges?
- 15. Overcoming Research Challenges Keeping close eye on business metrics 1) Define business objectives 2) Create goals in terms of metrics of these objectives 3) Start with Macro conversions 4) Setup micro conversions that form part of macro conversions 5) Create funnels of most important pathways Start Baselining measurements
- 16. Find Out Drop-Offs at Each Step of the Conversion Funnel VWO’s Plan layer
- 17. Overcoming Research Challenges Which methods to chose? 1) Web analytics 2) Surveys 3) Heatmaps and Clickmaps 4) Visitor Recordings (Session Replays) 5) User Testing 6) Form Analysis 7) Heuristic Analysis (Expert Reviews) 8) Customer Interviews 9) Click testing 10) 5 second test
- 18. 98% (103) views Heatmaps Session Replays Customer interviews Surveys Web analytics User Testing Expert Reviews Click testing Five second test Card sorting/Tree testing Form Analytics Valueofinsights Low High Difficulty in generating insightsDifficult Easy Maximum insights with minimum inputs
- 19. Overcoming Research Challenges • Understanding and documenting customer’s mind with data and evidence.
- 20. Overcoming Research Challenges • Connecting research tools to derive more value
- 21. Analyzing Heatmaps and Recordings where users dropped off
- 22. Analyzing Recordings where users saw particular variation
- 23. Analyzing Recordings where users converted to a particular goal
- 25. WHAT IS WRONG? Quantitative Analysis WHY IS IT WRONG? Qualitative Analysis Define business goals and funnels Define baseline conversion and try to find WHAT is wrong? E.g. Click on sign up CTA is too low. Look at qualitative capabilities such as heatmaps and recordings to find out WHY is it wrong? E.g. Try to find user behavior of visitors who are not clicking on sign up CTA
- 26. Strategy to analyze Session replays Exit PagesBounce pages Converted vs Non converted Custom data/Labels
- 28. Form Analytics Go beyond tracking form submission rates. Form analytics help you understand how visitors interact with every form field. It gives you insights on where visitors hesitate and drop off 98% (103) views 58% (61) views 27 % 34 % of the visitors submitted the form 20.5 seconds Total time spent on completing this field 10.3 seconds Time spent by visitors hesitating to give this data 20 % of the visitors drop off on this field of the visitors refill this field
- 29. Heat & Scroll Maps Find conversion barriers Findability/Visitor Flow Dead links Navigation menus/Hidden elements In page funnel Engagement with content
- 30. Conversion barriers Navigation bar distracting users from the primary goal of this page which is to download the app
- 31. Visitor Flow A heatmap report of the HIAA home page revealed interesting insights—68% of all clicks on the home page were on the Departures tab and only 6% clicked back to the Arrivals tab
- 32. Findability - Navigation menus
- 33. In page funnel A sharp drop-off of 40 %age points happens here and content below this is going to be ineffective. Also acts as a in page funnel representation
- 34. A lot of visitors are clicking on the cube and its branches thinking that these are links. Dead Links
- 35. Discovering visitors’ intent using Surveys Tirecraft.com conducted an exit intent survey to understand why visitors leave and found out that non-availability of Pricing is the biggest reason I prefer to buy tires in person There was no pricing information available I want more information about the tires I cant buy the tires I want online Other
Editor's Notes
- Thanks for the introduction Utkarsh and welcome everyone. I am excited to see you all here today and would love to chat more and hear more about your research efforts post my session. Before we deep dive into research I would like to quickly walk you through the basic approach to increase conversions. A typical conversion optimization process starts with understanding what the user is doing on the website. In the research phase we try to find out what is wrong by defining business goals and funnels and why is it wrong by looking at qualitative data of the users. Once we have observations we use that data in hypothesis phase. In this phase we define problems and solution based on our research. In the next phase once we have a backlog of ideas we prioritize them based on different criteria such as confidence , importance and ease of implementation. Once we have a prioritized list of ideas we test them against existing version of the website and finally in the learning phase we deploy winning hypothesis and gather learnings for subsequent tests. In this webinar we will deep into the research phase and understand why research is important and how we can build a solid process around understanding user behavior.
- The harsh reality about A/B testing is that only 1 in 8 test creates a major impact on conversions. Why is this number so low? The biggest reason for this is that we “assume” that our customers would like a certain change on our website. In most cases we follow best practices or we use our own experience to understand user behavior. But all of that is just assumption. There is a huge diversity in the kind of users that we get on our website. One best practice may or may not work for different businesses. Most organization in the industry are not following data driven approach towards testing. Most companies these days do have advanced analytics capability setup for their business where they keep a track of key KPI’s around their business goals. But what we don’t realize is that this is just one part of understanding user behavior. We do get answers to ”What” is wrong but we often don’t focus on “why” part of user behaviour analysis. I will talk more the right process in next few slides.
- One common question that all of us might have is that should we really invest our resources and time into user behavior analysis? We often spend a lot of our marketing budgets on SEO to bring in a lot of traffic on our website and we are somehow satisfied with industry standard 2-5% conversion rate but do we realize the potential customers that we are loosing by not focusing on them? Conversion optimization is all about increase that 2-5% conversion rate. This is a very interesting piece of information that was shared by one of CRO influencers. Lets say you have 1 million monthly traffic. In the next couple of slides Abhishek will walk us through different research methods and challenges with these methods.
- Thanks Rahul. Hello people, I am Abhishek, one of the product manager at VWO. Rahul has helped us understand research potential and established that research is foundation of a successful optimization program. Now, let us look at some of the ways this foundation can be build, that is how visitor research can be conducted and associated challenges with them.
- Broadly, customer research for conversion optimization can be divided into 4 categories: Quantitative, Qualitative, Technical and Heuristic. Most of you are aware of these techniques, but for the new comers in the industry, let me take a couple of minutes to explain these categories and different ways to conduct visitor research.
- Quantitative techniques are those which involves collecting metrics about users on how they interact and engage with your web properties. These metrics or numbers are later used to understand visitors behavior, and any optimization effort is expected to move these numbers in the desired direction. The most common example of quantitative techniques is gathering analytics - from simple metrics like ‘number of sessions', bounce rates, time spend on the page, to conversion focused metrics like measuring how many users clicked on CTA button or filled the demo form. If you want to optimize your website, it is important to measure such conversion focused metrics, that we call it as goals and events. These objectives define the success or failure of your optimization program. Advance practitioners also adapt conversion funnel tracking as part of quantitative research, where they track how many users are following or rather dropping from your primary pathway. For example: tracking a simple funnel from search page to product page to checkout page for ecommerce guys. Another way to collect quantitative research is Form analytics - that is understanding how users are interacting with different elements of your form. All these quantified data points are easy to interpret — and of course the data can’t lie, right? However, this approach is wrong. Absolutely wrong. That’s because our visitors are human — so no amount of statistical data we collect will ever be enough to completely explain their behavior.
- The main objective of visitor research, is to answer the question of the “why”. And using purely quantitative data, the “why” of our users’ is difficult to understand. Sometimes, users act in ways we cannot easily explain. For example, imagine you’re trying to solve the issue of high cart abandonment on your product landing page. While analytics will tell you how many of your visitors leave without completing their purchase, you will remain in the dark as why they abandoned their purchase. With qualitative tools, not only do you get to know what people think and feel about your website firsthand, you can also understand the underlying story behind the dry data. Let us look at some of the qualitative tools – the real arsenal of CRO experts.
- Qualitative techniques involve collecting visitors behavior that needs to be analyzed and understood in subjective terms. Since, there are no numbers involved, the interpretation is often limited by skills of person who analyses the qualitative data, but the data will speaks for itself. Some of the popular qualitative tools are heatmaps and clickmaps, customer interviews, surveys, session replays, click testing, Customer support interactions etc. I am assuming you already have some idea about these qualitative tools. So, I will not go into details but we will discuss the use cases of these tools at the end of the webinar. Coming to the third and forth category of visitor research is Technical and Heuristic Analysis.
- Technical research is discovering the broken functionality and user experience on your website, which are actually your your main conversion killers. It is somewhat interesting. We call it a low hanging fruit. Anyone can do it – you don’t require special tools and it instantly gives to results to fix without much efforts. All you need is to find what browsers and devices your users are using and compare the conversion rates among them. If you notice a drastic drop, it means that your site is not optimised for that technical configuration. May be there is a video playblack issue on Internet Explorer or your site does not render properly in older android phones. Just find this data from your analytical tool and compare conversion rates. As easy as it gets. You also need to keep a close eye on page load speed. Multiple studies has already confirmed direct correlation between page load slowdown and sales. Lastly, we have heuristic analysis. To explain this in simple terms, a subject matter expert analyses the website or page and provides recommendations and best-practices to follow based on their experience. Though we don’t count opinions when it comes to science of visitor research, but heuristic research has proved to be an effective method to discover pain-points than randomly sharing stupid ideas. The main advantage of heuristic analysis is that it can be done in really short time frame. The results might not be far reaching but are good enough to be optimal.
- Let us now understand different challenges associated with these research methods.
- We have seen that there are plethora of research techniques. But each of them bring their own challenges. One of the foremost problem is to attribute impact of these research methods on the business. If you are optimizing landing page banner, is it really helping your bottom line? ---- While you are conducting visitor research, it is important to understand, how much effort you are devoting and how much impact these research exercises can bring to your business metrics. This brings us to the second challenge – Deciding which research methods to chose that gives maximum insights with minimum inputs? Next, we have understanding user problems and documenting these insights. With multiple tools, there is no single source of truth - something you can rely on - you have got hold of the user’s problem, how to document that so you can come back to it whenever required. And lastly, with multiple tools it becomes incredibly difficult to connect insights and derive more value out of them.
- Worry not! If there are challenges, then there are solutions. Let us see how VWO overcomes these research challenges.
- Now, for following our business metrics, it is important that you first understand and start collecting them. As they say - The first step in improving something is knowing how good (or bad) it is right now. Thus, the CRO process should begin by establishing a baseline measurement, which you can do using web analytics tools. You should know what your current conversion rates are and then you should aim to improve these conversion rates. To setup these baseline measurements, you need to setup metrics that reflect the success of your optimisation program. What could be these metrics be? It depends primarily on the nature of your business, but Sales are often the best indicator. To measure sales, you can start measuring supporting indicators like lead contact forms filled, or clicks on demo button or number of promotional videos viewed. For example, a macro conversion is increasing revenue for an e-commerce site. So, a user should first create revenue goal or payment form submission goal. Now, the micro conversions are the goals that support the macro conversions, like visits to the product page which has Add to cart button or clicks on Add to cart button. User should now analyze behavior of their customers at pages related to micro conversions and deploy experiments to improve micro and then macro conversions. You should also setup funnels of most important paths your users follow, to see and undersatnd where they are dropping off.
- With VWO’s Track capability you can measure your CRO objectives as Goals. You can now record website metrics on your landing pages, eCommerce shops, or travel websites, and then measure these as goals. With VWO Funnels, you can Identify and set up conversion funnels to know where visitors drop off After you’ve documented your baseline, it is time to start your research and identify the obstacles your visitors need to overcome to become customers. Now, for the second challenge to chose between different tools - Remember the value of insights vs the difficulty in generating insights findings - with VWO, you get almost all the tools in the first quadrant. And for the third challenge, understanding and documenting research insights, you can use VWO’s
- Now the second challenge was to chose a research method that gives maximum insights with minimum effort.
- To answer which methods to use, we have conducted a survey with CRO experts and found that : ds to be fixed. Heatmaps are the easiest method to generate insights and the value of insights generated is good enough Web analytics, when set up properly can tell you about every aspect of your website and the insights are incredibly high. Surveys, form analytics and session replays are could be little difficult when it comes to digging insights but they tell you insights that no other tool can tell, and the value of these insights is really good. Card sorting, 5 second tests should be the last resort for generating insights. So, the first quadrant are the tools that you should aim for. Another point to note that Technical analysis is a must do - it does not take a much time and tells you immediately what nee
- And for the third challenge, understanding and documenting research insights, you can use VWO’s Plan layer. Here, you can save your notes on visitor research as Observations anywhere in VWO. These Observations are saved with the exact snapshot of the report that was used to create them.
- All of us get fascinated by looking at a visitor actions by looking at research tools such as visitor recordings. But we often end up doing aimless research. Couple of questions that we get to hear all the time are. I have 10000 recordings . Which ones should I watch ? Research involves a lot of manual effort and time. What should we do? Answer to these questions is adopting the right process. If we aimlessly keep looking at recordings we might get fascinated in the beginning but sooner than later we loose interest and ofcourse time is money for everyone.
- The most important step before starting user research is to define business goals and funnels. This step involves prioritizing goals for a particular month or a quarter. Once the goals are defined we need to create a baseline conversion and observe trends over time. For eg One of our optimization goals for the previous quarter was to increase sign ups on our website. Now we observe the conversion rates for sign up over time and see if its increasing as per our expectation. At the same time we look at different funnels that lead up to sign ups. At this stage we try to find out “What is wrong”. Now we have the numbers to understand where the leak is happening but we don’t know why is it happening. In the next stage we try to look at qualitative data of users that are not converting. This will help us understand “why” those visitors are not converting. For instance by looking at heatmaps and recordings of visitors who did not convert for a goals we can figure out reasons why they are not converting. It could be due to some distraction on the page but we instead of just assuming the reason we can actually observe the actual user data and come up with a data driven hypothesis. In the next couple of slides Dinesh will walk you through some examples of how we achieve success using this approach.
- After initial process is done you can further deep dive into session replays and look at bounce pages which will tell you more about the reasons visitors are bouncing off. Then you can also look at recordings for a specific funnels. For eg We spoke about funnels that lead upto sign up. In this case we can specifically look for recordings of visitors becoming part of those funnels. Lastly you can also pull in custom data from different CRM systems or by tagging users doing a particular action to specifically look for recordings of a particular user group. A good example for it could be looking at recordings of visitors who are logged in vs visitors that are not. In the next couple of slides dinesh will talk more about use cases for different capabilities like forms , heatmaps and surveys.
- Thanks Rahul and Abhishek for those insights into the art of conducting research for a successful Website Optimization Program. Hi everyone. I am Dinesh Kasliwal, Success Manager here at VWO. I handle Enterprise clients based out of US and LatAm region for VWO. As Rahul mentioned I would like to discuss a few real life use cases so as to derive important and action worthy observations out of your website research initiatives. As part of our quarterly target we were looking to generate more leads out of our existing traffic. While running a Forms Analysis Campaign on our Free Trial Sign up form we observed that while a decent amount of traffic landing on the page was interacting with the form, the overall form submission rate was still low. Now Form analytics help you understand how visitors interact with every form field. It gives you insights on where visitors hesitate and drop off Therefore, we decided to investigate further into field level statistics to see potential areas of friction on the form.
- We were further able to observe Observation 1 - Time spent on the email field was higher than expected. Upon investigating further by using recordings which were also enabled for this form we were able to find out that many visitors ended up refilling this form field as we accept only work email ids here and not personal ones, which were the first choice for a lot of visitors. Observation 2 - Again a high refill rate was observed for the password field as well. Once again when we looked through recordings and realised that a lot of users were getting validation errors on password field. Reason - There is no validation message shown until user enters a password less than 8 characters. Observation 3 - Lot of visitors are either hesitating or dropping off on telephone number field – Visitors are hesitant in sharing personal information here.
- One of the other important qualitative tool that is used for deriving important behavioral insights from the traffic coming on the website is Heatmaps and Scrollmaps. Heatmaps and Scrollmaps can help us in – Finding Conversion Barriers Understanding Visitor Flow and general findability of different elements on the page Making sense of Visitor engagement with Navigation Menus Finding Dead links receiving visitor attention Understanding In page funnel, and Figuring out Engagement with content.
- Here’s a classic example from one of our clients websites - The primary goal of the page is to interest visitors in downloading the app, but when heatmaps was generated for this page it was discovered that the navigation bar at the top is acting as a distraction and is a conversion barrier for the primary goal to be accomplished.
- A heatmap report of the HIAA home page revealed interesting insights—68% of all clicks on the home page were on the Departures tab and only 6% clicked back to the Arrivals tab.
- Heatmaps can also provide valuable insights into visitor engagement with elements inside of a navigation menu. Helps understand if a particular element is easy enough to discover as an option on the navigation menu.
- Scrollmaps help in representing what we call is an in-page funnel. As we can see here a sharp drop-off of 40 % points happens in the highlighted section and content below this is going to be ineffective.
- Heatmaps also help in discovering Dead links – in this example - A lot of visitors are clicking on the cube and its branches thinking that these are links.
- Finally if the efforts you invest in going through qualitative tools like heatmaps and recordings to discover the user intent do not reveal the picture clearly, you can use trigger based surveys to directly ask questions from your clients. As an example - Tirecraft.com conducted an exit intent survey to understand why visitors leave and found out that non-availability of Pricing is the biggest reason
- Finally if the efforts you invest in going through qualitative tools like heatmaps and recordings to discover the user intent do not reveal the picture clearly, you can use trigger based surveys to directly ask questions from your clients. As an example - Tirecraft.com conducted an exit intent survey to understand why visitors leave and found out that non-availability of Pricing is the biggest reason