AB Testing White PaperWhat is AB Testing?AB testing, Split Testing or Bucket Testing is a process for testing two different versions of an element (A& B) and a metric which exactly defines success. To determine which version is batter you have toperform experimentation simultaneously, at the end of experiment you measure which version wasmore successful and select that version for real world use.The Picture above Explains that two versions of web page have been made, version a consists of Bluetitle with different placement of Navigation Bar, News Block, Signup Form & Content body while versionb consists of green title with entirely different placement of navigation bar, page title, signup form &content body. Visitors are distributed randomly for accessing the page. Stats shows that more visitorsland on version b comparable to version a.You Split your website traffic between two versions of entirely different design, navigation and measuretheir performance using matrices like Bounce Rate, Click through Rate, Sales etc. At the end of test youselect the batter performed version.Why AB Testing?According to various researches a typical website converts only 2% of its visitors into customers, which isvery low amount of conversion of sales into leads. Visitors cannot explain why they left your websites!Either designs issues, navigation issues, color scheme is not attractive or anything else! The major
AB Testing White Paperobjective of A/B Testing is to select the most appropriate theme with complete navigation and userfriendly interface. Small changes can make a huge impact on conversion rates which could be beneficialfor webmaster in the long run.After performing successful AB Testing you can enhance your site conversion rate up to 40%, Googlealso believes A/B testing Work wells you can use Google Website Optimizer for further details.How Does A/B Testing Works?First of All you have to start with the original page or the page which is under your control! Then youhave to determine that goal has been reached. For Instance if you want to see how many people makepurchase, in this scenario Thank You Page is the key indicator for observing the conversion rate.A Simple test would present one version of the page to one visitor, and another version to the next. Oryou can swap both of these test pages randomly, or could be based on other demographic orparameters, it’s all depends upon the testing program and what you are trying to achieve!One Important aspect in A/B testing is time domain; you have to prepare yourself for running the testfor a month or more to acquire the desired results. Online calculator is useful tool to determine the testduration, bounce rate and expected conversion rate.AB Testing Processes (Step by Step):The Purpose of AB Split Testing is to work with one of these at a time or possibly several remedies at atime in different environment, controlled, variations to find out the best possible combination of variouson page Factors.These Processes are quite simple and easy to perform: Measure Test Evaluate OptimizeThis Process needs to continue as far as many iterations as possible, ideally it is continuous processwhich never ends, However practical real world issues limits it at some extant where testing stops andmarketing begins.Measuring:Measuring is one of the most important aspect of A/B Testing, Nothing can be tested properly unless ithave been measured, There are lots of factors which can be measured but we have to focus on keyfactors first which are:
AB Testing White Paper Hits or Delivered visitors CTR : Click-Through-Ratio (number of times the prospect has clicked through to the sales page) Sales Conversion vs. CTR Conversion vs. HitsThe above must be tabulated with reference to the Medium (email, web page visits etc.). The initialmeasurement is taken against to the control. In the first iteration, the control will be the existing landingpage or email promotion.In subsequent iterations, it will be the best performing variation of the promotional piece. This will be acombination of the best performing A/B Variations that are established as a result of the Testingprocess.Testing:Testing is the procedure of creating two variation (Named A and B), and measuring their success againstone constant. This constant is usually equivalent to the number of page visits – it is the number of timeswhenever promotional page being viewed.By Conversion, the control page is denoted as A. Going to its First Iteration, there will only twovariations, both of them labeled as the control; each of them has equal chance of becoming the control.In Order to track the exact page from conversion comes through, Link Tagging should be used. BasicallyLink Tagging is the process of naming target link –A or –B so that they can be easily identified in thestatistics pages used to measure vectors.
AB Testing White PaperIn each iteration, Variations of the control page are introduced, and the link tagged with –B suffix. Thevariations should not be too subtle; otherwise the testing process could not be produce the worthwhileresults.During the testing process measurement should be taken, however the variations should be taken oncewhen delivery quota has been achieved on Both A and B Variations.Evolution:The Evaluation process compares the vectors that have been measured. The control is then set to thebest performing variation (A or B). This then becomes the new A. The exact definition of bestperformance will depend on the goals of the marketing process.Optimization:The optimization process takes the best performing A/B splits by category (Headline, CTA, etc.) andcombines them to produce the best overall content to test.It is quite important aspect to continue the test, even after the optimization is completed; the wholeprocess is not over because small changes can make huge difference for converting leads to variance.After creating a well optimized page, it should be established as a new control for Future A/B testing.Benefits of A/B Testing:Compared with other methods, A/B testing has four huge benefits.1). It measures the actual behavior for your site in real world conditions. You can easily conclude the endresult by performing A/B testing, For Instance; if version B performs batter comparable to version A,then version B Design you should be shown to all users in near future.2). The Second Big advantage of A/B testing to provide you the clear picture regarding conversion andgenerating leads into sales, you can easily find out the best suitable approach by performing A/B testingunder your own circumstances.3). It has ability to measure very small differences with high statistical significance because you canthrow massive amount of traffic at each design.4). It is comparatively cheap, Once you created two design alternatives, you simply upload both of themon the server & perform monitoring which page perform better, there is no need for expensive usabilityspecialist to monitor each user behavior or analyze complicated interaction design questions. The onlything you have to do is wait for statistics and the go with the design which have batter numbers.Limitation of A/B Testing
AB Testing White PaperWith these clear benefits why we don’t use A/B testing for each process because it has some limitationas well. Let me provides you some details regarding A/B Testing Limitations.A/B testing can only be used for project which has clear & measurable goals; furthermore these goalsmust be measurable by computers. For many sites ultimate goal cannot be measurable through useraction on the server, Goals like improving brand repetition or supporting the company public relationsefforts cannot be measured by A/B testing.Another downside of A/B testing is that it only works with fully implemented designs. It is quite easy totest design when it is on running condition but implementation of specific design can be taken longtime, before presenting it in front of new customer on your live website, you must fully debug anexperimental design, Thus A/B testing is suitable for only very small number of ideas.Results and Deliverables of A/B TestingOne of the most vital aspects of A/B testing is to measure the results and deliverables of A/B testing;Conversion Rate Optimization is normally used to measure the results;Conversion Rate OptimizationConversion rate optimization also termed as CRO is a critical process for determine the results in A/Btesting. The main objective of CRO is to enhance your marketing efforts, no matters how effective atdrawing visitors, will struggle endlessly if your website is not effective in converting visitors to yourdesired goals.Tools for A/B Testing:
AB Testing White PaperGetting Traffic for website through search engine optimization is simply the First step, next step is toconvert your sales into lead, all you want the visitors you get to stay on your site and performed thedesired action.Over the Past few years A/B Split testing has been booming, there are plenty of more user friendly toolsout there by now. Here is the list of some useful tools that is beneficial for A/B Testing:Google Website OptimizerSplit Test (A/B Test) CalculatorA/B Theme Testing Plugin for WordPressVisual Website OptimizerLiveball PlatfromVertster – Multivariate Testing and A/B Split Test Software