Decision-making: choose the right tool for the job


Published on

A decision tree to help you navigate through the multiple techniques available to collect opinions, analyze ideas, and reach consensus.

Published in: Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Decision-making: choose the right tool for the job

  1. 1. Decision making: choose the righttool for the job February 2013 www .t hi s i sxy .c o m
  2. 2. X&Y Partners Decision making: choose the right tool for the jobContacts:Romeu Gasparromeu.gaspar@thisisxy.comUK: +44 (20) 3239 5245 | PT: +351 210 961 834Skype: xypartners 2
  3. 3. X&Y Partners Decision making: choose the right tool for the jobDecision making: choose the right tool for the jobA decision tree to help you merits and limitations, and there is nonavigate through the m ultiple one-size-fits-all solution. The decisiontechniques available to collect tree presented in Exhibit 1 will helpopinions, analyze ideas, and you choose the best tool for the job atreach consensus. hands. Generally speaking:More often than not, decisions today § Prediction markets are particularlyare made by teams, based on a broad useful for collecting input from anumber of collected ideas and large number of people, especiallyopinions. Collecting these inputs and when it is important to rapidlythen reaching a consensus can be incorporate new information.made through a broad number of However, this technique is onlydecision-making techniques. In this effective with “Yes or No” questions;article we analyze five of the most § For multi-response or open-endedpopular ones: Predictive Markets, questions, a Survey or Poll might beSurveys/Polls, the Delphi Method, the a more viable option, provided thatNominal Group Technique, and the the question can be made clearlyvenerable Face-to-face Meeting. and factually: evidence shows that even slight changes in wording canEach one of these tools has its own lead to vastly different results;Exhibit 1 – Proposed decision tree to select the most appropriate decision-making tool 3
  4. 4. X&Y Partners Decision making: choose the right tool for the job§ The Delphi Method is usually used Let us now look into each one of these to address complex issues, or methods in more detail: matters that require quantified Prediction Markets answers. The structured discussion process used in this method has A prediction market turns events (e.g. proven to be effective in bridging “Will candidate A win the elections?” or gaps in assumptions and in “Will cold fusion be viable before mitigating bias. The Delphi can 2020?”) into tradable securities: just however be time-consuming and like in a stock market, participants buy becomes cumbersome with more securities if they think the event will than 10 respondents); occur, and sell them if they think the§ The Nominal Group Technique is event will not occur (with real money in often used at workshops to generate some markets, and play money in and prioritize ideas (e.g. In which others). The security price is then products should we focus our R&D converted into an implied probability. efforts on?). Compared with the For instance, a 0,8€ price for a Delphi Method, the Nominal Group security that rewards 1€ at maturity Technique remains practical with implies a probability of 80% that the more than 10 participants, but is less event will indeed occur. efficient in eliminating bias and in bridging gaps in assumptions; Research has questioned the validity§ To tackle multiple issues at once, of directly converting prices into and do to so with minimal probabilities, but prediction markets preparation time, the traditional have nonetheless proven to be Face-to-face meeting continues to effective in collecting the “wisdom of be a valid option. Often described as the crowds” in order to predict the an unstructured discussion (as outcome of events. For instance, opposed to the more rigid structure Exhibit 2 compares prediction markets of the methods above), a decision- and polls for projecting the outcome of making meeting should nevertheless US presidential elections, suggesting follow the same rules as any other that the former is more accurate, effective meeting. especially when forecasting with more than 100 days in advance. 4
  5. 5. X&Y Partners Decision making: choose the right tool for the jobExhibit 2 – Comparison between the accuracy of prediction markets and polls, for several USpresidential elections assess risks. The biggest obstacle forThe use of prediction markets in a the corporate use of predictionbusiness context is still not widespread markets is arguably engaging thebut has gained some traction over the appropriate number of participants: thelast years. Inkling, for instance, is used process itself requires a sufficientlyby companies such as Procter & large number of regular participants toGamble and Ford to bring together ensure liquidity, while some companyemployees to evaluate ideas andExhibit 3 – Example of how differences in wording can dramatically change the results of asurvey or poll 5
  6. 6. X&Y Partners Decision making: choose the right tool for the jobissues might be too specific or too characteristics of the respondents aresensitive for widespread discussion. representative of the target population.Surveys and Polls The Delphi MethodSurveys and Polls might not have the The RAND Corporation originallyaccuracy of Prediction Markets, but developed the Delphi Method in thethey hold important advantages: they 1950s to forecast the impact ofwork with a wide range and number of technology in warfare. Today thisquestions, they require minimal setup method is widely used to addresstime, and most respondents are complex issues, particularly those thatalready familiarized with the process. require quantified answers. At X&Y Partners we routinely use it for marketWhen preparing a survey or a poll, forecasting, for instance.structuring the questions in a concise,clear and factual form is critical, as In a Delphi, a group of experts iseven small variations in wording can asked to individually answer alead to vastly different results (Exhibit question (e.g. “What will be the cost of3). It is also important to ensure that solar energy in 2015?”). The answersthe responses are statistically valid and assumptions of the experts arei.e., that the number of responses is then discussed and compared, andenough to ensure a reasonable each of the experts makes a secondsampling error, and that the prediction. This second prediction isExhibit 4 – Quantitative results from a comparative study that asked 227 participants toestimate 10 data points, first individually, and then using one of three decision-making methods 6
  7. 7. X&Y Partners Decision making: choose the right tool for the jobExhibit 5 – Example of an output from a Nominal Group Techniquealso made individually, but now the the Nominal Group Technique (Exhibitexpert is able to leverage the new 4), but less user friendly than the latterinformation gained from the other (Exhibit 6).experts. The process is iterative and The Nominal Group Techniqueusually runs until the moderator issatisfied with the degree of Most of us have at some pointconvergence of the answers from the participated in a workshop based onvarious experts. One of the key the Nominal Group Technique. In itscharacteristics of the Delphi is that the basic form, participants are askedanswers are anonymous: experts can (individually or in smaller groups) tosee and discuss the responses from answer a question or to provideother experts, but do not know who is solutions for a particular issue (e.g.responsible for a particular answer. “How can we increase sales of our XThis contributes to eliminate peer product?”). The facilitator will then listpressure and other bias that might all answers and foster a discussion toarise when some of the respondents merge similar entries, clarify wording,are more influential than others. and ensure that everybody has a similar understanding of the resultingRecent research has suggested that propositions. Participants are thenfor quantitative judgment tasks the asked to rank these propositions, inDelphi Method is slightly more order to identify the ones with a wideraccurate than Prediction Markets and 7
  8. 8. X&Y Partners Decision making: choose the right tool for the jobbase of consensus. Exhibit 5 illustrates also important to note that, even withthe results of a Nominal Group an experienced facilitator, it can beTechnique, where participants were hard to guarantee a full participationfirst asked to identify company from the quieter contributors. Knowingstakeholders, and then to rank each the Briggs-Meyer Type (Exhibit 6) ofstakeholder in terms of company each participant is often a valuableinfluence and dependency. tool in preparing a Nominal Group Technique.Exhibit 6 – Results of the qualitative evaluation requested to the 227 participants in theexperience described in Exhibit 4 Face-to-Face meetingFor quantitative judgment tasks, theNominal Group Technique is All of the methods discussed aboveconsidered to be slightly more are fairly rigid: they require preparationaccurate than Prediction Markets but time and follow a structured path that,less so than the Delphi Method for better or for worse, allows little(Exhibit 4). In terms of ease of use, it room for only outranked by the traditionalFace-to-Face meeting (Exhibit 6). If you have answered “No” to all questions in the decision tree fromPreparing a Nominal Group Technique Exhibit 1, the unstructured discussioncan be time consuming, as this of a traditional Face-to-Face meetingmethod requires a carefully planned might be the most suitable option tostructure and adequate facilities. It is 8
  9. 9. X&Y Partners Decision making: choose the right tool for the job Exhibit 7 – The Briggs-Meyer Type, an indicator of how people perceive the world and make decisionsaddress the issue at hands. A lack of the most appropriate participants,formal structure does not mean agree on an agenda, keep within thehowever a lack of organization. A allotted time, wrap up the conclusionsdecision-making meeting should follow and follow-up actions, and distributethe same guidelines of any other meeting minutes.effective meeting: plan ahead, select 9