THE IMPACT OF EXPERIMENTATION ON PRODUCT DEVELOPMENT IN

      COMPANIES INVOLVED IN INTERACTIVE MARKETING

              ...
UMI Number: 3369490


                             Copyright 2009 by
                           Sundukovskiy, Sergey L.


...
© Sergey L. Sundukovskiy, 2009
Abstract



       Since its emergence, the Internet and Internet related technologies have permeated almost

all aspects ...
Dedication



        I dedicate this work to my wife Galina for her commitment, dedication and unconditional

love. Also,...
Acknowledgements



       Special thanks to my friend Boris Droutman for introducing me to the subject of

experimentatio...
Table of Contents


Dedication ..............................................................................................
Qualitative Data Display for Describing the Phenomenon ....................................................... 44
   Quali...
List of Figures


Figure F1. Response summary................................................................................
Figure F28. Experimentation product deployment risk impact and experimentation experience
cross-tabulation ..................
ix
CHAPTER 1: INTRODUCTION



                                    Introduction to the Problem

Internet Marketing

       The...
reached $19.5 billion, projecting to reach $36.5 billion in 2011. Companies had realized that

Internet marketing was not ...
marketing; (c) affiliate marketing; (d) display marketing; and (e) social media marketing (See

Appendix C). These Interne...
devices, interactive screen devices and e-books are capable of carrying a marketing message by

means of wireless or hard-...
personalization through data mining and information analysis. The data mining and analysis

constitutes the bulk of the pe...
and product up-sell. Each one of these goals could be considered a question that interactive

Internet marketing is suppos...
product development, such data does not exist or is very limited. As such, predictive modeling

marketing strategies by de...
a more advantageous interactive Internet marketing product development approach that places

product initiative in the han...
back-loading to frontloading. Depending on the nature of the product companies may utilize

several experimentation modes:...
and instead favor low-tech solutions. The paradox consists of a seeming contradiction between

relying on the technologies...
Research Questions

       This study aims to answer six main research questions. What is the impact of

experimentation o...
important to understand the fundamental benefits offered by experimentation as it relates to

interactive marketing produc...
marketing ideas have been tested using surveys and focus groups. However, this approach is long

and inexact. It also give...
optimize the marketing goals of the companies involved in interactive and Internet marketing.



                         ...
Phantom Flow: flow that does not cause a variation in behavior. It represents a virtual

duplication of any physical flow ...
Experiment itself. Traffic criteria are a collection of exogenous variables.

        Traffic Distribution: allocation of ...
This study also assumes that all participants of the study are not aware of their active

participation in the study. This...
Organization of the Remainder of the Study

       The remainder of the study consists of four additional chapters. Chapte...
CHAPTER 2: LITERATURE REVIEW



                                  Experimentation Introduction

       The process of expe...
coming up with an improved solution based on the previous experience from the preceding

experiment cycles); (2) build (th...
the best = df(n) / dt, taking the time of the day into consideration.

       Thomke (1998) defines three experimentation ...
always possible to represent multivariate experiments as a series of univariate ones, by

temporarily freezing all but one...
Depending on the nature of the experiment it is possible to find a smaller subset of variables that

have the greatest eff...
In practical terms, going back to the “home to the office” driving example, univariate

models consider the independent im...
time. In more recent years researchers have focused on experimentation in the context of product

development.

       Enl...
marketing.

       Some of the mentioned research in the areas of interactive marketing, experimentation

and product inno...
model of interactive marketing on the basis of numerous case studies, presented and analyzed in

succession. An additional...
affecting desired interactive exposure is predominantly related to interactive content richness.

Additionally, Raman (199...
experimentation process is data analysis driven. In his essays on interactive marketing Li (2003)

examined three cases of...
(1999) did not mention this concept as interactive marketing experimentation by name, he

hypothesized that interactive ma...
empirical work that is devoted to experimentation in the context of product development. This

dissertation capitalizes on...
CHAPTER 3: METHODOLOGY



                                  Description of the Methodology

        The study assessing th...
meaning for several individuals of their direct experience of a concept or a phenomenon. In this

particular case the phen...
phenomenological method as hermeneutical. Van Manen (1990) described the researcher of the

study as one of the participan...
Design of the Study

       The qualitative research paradigm employs interviews as its predominant data collection

instr...
otherwise would be purely quantitative data. In contrast to close-ended questions these questions

are open-ended.

      ...
Assuming that the chosen sources were both valid and credible, further research

credibility and validity depended only on...
segmentation); (b) interactive marketing product development lifecycle (product risk, product

innovation, product improve...
company where the research was conducted utilized multifaceted interactive content. More

specifically the company used em...
tested and modified according to the feedback from the pilot group and the mentors of the study.

The pilot group consiste...
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Interactive Marketing
Upcoming SlideShare
Loading in...5
×

Interactive Marketing

1,852

Published on

Published in: Business
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,852
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
32
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Interactive Marketing

  1. 1. THE IMPACT OF EXPERIMENTATION ON PRODUCT DEVELOPMENT IN COMPANIES INVOLVED IN INTERACTIVE MARKETING by Sergey L. Sundukovskiy H. PERRIN GARSOMBKE, Ph.D., Faculty Mentor and Chair LARRY KLEIN, Ph.D., Committee Member JOSE NIEVES, Ph.D., Committee Member William A. Reed, Ph.D., Acting Dean, School of Business & Technology A Dissertation Presented in Partial Fulfillment Of the Requirements for the Degree Doctor of Philosophy Capella University August, 2009
  2. 2. UMI Number: 3369490 Copyright 2009 by Sundukovskiy, Sergey L. All rights reserved INFORMATION TO USERS The quality of this reproduction is dependent upon the quality of the copy submitted. Broken or indistinct print, colored or poor quality illustrations and photographs, print bleed-through, substandard margins, and improper alignment can adversely affect reproduction. In the unlikely event that the author did not send a complete manuscript and there are missing pages, these will be noted. Also, if unauthorized copyright material had to be removed, a note will indicate the deletion. ______________________________________________________________ UMI Microform 3369490 Copyright 2009 by ProQuest LLC All rights reserved. This microform edition is protected against unauthorized copying under Title 17, United States Code. _______________________________________________________________ ProQuest LLC 789 East Eisenhower Parkway P.O. Box 1346 Ann Arbor, MI 48106-1346
  3. 3. © Sergey L. Sundukovskiy, 2009
  4. 4. Abstract Since its emergence, the Internet and Internet related technologies have permeated almost all aspects of the modern life. The impact of the Internet on the day-to-day activities of its users has been quite dramatic. However, its effect on the business community has been even more profound. Besides yielding additional opportunities for existing businesses, the Internet has facilitated a new era of companies that have the Internet at the center of their business model, businesses that simply would not be able to exist without it. As such Internet-centric and increasingly traditional businesses now rely on interactive marketing as a source of revenue, differentiation and competitive advantage. Despite the obvious importance to many businesses, the efficiency and improvement of interactive marketing has largely stagnated or at least proceeds at a low pace. This study examined interactive marketing through the prism of experimentation as a way of propelling interactive marketing forward and enabling it to keep pace with technological advances. Experimentation lies at the core of product development, improvement and innovation. Active experimentation has been utilized as a product strategy in numerous business fields, but it has been largely ignored in interactive marketing. This study examined a number of experimentation models and their applicability to interactive marketing. In addition it focused on the elements of interactive marketing that are conducive to experimentation.
  5. 5. Dedication I dedicate this work to my wife Galina for her commitment, dedication and unconditional love. Also, to my children: Aaron, Rebekah and Miriam for giving meaning to my life. Your love and support gave me strength to see this project through. To my parents: Valentina and Leonid, for always believing in me and forever encouraging me to see physical, emotional and intellectual limits as imaginary lines that are meant to be crossed. Last, but by no means least, to my brother: Aleksey, for seeing me the way I wish I could truly be. iii
  6. 6. Acknowledgements Special thanks to my friend Boris Droutman for introducing me to the subject of experimentation and helping me generate the ideas that made this dissertation possible. Thanks for encouraging me, listening to the things I say and hearing the things I do not say. iv
  7. 7. Table of Contents Dedication ...................................................................................................................................... iii Acknowledgements ........................................................................................................................ iv Table of Contents ............................................................................................................................ v List of Figures ............................................................................................................................... vii CHAPTER 1: INTRODUCTION ................................................................................................... 2 Introduction to the Problem......................................................................................................... 2 Internet Marketing ....................................................................................................................... 2 Interactive Marketing .................................................................................................................. 4 Product Development .................................................................................................................. 9 Background of the Study ........................................................................................................... 10 Purpose of the Study ................................................................................................................. 11 Research Questions ................................................................................................................... 12 Definition of Terms ................................................................................................................... 15 Assumptions and Limitations .................................................................................................... 17 Organization of the Remainder of the Study............................................................................. 19 CHAPTER 2: LITERATURE REVIEW ...................................................................................... 20 Experimentation Introduction ................................................................................................... 20 Experimentation Strategies ....................................................................................................... 21 Experimentation Models ........................................................................................................... 22 Interactive Marketing ................................................................................................................ 27 Experimentation ........................................................................................................................ 29 Product Innovation .................................................................................................................... 31 CHAPTER 3: METHODOLOGY ................................................................................................ 33 Description of the Methodology ............................................................................................... 33 Design of the Study ................................................................................................................... 36 Population and Sampling .......................................................................................................... 37 Measurement Strategy ............................................................................................................... 38 Instrumentation.......................................................................................................................... 39 Data Collection .......................................................................................................................... 40 Data Analysis Procedures.......................................................................................................... 41 v
  8. 8. Qualitative Data Display for Describing the Phenomenon ....................................................... 44 Qualitative Data Display for Explaining the Phenomenon ....................................................... 45 CHAPTER 4: DATA COLLECTION AND ANALYSIS............................................................ 46 Overall Response Analysis ....................................................................................................... 46 Research Objective One ............................................................................................................ 48 Research Objective Two ........................................................................................................... 53 Research Objective Three ......................................................................................................... 58 Research Objective Four ........................................................................................................... 61 Research Objective Five............................................................................................................ 63 Research Objective Six ............................................................................................................. 66 Validity and Bias ....................................................................................................................... 69 CHAPTER 5: RESULTS, CONCLUSIONS, AND RECOMMENDATIONS ........................... 71 Results ....................................................................................................................................... 71 Conclusions ............................................................................................................................... 74 Recommendations ..................................................................................................................... 75 Limitations of the Study ............................................................................................................ 76 REFERENCES ............................................................................................................................. 78 APPENDIX A. SURVEY TEMPLATE ....................................................................................... 82 APPENDIX B. CONCEPTUAL FRAMEWORK........................................................................ 88 APPENDIX C. INTERACTIVE MARKETING .......................................................................... 89 APPENDIX D. MARKETING ..................................................................................................... 90 APPENDIX E. EXPERIMENTATION CYCLE ......................................................................... 91 APPENDIX F. DATA ANALYSIS .............................................................................................. 92 vi
  9. 9. List of Figures Figure F1. Response summary...................................................................................................... 92 Figure F2. Job function breakdown .............................................................................................. 92 Figure F3. Job function and experimentation experience cross-tabulation .................................. 93 Figure F4. Experimentation impact breakdown............................................................................ 93 Figure F5. Experimentation impact and department cross-tabulation .......................................... 94 Figure F6. Experimentation impact and experimentation experience cross-tabulation ................ 94 Figure F7. Experimentation levers impact breakdown ................................................................. 95 Figure F8. Experimentation levers impact and experimentation experience cross-tabulation ..... 95 Figure F9. Experimentation levers impact and job function cross-tabulation .............................. 96 Figure F10. Experimentation conditions impact breakdown ........................................................ 96 Figure F11. Experimentation conditions impact and experimentation experience cross-tabulation ....................................................................................................................................................... 97 Figure F12. Experimentation conditions impact and job function cross-tabulation ..................... 97 Figure F13. Experimentation CPA impact and experimentation overall impact cross-tabulation 98 Figure F14. Experimentation CPI impact and experimentation overall impact cross-tabulation . 98 Figure F15. Experimentation CPA impact and experimentation overall impact cross-tabulation 99 Figure F16. Experimentation upsell impact and experimentation overall impact cross-tabulation ....................................................................................................................................................... 99 Figure F17. Experimentation CPI impact and experimentation overall impact cross-tabulation100 Figure F18. Experimentation product lifecycle impact breakdown............................................ 100 Figure F19. Experimentation impact on product lifecycle and job function cross-tabulation.... 101 Figure F20. Experimentation impact on product lifecycle and experimentation experience cross- tabulation..................................................................................................................................... 101 Figure F21. Experimentation product innovation impact breakdown ........................................ 102 Figure F22. Experimentation product innovation impact and experimentation experience cross- tabulation..................................................................................................................................... 102 Figure F23. Experimentation product innovation impact and job function cross-tabulation ..... 103 Figure F24. Experimentation product improvement impact breakdown .................................... 103 Figure F25. Experimentation product improvement impact and job function cross-tabulation . 104 Figure F26. Experimentation product improvement impact and experimentation experience cross-tabulation ........................................................................................................................... 104 Figure F27. Experimentation product deployment risk impact breakdown ............................... 105 vii
  10. 10. Figure F28. Experimentation product deployment risk impact and experimentation experience cross-tabulation ........................................................................................................................... 105 Figure F29. Experimentation product deployment risk and job function breakdown ................ 106 Figure F30. Experimentation competitive standing of the company impact breakdown ........... 106 Figure F31. Experimentation competitive standing of the company impact and job function cross-tabulation ........................................................................................................................... 107 Figure F32. Experimentation competitive standing of the company impact and experimentation experience cross-tabulation ......................................................................................................... 107 Figure F33. Experimentation revenue impact ............................................................................. 108 Figure F34. Experimentation profit impact ................................................................................ 108 Figure F35. Experimentation market size impact ....................................................................... 108 Figure F36. Experimentation market penetration impact ........................................................... 109 Figure F37. Experimentation marketing goals impact horizontalization.................................... 110 Figure F38. Experimentation marketing levers impact horizontalization .................................. 111 Figure F39. Experimentation conditions impact horizontalization............................................. 112 Figure F40. Experimentation product lifecycle impact horizontalization .................................. 113 Figure F41. Experimentation product innovation impact horizontalization ............................... 114 Figure F42. Experimentation product improvement impact horizontalization........................... 115 Figure F43. Experimentation product deployment risk impact horizontalization ...................... 116 Figure F44. Experimentation competitive standing impact horizontalization ............................ 117 viii
  11. 11. ix
  12. 12. CHAPTER 1: INTRODUCTION Introduction to the Problem Internet Marketing The introduction of the World Wide Web, enabled by the Internet, saw an explosive user growth (Newman, 2001). Consequently both online based and traditional companies with significant Internet presence started to realize the full potential of World Wide Web as a marketing medium. Since the popularization of the Internet and Internet related technologies, several new marketing paradigms have emerged that take advantage of Internet. Such emergence was not planned and occurred spontaneously as an offshoot of traditional marketing (Mark, 2003). Internet marketing was formulated as one form of non-traditional marketing. Due to early disappointments and lack of experience with a new non-traditional marketing medium Internet marketing was initially used to supplement traditional marketing efforts. According to eMarketer Research (2007), US online marketing in 2002 shrunk by 15.8%. Companies did not see Internet marketing as a separate marketing channel, with a unique set of characteristics and considered it the same as radio and television (Raman, 1996). Internet marketing was combined with traditional television and radio campaigns because at first, it was not considered capable of carrying the full weight of responsibilities. However, over time marketers began to realize that traditional marketing campaigns consisting of television and radio advertisements, and print media designed to attract users to their websites had a limited success, whereas the Internet itself proved quite promising in that regard. According to eMarketer Research (2007) online advertizing spending in 2007 had 2
  13. 13. reached $19.5 billion, projecting to reach $36.5 billion in 2011. Companies had realized that Internet marketing was not only a strong and independent marketing medium, but also had a number of advantages over traditional forms of marketing, such as television, radio and print media (Mark, 2003). In order to earn its independence and be considered comparable to one of the traditional dominant marketing channels, Internet marketing had to be evaluated against a multitude of traditional marketing problems. It had successfully answered a number of questions that were historically asked by marketing science. Internet marketing showed its potential for consumer accessibility, defining corporate identity, promoting brand awareness, enabling market segmentation and achieving market localization and internationalization (Jebakumari, 2002). In addition, Internet marketing capabilities such as personalization, interactivity and traceability were formulated among its advantages over traditional dominant medium channels such as television, radio and print media. The Internet allowed synchronous and asynchronous user behavior analysis, which in turn allowed for previously unavailable personalization capabilities (Milley, 2000). It had the potential to reach a vast number of World Wide Web users that otherwise would be either difficult to reach, or in some cases completely unreachable. According to Internet World Stats (2007), by the end of 2007 the Internet had reached 1,319,872,109 users with the largest percentage growth accounted for by countries in Middle East, Africa and Latin America. Since its inception, the Internet has developed into one of the strongest and mature Marketing channels that is segmented into several sub-channels. At the present time Internet marketing could be considered as a conglomerate marketing methodology, providing a common umbrella for a set of diverse marketing sub-channels. The following distinct Internet sub- marketing channels have now emerged: (a) search engine marketing (SEM); (b) e-mail 3
  14. 14. marketing; (c) affiliate marketing; (d) display marketing; and (e) social media marketing (See Appendix C). These Internet marketing approaches, founded upon sociological, economic and cultural marketing aspects, are now interwoven into Internet marketing fabric as a cohesive entity. At the same time each one of these Internet marketing sub-channels has a set of unique characteristics, based on its execution and marketing strategies. These characteristics distinguish it from other tangential marketing approaches (Mark, 2003). Interactive Marketing In addition to being part of Internet marketing, social media marketing, affiliate marketing, search engine marketing and e-mail marketing sub-channels have another aspect in common, their interactivity. Each one of these sub-channels relies on the user to make an interactive step directly related to the advertisement stimuli. As such these marketing sub- channels could be placed under another umbrella called “Interactive Marketing”. In its purest form Deighton (1996) defines “Interactive Marketing” as an ability of the computer based system to interact with the user for marketing based purposes. In many cases however interactive marketing consists of a multi-transactional interaction that is based on user purchase and visit history, declared or implied preferences and demographic information consisting of age, sex, income level and even race. Most of the interactive marketing accomplished through the World Wide Web is enabled by the Internet. However, it would be incorrect to consider Internet marketing as synonymous with interactive marketing (Deighton, 1996). Based on the Veen diagram (see Appendix D) not all Internet marketing is interactive and not all interactive marketing is conducted on the World Wide Web or is Internet based. For instance display marketing, one of the Internet marketing sub-channels, is not interactive in nature. In the same vain iPods, mobile devices, gaming 4
  15. 15. devices, interactive screen devices and e-books are capable of carrying a marketing message by means of wireless or hard-connected communication. Each of these devices has its own marketing specifics related to its size and interactivity, tracing power, display and communication capabilities. However, based on the Veen diagram (see Appendix D) it is evident that a great amount of overlap exists between the Internet and interactive marketing. Chulho (1998) defined the “Interactive Age” as an age of full interactivity between humans and machines and technology connected humans. Similar parallels can be drawn between the “Information Age” and “Participation Age”, coined by Schwartz (2005). Blogging and Social Networking are two recent examples of the Interaction or Participation Age that are utilized by social media marketing. At the heart of interactive marketing are the two main concepts of “Personalization” and “Customer Engagement”. The interactive marketing itself could be fully defined in terms of these concepts. More specifically, this researcher proposes an alternative definition of “Interactive Marketing” describing it as a process of customer engagement through personalization. In turn, Steinberg (2006) defines “Customer Engagement” as a process of engaging potential or existing customers in inter-client or inter-customer communication by means of the push or pull marketing model. According to Haag (2006) “Personalization” is a process of generating unique experience for an ever finer grain segment of customers. Taken to the logical extreme, the process of personalization is a process of targeting a single individual with the stimuli uniquely tailored to that individual and that individual alone. The logical conclusion of these definitions is the hypothesis that the most effective customer engagement could be achieved through granular personalization activities. The companies involved in interactive Internet marketing endeavor to achieve 5
  16. 16. personalization through data mining and information analysis. The data mining and analysis constitutes the bulk of the personalization effort in both temporal and relative importance perspectives. Ozdemiz (2002) describes personalization as an effort consisting of three major steps: (a) data gathering; (b) data analysis; (c) personalized content generation. According to comSource (2007) the data gathering portion of the personalization process undertaken by major interactive marketing companies such as Yahoo, Google and Facebook has reached a significant, earlier unseen level. According to comSource (2007), in a period of a single month in 2007 Yahoo on average collected data 2,520 times for every visitor to its site. Mark (2003) posited that Internet marketing companies collect as much data a possible even without a specific goal in mind. Since, it is not always clear what customer data would become useful in the future, all available data is collected (Mark, 2003). It usually consists of webpage visits, visit history, purchasing history, abandonment and completion history, self-declared or inferred preferences, computer information, web browser information, self-declared or inferred demographic information self-declared or inferred financial information, geographic information, age and sex information (Mark, 2003). The collected data is carefully cataloged and analyzed. The purpose of this data analysis is to determine the most appropriate marketing strategy for a particular user and other users that fit similar characteristics. Milley (2000) describes this approach as a “Data Empowered” marketing strategy. The intention of any marketing strategy in general and interactive Internet marketing in particular is to achieve a desired goal. In general interactive Internet marketing tries to achieve a goal of conversion where the user is synchronously driven to a purchasing decision. According to Macias (2000) even though conversion is certainly the dominant interactive marketing goal, other goals may include customer retention, customer acquisition, lifetime value maximization 6
  17. 17. and product up-sell. Each one of these goals could be considered a question that interactive Internet marketing is supposed to answer. Namely, interactive Internet marketing is required to provide answers to the following questions: (a) what is the most effective customer retention strategy? (b) what is the most effective to driver customer to a purchasing decision? (c) what is the most effective customer acquisition strategy? and (e) what is the most effective product up- sell strategy? By analyzing collected user data interactive marketing researchers try to answer these questions in a predictive manner. The predictive data analysis, which is at the core of the data empowered marketing strategy, is accomplished through diverse data modeling techniques. Dou (1999) described the use of the Catastrophe Theory to model online store sales. Other techniques include the Recency-Frequency-Monetary Value (RFM) model designed to assess the customer purchasing decision. However, these and other models have several fundamental problems. First of all, these models are predictive and as such carry a fair amount of statistical error due to a need to make assumptions and use guess work. Consequently, these models are subject to Simpson’s Paradox where a lack of understanding of data granularity could lead to incorrect conclusions. Secondly, the Internet is an open system where not all of the variables and their effect are clearly defined (Dou, 1999). As such confounding variables become a real issue. Thirdly, the feedback loop between data collection, data analysis and interactive Marketing changes is quite long and inefficient (Dou, 1999). In addition to the limitations mentioned above, predictive modeling strategies are fundamentally unsuitable to the set of marketing tasks related to product development and risk mitigation. The data empowered marketing strategy absolutely requires extensive amounts of data in order to come up with a predictive marketing model. However, at the stage of new 7
  18. 18. product development, such data does not exist or is very limited. As such, predictive modeling marketing strategies by definition are not suitable for new product development. In the past marketers had to rely on focus groups to gain some guidance on new and existing product development (Mark, 2003). However, the focus group method has proven to be extremely error prone due to the Survey Paradox where people say one thing and then another. According to Mark (2003), in many cases companies would also fail to generate a focus group of the required diversity. As such it was not uncommon to see focus group results being skewed to a particular group of users. In addition, the focus group approach to product development resulted in a monolithic product development driven by corporate executives. In this product development model, product ideas were generated by company managers and corporate executives. Kohavi, Longbotham, Henne and Sommerfield (2007) described this model as the “Highest Paid Person’s Opinion” (HiPPO), where the most senior member of the team comes up with the product idea or is responsible for choosing among multiple product options. When the product idea is selected, the resultant product is manufactured in the monolithic fashion and subsequently demonstrated to the focus group. One possible name for the risk profile in this type of product development is “Risk Backloading”, where the bulk of the risk is born at the end of the product development lifecycle. Consequently, HiPPO initiated and focus group validated product development is extremely deficient in both risk mitigation and consumer-product fit. The main goal of this study is to propose an alternative interactive Internet marketing product development strategy that mitigates the shortcomings of predictive data modeling and HiPPO-Focus Group product development. It addresses data modeling shortcomings related to statistical uncertainly, data feedback loop and confounding variables. In addition it recommends 8
  19. 19. a more advantageous interactive Internet marketing product development approach that places product initiative in the hands of the end-users and allows for a more advantageous risk profile related to product innovation and product development. Even though this study treats Internet and interactive marketing synonymously, it more specifically considers only the part of interactive marketing that is enabled by Word Wide Web and Internet technologies. The findings of the study should however be applicable to the interactive devices mentioned above. Product Development As mentioned earlier, the use of experimentation to investigate the problem has been described by several researchers. It is possible to consider product development as a particular type of problem and so the process of experimentation should be fully applicable to product development. According to Thomke (1997), experimentation as a method of product development has been utilized by a number of industries including Automotive, Pharmaceutical and Electronics. Furthermore von Hippel (1998) posited that experimentation facilitates product development by end-users of the product. User-driven product development allows companies to get away from the HiPPO principal, where product decisions are made by the people who are furthest removed from the product itself. Product users themselves consciously or subconsciously galvanize and propel product development forward. More importantly they do so with their actions instead of the words and in this way they eliminate the Survey Paradox. Thomke, von Hippel, and Franke (1997) argued that employing experimentation in product development allows companies to improve their competitive standing. Since products are shaped by the end-users directly, the chances of product failure are greatly decreased. The experimentation process allows controlled failures early and often when the cost of change is minimal. By employing the experimentation approach companies reverse the risk profile from 9
  20. 20. back-loading to frontloading. Depending on the nature of the product companies may utilize several experimentation modes: (a) simulation; (b) prototype; and (c) sample. Experimentation modes depend on such factors as experiment result applicability, experiment cost and experiment efficiency. According to Thomke (1997), experimentation modes constitute a continuous process optimized based on the efficiency vs. number of experimental cycles. Based on the applicability of experimentation to product development in other areas it would seem logical that the process of experimentation would be pertinent to Internet marketing and interactive marketing as well. However, very little if any empirical studies have been conducted to explore this matter. Researchers such as Li (2003), Raman (1996), Mark (2003), among others, have broached the subject from the predictive modeling point of view. Even if experimentation is mentioned it lacks continuity with interactive marketing itself and is always driven by reactive analysis. Background of the Study This study examines the applicability of experimentation to product development in the confines of Internet and interactive marketing. Since, empirical evidence of experimentation in the field of interactive marketing is sorely lacking it draws on experimentation experience from other industries and product categories. Its background is rooted in the fields of experimentation, product development, production innovation, Internet development, Internet marketing and interactive marketing. The narrative of the study is centered on the product development strategies in interactive Internet marketing. It is designed to address what can be called the Information Age Product Development Paradox, where companies that have a significant Internet eCommerce presence tend to ignore technology-centric product development strategies 10
  21. 21. and instead favor low-tech solutions. The paradox consists of a seeming contradiction between relying on the technologies as the source of survival, yet disregarding them when it comes to product development. With regards to product development, companies involved in the Internet and the interactive marketing tend to ignore product development strategies that have been developed in other fields. As such they face problems related to product risk, product improvement and product innovation. Purpose of the Study The main purpose of the study is to examine the applicability of experimentation as the product development strategy in the companies involved in interactive marketing. It examines the use of experimentation in all parts of the interactive marketing product development lifecycle: (a) idea generation; (b) idea screening; (c) concept development and testing; (d) business analysis; (e) beta testing and market testing; (f) technical implementation; and (g) commercialization. In addition this study examines the impact of experimentation on product evolution as a function of user interactivity. It aims to contribute to the experimentation body of knowledge as well as establishing an empirical base for interactive marketing experimentation practice that is quite limited at the present time. Furthermore, this study looks at the unique elements of interactive marketing experimentation. It attempts to define the majority of the exogenous and endogenous interactive marketing factors and their effect on interactive marketing goals. The secondary purpose of the study is to point out the gaps and propose improvements in the Design of Experiments Theory, as it pertains to interactive marketing. 11
  22. 22. Research Questions This study aims to answer six main research questions. What is the impact of experimentation on interactive marketing goals? What are the key experimentation levers pertaining to interactive marketing? What is the impact of experimentation on product innovation in interactive marketing? What is the impact of experimentation on interactive marketing product improvement? What is the impact of experimentation on interactive marketing product development and deployment risk? What is the impact of experimentation on the competitive standing of the companies involved in interactive marketing? Online conversion is one of the key goals of interactive marketing companies. Even though conversion is not the only goal, the majority of interactive marketing companies are striving to achieve higher conversion. Even though conversion events of interactive marketing companies are quite diverse, in essence, the conversion event is equivalent to a revenue generating event. It is paramount to determine if the use of experimentation has a positive effect on online conversion. This study examines a number of exogenous and endogenous variables pertaining to interactive marketing and analyses their impact on the interactive marketing goals. The aim of the study is not to define an exhaustive list of exogenous and endogenous factors across all interactive marketing products and environments, but rather to determine an effective subset usable across interactive marketing companies of different types. A significant number of the research questions posed by this study are devoted to the analysis of the impact of experimentation on product development lifecycle. More specifically the study aims to examine the impact of experimentation on product innovation, product improvement, product development and product deployment risk. Similar to the goals, the interactive marketing companies’ products are equally as complex and as diverse. However, it is 12
  23. 23. important to understand the fundamental benefits offered by experimentation as it relates to interactive marketing product development regardless of the product type. Lastly, this study will examine the impact of experimentation on the competitive standing of companies in the interactive marketing space. The aim of the study is to show that companies in the interactive marketing space that abstain from experimentation, in relation to product development, ignore it at their own peril and put their competitive standing and quite possibly long term survival at risk. Nature of the Study Experimentation in the interactive marketing field is a social phenomenon since it is designed to achieve the best results through social interaction. Despite the fact that the social interaction in this context occurs between man and machine, the social context of the interaction is preserved. As such the use of a qualitative design approach is consistent with social constructivism and fully justified as the research approach for this dissertation. It is would have been absolutely acceptable to employ a quantitative or mixed-method design approach since experimentation relies heavily on statistical analysis and statistical data modeling. However, the topic of this dissertation and the state of the empirical research in the experimentation field of interactive marketing are more conducive to qualitative analysis. This study is theory forming in nature and as such it is consistent with one of the strong fundamentals of qualitative analysis. Significance of the Study The experimentation approach can give companies involved in interactive marketing the means to test marketing ideas without putting the majority of its traffic at risk. In the past 13
  24. 24. marketing ideas have been tested using surveys and focus groups. However, this approach is long and inexact. It also gives rise to what can be called a “Survey Paradox”, where people say one thing and do another (Sheffrin, 1996). The process of experimentation presents qualitative and quantitative results and represents an innovative development in testing Internet marketing and interactive marketing ideas. However, very little or virtually none of the empirical research confirms positive effects of experimentation by online conversion when used as part of interactive marketing strategies. Furthermore, due to a lack of solid theoretical research companies involved in experimentation in the field of interactive and Internet marketing are forced to develop a practical base behind experimentation through trial and error. Such approach results in limited success and might lead to companies abandoning experimentation altogether. In addition companies that do successfully pursue experimentation, in the context of interactive marketing, quite limit their success due to poor alignment between the Business and Information Systems designed to conduct experimentation. As such experimentation is done haphazardly, lacks continuity and does not result in a cohesive strategy. The significance of this study is in establishing a theoretical base behind experimentation in the interactive and Internet marketing fields. One of the key implications of the study is the assertion that companies that are involved in interactive and Internet marketing, outside of the experimentation framework, negatively impact their competitive standing, increase product development risk and reduce their likelihood of success in achieving their marketing goals. Another essential contribution of the study is an outline of the implementation process in the context of the experimentation framework that allows achieving cohesion between departmental units involved in the interactive marketing experimentation. This study positions experimentation as a product development and product improvement method designed to 14
  25. 25. optimize the marketing goals of the companies involved in interactive and Internet marketing. Definition of Terms Base Flow: current web site execution path with the highest fitness value. All related flows performance is compared to the Base Flow. Challenger Flow: variation of the Base Flow designed to achieve a higher fitness value. Confirmation Flow: previous Base Flow designed to confirm its losing status to the current Base Flow. The introduction of the Confirmation Flow can be classified as confirmation testing, where results of the previous Experiment are reconfirmed. The purpose of the confirmation testing is to make sure that the previous Base Flow had lost to the current Base Flow in the head to head comparison and was not a result of a statistical error or impact of unaccounted or unknown confounding variable. Experiment: manipulation on one or more of the endogenous variables in the context of the exogenous variables with the purpose of achieving Silo Goals. The purpose of the experiment is to reject or accept the null hypothesis. Experimentation Levers: collection of exogenous variables that is intrinsic to the Experiment. Flow: single variation of the web site execution path. Typically the experiment consists of several Flows that manipulate a single variable for a side by side comparison. Multivariate Experiment: experiment consisting of manipulation of multiple endogenous variables simultaneously. Multiple Singlevariate Experiments: Multiple experiment sets affecting a single endogenous variable per set. 15
  26. 26. Phantom Flow: flow that does not cause a variation in behavior. It represents a virtual duplication of any physical flow for achieving statistical significance. Silo: area under experimentation. The Silo usually consists of a collection of endogenous variables that can be manipulated in a particular context. In typical interactive marketing Silos consist of landing web pages, up sell web pages, offer pages, etc. Silo Goal: purpose of the experimentation. In the majority of interactive marketing cases the Silo Goal is represented by conversion also known as a purchasing decision. Other Silo Goals may include link offs, up sell actions, subscriptions, content contribution, etc. In general the terms the goal of the experimentation is to achieve the highest fitness value. Singlevariate Experiment: experiment consisting of manipulation of a single endogenous variable. View: single web page alongside the web site execution path. Typically Flow consists of several Views that contain one or more endogenous variables that are being manipulated. Imposed Flow: flow that allows predictive execution of the web site path without distribution and analysis. Imposed Flow is often utilized for regulatory or testing purposes. Silo Visit: represents a visit to one of the areas under experimentation, typically represented by a collection of web pages. Site Visit: represents a visit to the web site or any other Traffic Origin of the interactive marketing company. Traffic Origin: source where web traffic originates. Traffic Origin is often represented by one of the interactive marketing channels such as email, search, display, affiliates, call center and social media. Traffic Origin is one of the key exogenous variables. Traffic Criteria: criteria that is intrinsic to the incoming web traffic, but extrinsic to the 16
  27. 27. Experiment itself. Traffic criteria are a collection of exogenous variables. Traffic Distribution: allocation of the web traffic between Experiments and Flows according to the distribution algorithm. Traffic Distribution between Experiments and Flows must add up to 100% otherwise some of the incoming traffic will fail to be distributed. Vertical: product centric collection of interactive marketing assets. View Visit: represents a display web page or any other asset of interactive marketing such as email, banner, etc. Visitor: represents a unique visitor to the web site. Assumptions and Limitations In addition to paradigmatic and methodological assumptions this study has a number of domain assumptions. First, it assumes the synonymous nature of following terms: A/B Testing, Split Testing, A/B/C Testing, Singlevariate Testing, Singlevariate Multi-Experiment Testing, and Multivariate Testing. All of these terms are aimed at describing the same phenomenon and had evolved under a common umbrella (see Appendix C). However, it needs to be noted that the term multivariate testing refers to the experimentation technique related to univariate testing strategies, yet it is not synonymous with them. However, for all intent and purposes the practice of multivariate testing as well all univariate testing techniques will be assumed to be describing a common phenomenon. In cases where the study discusses the intricacies of a particular experimentation approach the name of the approach will be explicitly noted. In the same vain such terms as Internet marketing and interactive marketing are treated synonymously and interchangeably. In cases where a distinction between these concepts needs to be made it is made explicitly. 17
  28. 28. This study also assumes that all participants of the study are not aware of their active participation in the study. This assumption is essential for illuminating participant bias. It is also assumed that all participants of the study have at least cursory knowledge of interactive and internet marketing as well as a practical understanding of experimentation as it applies to the above mentioned concepts. Besides listed domain assumptions this study has a number of paradigmatic and methodological assumptions. In some respects it follows the basic assumptions of the systems approach. It assumes objective reality that is independent of the observer. Similar to the systems approach the key aim of experimentation is to model the real world phenomenon. As such it employs inductive and deductive reasoning as well as verification as the basis for its conclusions. It assumes that the results of any given experiment may be a result of casual relationships and must be verified by further experimentation. The methodological assumptions of the study are related to phenomenological inquiry. According to Becker (1992), the researcher of the phenomenological study is viewed as a co- creator of knowledge alongside the participants of the study. Even though such researcher involvement is beneficial, in this particular case discrepancy between the participant’s and the researcher’s knowledge of the studied phenomenon is too great. As such, the level of threat to researcher bias, validity, generalization and repeatability of the study is too great. Consequently, the researcher employed a combination of transcendental and experimental phenomenological methodology that allowed the researcher to detach his experience from the experience of the participants of the study. 18
  29. 29. Organization of the Remainder of the Study The remainder of the study consists of four additional chapters. Chapter Two examines literature related to interactive and Internet marketing, product development and experimentation. The literature has been selected to describe each of the aspects of the study independently yet leading to cohesion between experimentation, interactive marketing and product development concepts. Chapter Three describes the design methodology that was chosen to conduct the study. It defends the choice of research methodology by examining the methodological fit to the research approach and the subject of the study. It also describes elements of the study including the data collection procedures, data sampling, data collection instruments and data coding. Chapter Four illustrates the findings of the study and undertakes a detailed analysis of the results of the study. It consists of data analysis and data display for describing and explaining the phenomenon of the study. Chapter Five summarizes the findings of the study and introduces an alternative hypothesis for the findings of the study, validity and bias as well as examines the trend in the findings of the study. It also proposes topics for future research and the areas of inquiry in the subject area. 19
  30. 30. CHAPTER 2: LITERATURE REVIEW Experimentation Introduction The process of experimentation in general is at the center of scientific discovery. In fact science or knowledge derived from experimentation is called “experimental science”. People have utilized experimentation since ancient times with one of the classic examples of early experimentation originating from Egypt around 2613-2589 BC. When Egyptians attempted to build a smooth-sided pyramid they engaged in the process of experimentation. They initially started building a pyramid at Meidum which collapsed due to angle acuteness. Based on the result of this failed experiment Egyptians altered the angle of the Bent-Pyramid at Dahshur more than half way through to save it from collapse, resulting in a bent shape. Subsequent smooth- sided Egyptian pyramids have utilized the correct angle from the outset. By looking at the experiment conducted by the Egyptians it is possible to extract several observations: (a) experiments often fail (sometimes they are designed to fail); (b) full scale experiments can be quite expensive; (c) conducting experiments consecutively may take a long time; and (d) conducting experiments concurrently allows effective side by side comparison. Unbeknown to them, this Egyptian experiment created the basis for the Design of Experiments theory. This early experiment and subsequent application of the experiment results contained the majority of elements founds in modern product experiments. Thomke (1997) described experimentation as one of the forms of problem solving. In short Thomke (1997) defined experimentation as a trial and error process. According to Thomke (1998), the experimentation process consists of four major steps: (1) design (design consists of 20
  31. 31. coming up with an improved solution based on the previous experience from the preceding experiment cycles); (2) build (the build step consists of modeling and constructing products to be experimented upon); (3) run (the run step of the experimentation process consists of executing the experiment in the real or simulated environment); and (4) analyze (analysis consists of data mining and data investigation collected during experiment execution). Thomke (1998), pointed out that the experimentation process changes under pressure from exogenous elements. These exogenous elements consist of uncertain requirements, environmental and technological changes. Experimentation Strategies The process of conducting experimentation is not uniform. The choice of the experimentation strategy depends on the problem it is trying to resolve. In order to understand experimentation strategies it is important to look at what biologists define as the “fitness landscape” (Beerenwinkel, Pachter & Sturmfels, 2007). Quite often the solution to a particular problem is not singular. Fitness landscape consists of all possible solutions to a particular problem. A fitness function defines the quality of the solution in relation to other solutions to the same problem. The optimal solution to the problem is thought to have the highest fitness value. A real world demonstration of the fitness landscape could be observed by looking at the simple problem. For example, the way between home and the office consists of n routes. The set of all routes constitutes a fitness landscape. Among all of the possible routes one of the routes is the “best”, where best = f(n). Since, best is a relative term it must be defined in the context of the fitness landscape. It is likely that the majority of people would consider the best route to constitute shortest time. However, it is possible that some would choose a scenic route to be the best. It is also possible to have multiple optimal solutions where the best route changes based on exogenous elements such as time of day, day of the week and weather conditions. In that case, 21
  32. 32. the best = df(n) / dt, taking the time of the day into consideration. Thomke (1998) defines three experimentation strategies: (a) parallel experimentation; (b) serial experimentation with minimal learning; and (c) serial experimentation with learning. It is possible to demonstrate these experimentation strategies by continuing with the “driving from home to office” problem. To find out the route that yields the shortest driving time between the home and the office and is the quickest way would require employing a parallel experimentation strategy. However, it is impossible to accomplish this using a single driver, since it requires taking all possible routes simultaneously. The driver of the car would need help from his friends. If the driver of the car was determined to find the shortest route himself/herself, he/she could employ serial experimentation with minimal learning or experimentation with learning. The serial experimentation with minimal learning would require the driver to follow a predefined plan of taking different routes until all possible routes where exhausted. After all routes have been tried, the driver would have analyze the time each has taken to determine the one with the shortest time. The serial experimentation with learning would allow the driver to avoid trying all routes by analyzing results of initial experiments and illuminating routes that certainly would not yield good results. Experimentation Models The experiments are further complicated by endogenous and the exogenous elements. It is certainly possible to affect the experimental results by changing the endogenous elements of the system (see Appendix E). For instance besides the different routes, changing drivers, car type, fuel type, and the number of drivers among many other things would possible affect the driving time. The experiments with multiple changing endogenous elements are called multivariate. The experiments with a single endogenous variable are called univariate. It is 22
  33. 33. always possible to represent multivariate experiments as a series of univariate ones, by temporarily freezing all but one variable. Ideally, in order to understand the impact of all exogenous variables each of them requires a separate experiment. In other words in order to understand the impact of the time of departure from the office on the overall travel time, the driver needs to conduct experiments by leaving the office every hour on the hour. In the same vain all weather conditions must be tested as well. Based on the analysis of the experimental models it is quite obvious that the complexity of experimentation grows exponentially based on the number of endogenous and exogenous elements, such that Number of Experiments (E) = k * (m^n)!, where k is the number of the steps in the process, m is the number endogenous variables and n is the number of exogenous variables. Based on formula above it is possible to arrive at the conclusion that the chance of guessing the option with the highest fitness value, when the number of endogenous and the exogenous elements are beyond trivial, is infinitesimally small. At the same time running experimentation of all combinations of endogenous and the exogenous factors in order to determine the best possible combination yielding the highest fitness value would constitute a full factorial experiment (Xu &Wu, 2001). However, even with a trivial number of exogenous and endogenous factors the number of resulting experiments that satisfies full factorial design is truly staggering. By applying the formula listed above a single step process with 3 endogenous factors and 3 exogenous factors would result in 1.0888 E 28 number of experiments. For all intent and purposes running full factorial experiments beyond a trivial number of factors is simply not practical. In order to resolve the conundrum of no experimentation and full factorial experimentation it is necessary to look at the nature of exogenous and endogenous factors. Not all of the variables involved in the experiment affect the outcome of the experiment equally. 23
  34. 34. Depending on the nature of the experiment it is possible to find a smaller subset of variables that have the greatest effect on the experiment. Anderson (1972) called this approach to experimentation partial factorial. There are number of statistical methods of determining what variables truly matter to the outcome of the experimentation. Li (2003) identified the following partial factorial reduction models: (a) Univariate Poisson (relies on the analysis of the variable of all of the involved variables); (b) Univariate Tobit without Log Transformation; (c) Univariate Tobit with Log Transformation; (d) Discretized Univariate Tobit with Log Transformation; (e) Discretized Univariate Tobit with Heteregeity; (f) Multivariate Count; (g) Multivariate Count with Mixture; and (f) Multivariate two-state hidden Markov Chain Tobit. Even a cursory look at the listed models allows them to be separated into two categories: (a) univariate; and (b) multivariate. Fundamentally univariate models, where ANOVA tests are applied in succession, are designed to ascertain the effect of the independent variables on the dependant variables (Biskin, 1980). In the context of the experimentation univariate models are designed to highlight the exogenous and endogenous factors that have a significant impact of the outcome. On the other hand multivariate models, where a MANOVA test is conducted, are designed to come up with sets of independent variables that have an impact on the dependant variables (Huberty, 1986). Again by taking the experimentation context into account the multivariate models are designed to highlight sets of exogenous and endogenous factors that have a significant impact on the outcome of the experimentation. According to Huberty and Moris (1989) the fundamental difference between multiple univariate ANOVA tests and multivariate MANOVA tests consist in the consideration of the effects of the independent variables on each other and their compound effect on the outcome. More specifically univariate models tend to ignore the relationship between exogenous and endogenous factors and their compounding effect, where multivariate models take this relationship into account. 24
  35. 35. In practical terms, going back to the “home to the office” driving example, univariate models consider the independent impact of the weather conditions and the time of the departure on the driving time, where multivariate models would consider these two factors in conjunction with each other. Even though interactive and Internet marketing are relatively new phenomena they have generated a fair amount of popular and scientific foment. It is even fair to say that the scientific community has been lagging behind interactive and Internet marketing practitioners who have pushed exploration boundaries. At the same time, in recent years, interactive and Internet marketing, as a subject of scientific inquiry, have seen an increased rate of exciting empirical research. These studies have focused on interactive and Internet marketing from social, physiological and physiological perspectives (Jebakumari, 2002; Milley, 2000; Macias, 2000; Liu, 2002; Newman, 2001; Mark, 2003; Bezjian-Avery, 1997; Raman, 1996). Unlike Internet and interactive marketing, experimentation is not a new phenomenon. There is abundant evidence that ancient Egyptians conducted experiments during pyramid construction. In a more recent example of experimentation James Lind, while servicing onboard the Salisbury, conducted an experiment of using citrus to cure scurvy. One of the notable differences between early experimentation efforts and the experiments conducted by Lind was the use of control and treatment groups. The results of the control group were compared with the results of treatment groups in order to confirm or reject the hypothesis of the experiment. In the early 20th century Ronald Fisher formulated a mathematical method for designing and analyzing experiments. He had introduced “factorial” as the term applicable to experiments involving several factors or variables at the time (Fisher, 1935). Fisher (1926) initially used “complex experimentation” as the term describing experimentation with multiple variables at the same 25
  36. 36. time. In more recent years researchers have focused on experimentation in the context of product development. Enlarged experimentation methods have been researched in the context of Electrical and Mechanical Engineering (Wang, 1999; Hansen, 1989; Donne, 1996). It is certainly not surprising that experimentation practice has been widely employed in industrial manufacturing since manufacturing product commitment is quite expensive and may result in significant losses and even, in some cases, impact on the long-term survival of the company. The product development lifecycle often requires experimentation to be part of the product development process. The deceptive ease of change in the Internet and interactive marketing product development has resulted in the situation where experimentation best practices found in the industrial product engineering are ignored. There is certainly a glaring lack of empirical research into experimentation in the Internet and interactive marketing fields as it pertains to product development. There are a few empirical works that have broached this subject (Dou, 1999; Li 2003; Ozdemir, 2000); however, these studies are primarily dedicated to data mining and predictive modeling rather than experimentation as a continuous practice. It is important to note that in recent years several researchers have focused on interactivity in general and interactive experimentation in particular as a key driver in product innovation (Thomke, Von Hippel & Franke, 1997; Thomke, 1998; Thomke 2001; Von Hippel 1998; Thomke; 1995). These studies have asserted that product innovation is driven by product users themselves through interaction and experimentation. Product innovation was particularly highlighted in these studies and was considered separately from the remaining phases of product development lifecycle. In large, this part of this dissertation capitalizes on the mentioned studies in the context of applying findings of above mentioned research both to Internet and interactive 26
  37. 37. marketing. Some of the mentioned research in the areas of interactive marketing, experimentation and product innovation is examined in more detail below. Interactive Marketing If the Internet timeline could be separated into three decadal stages: (a) mid 1990s – introduction stage; (b) early 2000 – development stage; and (c) late 2000 – maturity stage; then the Jebakumari (2002) study could classified as a study of the stages of Internet development. It was during this time that Internet interactivity came into strong researcher and practitioner focus. The overall purpose of the Jebakumari (2002) study was to describe interactivity in the context of Internet marketing. Lyons (2007) offered several research questions: (a) what are the nature, characteristics and components of interactivity? (b) what are the shortcomings of the traditional marketing models in context of the interactive medium? (c) how is interactivity related to comprehension? Jebakumari (2002) examined traditional marketing and its shortcomings to explain this interactive phenomenon. Both traditional and interactive Internet marketing were compared and contrasted. The conclusions reached by Jebakumari (2002) were reminiscent of a similar study conducted by Mark (2003). Jebakumari (2002) found that a number of traditional marketing techniques were inconsistent with the interactive media and did not adequately address the interactive audience. A study by Milley (2000) could be attributed to the late introductory and early development stages of Internet marketing. Miley (2002) explored, what he calls, Web-enabled consumer marketing, its intricacies and specifics. Miley (2002) tried to formulate a theoretical 27
  38. 38. model of interactive marketing on the basis of numerous case studies, presented and analyzed in succession. An additional focus of the study was related to the operational recommendations of running a consumer oriented interactive web site. Miley (2002) proposed the following research questions: (a) what is the theoretical basis of Web-enabled consumer marketing? and (b) how should the company align its operations to be congruent with the Web-enabled consumer marketing model? Miley (2002) reached the conclusion that Web-enabled consumer marketing requires analysis of behavioral user data to guide future actions and marketing decisions. He also concluded that in order to facilitate comprehensive data analysis, interactive user data must be both extensive and complete. As such, Web-enabled consumer marketing or interactive marketing companies must position their human and systems resources, as well as establish operational practices conducive to data capture. The Raman (1996) study could be attributed to the introductory period of interactive and Internet Marketing. Raman (1996) explored interactivity on the Web, at the time when it was emerging phenomenon. In particular, Raman (1996) examined the desired customer exposure to online banners. Similar to the later studies by Mark (2003) and Jebakumari (2002) that focused on the comparison and contrast between traditional and interactive marketing, Raman (1996) contrasted banner exposure in traditional and interactive marketing models. The Raman (1996) study is similar to parallel study by Bezjian-Avery (1997) which attempted to define interactive marketing and its core concepts. Raman (1996) proposed the following research questions: (a) what are the factors affecting the desired interactive exposure? and (b) how do the levels of interactive exposure affect the desired advertising outcome? Raman (1996) concluded that the dominant factor 28
  39. 39. affecting desired interactive exposure is predominantly related to interactive content richness. Additionally, Raman (1996) concluded that an interactive advertisement that speaks to the consumer on an individual level at the same time as being pertinent and engaging has a high chance of achieving the designed interactive outcome. Experimentation As mentioned earlier, the body of knowledge regarding experimentation and experiment design is heavily focused on major engineering disciplines. The empirical research on the subject of experimentation in interactive and Internet marketing is scarce and tangential. This study relies on several seminal works on Design of Experiments, data modeling and product innovation. In the area of Design of Experiments this study examined several research papers related to the Taguchi Method. Weng (2007) presented a detailed analysis of experiment optimization methods. These methods were compared on the bases of (a) global optimization; (b) discontinuous object function; (c) non-differentiable function; and (d) convergence rate. Weng (2007) found that the Taguchi Method scored extremely well in all of the compared categories. Weng (2007) gave a detailed review of the Taguchi Method itself and its benefits over other optimization methods. Weng (2007) also suggested several improvements to the Taguchi Method that can be applicable to experimental design in interactive marketing. In addition to the design of experiments in Electrical and Mechanical Engineering this dissertation is based on another tangential topic related to data mining and data modeling in interactive and internet marketing. It is important to note that it is impossible to conduct experimentation without being engaged in some form of data mining and data modeling. The experimentation is enabled by data analysis and data mining. More specifically, the 29
  40. 40. experimentation process is data analysis driven. In his essays on interactive marketing Li (2003) examined three cases of interactive marketing. In the first essay Li (2003) described the functionality of cross-selling services on an interactive banking web site. Li (2003) analyzed the behavioral reasoning behind online user actions as they pertain to the purchasing of products and services offered by the interactive banking web site. In conducting behavioral analysis Li (2003) utilized several multivariate probit models implemented by Hierarchical Bayer framework. This dissertation examined the applicability of the models proposed by Li (2003) in conducting interactive real-time online experiments. In his second essay Li (2003) analyzed the browsing behavior of users on several interactive web sites. In order to predict future browsing paths Li (2003) utilized several Poisson and discretized tobit models. These models were compared and contrasted in the context of their ability to accurately predict user browsing behavior. This dissertation utilizes the modeling technique findings presented by Li (2003). In his third essay Li (2003) analyzed purchase and conversion data from several eCommerce web sites. He used this data to build a predictive purchase model. Li (2003) concluded that his Hierarchical Bayer framework supplemented with hidden Markov model could accurately predict a path reflecting user goals ultimately leading to a potential purchase. This dissertation capitalizes on the findings of this essay during the set up and analysis of the effect of experimentation on reaching interactive marketing goals. Similar to the Li (2003) study, research by Dou (1999) utilized similar statistical analysis for modeling online sales. Dou (1999) examined the applicability of the Catastrophe Theory to modeling actual behavior and predicting potential purchasing online decisions. Dou (1999) explored what he termed the data empowered marketing strategy, where data was mined through tracking users to guide the interactive marketing decisions of the company. Even though Dou 30
  41. 41. (1999) did not mention this concept as interactive marketing experimentation by name, he hypothesized that interactive marketing data can be used to alter the interactive user experience in real time as more of the user data was collected and analyzed. Dou (1999) called this approach adaptive marketing communication, where consumer behavior is analyzed through continuous observation. Dou (1999) proposed that interactive marketing data can be modeled using the Catastrophe model. He hypothesized that Catastrophe Theory is eminently suitable for this type of analysis and predictive modeling. Dou (1999) concluded that it was indeed possible to model and ultimately predict the browsing and purchasing behavior of users on interactive marketing web sites. The significance of both the Li (2003) and Dou (1999) studies is the fact that interactive marketing data is been actively analyzed using a multitude of statistical models in the context of interactive marketing. However, it is import to note that use of the Taguchi Method for similar analysis has not been empirically researched. Additional computing paradigms for predictive data modeling such as Evolutionary Computing and Genetic Algorithms have been explored by several researchers (Ozdemir, 2002). Ozdemir (2002) argued that Evolutionary Computing offers real potential in deriving a best fitness value. As such it holds significant promise for online data modeling and interactive marketing experimentation. Product Innovation A significant portion of this study is devoted to analyzing the impact of experimentation on the product development lifecycle in the context of interactive marketing. Even though there are few empirical studies that directly deal with experimentation in interactive marketing, emphasizing the web site as an interactive marketing product, there is a significant body of 31
  42. 42. empirical work that is devoted to experimentation in the context of product development. This dissertation capitalizes on the several seminal works by Thomke and Von Hippel. Thomke (1995) hypothesized that the mode of experimentation such as prototyping and simulation has a significant impact on the economics of experimentation. More specifically, Thomke (1995) proposed that the use of simulation experimentation is more economical and therefore far more likely to be used in product development. Consequently simulation experimentation could be viewed as a product enabler and innovation driver. Thomke (1995) presented two case studies where experimentation was used in the design of new pharmaceutical drugs and integrated- circuit based systems. Thomke (1995) proposed experimentation design cycles consisting of designing, building, running and analyzing activities performed in a contiguous manner. Each successive cycle was built taking into account the findings of the previous cycle. This study hypothesized the applicability of this cycle in general, and the process in particular, to interactive marketing product development (see Appendix E). Thomke (1995) found that switching between prototyping and simulation experimentation modes significantly affected experimentation economics, resulting in a substantial reduction in design cost. In a seemingly unrelated study Von Hippel (1998) argued that product innovation should be driven by the people who would benefit from the end product of innovation, end users of the product themselves. Von Hippel (1998) described what could be called the Von Hippel paradox, where product specialists should not be primarily responsible for product innovation, but rather defer to product users as a source of ultimate innovation. Von Hippel (1998) described this paradox as a shift in locus of problem-solving. In this dissertation Von Hippel’s ideas are combined with the approach proposed by Thomke (1995), where interactive marketing product development is driven by users through interactive experimentation. 32
  43. 43. CHAPTER 3: METHODOLOGY Description of the Methodology The study assessing the impact of experimentation on interactive product development utilized a qualitative research paradigm. The choice of qualitative research methodology was related to the nature of the topic and the innate characteristics of the field of the study. Employing qualitative research methods makes the quality of the data of paramount importance. Consequently, emphasis is placed on how and under what circumstances the data is collected (Morgan & Smircich, 1980). In contrast to quantitative research methods, it is rare to see a qualitative researcher working with large quantities of data. This is the case with the current study as all analyzed data comes from a single organization. Maxwell (1992) defined the qualitative research methods as theory forming. These methods are used to generate new theories or introduce new hypotheses. Maxwell (1992) called qualitative research a paradigm that is concerned with a “breadth first” approach as opposed to a “depth first” as is the case with quantitative research. More specifically, qualitative research is preoccupied with describing a phenomenon as thoroughly as possible, and forming a theory behind it. Based on the paradigmatic characteristics provided by Maxwell (1992), the use of qualitative research methods was consistent with the goals of the study and the state of knowledge in the field of experimentation in the context of interactive marketing. This study utilized a phenomenological method as one of the research methodologies under the qualitative paradigm umbrella. The phenomenological method was first formulated by Husserl (1983). Creswell (2007) defined the phenomenological method as a description of the 33
  44. 44. meaning for several individuals of their direct experience of a concept or a phenomenon. In this particular case the phenomenon is a process of experimentation in the context of interactive marketing. Husserl (1983) described the application of this phenomenological approach as an execution of three consecutive steps. The first step consisted of adopting a phenomenological method that encouraged the researcher to infuse quantitative data with the qualitative context that allowed the data to be meaningful. The second step consisted of seeking out an instance where the phenomenon can be studied in its natural context in order to distill the essence of the phenomenon. The third and final step was described by Husserl (1983), and consisted of describing the discovered meaning of the phenomenon. The experimentation phenomenon in the context of interactive marketing sits well in the phenomenological inquiry. This researcher wanted to discover meaning behind the experimentation phenomenon in the context of interactive marketing by studying individuals who have experienced the phenomenon first hand. Even though participants of the study have experienced the phenomenon they are not necessarily aware of its meaning (Georgi, 2006). This point of view is certainly consistent with the description of the experimentation phenomenon. The participants of this study have certainly experienced the experimentation phenomenon in the context of interactive marketing, but by and large they are not aware of its meaning and its fundamental characteristics. By utilizing phenomenological tools as a bracketing, horizontalization, clustering, delimiting and imaginative variation, phenomenological tools allow the researcher to extract meaning from the experiences of the participants of the study. The phenomenological method allows the researcher of the study to quantify his/her own experience by supplementing findings of the study with his/her own observations and interpretations in the context of the experience. Creswell (2007) described this type of 34
  45. 45. phenomenological method as hermeneutical. Van Manen (1990) described the researcher of the study as one of the participants of the study. This researcher has an extensive experience with experimentation in the context of Internet and interactive marketing. The phenomenological method allows researcher understanding of his own experience while maintaining a strong relationship to the topic (van Manen, 1990). However, in order to address generalization, validation, validity and bias, the researcher must employ bracketing to distinguish his own experience from the experience of the participants of the study. As such this researcher tried to deemphasize his own experience. This researcher employed a combination of transcendental and experimental phenomenology rather than hermeneutical phenomenology, where emphasis is placed on the experience of the participants of the study and the experience of the researcher is bracketed (Moustakas, 1994). Use of experimental phenomenology allowed the researcher to focus on the practical application of the phenomenon rather than the philosophical side of it. It is important to note the distinction between other qualitative methodologies such as grounded theory or other narrative approaches and phenomenological methods. Creswell (2007) made a distinction between narrative study and phenomenological study, where the former is experienced by several study participants individually, as opposed to as a group in a latter case. Even though participants of the study were selected from several groups participating in interactive marketing experimentation, individual group representatives described an experience of the group to which they belong. According to Cresswell (2007) phenomenological methods place emphasis on the shared experience of the phenomenon. It is critical to understand experimentation in interactive marketing in the context of a particular group as well as the organization as a whole. 35
  46. 46. Design of the Study The qualitative research paradigm employs interviews as its predominant data collection instrument (Babour, 1998). When an interview is conducted in a purely qualitative manner, the researcher then takes an active participation in the interview process. In that case, the researcher is considered to be an actual instrument of the study. Babour, (1998) pointed out that participants in a study often receive major guidance from the researcher throughout the interview process. The qualitative research paradigm thus encourages researcher participation in order to reduce language ambiguity and supplement the possible lack of context associated with quantitative data collection instruments. This study however, did not employ interviews as the means of data collection. The major concerns of the study were related to credibility, validity and bias. Since the researcher of the study is employed by the company where the research is being conducted, actual or potential undue influence was a paramount concern. The researcher struggled to maintain the balance between extricating himself from the data collection process on one hand, yet maintaining the qualitative nature of the study on the other hand. In order to address validity, credibility and bias concepts as well as maintaining the qualitative nature of the study, the researcher employed a research instrument used in mixed- method research studies. More specifically this study utilized a mixed-method survey as a research instrument. Johnson and Onwuegbuzie (2004) also described a mixed-method survey that embodies both qualitative and quantitative aspects. Mixed-method surveys typically contain questions found in fixed surveys. These types of questions are referred to as close-ended, where the set of responses are limited. In addition to quantitative questions, these surveys contain corresponding sections that allow freehand expression, allowing for a qualitative context to what 36
  47. 47. otherwise would be purely quantitative data. In contrast to close-ended questions these questions are open-ended. The resultant survey consisted of 15 open-ended questions and 17 close-ended questions. In order to eliminate possible researcher influence the survey was administered over the Internet in an anonymous fashion. In addition data was collected under false pretenses. The participants of the study were not told that the data was being collected for the purposes of research due to the possibility of participant bias. The survey was positioned as providing helpful feedback on the experimentation efforts of the company in the context of interactive marketing. Population and Sampling The study included 23 human participants. The participants of the study work in the same organization as the author of the study. A particular set of participants was chosen from all of the groups involved in the interactive marketing experimentation. The technology group was excluded from study participation, since the author of the study works for the technology group and may exert undue influence on the participants of the study. The participants of the study were randomly chosen from Interactive Marketing, Business Development, User Experience, Data Analysis and Creative Design groups. The participants from each of the mentioned business units provided information relevant to the results of experimentation and its impact on various aspects of interactive marketing. They were asked to elaborate on their experiences of experimentation in the context of the interactive marketing. The participants of the study were solicited on the perceived success of experimentation relative to its goals such as improved conversion, product innovation, product improvement, risk mitigation and improved competitive standing. 37
  48. 48. Assuming that the chosen sources were both valid and credible, further research credibility and validity depended only on the researcher himself/herself. One of the ways of selecting credible and valid sources is by selecting them at random. More specifically only a single representative of each business unit was selected at random. This type of selection method helped reduce personal bias. Measurement Strategy This study surveyed interactive marketing professionals in the confines of a single company. The study participants were selected at random to represent their interactive marketing channel. The respondents of the study were asked to complete a mixed-method survey consisting of 40 questions related to their experience with experimentation in the context of interactive marketing. The research questions of the survey were designed to understand the relationship between the corresponding dependent and independent variables. Since the number of independent variables was too great they were grouped under common categories. For instance, independent variables related to experimentation such as color, font, font size, and images were grouped under a visual category. It is important to note that independent variable categories were classified as either endogenous or exogenous. The resultant survey included five categories of endogenous independent variables (see Appendix B) (a) visual; (b) functional; (c) positional; (d) informational; and (e) behavioral, as well as six categories of exogenous independent variables (see Appendix B) (a) temporal; (b) demographical; (c) seasonal; and (d) contextual. In addition to the independent variables each of the research questions had a number of dependent variables associated with it. These dependent variables were assigned as follows (see Appendix B) (a) competitive standing (revenue, market size, profit, market share, and market 38
  49. 49. segmentation); (b) interactive marketing product development lifecycle (product risk, product innovation, product improvement, product life cycle, and product targeting); and (c) interactive marketing goals (cost per acquisition, cost per impression, cost per action, upsell, and click- through rate). Each research question was directly represented in the survey in the form of several survey questions. In addition to asking participants of the study to answer research questions directly, each of the dependent variables was investigated in isolation (see Appendix A). The survey questions were formulated to draw a connection between independent variables in an amortized form and the dependent variables associated with research questions. The amortized independent variables were referred to as “Experimentation” (see Appendix A), where dependent variables were called out in exactly the same way as they were specified in the “Conceptual Framework” (see Appendix B). Even though the quantitative questions were utilized alongside qualitative questions, qualitative data was not used in drawing conclusions of the study. The point of analysis associated with the quantitative data was to ascertain consistency between qualitative and quantitative answers. The quantitative survey questions used single and multiple choice scales. The qualitative survey questions utilized a measurement strategy associated with the phenomenological method consisting of horizontalization, clustering, textual and combined description. Instrumentation The survey (see Appendix A) contained 40 questions designed to solicit information related to the experimentation efforts of the company in the context of interactive marketing. The 39
  50. 50. company where the research was conducted utilized multifaceted interactive content. More specifically the company used email, search, social, display, internet and affiliate interactive marketing approaches. The participants of the study were asked to fill out the survey relating to their experience of product development through experimentation in each of the respective interactive marketing areas. The questions of the study were crafted to meet the objectives of the study. The close-ended questions of the study were not used in the final analysis of the study, but rather they were designed to ensure consistency of a corresponding open-ended question as well as to guide the user to stay in the confines of the intended question. The survey was designed and implemented using online survey software and conducted over the Internet. The participants of the study were invited by the CEO of the company to complete the survey via. The email contained the link to the online survey as well as an explanation of the purpose of the survey, ensuring that participation in the study was voluntary and anonymous. The survey contained five major sections: (a) introduction (questions related to overall experimentation experience); (b) interactive marketing goals (questions related to the impact of experimentation process on the goals of various interactive marketing channels); (c) interactive marketing product (questions related to the impact of the experimentation process on the various aspects of product development lifecycle of various interactive marketing channels); (d) competitive standing (questions related to the impact of experimentation process on the various key competitive indicators); and (e) sustaining effects (questions related to the sustaining effects of experimentation in the context of interactive marketing) Data Collection The data was collected via the SurveyMonkey.com web site. The initial survey was pre- 40
  51. 51. tested and modified according to the feedback from the pilot group and the mentors of the study. The pilot group consisted of three members chosen from a pool of potential participants. The participants of the study were given a week to complete the survey with multiple reminders sent two days and on the day before the survey expiration period. All questions in the study were designated as mandatory and the only two ways to exit the survey were to either to complete it or abandon it. If the survey was abandoned in order to proceed with the survey at the future date the participant of the study had to start the survey over again. According to the SurveyMonkey.com statistics none of the surveys were abandoned and the effective survey completion was at 100%. Due to employing the survey online participant anonymity was preserved. After all surveys were completed the survey results were downloaded onto the researcher’s computer and analyzed. At all times the survey results were protected from inadvertent or intentional disclosure. The surveys were conducted online over secure protocol and access to the survey results was username and password protected. When the results of the survey were downloaded to the researcher’s computer access to the computer itself was username and password protected as well. Data Analysis Procedures The data analysis procedures roughly consisted of the steps outlined by Creswell (1998) with slight adaptation for the needs of this study. These steps consisted of (a) horizontalization and bracketing; (b) clusters of meaning; (c) textual description; (d) composite description. It is important to note that the usual phenomenological step of transcription was omitted since the data was collected via an open-ended survey administered over the Internet. As such the data transcription consisted of downloading the results of the completed surveys. The data analysis 41

×