Addressing Large, Complex, Unstructured Problems


Published on

Published in: Education, Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Addressing Large, Complex, Unstructured Problems

  1. 1. Addressing Large, Complex, Unstructured Problems Written by Alexa DiBenedetto Advised by Roger Hoerl Senior Thesis March 2014
  2. 2. 2 I. Introduction Abstract We are all faced with problems throughout our lives, not only in our educations and careers, but also in our personal lives and relationships. Despite the universal need for effective means of addressing problems, there is surprisingly little agreement in the statistical or scientific literature as to how to approach problems in general. The problem solving frameworks that do exist tend to be designed for narrow, well-defined problems. Unfortunately, the most significant problems faced by modern society tend to be large, complex, and unstructured, that is, not well defined. I have identified major themes in the problem solving literature across disciplines, which I have divided into two macro- themes: six key phases that any problem solver must pass through and consciously consider during the problem solving process, and the key considerations that should be kept in mind during every step of the problem solving process. In addition, I have made some recommendations to problem solvers who face large, complex unstructured problems, such as understanding that there is no one best method for solving every problem, being able to use a variety of tools and techniques, even if we aren’t familiar with them, and allowing yourself more time than you think is necessary to work through the problem solving process. Large, Complex, Unstructured Problems The term “large, complex, and unstructured problem” itself can be a bit ambiguous. Thus, I explain what I mean by large, complex, and unstructured in terms that are general enough they can be applied in any field, not just statistics. One characteristic of these types of problems is that they are large in scale. For instance, solving a textbook problem
  3. 3. 3 usually involves analyzing manageable data sets that we can simply enter into Excel or statistical software, and apply regression, hypothesis testing, and so on. On the other hand, real-world problems usually are not neat and tidy in this way, as there are often extremely large amounts of data and information being gathered over a large span of time or from many sources. These problems are also complex. While this term is somewhat ambiguous, in this instance it can mean that the thing we are trying to “solve” is complex, or the tools needed to solve it are complex; clearly, there is no “correct” or standard solution. In most cases, there are multiple stakeholders who each have a different objective within the problem. Usually, the relationship between these stakeholders is unclear or undefined, which causes even more complexity. There also may be legal or ethical issues in these kinds of problems, in addition to questions about who should be responsible for conducting and funding the research necessary to solve them. By highlighting all of these complexities, it becomes clear why we must treat large complex problems differently than we do regular “textbook” problems. Lastly, these problems are unstructured, meaning that a problem is not necessarily precisely defined, or set up with a clearly defined set of data, background information, problem, or solution. Sometimes, researchers do not agree on exactly what question they are trying to answer, or what problem they are solving. In this case, it becomes impossible to simply plug in equations and regression models etc. Many preliminary steps must be taken to assess the root of the problem being posed, such as why it is being asked, what is important about it, what is the significance of the results we are aiming to obtain, and so on. This aspect of problems makes them most difficult to address in a
  4. 4. 4 cookie cutter way. However we can hopefully establish a way to streamline this process so that it becomes easier and more manageable to attack such problems. Motivation While there is not much research being done on methods of solving large, complex, and unstructured problems, there has been some thought into the challenges caused by not having a systematic way of attacking such problems. Hoerl and Snee (Closing the Gap 52) highlighted one of the main issues regarding this type of problem solving: often, most of the emphasis is placed on the tools and methods used to solve problems, and there is little done to teach students and researchers to apply these tools in a more broad setting where the problems are not neatly laid out like they would be in a textbook. This process, sometimes referred to as “statistical engineering,” might prove to be the most important aspect in solving large, complex, unstructured problems, as it focuses on creatively applying the tools we learn from doing “cookie-cutter” examples and problems in school. This article actually suggests that the statistical problem solving process only relies 20% on studying the “pure” science of statistics, and that 80% is dependent on statistical engineering, where we consider how to link and integrate these tools. Thus, the need to establish a structured way of solving large, complex, and unstructured problems becomes clear. The Mathematics Department at Harvard University recognized the need to develop courses to prepare students to address the kinds of problems they would face in the “real world” as statisticians or researchers. Thus, a new course was created, titled “Statistics 399: Problem Solving in Statistics,” with the goal of “helping Statistics PhD students transition through the qualifying exams and into research”(Harvard Statistics
  5. 5. 5 website). The professors who created this class point out that well written exams not only benefit the students taking them, but also help the professors who are writing them (Blitzstein, Meng 1-2). They note that there is often no textbook or library to simply copy and paste exam questions that are unique and also stimulating. Thus, there is a need for creative problem generation, and even more importantly, creative problem solving. None of these problems have cookie-cutter solutions. The goal of a course like Stat 399 is to allow students to develop skills that will not only help them to solve simple, straight- forward problems like the ones they’ve seen in class, but also recognize the similarities and differences between various problems in order to allow them to solve problems that are not as straightforward. These problems, called “nano-research problems,” are meant to mimic real world examples and research. Example For the purposes of this paper, we can study large, complex, and unstructured problems through the analysis of a particular example, making it easier to understand key concepts and pieces of advice. An example of a real-life problem that is large, complex, and unstructured is determining how to provide an affordable yet comprehensive healthcare system to Americans. The amount of data that is available to study is immense – patient records, insurance claims, Medicare/Medicaid records, and more, gathered over the last few decades. Additionally, the scope of the problem in itself is large – it involves all Americans. In addition, the problem is unstructured – no one solution is “correct” because there are many varying viewpoints on what is considered a “good” healthcare system and what we even mean by the word “good.” Who is the judge of the “goodness” of the
  6. 6. 6 healthcare system? The government? The citizens of America? Insurance companies? Doctors? The list is almost endless. Further, what exactly do we mean by “Americans”? Do we mean people who were born in America? Anyone who lives in America, including or not including illegal immigrants? Additionally, how do we define the problem? Are we searching for a solution that is beneficial to the most people for the least amount of out- of-pocket expense? Most profitable for the medical profession or private insurance companies? Least expensive to the US Government? Has the highest rate of surgical success? Least number of fatalities? Clearly, this list goes on indefinitely. Our problem is also complex. We also have no metric to measure the success of the healthcare model we come up with, because our original problem was not well defined. There are multiple stakeholders, each with different objectives: doctors, patients, the US government, insurance companies, pharmaceutical companies, hospitals, and more. There are many legal and ethical issues regarding the healthcare system: is it ever appropriate to refuse treatment to someone? Should cost be considered as a factor, or is age more important? How important is the risk of failure or probability of success? Who should pay for medical research to create this healthcare model? What is the role of individual responsibility – if a person chooses to participate in inherently dangerous activities, should they pay a higher premium? Who should pay for medical insurance – an individual’s employer, the government, or the individual? This list is almost endless. It is clear from this real life example that large, complex, unstructured problems could benefit from a systematic approach, and that this would make solving problems in the future much more consistent and simplified.
  7. 7. 7 II. Literature Review and Interviews Researchers in almost every discipline face large, complex, and unstructured problems. Thus, an important step in creating guidelines for solving such problems that can be applied to a variety of fields is researching and analyzing the available literature across disciplines. While we have not interviewed professors in all disciplines or researched all fields, we have taken a sampling of relevant disciplines that have something to say about the process of problem solving, and acknowledge that our list may leave some input out, and is certainly not exhaustive. Input from Psychology There has been some research on the problem solving process in Psychology, which focuses on brain activity and the personality traits that contribute to the way we solve problems. One finding has been about the “aha moment,” or the sudden moment of clarity during which a person realizes the answer to the problem they have been trying to solve. In their article, called “Gaining Insight into the Aha Experience,” Topolinski and Reber define and interpret the “aha moment.” The four key components of the “aha” experience are: suddenness, ease, positive affect, and the feeling of being right (403). Suddenness refers to the idea that this “aha” moment comes unexpectedly and immediately. The insight is gained and processed easily, and is gratifying. Lastly, the problem-solvers have confidence in their solution, before they have formally tested the “correctness” of it. In other words, “insight is an experience during or subsequent to problem-solving attempts, in which problem-related content comes to mind with sudden
  8. 8. 8 ease and provides a feeling of pleasure, the belief that the solution is true, and confidence in this belief” (402-3). Another interesting idea that this article proposes is the fact that when a general- knowledge question is more easily processed, people feel more confident in their answers, independent of their actual knowledge. Additionally, the sudden onset of a solution is sometimes enough to make a person feel more confident in his or her answer (403-4). In conclusion, the authors state, “when a solution to a problem pops into a person’s mind, information that has been difficult to process can be processed more fluently” (404). One negative aspect of this moment of insight is the fact that most of the time it cannot be artificially induced, and its onset cannot be predicted. Thus, while it can be a helpful aspect of solving a problem, we cannot rely on experiencing it when we are solving problems. In addition to exploring the “aha moment,” researchers in Psychology have attempted to understand the way people make decisions based on different personality characteristics. Professor Graham Wilson describes the way our personalities affect our decision-making styles in a chapter of his manual (Decision-Making and Problem Solving). The objectives of this chapter are: to identify your personal psychological type and relate it to your personal preferences, to describe factors and personal styles that have an impact on decision making, to distinguish between situations requiring individual decisions and those requiring group decisions, and to identify the attributes of effective decision makers. This unit is based on the Myers-Briggs Type Indicator (MBTI), and highlights the different mental processes we use to think, as established by psychiatrist Carl Jung: taking in information (perceiving), and organizing information and drawing
  9. 9. 9 conclusions (judging) (3-1). Jung also differentiated between two different ways people either perceive or judge: people take in information (perceive) by either using their sense or their intuition, while people organize information by either thinking or feeling (3-2). Combining these with one other set of preferences, we have the MBTI, which assesses preferences on the following scale: extroversion vs. introversion (where energy is derived and focused), sensing vs. intuition (how information is obtained), thinking vs. feeling (how decisions are made), and judging vs. perceiving (how a person is oriented toward the external world). Combining these four scales, there are a total of 16 psychological types, none of which is “right” or “wrong,” “better” or “worse” (3-3-4). This manual suggests that our preferences inherently affect the way we make decisions. The results of sensing vs. intuition (S/N) and of thinking vs. feeling (T/F) have a lot to do with our personal decision making styles. Sensing favors stability, basing decisions on past experiences, while intuition favors innovation, basing decisions on creativity and insight. On the other hand, thinking favors effectiveness, objectivity, and logic, while feeling favors integrity, and considering people’s values and needs (3-4). The critical thing to take away from this chapter is that a person cannot effectively make decisions if they do not combine all of these aspects, using sense and intuition, thinking and feeling (3-5). Another important consideration is who should be making the decisions. Four categories are presented: individual, consultation, group, and delegation (3-6). In individual decision-making, the leader must make the decision alone, collecting input from others only when it is relevant and necessary. When decision making through consultation, the leader shares the issue with one or more people, who all provide input,
  10. 10. 10 although the final decision may or may not take this input into consideration. Group decision-making is similar, however all participants have an equal say in the final decision. During delegation, the leader chooses someone to make the decision within certain guidelines set forth by the leader (3-6). One thing to avoid during group decision- making is a phenomenon called “groupthink,” where members “let their need to agree with each other interfere with their ability to think about the decision critically” (3-8). This is where the group leader plays a huge role in the decision making process, as he or she has the authority to prevent “groupthink” from happening (3-9). The last aspect of decision making that is addressed in this chapter are the attributes that make an effective decision maker. These attributes are: knowledge, initiative, advice seeking, selectivity, comprehensiveness, currency, flexibility, good judgment, calculated risk-taking, and self-knowledge. Most of these are self-explanatory. However, some of them need to be explained in greater detail. For instance, selectivity means that an effective decision maker seeks only pertinent data, and avoids getting bogged down by extraneous information. Currency implies considering current conditions and taking advantage of current opportunities. Finally, self-knowledge means knowing yourself and your strengths and weaknesses. (3-14-5). Two ways of judging whether someone is a good decision maker are: whether they make competent and confident decisions, and whether most of their decisions work out right (3-13). The IESE School of Business’s six-step process to resolving unstructured problems explains that the benefits of their process is its ability to bring together two different types of people - those who see the world in terms of numbers, and those who see the world in terms of words and emotions. Problem solving that uses only one of
  11. 11. 11 these approaches is problematic because we may be missing key statistical or quantitative analysis, or we may be missing key insight into why the problem exists, or how to solve and implement the results of the problem. When both approaches are recognized and utilized together, we have the highest chance of sustained success (4-5). Thus, we cannot rely on one type of person to solve any given problem, and combining the strengths of different kinds of people will help us achieve the greatest possible results. Erika Wells, Visiting Assistant Professor of Psychology at Union College, gave her perspective on the problem solving process in Psychology. She tried to give me an explanation for why and how people make decisions when it comes to problem solving. She believes that even when we think a problem is complex and unstructured, that it actually is not. She explains that with enough work, any problem can be stated in more simple and easy to understand terms. Further, Professor Wells argues that most of the time, when we are solving a complex problem, we reach an “Aha!” moment, or the moment of insight, during which the solution suddenly becomes clear, which is supported by Topolinski and Reber. One of the methods she suggests, when we do now know exactly what our problem is, is trial and error. By trying simple analysis, even if we don’t know the exact problem we are trying to solve, we can usually at least rule out the things that do not work or are not correct, which brings us one step closer to solving the problem. One of the important aspects of problem solving according to Professor Wells is the size of the problem space. The larger the problem space, the longer it takes for us to acknowledge all of our variables and alternative approaches and test them. Thus, we
  12. 12. 12 should always take steps to try to reduce our problem space as much as we can, which usually involves a subject matter expert. Input from Economics Lewis Davis, Associate Professor of Economics at Union College, suggests that the easiest way to solve a large, complex, unstructured problem is to try to add structure to it. He argued that oftentimes, what appears to lack structure actually does have structure, but it is just hidden. Professor Davis’s focus is on Econometrics (applied statistics) and Economic Development, which focuses on the policies that attempt to better the lives of communities. In this field, many of the variables studied rely on another variable, which relies on another variable, which leads to a never-ending chain of dependence. To avoid this problem, he tries to find variables that are truly external and provide “exogenous shocks to the system.” To do this, he notes that economists look for “natural experiments” which provide us with limited knowledge that we can apply to a wider question at least to gain a little insight, if nothing else. Professor Davis admitted that most of the time, Economists and Economic students do not actually systematize undefined/unstructured variables, but rather agree to use metrics that have been developed by experts in other areas (i.e. using a political scientist’s method for measuring how “democratic” a society is). He notes that the hardest aspect of writing an Economics Senior Thesis is developing a problem that is appropriate and researchable, and he often helps his students significantly with this step. Another concern relating to problem solving in Economics is the fact that creativity cannot be effectively “taught,” but it is often what separates the successful
  13. 13. 13 students from those who are not successful. He questions whether this type of creativity can be taught or if it is ingrained in us from a young age. Input from Engineering In addition, there is a lot to learn from the problem solving process in Engineering, specifically Mechanical Engineering. Bradford Bruno, Associate Professor of Mechanical Engineering at Union College, explained to me that the problem solving process in engineering is similar to the process of solving problems in any field. The first step, which is often overlooked, is defining the problem. The next step is generally brainstorming and listing various alternative methods for solving the problem. Then, testing is done to determine which alternative is most likely best. This step can be difficult, however, because you must define your criteria for judging the “goodness” of the model, which can be one of the harder steps of the process. For instance, in trying to design a bridge, would the best bridge be the one that is cheapest to make and supports the most weight? Or would it be made of the lightest materials and the longest lasting? Clearly, there is a lot of freedom for the mechanical engineer to decide which criteria are most important, and which ones can be compromised. The next steps are pretty standard: building the model, testing the model, making changes to the model, and repeating the process until an acceptable result has been achieved. Professors of Mechanical Engineering at Union try to weave creative problem solving into all of their courses, but it becomes more of a main focus during the senior capstone project, where students must create and solve a problem all on their own (with the help of an advisor). During this class, the engineering and mathematical tools are emphasized along with the problem solving techniques mentioned above. This class is
  14. 14. 14 meant to help prepare students to leave the college world where problems are laid out and well defined, and enter the real world where this is usually not the case. William Keat, Associate Professor of Mechanical Engineering at Union College, focuses on engineering design, which is the process of studying a problem or task and figuring out how to solve it, including the brainstorming, testing and evaluation stages. The three main points in this process are design, building, and testing. However, within the context of college-level courses in engineering, most problems students are asked to solve are not large, complex, or unstructured. In general, engineers follow a set of guidelines for solving problems. The six main steps in this process are: recognizing the need to find a solution, defining the problem, brainstorming possible solutions, evaluating these solutions in order to pick what we think is best, developing a detailed solution, and testing and evaluating the solution. For instance, Professor Keat is working with a senior thesis student to develop a portable piece of equipment that helps senior citizens get out of chairs. The problem needs to be solved because there is human need for it - there is no such device that we know of that is affordable and easy to transport. The problem as they have defined it is inventing a portable and affordable machine to help disabled people or senior citizens get out of chairs more easily. Then, they had to come up with the requirements that they were going to use to measure how well this piece of equipment works and how they will measure these criteria. Professor Keat says that brainstorming often entails breaking down the problem at hand into the smallest possible pieces, so that it is manageable to work with. Then, if we can individually solve each piece, we can attempt to put the pieces together to form a
  15. 15. 15 solution of the whole problem. Next comes preliminary testing and evaluation, and then testing of the final model to check if it passes our original criteria. One unique difference between engineering and other fields is the emphasis on teamwork. Most problems require a significant amount of background knowledge in a variety of fields, which would be unreasonable for one person to know. So, combining efforts allows us to come up with creative and helpful solutions. At the end of our discussion, Professor Keat listed the two fundamental design principles for engineers, one of which is the Information Axiom, which basically states to keep it simple. I think this principle could also apply for solving large, complex, unstructured problems. Professor Keat wrote “Engineering Design in General Education,” in which he explains how he created a sophomore research seminar (SRS) that was targeted towards engineers and non-engineers alike, which aimed at teaching students how to think as an engineer. In this course, titled “Impossible Missions Design Teams,” Professor Keat hoped to teach students with a variety of backgrounds about the fundamental principles of engineering design. Professor Keat states, “The objective was for students to learn basic research skills in the context of creative problem solving and engineering design.” Professor Keat cites a book written by John Dewey, titled “How We Think,” listing a five-step analysis of the thought process: felt difficulty, its location and definition, suggestion of possible solutions, development of reasoning, and further observation and experimentation leading to acceptance or rejection. He claims that these steps can be directly compared to the steps an engineer takes when attempting to solve a difficult or complex problem. The goal of this SRS was to use these steps, with the
  16. 16. 16 “team” nature of these kinds of problems in mind, and develop a systematic approach that everyone could use to solve the problems. Input from Quality Management and Statistics One field that commonly faces problem solving is quality management, where researchers are constantly faced with creating solutions and analyzing them to determine how to optimize their results. In “One Size Does Not Fit All,” Hoerl and Snee propose attempting to break down large unstructured statistical problems into manageable categories so that they can be analyzed and solved in a structured way. The first characterization of a problem comes from two variables: is the solution known, and the complexity of the problem. The answer to the first question is a simple “yes” or “no” (1). However, simply knowing the solution to a problem does not make it easy to solve. In the article, the example is given where the “solution” is that a particular person needs to lose weight, but the “how,” i.e. ways to keep weight off, becomes the important factor, as knowing a solution is meaningless if we cannot implement it. On the other hand, when the solution is unknown, the way we approach the problem changes; instead of figuring out how to implement the solution we already know, we must shift our attention to figuring out what the solution even is (i.e. why is a person overweight, vs. how to maintain weight loss). We can also categorize a problem based on how complex it is, either low or high based on the article. As discussed, low complexity problems involve isolated instances of something going wrong, and they are usually easy to fix once the solution is known. High complexity problems, on the other hand, do not necessarily have a particular isolated problem, but rather the problem is on a much larger scale (2). When combined, these two
  17. 17. 17 variables with two levels create a two by two matrix, which can be used to organize problems that are seemingly very different into easy to understand categories. Once a problem is categorized, it becomes much easier to start the problem solving process. Hoerl and Snee study this process further in “Closing the Gap.” They state three main problems in the field of statistics; that many times, statistical projects do not have a high enough impact, that there is often a disconnect between statistical thinking and the tools and methods used to implement such thinking, and that statisticians have the potential to play a much more active role in whatever field they are in, instead of just playing the passive role of a consultant (52). The term “statistical engineering” is discussed, meaning in the literal sense the creative application of (statistical) knowledge. For example, a chemist inventing a new substance is an example of science; while a chemical engineer figuring out how to mass-produce it and use it in creative ways is the goal of engineering. This idea can be applied to statistics just as easily. Thus, the role of a statistical engineer is not to “advance fundamental laws of science,” but rather is to figure out a way to use these laws of science in the most beneficial way to society as a whole. The article suggests that the balance needed is actually 80% statistical engineering and only 20% of statistics as a pure science (52-3). The article suggests Lean Six Sigma (LSS) as an example of engineering: it did not create any new tools as in science, but rather integrated existing tools in a more efficient and beneficial way to generate enhanced results. LSS, as expressed in the article, has not invented anything new, but rather provides a roadmap to address various types of business problems (53). The article states that there are many people who are experts in
  18. 18. 18 statistics and statistical tools, but lack the ability to apply these tools in a creative, efficient, and meaningful way. This then, is the role of statistical engineering. In “Tried and True,” Hoerl and Snee focus on the potential uses of statistical engineering. The article opens with an example of using statistical engineering to increase the efficiency of a transactional process. While many people can come up with many correct methods for solving this problem, statistical engineering allows researchers to streamline their results and create a methodical approach to problem solving (in this case, it was designing an experiment). They realized that they could use methods they already knew and used in other fields and apply them to their problem (in this case, financial collections) (58). Another example discussed is in the context of quality control. In this case, the Product Quality Management System (PQM) was developed in the DuPont Company, and over $30 million was gained in operating costs over two years. Many varying statistical tools were woven into PQM, such as DoE, ANOVA, response surface methods, and more. Again, no new tools were invented, but existing tools were linked and sequenced in a novel way to achieve significant results (58). Together, statistical engineers are able to create new and creative solutions utilizing existing methods to confusing or unique problems. Statistical engineering can be seen as a more basic example of simply streamlining the way we approach any statistics problem. In the area of experimental design, three steps – screening, characterization, and optimization – provide an overall strategy of experimentation that goes beyond any one design. The screening phase involves looking at large numbers of variables with the goal of narrowing them down and
  19. 19. 19 figuring out which ones were most important to the problem at hand. During the characterization phase, emphasis is put on measuring linear effects and interactions, and finding good operating conditions. Lastly, in optimization, we use response surface designs to make predictive models (59). However, it is important to note that no one step can be used to solve a large, complex, and unstructured problem, but knowing the three different stages of a problem may be a good place to start, and this methodology relates to the generally sequential approach of solving problems. In “Statistical Thinking: Improving Business Performance,” Hoerl and Snee focus on adapting the Lean Six Sigma techniques to be more broad and applicable to more types of problems. The overall strategy, called DMAIC, is an acronym for “Define, Measure, Analyze, Improve, and Control.” An important aspect of Lean Six Sigma and the DMAIC process is the fact that two entire phases are devoted to defining the problem precisely, and quantifying it. Picking a project and clearly defining the problem characterize the “define” stage. In the case of large, complex, unstructured problems, this may prove to be the hardest step, as once the problem is well defined it usually becomes easier to solve. The next step, measure, is where we pick the output(s) that we believe to be important or need improvement, and determine whether they are quantifiable. We gather as much data as we can on these variables (130-32). Next comes the analyze step, which is where we use our statistical tools to run various tests and create models to represent and explain our data (132-34). The next step is the improvement step, which is when we propose a way or ways to improve our problem based on the data we analyzed previously, and then test the improvements to make sure they worked. This step and the one before it are generally
  20. 20. 20 iterative, as one run of analysis and improvement usually do not solve a problem completely, while they so usually point us in the right direction (135). Lastly, we control our solution, so that we can sustain it once we are done working on the problem (135-36). It is crucial to understand that this framework does not work on every kind of problem, specifically where we don’t have a well-defined problem or we don’t have a clear way of measuring our variables. This framework should not be used as strict science, but rather as a suggested set of guidelines to help make the problem solving process easier to attack. A workshop presented by the National Science Foundation, titled “Discovery in Complex or Massive Datasets: Common Statistical Themes,” focused on the application of statistics in various fields, specifically in the case of large data sets. The areas discussed were biology, finance, computer science, astrophysics, and more. Two key points that were brought up in all cases were sparsity and machine learning. Sparsity reflects the way we pare down information from extremely large data sets into a more manageable size (5). In the case of the examples in this article, this usually meant compressing data and information. Machine learning highlights the role of computers and models in analyzing the data we have, which becomes a much more difficult and important task when we have big data sets (6). This workshop takes time to point out that statisticians work in all kinds of fields and disciplines, and that even some of the examples given in this workshop come from people who would not consider themselves “statisticians,” but use statistics in their everyday life enough to understand its role and importance. The author(s) argues that
  21. 21. 21 statistics can achieve the most success when it is applied to subjects outside of its origins(7). Statistics is meant to model situations that happen in everyday life, in all fields such as the ones listed above. The contextual knowledge comes from all of these disciplines, but the statistical tools used to solve these problems are all the same. Some ways statisticians attack really large problems is by clustering, which is splitting the problem up into smaller, more manageable groups. In this case, the way the data is split up relies on the subject matter knowledge of the person running the experiment or analysis, and the statistician’s role is to use this knowledge to make his job more manageable. One example, statistics used in biological data, shows that different kinds of data are combined, resulting in large sets of often times confusing information. What makes this type of problem possible to solve is the role of the scientist who has background knowledge on the problem at hand (8-9). Without that, the statistician can only make assumptions and guesses, and if he or she is not correct, then his outcome means nothing. Statisticians also often need the help of a computational expert, and the teamwork among these three groups of people is what makes large problems solvable. The article concludes by addressing the fact that most fields deal with large and complex data sets regularly, whereas they were not as major before the invention of computers and the type of computing we do today. We also have to be aware of the effect computers have on randomness in our experiments and data analysis. Some of the other major questions and concerns brought up in the article are: how do we decide which variables are important or not important? What role does traditional data analysis play in
  22. 22. 22 analysis or large data sets? How can we systematically analyze the outputs of modern statistical methods(20-1)? Input from Science Another field that applies problem-solving techniques in new and interesting ways is Biology or Biochemistry. Brian Cohen, Lecturer of Biology and Biochemistry at Union College, explains that in general, biologists have more data than they even know what to do with (for example, DNA or brain activity data). Thus, he proposes trying to break the problem down into the smallest possible pieces, if this is possible, i.e. “working from the bottom up.” He suggests that this approach is more beneficial and efficient than trying to take all of the available information and developing many pathways to the “solution,” and then trying to test them all. Professor Cohen believes that one of the most important steps is defining our target precisely, and then thinking about the ramifications of making the changes we need. Professor Cohen suggested that I look into systems biology, which is an emerging approach to solving complex problems in biology, particularly biomedicine, which focuses on interactions using a holistic approach. This approach is generally in opposition to the scientific method, as it emphasizes the holistic approach as opposed to breaking the problem down into manageable parts and solving them individually in a systematic way. The student research that Professor Cohen advises is usually fairly well defined, and the students generally each work on a little piece of the bigger problem that he is trying to solve. Thus, the problem solving process in this case is defined but still requires the creativity to come up with various ways of solving it, since the “answer” is not always straightforward.
  23. 23. 23 Another method for solving problems in any scientific field is the scientific method, a process used in many analytical scientific disciplines to help acquire knowledge or investigate phenomena. This method for problem solving was first seen in Parmenides’s writings, where he uses deduction to solve problems. Later, the atomists Leucippus and Democritus built upon this, and Aristotle formalized the scientific method. The whole premise of the scientific method is to build upon previous advancements in science by applying what we already know in order to learn something new. Importantly, not all of the steps of the scientific method will always be used, and they don’t necessarily need to be used in the order presented below (Edmund). The scientific method consists of five steps, each of which requires significant thought and creativity. The first step is formulating your question. As we have seen numerous other times, this step generally seems trivial, but usually ends up being one of the most important steps in the problem solving process. Sometimes, our questions are straightforward (i.e. how can I bake a moist and delicious cake). Other times, the questions are open-ended, and this is the case we are more interested in studying. This stage also requires some research into similar studies or experiments done in the past. The next step in the scientific method is creating a hypothesis. A hypothesis is a conjecture about what we think will or will not happen. It is important that we set up our hypotheses in a way such that we can draw meaningful conclusions from them. In statistics, this means that our null hypothesis must be some statement of equality, which is usually what we believe to be false. Thus, if we conclude that our null hypothesis fails, we can accept our alternate hypothesis, which is what we believed would be true based
  24. 24. 24 on our initial research and/or subject matter knowledge. Our hypotheses must also be mutually exclusive and exhaustive of all the possible outcomes of our question (Harris). Thirdly we make more predictions after we disprove or fail to disprove our hypothesis. We want to continuously test our predictions until we have concluded something worthwhile (or failed to do so…). Once we predict our outcome, we test it. This is the part of the process where we see if what we predicted would be true is actually what happens naturally. Our last step is analysis. Once we conduct our experiment or test, we need to analyze the outcome and determine what our next steps should be. In general, this process cannot be used only once to solve a problem, but rather will be needed in an iterative way until we finally reach some sort of a conclusion. If we realize our hypotheses were wrong, we should make new ones. If we notice flaws in our experiments, we should design new ones. It is also important to note that whatever conclusions we draw about our sample cannot be applied to anything outside of this sample (Harris). We can use these results to help guide us but should not simple apply our results to our target population if this is not the sample we originally tested. These steps should be considered suggestions, which can help us in the problem solving process, rather than strict rules that we must follow exactly. Wolkenhauer et al. describe the role of systems biology in solving problems, specifically in the field of medicine, in their article titled “The Road from Systems Biology to Systems Medicine.” This article emphasizes the ways in which we can use interdisciplinary techniques to solve biomedical problems, specifically in the context of big data analytics where we are often trying to dig deeper into massive data sets. The article claims that an interdisciplinary approach is necessary to “face the challenges of
  25. 25. 25 integrating basic research and clinical practice” (502). Further, the article states: “advances in measurement technologies can generate large-scale, multi level but also heterogeneous datasets, requiring not only new computational platforms to manage data but most importantly, requiring new ways of thinking, including the application and development of methodologies from the mathematical sciences” (502). The importance of systems biology is the multidisciplinary aspect, as it combines experimental work with mathematical and computational analyses. One of the motivations for studying systems biology was the time-sensitive nature of the problems: cell processes happen quickly and standard statistical and bioinformatics approaches do not work under these circumstances (503). Further, often times many varying data sets are combined, which makes it more difficult to apply standard techniques and necessary to apply new techniques that can handle heterogeneous data sets. This article also mentions the fact that mathematical modeling has not reached its full potential within the field of biomedical research, but one of the positive aspects of mathematical modeling is the fact that it can be done outside of the experiment being conducted (504). There is a lot of emphasis placed on the relationship between the “modeler” and the “experimenter,” as they must work together and utilize interdisciplinary tools to solve these kinds of problems. Recently, there has been a focus on “data-driven modeling,” and then “model driven experimentation,” which is a key feature of big data analytics (504). Thus, not only is the actual experiment important, but equally important is the step that comes before it, where data is analyzed without performing actual experiments, to gather background knowledge on the problem at hand.
  26. 26. 26 One of the common themes of the article is the need to have an interdisciplinary approach to solving large and complex problems in biomedicine, as described: “many scientific questions require a range of expertise from different fields. Therefore, integration across disciplinary boundaries is crucial. The expertise for particular technologies and experimental systems is rarely found in a single laboratory, institute, or country, and this raises the need for standards and ontologies that support the sharing and integration of data and models” (505). Input from the Problem Solving Discipline Graham Wilson, author of “Decision-Making and Problem Solving,” focuses on decision making and problem solving in emergency situations. Thus, there is emphasis on determining the problem (and how this is different from the symptoms of the problem), and knowing how to react in a timely and appropriate manner. The book also has sections that use personality traits to help determine what kind of decision maker you are and how this affects the way we should proceed in these kinds of situations. The introduction chapter highlights the fact that making poor decisions during the early stages of a crisis or event can create more problems later on and make the decision-maker’s job more difficult. The six units covered in this text are: introduction, the decision-making process, identifying decision-making styles and attributes, ethical decision making and problem solving, decision making in an emergency, and a course summary. The second chapter focuses on the decision-making process. The objectives of this chapter are to acknowledge the importance of making decisions before a crisis happens as opposed to after, defining the steps of the decision-making process, and distinguishing the causes from the symptoms of a real-life case study. The first thing
  27. 27. 27 discussed is the difference between problem solving and decision-making, where problem solving is “designed to analyze a situation systematically and generate, implement, and evaluate solutions” (2-2). On the other hand, decision making is “a mechanism for making choices at each step of the problem-solving process,” and it a necessary part of the problem solving process (2-2). The five steps that are suggested to solve problems in this book are: identifying the problem, exploring alternatives, selecting an alternative, implementing the solution, and evaluating the solution. Wilson emphasizes the importance of considering all of these steps, while it may not always be necessary to write down or otherwise document them all. Wilson breaks down each step into smaller pieces to make it more clear and easy to understand. Wilson acknowledges, as we have seen many times before, that problem identification is often the most important and most difficult step in the decision making process. Every step we take after defining the problem will be based on this step. Wilson defines a problem as “a situation of condition of people or the organization that will exist and is considered undesirable” (2-6). Further, many people confuse the problem for its solutions, by stating the problem in terms of its solution rather than the actual problem at hand. Another important piece of this step is identifying the parameters of the problem, which usually involves answering questions like, “what is happening?” “who is involved?” or “what are the stakes?” (2-7). When considering complex and unstructured problems, we also have to determine the best metric to define all of these parameters, which can be surprisingly tricky. The second step in the problem solving process is exploring alternatives. This step actually consists of two parts: exploring alternatives, and evaluating alternatives (2-11).
  28. 28. 28 Wilson suggests three methods for coming up with alternatives, which include brainstorming, surveys, and discussion groups. Brainstorming consists of thinking out loud either individually or in a group, while a survey is meant to poll a large number of respondents. Discussion groups consist of those people who are directly involved in the problem solving process and is similar to the brainstorming process but is more formal. Once the alternatives are created, there must be some way to test them initially in order to pick the “best” one to pursue. In order to evaluate alternatives, the manual suggests 6 approaches to consider: identifying the constraints, determining the appropriateness of the solution, verifying the adequacy of the solution, evaluating the effectiveness, evaluating the efficiency, and determining the side effects of the solution (2-13). One other aspect of this part of the process, which hasn’t been mentioned in other literature, is identifying contingencies, or trying to predict what might go wrong (2-12). Next, we select an alternative. Whichever alternative we decided was the “best” in the last step becomes the subject of our main attention during this step. There are many factors that should be considered when selecting an alternative, such as political, safety, financial, environmental, and ethical factors (2-15). Also, sometimes it is difficult to implement this alternative once it is chosen, as there are often unforeseen repercussions or consequences. This manual provides a worksheet that can help us determine the best solution by listing the solution and its limiting factors, and considering combining solutions if one does not stand out as a clear winner (2-16). After we pick the best alternative, we must implement it, which generally consists of 5 steps: developing an action plan, determining objectives, identifying needed resources, building a plan, and finally implementing this plan. Developing the action plan entails creating a series of
  29. 29. 29 steps to follow, as well as who will be responsible for each step. Then, we determine objectives, which are “measurable targets that we use to monitor progress and establish priorities” (2-17).We identify the resources we will need, considering cost, time, and any special requirements. Then, we build our plan, which should state “who will do what by when, where, and how,” (2-17). Finally, we put this plan into action. The manual includes a checklist that can aid us in following through with the implementation and action plan process. The last step is evaluating the solution, which involves two parts: monitoring the progress, and evaluating the results. To monitor the progress, we must ask whether the situation has changed, are other resources required, and are there other alternative solutions required (2-21)? This step is an ongoing process that does not end at the end of our project, but must be continued to ensure our situation doesn’t change (2-21). Chapter 4 focuses on the role of ethics in the decision making process, specifically in emergency situations, and attempts to identify potential ethical issues that may come up and ways which we can apply the problem-solving model to these ethical issues. Ethics is defined as a “set of standards that guides our behavior, both as individuals and as members of organizations; principles of right and wrong, such as being honest, fair, and treating others with respect (4-2). It is important to note that ethics do not imply legality. This manual offers a list of ethical “don’ts” that specifically apply to emergency situations: don’t exceed your authority or make promises, and don’t use your position to seek personal gain - it even goes as far as to make sure you not only act ethically, but are aware of the appearance of being unethical as well (4-3). In addition, some ethical “do’s” are listed: do place the law and ethical principles above personal
  30. 30. 30 gain, do act impartially, so protect and conserve agency property, and do put forth honest effort (4-4). Next, the manual outlines four components of ethical decision-making: commitment, consciousness, and competency. Commitment entails “demonstrating a strong desire to act ethically and to do the right thing, especially when ethics impose financial, social, or psychological costs” (4-6). Consciousness involves “seeing and understanding the ethical implications of our behavior and applying our ethical values to our daily lives.” Finally, competency can be broken down into three parts: evaluation creativity, and prediction. Evaluation is the ability to “collect and evaluate relevant facts and [to know] when to stop collecting facts,” creativity is the “capacity to develop resourceful means of accomplishing goals in ways that avoid or minimize ethical problems,” and prediction means the “ability to foresee the potential consequences of conduct and assess the likelihood or risk that persons will be helped or harmed by an act” (4-7). The fifth unit focuses specifically on decision making in an emergency, which changes the way we go about making decisions. Some impediments people face in these situations are due to stress, such as time pressure, fatigue, lack of information, and uncertainty. Often, people who must make decisions under stress experience conflict with other key players, perceive selectively because of sensory overload, or experience perception distortion and poor judgment (5-2). In addition, these people are usually less tolerant of ambiguity and are susceptible to making premature decisions, have a decreased ability to handle difficult tasks or work effectively, and experience a greater tendency toward aggression and escape behaviors (5-2).
  31. 31. 31 The last chapter summarizes the key concepts from the rest of the manual. It defines problem solving versus decision-making, and outlines a five-step problem solving model: identify the problem, explore alternatives, select an alternative, implement the solution, and evaluate the solution (6-2). Next, it lists the factors that affect decision making, which are political,, safety, financial, environmental, and ethical (6-3). The manual also lists four decision-making styles, based on the MBTI test: sensing, intuition, thinking, and feeling (6-4). The four ways to make a decision as outlined by Wilson are individual, consultation, group, and delegation (6-5). The manual lists many attributes of a good decision maker: knowledge, initiative, currency, flexibility, self-knowledge, calculated risk-taking, and good judgment are just a few (6-6). Next, some ethical “do’s” and “don’ts” are listed: don’t exceed your authority, or use your position to seek personal gain, but do place the law above personal gain and act impartially, while putting forth an honest effort (6-8). Finally, the three components of ethical decisions are commitment or motivation, consciousness or awareness, and competency or skill (6-9). In their article, titled “The Process of Solving Complex Problems,” Fischer et al. discusses Complex Problem Solving (CPS) and how it applies to various fields. Five key points of this process are highlighted: information generation, information reduction, model building, dynamic decision-making, and evaluation. The article’s first sentence does a good job of summing up the motivation behind this research: “In times of increasing globalization and technological advances, many problems humans have to face in everyday life are quite complex, involving multiple goals as well as many possible actions that could be considered, each associated with several different and uncertain consequences, in environments that may change dynamically and independent of the
  32. 32. 32 problem solvers’ actions” (20). Thus, the aim of the article is to come up with a systematic way of solving complex problems that can be applied in any field. The article tries to more clearly define exactly what is meant by “complex problem solving” by describing some of the important characteristics of such problems. For instance, a problem is defined as complex when it has many elements or parts. A problem exists, as described by the article, when we have goals but do not know how to reach them(22). Thus, problem solving is trying to find the means of achieving our goals. One of the most important aspects of problem solving in any field is that the researcher should have knowledge on the tools needed to actually solve the problem (i.e. applying equations, models, etc.), but also should have subject matter knowledge (or should have an expert nearby). The article continues, outlining the most important aspects of complex problem solving (CPS), the first of which is human problem solving (23). The most important aspects of this are creating an internal representation of the problem space. Additionally, we must try to formulate a method for achieving our goal, given our internal representation of it. This process is iterative and often requires us to go back to the beginning if our methods do not work out. Frequently, the problem solver must define parts of the problem him or herself, which is consistent with what we have to do to solve unstructured problems. Another aspect of CPS is expertise. Having expertise in a given field greatly reduces the amount of things we must filter through to find the most important things. It allows us to make educated decisions (even guesses sometimes) and helps guide our technical analyses. In general, experts can solve problems faster than novices in their
  33. 33. 33 given fields (25-8). Next, the article outlines decision-making strategies. The article concluded that the important part of this discussion is not which one strategy is best, but how to discern when to use which strategy depending on the circumstances. Problems solvers base their decisions on the known solutions if they are available. If not, they focus on solving the problem at hand given the circumstances, or trying to find new information that would help them solve the problem. Information reduction is also important in this process because it can simplify a problem that at first seems really complex and large. The article acknowledges that a lot of research needs to be done in the field of complex problem solving. However, the techniques listed above provide a good start (37). Similarly, TRIZ, also commonly referred to as the “theory of inventive problem solving/TIPS,” is a problem solving technique, which was developed by Soviet inventor Genrich Altshuller and his colleagues around 1946. The basis of this theory is that we can use already established principles and insights from other fields that many not necessarily be related to what we are studying, but if they are similar enough we can apply these techniques to help us solve our new problems. Specifically, TRIZ is often used where we have to somehow avoid an inherent conflict within our problem, such as creating a high- power engine that is also light. These two qualifications seemingly contradict themselves - the most high-power engine we could make is probably really large and heavy, while the lightest engine we could make is not high-powered (Technical Innovation Center). The idea behind TRIZ is to take our problem, find a general problem that has been solved that can give us insight into our own problem, and apply the solution to this general problem somehow to our specific problem. Technical contradictions are solved
  34. 34. 34 using 39 elimination principles, while physical contradictions are solved using 4 basic principles, which study supersystems, subsystems, and separation of time and space. All of this information was put together into a matrix, which attempts to systematically approach the way we attack problems (Technical Innovation Center). Thus, we can use TRIZ to solve large, complex unstructured problems because we can use previous results from similar problems and apply them in new and creative ways. III. Key Themes from Literature I have found many recurring themes that appear throughout the literature for large, complex, unstructured problems. I have organized them into two macro-themes – the first contains six key phases that, regardless of the problem solving method used, a researcher must pass through and consciously consider. I believe that these phases cannot be skipped and must be taken into consideration regardless of the type of problem. The second macro-theme contains the key considerations that a researcher must take into account during all phases of the problem solving process. Thus, while there is no “best” or “correct” method of problem solving, these key themes will always be encountered and must always be taken into consideration. Within each macro-theme, I have identified additional sub themes. In macro- theme one, I suggest that there are six phases that a problem solver will necessarily encounter (in order): identifying the motivation for solving the problem at hand, problem definition, understanding the context, strategizing, brainstorming, testing, and evaluating alternatives, and maintaining the solution. Within macro-theme two, I distinguish three important things that all researchers much consider throughout the problem solving
  35. 35. 35 process: gathering subject-matter knowledge, assembling a team of diverse perspectives, and using diverse methods. I. While there is no “best” method that can be applied to every problem, there are certain phases that appear in any problem solving process that cannot be skipped or ignored. 1. Identifying the motivation for solving the problem at hand. The first step of solving any problem, which cannot be skipped or ignored, is understanding the need to solve the problem at hand. As Fischer et al. explain, “in times of increasing globalization and technological advances, many problems humans have to face in everyday life are quite complex, Figure 1: Outlining the six key phases of the problem solving process, and the three key considerations that should be applied during all six phases
  36. 36. 36 involving multiple goals as well as many possible actions that could be considered, each associated with several different and uncertain consequences, in environments that may change dynamically and independent of the problem solvers’ actions” (20). Clearly, whether we realize it or not, we are faced with large, complex, and unstructured problems in our daily lives. Whether it is in our professional lives, our social lives, or our personal lives, these problems arise constantly. However, there are some problems that need to be solved immediately, and some that are not a high priority for us to solve. Further, we must identify why this is a problem worth or not worth solving, by considering the consequences of solving or not solving any given problem. For instance, we might not have enough time to solve a given problem, thus it is not worth solving. Or, we might only be able to slightly make a situation better, like in the case of quality management for example. If we can only reduce mistakes in a certain piece of machinery by such a negligible amount that it wouldn’t make a noticeable difference, perhaps it is not worth spending the time, money, and manpower to fix the problem. In other words, given the many challenges of modern life, we must “choose our battles” carefully. 2. Problem Definition We cannot attempt to solve a problem that has yet to be defined. As in the US Healthcare system example, we cannot begin to try to come up with a
  37. 37. 37 “better” healthcare model if we do not know exactly what problem we are trying to solve. The first aspect of this stage is defining the actual problem statement. As Hoerl and Snee state in “One Size Does Not Fit All,” this step is often the hardest step especially in the case of large, complex, and unstructured problems. In general, most people are comfortable solving problems that are well defined, such as textbook problems in a college course, because once the problem is defined they can easily identify the appropriate methods to use. However, in the case of unstructured problems, we have to define the problem before we can continue the problem solving process. Wilson describes how often people fail to clearly define the problem, and sometimes confuse the problem for its solutions, which we should always avoid (2-6). Another important aspect of this phase is defining the goals of our research and problem solving process. In the case of the US Healthcare system, do we want to cover the largest number of Americans? Increase life expectancy? Or achieve some other goal? We must consider the multiple stakeholders and ensure our problem is defined in terms of everybody’s interests. Similarly, we must make sure that we are determining the necessary resources we will need to solve the problem, so that we can define the problem and our goals keeping this in mind. As explained by
  38. 38. 38 the IESE, if we define the problem in such a way that it becomes impossible to solve, then our efforts will be futile. 3. Understanding the context One important aspect of a large, complex, unstructured problem is that its solution often requires deep understanding of its context. Big problems have defied solution for a reason; to address them we need to understand how they got there, and what sustains them. For instance, in the case of the US Healthcare system, it is important to consider the economic status of the US, the political situation, and the social condition of the country, as well as the history of healthcare in the US. This could significantly enhance the way we approach the problem and the solutions we come up with. In addition, we must know what has already been attempted or solved in relation to our problem. For instance, using the methodology of TRIZ, we should look for any information or advancements that relates to our project, even if it does not directly answer the question we have posed. Using TRIZ, we could apply similar techniques to a portion of our project that is similar to a problem that has already been solved. In general, we can apply strategies from various fields and problems that have been solved in the past that relate to our field, regardless of whether or not it directly answers our question. Any advancements we make simply by researching what has been done already will help us achieve our results. 4. Strategizing
  39. 39. 39 First, we must acknowledge the need for a strategy. Fischer et al. point out that in order to create a strategy, we must first create the problem space that we will be working within, which is defined as, “a set of possible states of the problem, given the initial state, the applicable operators, and certain goal states” (23). In small, structured, or simple problems, i.e. textbook examples, this step is often assumed and not emphasized. However, especially for complex problems, this step is crucial, as we cannot continue with the problem solving process unless we systematically classify the problem. Frequently, we realize that the current format of the problem and the problem space do not promote our creativity and ability to solve the problem, so we must go back to the problem definition and restructure the problem space. It is in this way that we utilize a sequential approach to problem solving. Once we lay out the problem space, we can try to break up the problem into smaller, sequential “steps” that make resolving the problem easier. For example, in “Tried and True,” Hoerl and Snee argue that we can use three steps in a sequential approach to solve experimental design problems: screening, characterization, and optimization. During the screening phase, our goal is to identify the most important variables, so that in the characterization phase we can measure linear effects and interactions. Finally, in the optimization phase, we make predictive models from the data analysis in the first two steps. Approaching a problem this way can also be helpful on a larger scale: first we want to
  40. 40. 40 narrow down the factors we are studying, and then measure the effects they have on the given question. Finally, we want to create some sort of final model or solution based on this process. Importantly, we cannot just use one step of this approach to solve a large, complex, unstructured problem, and they must be used in this arrangement to achieve the desired results. 5. Brainstorming, testing, and evaluating alternatives One crucial step in the problem solving process is defining what our criteria will be for determining how “good” our solution will be. According to the IESE’s six-step problem solving process, “the key to successfully solving a problem is choosing which if all of the possibly important issues will really be the most critical in making a decision” (2). In our example of the US Healthcare system, this means determining which criteria will be used to judge the “goodness” of our healthcare model – will we be most interested in reducing citizen’s out-of-pocket payments, decreasing overall fatalities, least amount of government spending, or some combination of these and many other criteria? We must carefully consider this step before we move on, as it impacts the way we judge the “goodness” of our solutions, which ultimately is the end result of our problem solving process. The next logical step in this process is identifying alternatives. The IESE states that this step is more difficult in a large, complex, unstructured problem, but it is critical nonetheless (3). This step is rarely mentioned in
  41. 41. 41 the quality management and statistics literature, but is mentioned in the engineering, biomedical, and psychological literature. I also think it is important to note, as the IESE article explains, researchers must choose the criteria before selecting their alternatives, as bias is introduced when this process works in reverse (3). We want to avoid bias at every stage of the problem solving process, and it is a quality found in human nature that we will bias our criteria if we already have the alternatives laid out. Wilson suggests various methods we can use to produce alternatives – brainstorming, surveying, and utilizing discussion groups – each of which having its benefits and unfavorable effects (2-13). Another important aspect of this phase is the role of creativity. We cannot teach people how to be creative, yet it is crucial that the people who are working on the problem at hand are creative enough to think outside of the box when it comes to alternative development. If we can only come up with solutions that have been used in the past or that are similar to things we have seen or done before, we are severely restricting our ability to solve a large, complex, unstructured problem. Once we pick our alternatives, we must test them to determine which one or ones are the most likely to be successful, by using our criteria. This is where researchers have the most freedom to use whatever tools and methods are appropriate for their particular project, but this step can also be confusing for those who are studying large, complex unstructured problems because the correct method might not be apparent.
  42. 42. 42 We continue to narrow down our alternatives by analyzing them and comparing them to the criteria we selected until we are happy with our results and our final solution. Wilson suggests various ways of determining which alternate solution is “best,” in addition to measuring the criteria we set forth. The six things he recommends considering are: identifying the constraints, determining the appropriateness of the solution, verifying the adequacy of the solution, evaluating the effectiveness, evaluating the efficiency, and determining the side effects of the solution (2-13). In addition to our criteria, these guidelines can be a good way for us to determine which alternative is the best and are a good place to start especially in the case of large, complex, unstructured problems. One last step we should always consider during this phase is whether or not it would be feasible to actually implement our solution. Wilson constructs a five-step method for implementing a solution once we have picked the best alternative: developing an action plan, determining objectives, identifying needed resources, building a plan, and finally implementing this plan (2-17). We should always consider these steps, in order, to ensure that since we have gone through the trouble of solving our problem, we can successfully implement it. 6. Maintaining the solution I believe that ensuring we can maintain whatever solution we decide to implement is just as important as being able to come up with the solution
  43. 43. 43 in the first place. Hoerl and Snee, Wilson, and the IESE all emphasize the importance of sustaining our results, stating that we often need to come back to our problem and reevaluate the solution to ensure it is still successful after some time has passed and the problem has been closed. Similar to the problem solving process as a whole, sustaining the successful result is usually iterative and requires constant maintenance, but is necessary to guarantee that all of the hard work it took to create the solution was not in vain. II. As we saw in Figure 1, the following key considerations should be kept in mind during all of the stages of the problem solving process. 1. Gathering subject matter knowledge There are many times during the problem solving process in which it would be beneficial to have subject matter knowledge. For instance, when defining the problem and the criteria, it would be helpful to have a subject matter expert there to advise the researcher on how to appropriately define the problem. For example, in the US Healthcare problem, it would be appropriate to gather doctors, insurance companies, and government officials to help the researcher understand exactly how the current healthcare system works, so that he or she can correctly define the problem. The NSF workshop mirrors this idea, highlighting the role of the subject matter expert. Without the expert’s advise, a statistician cannot make informed decisions. Specifically in the case of large, complex, unstructured problems, data is often combined from multiple sources and
  44. 44. 44 it is up to the subject matter expert to make sense of it all so the statistician or researcher can perform his or her analysis. Fischer et al. highlight another important aspect of consulting experts during the problem solving process: it allows us to filter through large amounts of information more quickly with less wasted time (26). They explain that problems become easier to solve when we have a wealth of subject matter knowledge, and something that seems complex may actually be simple when we consult an expert. Further, consulting experts allows us to make educated guesses instead of random guesses, and can usually help guide our analysis. Before we even analyze any data, a subject matter expert can help us make decisions that will directly impact our analysis. Such expertise is critical at each subsequent phase of the problem solving process. 2. Assembling a team of diverse perspectives A significant amount of time should be spent constructing a team of diverse individuals who will all bring something meaningful to the problem solving process. Diversity can refer to different personalities as well as disciplines and levels of experience. As the IESE describes, there are two types of people – those who see the world in terms of numbers and those who see the world in terms of emotions – and both types of people should be involved in the problem solving process together (4-5). Further, consideration should be given towards who should be making the decisions and solving the problem.
  45. 45. 45 Wilson presents four categories of decision-making: individual, consultation, group, and delegation (3-6). Based on the context of the problem, an appropriate method should be used, and this step should not be overlooked. We should take extra care to ensure we avoid the “groupthink,” where the desire to conform to the group drives people to make biased decisions. Groupthink can be extremely detrimental to the problem solving process, as creative solutions can be stifled by leaders who are not open to new ideas. Leaders must be careful to avoid only working with people who think like them – something that unfortunately happens all too often without leaders and bosses even being aware of it. By intentionally picking individuals who bring different perspectives to the process, problem solvers can increase the chances of coming up with a successful result. 3. Using diverse methods I believe that one of the most important themes in the literature has been that regardless of the problem solving technique, researchers must be willing to be creative and use methods and techniques that are not familiar to them. Sticking to what is comfortable and familiar will not foster “good” problem solving. Rather, it prevents us from solving problems, specifically large, complex, unstructured problems. In addition, problem solvers must understand that only rarely will one single method or technique solve any problem, regardless of the size, complexity, or structure or lack thereof. So, it is important to keep a large
  46. 46. 46 toolbox of methods and techniques available to use whenever necessary, as well as the ability to be flexible and creative. A combination of technique and creativity is important in problem solving, and no one method or technique, when used alone, is universally “best,” which is controversial to much of the literature that attempts to compare various methods and pick the “best” one. IV. Recommendations for Problem Solvers After compiling as many of the problem solving techniques from various disciplines, I have created my own list of recommendations that I would suggest to people trying to solve large, complex, unstructured problems. This list is not exhaustive, and is meant to be supplemental and additive to the key themes I outlined in the previous section. My first recommendation for problem solvers is also the most important in my opinion – it is crucial to understand that while I have provided a list of phases and techniques that should be applied, this methodology is not meant in any way to be the “correct” or “only” way of solving a problem. It is meant, however, to be a good starting point for people who do not know what the steps are for solving a large, complex, unstructured problem. One issue too often seen in the problem solving literature is that it spends too much time defending different methods and techniques as the “best” ones. However, I believe that this is a waste of time, as no single method can solve every problem. This brings back the role of statistical engineering, where researchers and problem solvers should focus on utilizing tools they have in inventive and creative ways,
  47. 47. 47 rather than attempting to figure out which tool or tools are the “best” and work for every problem. In addition, I recommend that those responsible for addressing big problems spend a significant amount of time gathering the right group of people to assist them during the problem solving process. This step is often overlooked because natural “teams” can form through existing relationships and problem solving teams in the past. However, I advise that each problem be thought about individually, so that various personalities and experts can be assembled to form a cohesive and productive problem solving team. As I mentioned earlier, the phenomena of “groupthink” should be avoided at all costs, so that the same mistakes are not made consistently and new approaches can be utilized. No one person, regardless of their experience or expertise, can solve every problem, and it is always beneficial to have diverse people on the problem solving team. This is especially true for large, complex, unstructured problems, where there is often a diverse amount of information necessary to make informed decisions, and it would be inefficient to try to learn all of the required background information. In particular, we need to include those with subject matter knowledge, those with data analysis skills, and those who can creatively apply the tools and techniques they know. Another important recommendation is being comfortable with using tools and techniques that you are not familiar with. It is easy to rely on the tools that you know and have worked in the past, but when dealing with large, complex, unstructured problems, it is important to look outside of your “toolbox” when thinking of ways to solve the problem at hand. This is another reason why it is so important to gather a diverse team, each of whom brings a different viewpoint and skill set to the table. Being able to adapt
  48. 48. 48 and adjust your skill set and use unfamiliar tools will make you more likely to succeed. Having a diverse toolkit will enable problem solvers to select the method based on the unique nature of the problem, as opposed to tailoring the problem to fit within a certain framework so they can use the tools they are familiar with. In other words, it will help problem solvers avoid viewing every problem as a nail because the only tool they have is a hammer. Often, this will require researchers to read diverse literature outside of their own fields. I suggest reading the manual by Graham Wilson, referenced several times in this thesis (“Problem Solving and Decision Making”). While this manual is meant to help students teach themselves the methods of problem solving, it focuses on emergency situations. It may not seem relevant to the process of solving large, complex, unstructured problems, but it actually does a great job of breaking down this process into its most basic definitions (i.e. what do we define as a “problem”?), and then providing various steps and methods for working our way through the problem solving process. If you are looking for a comprehensive guide to the basics of problem solving, especially in an emergency situation, this is a great place to start, and it provides worksheets that a problem solver could easily fill out with the necessary information to be even simpler to use and understand. Another recommendation is to give yourself more time than you think you might need to solve the problem. Problem solvers should take the time to carefully define the problem, as this is the “make or break” aspect of the problem solving process. Additionally, researchers and problem solvers could benefit from having time to create alternatives and analyze them, and then have more time to contemplate these solutions
  49. 49. 49 and reflect on what they have accomplished. As mentioned in the article on the “Aha moment,” you cannot force or expedite these moments of clarity; often just having more time to think about the problem will prove beneficial (Topolinski and Reber). Going along with this, do not get frustrated if your first few solutions do not work out. If problem solvers could solve any problem in a few simple steps, large, complex, unstructured problems would not even exist. So, do not let a temporary failure or minor setback cause you give up on the problem solving process as a whole. One final recommendation I have for problem solvers is to make sure that once you have a solution, you can successfully implement and sustain the results. Our efforts would be futile if we spent our resources on solving a problem, and could not implement and sustain the results. This implies that throughout the problem solving process, including deciding which problems to work on, we consciously consider the long term view of sustainability. V. Conclusion In this thesis, I researched available literature and interviewed experts about the problem solving process in various disciplines, such as psychology, economics, biology, engineering, quality management and statistics, and the problem solving discipline. I organized the wealth of information into two macro-themes: six key phases that a problem solver will necessarily encounter during the problem solving process, and key considerations that must be consciously applied during all stages of the problem solving process. The six phases of the problem solving that I have identified are: identifying the motivation for solving the problem at hand, problem definition, understanding the
  50. 50. 50 context, strategizing, brainstorming, testing, and evaluating alternatives, and maintaining the solution. Within each of these phases there are other important things to consider. These steps are not meant to be definitive and I do not claim that this is the “best” process of problem solving. Rather, I believe that these steps are necessary regardless of the kind of problem, and that problem solvers should seriously reflect on all of them. The three key considerations that I believe should be employed during all stages of the problem solving process are: gathering subject matter knowledge, assembling a team of diverse perspectives, and using diverse methods. These themes should not be overlooked, and researchers should make a conscious effort to utilize them during every stage of the problem solving process. In addition to the key themes above, I have also come up with a list of recommendations to problem solvers, especially when attempting to solve large, complex, unstructured problems. The first one, which I believe to be the most important, is to take everything I have gathered and organized lightly. I do not intend this to be a comprehensive method for solving large, complex, unstructured problems. Rather, I suggest this paper as a good starting place for those who are unsure of how to begin solving a large, complex, unstructured problem. While much of the literature on problem solving in various disciplines spends time debating which methods and techniques are best, I argue that there is no one best method. However, I believe that the phases I have outlined will be present regardless of the type of problem being solved. In addition, problem solvers must be flexible in all aspects of the problem solving process. Whether this means working with people who have a variety of subject matter backgrounds, people with different personalities, or being able to utilize tools and
  51. 51. 51 techniques that they are not comfortable with or used to using, researchers and problem solvers must be able to step out of the box if they want to solve a large, complex, unstructured problem. Finally, researchers should ensure that they have the necessary resources available. These resources come in many forms, for instance time, money, and manpower. Problem solvers should always give themselves more time than they anticipate needing to solve a big problem, to allow them to take the problem definition stage seriously, as well as all of the subsequent stages. Further, researchers should make sure that they have access to all of the necessary resources they will need to not only solve the problem, but also implement and sustain their results. If results cannot be implemented or sustained, then the problem solving process was ineffective. While there are many problem solving techniques presented within various disciplines, my goal was to condense these methods to identify important themes that could help people solve big problems across disciplines. By using the guidelines and consciously considering the key themes I have proposed, I believe we can successfully solve large, complex, unstructured problems.
  52. 52. 52 Works Cited Bruno, Brad. Personal interview. 23 Jan. 2014. Cohen, Brian. Personal interview. 12 Feb. 2014 Davis, Lewis. Personal interview. 27 Jan. 2014. Edmund, Norman W. "The Scientific Method Today." The Scientific Method Today. Web. 16 Mar. 2014. Fischer, Andreas, Samuel Greiff, and Joachim Funke. "The Process of Solving Complex Problems." The Journal of Problem Solving 4.1 (2012): n. pag. Print. Harris, William. "How the Scientific Method Works." HowStuffWorks., 14 Jan. 2008. Web. 16 Mar. 2014. Hoerl, Roger W., and Ronald D. Snee. "Closing the Gap." Quality Progress (2010): 52- 53. Web. Hoerl, Roger W., and Ronald D. Snee. "One Size Does Not Fit All." Quality Progress 46.5 (2013): 48. Web. Hoerl, Roger W., and Ronald D. Snee. "Tried and True." Quality Progress (2010): 58-60. Web. Hoerl, Roger, and Ronald Snee. "Statistical Engineering: Tactics to Deploy Statistical Thinking." Statistical Thinking: Improving Business Performance. 2nd ed. Hoboken, NJ: Wiley & Sons, 2012. 93-138. Print. Keat, Bill. Engineering Design in General Education. Symposium on Engineering and Liberal Education. Web. Keat, Bill. Personal interview. 30 Jan. 2014.
  53. 53. 53 Meng, Xiao-Li, and Joseph Blitzstein. Nano-Project Qualifying Exam Process: An Intensified Dialogue Between Students and Faculty.Department of Statistics, Harvard University. Web. National Science Foundation. Discovery in Complex or Massive Datasets: Common Statistical Themes. Proc. of Mathematical Association of America (MAA), Washington, DC. 2007. Print. Rosenberg, Mike. "IESE’s Six Step Process for Resolving Unstructured Problems." IESE Business School. Web. "Technical Innovation Center, Inc." Technical Innovation Center, Inc., Oct. 2012. Web. 16 Feb. 2014. Topolinski, S., and R. Reber. "Gaining Insight Into the "Aha" Experience." Current Directions in Psychological Science 19.6 (2010): 402-05. Print. Wells, Erika. Personal interview. 29 Jan. 2014. Wilson, Graham. Decision Making and Problem Solving: Student Manual. 2002. Print. Emergency Management Institute. Wolkenhauer, Olaf, Charles Auffray, Robert Jaster, Gustav Steinhoff, and Olaf Dammann. "The Road from Systems Biology to Systems Medicine." Pediatric Research 73.4-2 (2013): 502-07. Print.