Ask how many people have created risk matrices for their programs? It better be everyone in the room…
Just to give a brief history of early spaceflight, put next the subsequent slides in context, in terms of where technology stood at the time.
This is just an example of the type of program that was going on a the time.Solar Panels – In 1839 Alexandre Edmond Becquerel discovered the photovoltaic effect 1941, RusselOhi invented the solar cell, shortly after the invention of the transistor July 10, 1962 – Telstar launches as the first solar powered satellite, a result of an agreement between AT&T, Bell Telephone Laboratories, NASA, the British General Post Office and the French National PTT (Post, Telegraph, and Telecom)
These days, it takes a year to get through Phase A. Another year to get from Phase A to PDR!!!Now think about what the team building the fleet of MiDAS spacecraft accomplished: Developed a new instrument technology for space Built 12 spacecraft in less than 3.5 yearsThe MIDAS-A Spacecraft weighed 5001 pounds fueled (4780 pounds dry mass) Proved that missile launches could be detected from orbit
Some of this information came from Space News.
I’m not saying that Risk Aversion was the only reason NPOESS failed, but it was certainly a major contributing factor.Where the tri-agency management structure truly failed was that no one was willing to accept any level of risk, because they didn’t want the blame if something went wrong.
This is an example of how Setting Expectations would
NEAR engineers gave three-minute reports andused a simple 12-line schedule. Many so-called “good ideas”were rejected during the design phase because they wouldhave increased the cost, schedule, or complexity of the project.
Held 1,093 US patents in his name, as well as many patents in the United Kingdom, France, and Germany.
“Wind tunnels capture valuable, relevant hypersonic data and can operate for relatively long durations up to around Mach 15. To replicate speeds above Mach 15 generally requires special wind tunnels, called impulse tunnels, which provide milliseconds or less of data per run,” Schulz said. “To have captured the equivalent aerodynamic data from flight one at only a scale representation on the ground would have required years, tens of millions of dollars, and several hundred impulse tunnel tests.” According to Schulz, impulse tunnel testing is required to create a portion of Mach 20 relevant physics on the ground.”Essentially, they are taking a commercial company approach, build the darn thing with the best knowledge you have, and see if it works.
Article author’s quote: Probably the force of the hypersonic gale screaming past it as it sped through the air overcame the thrusters attempting to maintain controlled flight, and it spun out and blew apart.When questioned about the failure, Project Manager USAF Maj. Chris Schulz didn’t shake and quiver, he didn’t apologize, or talk about the extensive investigation that will be conducted to root out the problem and eliminate it.
Risk as an Essential Part of Technology R&D Greg FletcherSpace Science and Engineering Division Southwest Research Institute All information contained herein was obtained from open sources published in print and on the web All opinions stated herein are strictly the author’s and not that of any institution or group February 2012
Risk DefinedRisk –Expose (someone or something valued) to danger, harm, or loss: ―he risked his life to save his dog‖.
Early Spaceflight Sputnik was launched by the USSR on October 4, 1957 Ignited the space race, and proved the Soviet Union had perfected the ICBM Identified upper layers of the atmosphere Explorer I was launched by the US Army on January 31, 1958 Demonstrated US ICBM capability Discovered the Van Allen belts (named for James Van Allen, who flew the instrument that made the detection) Space Race was on, and the decade that followed saw an unprecedented revolution in technology
Missile Defense Alarm System (MiDAS) In February of 1959, the US government began a program to put a missile defense warning system in orbit around the Earth (MiDAS) Humans had only begun to put objects in Earth orbit Infrared imaging technology was under development and had never flown in space (was used in the Falcon air to air missile in service starting in 1955) Had to develop automated detection algorithm, because at that time they couldn‘t transmit images to the ground (due to limited RF bandwidth) Infrared Sensor assembly from Battery powered, so they only lasted a MiDAS spacecraft few weeks in orbit
MiDAS In February of 1960 the first MiDAS spacecraft launched First launch just ONE YEAR after the program was initiated!! By July of 1963 (just short of 3.5 years), nine MiDAS spacecraft were in orbit Since they were battery powered each one only lasted three weeks Three had launch failures but they succeeded in proving that it was possible to detect missile launches from Earth orbit Considered a major success at the time Launch failures later spurred an effort to prevent future failures
To raise new questions, new possibilities, to regard old problems from a new angle, requires creative imagination and marks real advance in science.‖ - Albert Einstein
Building Spaceflight Hardware Takes to long! Schedules slip and costs grow Examples: NPOESS MSL (finally launched Nov 2011) JWST SBIRS (Space Based Infrared System) Many other examples available
NPOESS Overview Contract award in 2002 Program cost was $6.1 billion Managed by DoD, NASA and NOAA Expected a risk reduction demonstrator satellite to launch in 2006 First (of six) NPOESS satellites intended for 2009 launch Intended to replace DoD‘s DMSP (Defense Meteorological Satellite Program) and NPAA‘s POES (Polar Operational Environmental Satellites)Credit: Some information came from article: F. G. Kennedy,Space and Risk Analysis Paralysis, AIAA, Nov 2011
NPOESS Overview (cont.) By 2010 the Demonstrator slipped five years to 2011 First spacecraft scheduled in 2014 (and reduced to 4 spacecraft) Costs were overrun to $11 billion (that‘s nearly $5 billion overrun After eight years, we hadn‘t managed to put one demonstrator in earth orbit We put men on the moon in ten years!!!
NPOESS Overview (cont.) White House announced in February 1, 2010 that the NPOESS satellite partnership was to be dissolved Two separate lines of polar orbiting satellites to serve the military and civilian communities would instead be implemented NOAA/NASA portion is called the Joint Satellite System (JPSS) DoD portion is called Defense Weather Satellite System (DWSS)
NPOESS Monday Morning Quarterbacking What went wrong? Blame was placed on the inter-agency management structure Risk aversion hampered progress Processes designed to mitigate risk, hampered progress Tri-agency management structure meant that no one was willing to accept any risk, for fear of being blamed if there were problems later Failure is not an option, means that if you don‘t fly, you can‘t fail
So what happened in between MiDAS and later missions like NPOESS?
Faster, Better, Cheaper Pick any two!However: First 9 out of 10 missions successful Innovative missions that came in on time and under budget Flew 16 missions for less than $3B!!
Faster, Better, Cheaper (cont.) NEAR (Near Earth Asteroid Rendezvous) Estimated at $200M and came in at $122M 27 months of start of funding to launch! Took 10 Times the expected data Not designed as a lander, but coasted to a stop on Eros, the first time this had ever been done Mars Pathfinder First successful rover on another planet 17,000 images 1/15th the cost of Viking 20 years earlier
Faster, Better, Cheaper (cont.)And then in 1999 – 4 out 5 five missions crashed and burned (some literally) Bad press was relentless (maybe rightfully so) Findings indicated that FBC programs that failed had reduced cost and schedule, but not lessoned complexity accordingly PM‘s of successful FBC missions insisted on simplicity both technically and organizationally
Long Term Result of FBC In order to avoid further embarrassment, programs adopted ‗rigorous‘ risk mitigation plans Fear of failure became so great, missions delayed in order to mitigate risk, which then caused overrunsCredit: Lt. Col. Dan Ward, USAF, “Faster, Better, Cheaper Revisited –Program Management Lessons from NASA”, Defense AT&L, March-April 2010
Risk Aversion –The reluctance of a person to accept a bargain with an uncertain payoff, rather than a bargain with a more certain, but possibly lower, expected payoff .
Innovation Innovation – Something new or different introduced (from Dictionary.com) Three keys to innovation Seek out new ideas Test these ideas on a scale where failure is survivable Constantly monitor these trials for feedback Credit: Tim Harford, Adapt – Why Success Always Starts with Failure
“Results? Why, man, I have gotten lotsof results! If I find 10,000 wayssomething wont work, I havent failed. Iam not discouraged, because everywrong attempt discarded is often a stepforward....” -Thomas Edison
DARPA‘s HTV2 DARPA‘s Hypersonic Test Vehicle 2 is designed to launch from the US and land anywhere on the globe in under an hour Re-enters atmosphere at speeds up to Mach 20 (~13,000 mi/hr) withstanding temperatures of 3500 degrees Fahrenheit Quote from HTV-2 Website – ―At that speed air doesn‘t travel around you – you rip it apart‖ Quote before second test flight – ―It‘s time to conduct another flight test to validate our assumptions and gain further insight into extremely high Mach regimes that we cannot fully replicate on the ground.‖
DARPA‘s HTV2 (cont.) CNN Headline – ―Flight failure won‘t stop ‗Mad Scientists‘ Quote from Article – ―The failure is not surprising; permission to fail is what has enabled the agencys spectacular success over its 53-year history‖ Quote from Air Force Maj. Chris Schulz after second catastrophic failure "We do not yet know how to achieve the desired control during the aerodynamic phase of flight. It‘s vexing; I‘m confident there is a solution. We have to find it.‖
DARPA‘s HTV2 (cont.) After the second catastrophic failure, CNN and other news agencies hailed DARPA as bold, forward thinking and visionary, daring to do what others would not!! What would they have said if it was a NASA re-entry vehicle test failure?NASA screws up again!!!(even though it may have been years since a failure of any kind)
Headlines from NASA Missions Popular Science‘s ‗The Top 10 Failed NASA Missions‘ ―In space, no one can hear you screw up‖ DART – ―Fear and loathing in orbit‖ Genesis – Genesis space capsule crashes Spacecraft carrying solar samples slams into Utah desert UARS (Re-entry) – The Sky is Falling (But We Don‘t Know Where)
Setting Expectations DARPA says openly and publicly that not only is failure an option, but it‘s expected and accepted as part pushing the technology envelope Quote ―We learn as much from our failures as we do our successes‖ When NASA says failure is not an option, that‘s what the public expects! There are times when failure is not an option (manned flight) Experimental missions, failure should be an option (though not a goal)
Stigma of Failure Stigma of Failure holds many government agencies back from innovation Internal cultural practices of not sticking your neck out and just waiting out the latest change effort Warranted in many cases, since some agencies cannot fail in their primary mission (defending the nation, or sending social security checks) Failure to innovate is a mission failure for NASA Innovation requires pushing the limits and risking setbacks through failure Yet failing at something even if it‘s risky is viewed a mission failure Tell me again why we do this?!
―We choose to go to the Moon in this decade and do the other things, not because they are easy, but because they are hard, because that goal will serve to organize and measure the best of our energies and skills…‖ -John F. Kennedy
Failure is an Option Tim Harford states in his book that today‘s world is to complex for top-down ―big project‖ innovation based purely on expert judgment Best path to innovation is to try a lot of ideas simultaneously (even if they contradict each other) Build in robust feedback loops Use the winning ideas to start a new round of trials This is not new, in fact it‘s the oldest method of innovation (think evolution) Harford concludes that the organizations that survive the best are ones that make incremental changes, and occasionally take on long-shot ideas
Failure is an Option (cont.) Harford states that this innovation method does not work with government agencies because of several barriers There is not enough time for political appointees to fully see these experiments through before a new administration comes in office Process depends on a large number of failures for innovation but failure carries a high stigma in governmentThis is true, but despite the facts,occasionally the US Government does someinnovative and amazing work
Historically Innovative Government Works Numerous government projects that have been extremely innovative and successful Hoover Dam Rural Electrification Interstate Highway System Moon landings Space Shuttle The Internet
"The things we fear most in organizations—fluctuations, disturbances, imbalances—are the primary sources of creativity.‖ - Margaret J. Wheatley
Risk and Innovation Amount of risk associated with a new technology depends on the type of technology and the magnitude of the leap from what currently exists In research, learning from failure often results in success Acceptable level of risk depends on several factors What is the cost of failure (cost can be monetary or other assets, including humans) What is the return if the risk pays off (break through/game changing technology, knowledge, etc)
Individual Risk Tolerance Risk tolerance varies quite a bit from person to person Generally, individual people are fairly risk tolerant Groups of people tend to be less risk tolerent Organizations become less and less risk tolerant as they grow in size One way Mars mission (from Jan-2011)
Heritage Most missions in the last 10 years have required that components, subsystems and instruments have spaceflight heritage Can‘t fly without heritage Can‘t get heritage without flying Most proposals are considered high risk if there is anything below TRL 7 or 8 (TRL 9 is preferred) Explains why we‘re still flying the 386 processors on new missions
Heritage (Cont) How can we move technology forward if we don‘t fly new hardware? This is one of the major symptoms of an overly risk averse environmentSo what do you do about it?
Awareness of the Problem Hi, I‘m Greg I have a problem with Risk Aversion NASA is aware that excessive risk aversion has hindered innovation Also aware that is has caused cost overruns You can actually find quite a bit written about it on the NASA web sites
Story about a personal experience managing aprogram that was risk averse tothe point of paralysis (if there‘s time)
What is NASA Doing NASA has tasked the Office of the Chief Technologist with fostering innovative ideas Low TRL Game Changing Cross Cutting NAIC Concepts are encouraged to be wild and out there Submit a two page whitepaper Whitepapers are selected for proposals (10 pages) for $100k concept study Concept can be funded to build hardware
Final Thoughts Cubesats and Nanosats can offer a low cost option to fly new technologies Free launches are available as secondary payloads Program costs are low (in many cases less than $200k, depending on how much development is required for hardware and payload) Drawback is the hardware has to be small enough to fit the form factor