A short Report on the failure of the Mars Polar Lander and Deep Space 2 Mission by NASA. The report is from the Software Engineering and Management point of view. The ripple effects of the Design and architectural decisions. The environment under which the project was bid, its effects on organization that was assigned to develop hardware/software for the projects.
Evaluation of the Report Published by NASA on the Loss of the Mars Polar Lander and Deep Space 2 Missions.
1. Evaluation of a Report published by NASA on the
Loss of Mars Polar Lander and Deep Space 2
Missions
Om Shukla
1001171582
2. 1
CSE 5325- Software Engineering-II
Class Project:
Evaluation of the Report Published by NASA on the Loss of the Mars Polar Lander
and Deep Space 2 Missions.
Name: Shukla, Om
Student ID: 1001171582.
Date: 11/24/2015
4. 3
Introduction:
Through history, people have wondered whether there is life beyond Earth. And from the
last two decades we have been continuously advancing our ability to pursue this question. Mars
is a nearest planet feasible enough for the human exploration. The Mars Pathfinder landing on
July 4th, 1997, demonstrated extraordinary public interest in Mars, setting a record of the number
of visits to a Web site. Hence the Mars Program Independent Assessment Team found no reason
not to continue the Mars exploration. While the challenges were tough, the deep space success
demonstrated that the high risk were manageable and acceptable. In 1998, NASA conducted a
review of multi-year Mars program, calling in the outside experts help evaluate and refine the
architecture for the direction of the effort. The review resulted in the series of sample-return
missions, in the early next decade. The initial plan was to launch the missions almost every year
from 1999 continuing next decade till 2013.
Thus, Mars Polar Lander was launched on January 3, 1999 at 3:21 p.m. Eastern Standard
Time, on Delta II rocket from Space Launch Complex at Cape Canaveral Air Station, Florida.
The Mars Polar Lander was the part of Jet Propulsion Laboratory’s (JPL) Mars’ 98 Development
Project. Mars Polar Lander was with two Deep Space 2 Probes, designed by NASA’s New
Millennium Program, whose purpose is to flight test new technologies and demonstrate
innovative approaches for future Missions. Deep Space 2 probe were as the size of basketball, its
challenge was that a miniature components could be sent to other planet to conduct the science
experiments. The Mars Polar Lander approached the Mars on 3 December 1999, until then the
communications were up, then there was a small trajectory-correction maneuver, this made the
Mars Polar Lander onto entry trajectory and thus the antenna pointed off from Earth, and signal
was lost as expected. After twelve minutes the communications were supposed to have
established and data should have transferred to Earth, 24 hours later. Therefore it was expected
that the first data would be received on 4th December at 7:25 p.m. PST, about seven hours after
the touchdown. However no communications were established and none data received.
Attempts to communicate with Mars Polar Lander continued till mid-January without any
success. On 17 January 2000, the flight team announced that effort to recover the Mars Polar
Lander has concluded. The JPL Special Review Board and its consultants identified number of
failure scenarios, which for convenience were organized by mission phase. The Board divided
and organized itself into the each of the Mars Polar Lander’s development areas. Each Review
Team provided an assessment in their respective areas related design and test practices relevant
to the hypothesized failure. The Review team conducted their investigations through meetings
with Mars Surveyor 98’s personnel from Lockheed Martin Aeronautics (LMA) and JPL and
Deep Space 2 project personnel.
5. 4
Summary of Report:
In 1992, Daniel S. Goldin became the NASA administrator, after two months he
introduced a new way of working to the employees of JPL, he challenged the employees of JPL
to revolutionize the future NASA space missions to provide American people more cost-
effective space science programs. (Daniel Goldin, 1992) “How can we do everything better,
faster and cheaper, without compromising the safety?” And thus began the revolution of space
missions, the Faster Better Cheaper (FBC) strategy introduced concept of smaller spacecraft and
thus more frequent missions. The FBC strategy also distributes the risks over larger number of
small missions oppose to one large mission. Utilization of new technology was integral to the
FBC success.
REVIEW AND ANALYZE RECENT MARS AND DEEP SPACE MISSIONS
Missions Successes Failures
1. Mars Global Surveyor. x
2. Pathfinder. x
3. DeepSpace 1. x
4. Mars Climate Orbiter. x
5. Mars Polar Lander. x
6. Deep Space 2. x
Table: 1- Caparisons of Mars Missions.
However, NASA, JPL and LMA have not completely made the transition to FBC. They
had not documented the policies and procedure that made up their FBC approach. Rather project
managers had their own different interpretation. Under this environment the Mars missions
started. These different interpretation caused management to miss few steps resulting in the
failure to effectively implement FBC. Like all major changes, converting to FBC was a serious
management and leadership problem. This caused high failure rates in Mars Missions as shown
in Table 1 above. For the Mars Polar Lander, the JPL and LMA committed to the overly
challenging problematic goals during bidding for this project. The JPL management perception
was that, no cost increase is permissible. On the other hand the aggressive pricing strategy of
LMA exacerbated this problematic situation. This massive pressure of meeting the cost and
schedule goals resulted in environment of increasing risk in which too many corners were cut,
even in applying the proven engineering practices and applying the necessary checks and
balances. The Mars Program Independent Assessment Team (MPIAT) in its “Report on the Loss
of the Mars Polar Lander and Deep Space 2 Missions” dated 3/22/2000, provided the examples
such as: incomplete systems testing, lack of critical event telemetry and requirements creep.
6. 5
According to MPIAT, the organizations JPL and LMA also failed to ensure adequate
independent reviews and adherence to the established policies and procedures.
Fig-1
The Figure above illustrates the overly constraint conditions of Mars’ 98 project. The
cost, schedule and technical requirements and launch vehicles margins were inadequate. And as
shown in figure the only remaining variable was Risk. However even in high cost constraint
environment great care should be taken for cost-risk tradeoff. Accordingly for this mission the
management was facing excessive risks, mostly which were accompanied by integrating the new
technologies. As the matter of fact management did not manage the risk quite well as it should
have been. The MPIAT in its report mentioned that there was lack of adequate risk identification,
communication, management and mitigations, and which compromised the mission’s success.
The JPL Special Review Space Board in its report “Mars Polar Lander/Deep space 2 Loss – JPL
Special Review Board Report” found few choices that were further resulted in unanticipated
design complexity and consequences. One of those decision to use the four smaller off-the-shelf
engines for stability and control for which the Lander only required at least two canted engines in
each of three locations. Another decision was to use the pulse-mode control for the descent
engines to avoid the cost and cos-risk of developing and qualifying the throttle valve and
somewhat more difficult terminal descent guidance system algorithms. This introduced other
risks in the propulsion, mechanical and control areas.
From the beginning the Mars Polar Lander project was under considerable funding and
schedule pressure. In order to meet this challenges, the Laboratory decided to manage the project
with small JPL team and to rely heavily on LMA’s management and engineering structure.
Consequently, there was no JPL line management involved into the project. The LMA first- and
7. 6
second-level technical managers provided day to day technical oversight of the project. The JPL
team of approximately 10 technical and management people provided higher level technical
oversight. The result was minimal involvement by the JPL technical experts. On the other hand
the LMA used excessive overtime in order to complete the work on schedule within available
workforce. According to the JPL Special Review Board Report the record shows that
development staff worked 60 hours per week, and in which some of them were working almost
80 hours per week. Another consequence of the tight funding constraint was that many key
technical areas were staffed by a single individual. It is the Board’s assessment that this staffing
conditions led to a breakdown in inter-group communications, and there was insufficient time to
reflect on what may be the unintended consequence of day-to-day decision would be. In short
there was insufficient time and workforce available then normally found in the JPL projects.
The project did not have documented review plan but did hold many reviews, both formal
and informal. Subsystems Preliminary Design Reviews (PDRs) and Critical design Reviews
(CDRs) were conducted in a manner that they have reduced the level of formality but still
covering the appropriate depth and breadth of the Technical oversight. Most of the PDRs and
CDRs were included in-depth penetration by technical experts but some did not. According to
the JPL’s Special Review Boards report, in case of Propulsion subsystem, the thermal control
design interfaces were not mature enough to evaluate at CDR’s. A delta review should have been
held, but was not. Such a review could have been discovered the problems faced during the
flight. This limitation on technical penetration of action item and their closures were not typical
of JPL’s projects and was probably the unintended consequence of Project funding limitations.
The subsystems PDRs and CDRs were adequate in identifying most of technical issues.
Although all actions and recommendations were closed out formally prior to launch, these
closures were usually approved by the projects based on the LMA closures without any
independent technical support. The JPL’s Special Review Board reviewed these closure of some
action items that related to the potential failures, and found that while appropriate concerns were
raised at the reviews, the actions taken by the project did not adequately addresses these concerns
in all the cases. Why was such concerns remained unaddressed? Did they ignored such concerns,
under the pressure of schedule and budget limitations, this might be considered as unethical,
wrapping up the review process just to complete a formality!
8. 7
Conclusion:
Figure-2: Development Cost comparison Between Pathfinder & MCO & MPL.
The most probable cause of the Mars Polar Lander failure was: premature shutdown of
lander engines due to spurious signal generated at the lander leg deployment during the descent.
The spurious signals would be a false indication that the lander has landed. This in turn will
result in the crash of Lander due to the over speed during descent. Because of the absence of
flight data there is no way to know that whether Lander reached the terminal propulsion descent
phase of the mission. If it did, the extensive test results show that it would almost certainly have
been lost due premature engine shutdown. In short the most probable cause of the Mars Polar
Lander resulted from inadequate checks and balances that permitted an incomplete systems test
and allowed a significant software design flaw to go undetected.
As we have discussed, the attributes which led to the failure:
Inadequate JPL management Oversight.
Inadequate margin from the beginning:
o Only variable was risk
o Overly aggressive LMA’s cost proposals
o Excessively optimistic project implementation
o Inadequate staffing: single individuals implementing many
important activities.
Requirements Creep, inadequate Requirements management
No Entry, Descent and Landing Telemetry (EDL)
o Impedes failure analysis, and limits ability the corrective actions.
9. 8
When it comes to the Risk Management, it starts from the systems architectural design
phase. It is the role of the systems architect or a team of architects to evaluate the risks and
prepare risks assessments, later starts the plan for the risk identification, managements and then
avoidance or contingencies, which depends on the probability and the risk-cost. The LMA’s
Management was unable to go through all such process due to constraints we discussed, as the
project was understaffed, many key technical decisions and activities were carried out by the
single individuals. Peers working together is the best line of defense against errors. The LMA’s
negligence to such procedures and proper risk management activities attributed to the failure of
Mars Polar Lander Project.
As we can observe from the statistics of the Development cost comparison between the
Pathfinder project, which was the success with the Mars Polar Lander and Mars Climate Orbiter
project, both were failures. There is a considerable cost difference in Project management and
the Mission Engineering and Operations Development. The cost for Project Management was
less than half of that of a Pathfinder’s Project management cost. On the other hand they spend
almost same amount in science and instruments development. It could have been better if they
had found a proper balance between these spending. The technology which was being developed
should have been developed and planned such that it would be enough funding and time to carry
out reviews and testing, to make sure that the technology is ready for the mission. Which was
clearly not the case for the Deep Space 2 probes, the JPL Special Review Board found out that
the Deep Space 2 probes had not been tested enough and were not ready for the launch.
The Communication between the organizations were ineffective, which should have.
They should have conducted the high level technical and current status review with the technical
experts and senior management personnel from each organization, NASA, JPL and LMA. The
alternatives should have been explored, which would address the cost and schedule constraints of
the project. As explained by senior lecturer at University of Texas at Arlington, Dr. John Robb in
his Software Testing class (Robb, 2015) that “almost 90 % of the defects can be found by
Technical reviews”. He also explained that to find significant amount of defects in these
technical reviews, it is important to plan the review and give appropriate time for preparation to
the review members. These Technical reviews should have been carried out more frequently by
the JPL and LMA’s development team. They also should have revised the organizational
procedures for these reviews to make sure that the issues raised in these technical reviews should
be addressed by appropriate actions and notified to the issuer about these actions.
The decisions that were taken at the early phase of development should have been
documented properly, this document would have helped understanding the reasons behind those
decisions. For example the decisions of not implementing the EDL telemetry. Because the EDL
telemetry was not implemented there was no proper way to test the system, which in this case
would have helped significantly, and even could have prevented the mishap. One of the most
probable reasons that the Mars Polar Lander crashed was because, the software that should have
been designed to ignore the spurious signals was not designed to do so after all.
10. 9
References:
1. JPL Special Review Board, “Report of the Loss of the Mars Polar Lander and Deep
Space 2 Missions” Parts [1] [2] [3] [4] [5].
2. Mars Program Independent Assessment Team (MPIAT) “Report on the Loss of the Mars
Polar Lander and Deep Space 2 Missions” date: 03/22/2000.
3. NASA Press Release.
4. Hans van Vliet (2008), Software Engineering: Principles and Practice, John Wiley &
Sons.
5. Dr. John Robb (2015), Lecture Slides on Software Testing:
https://elearn.uta.edu/webapps/blackboard/content/listContent.jsp?course_id=_268351_1
6. Paul C. Jorgensen (2013), Software Testing: A Craftsman’s Approach, Fourth Edition,
CRC Press.
11. 10
Appendix:
1. NASA: The National Aeronautics and Space Administration (NASA) is the United States
Government agency responsible for the civilian space program as well as aeronautics and
aerospace research.
2. JPL: The Jet Propulsion Laboratory, is a federally funded research and development
center and NASA field center located in Pasadena, California, United States.
3. LMA: Lockheed Martin Aeronautics is an American global aerospace, defense, security
and advanced technologies company with worldwide interests.
4. Mars Pathfinder: Mars Pathfinder is an American robotic spacecraft that landed a base
station with a roving probe on Mars in 1997.
5. MPL: The Mars Polar Lander, was a 290-kilogram robotic spacecraft lander launched by
NASA.
6. DS2: Deep Space 2 was a NASA probe part of the New Millennium Program.
7. MCO: Mars Climate Orbiter was a 338 kilogram robotic space probe launched by
NASA.
8. EDL: Entry, Descent, and Landing.
9. MPIAT: Mars Program Independent Assessment Team.
10. PDR: Preliminary Design Review.
11. CDR: Critical Design Review.
