Here are the key lessons learned from Raytheon's perspective: 1. Hardware safety is the top priority. Don't over-test and stress the hardware beyond its limits. 2. Thorough preparation and readiness are essential for successful environmental testing. This includes getting customer concurrence on test plans. 3. Rigorous review and analysis of test equipment is as important as the flight hardware. All engineering work products, especially those not previously validated, require in-depth reviews. Glory APS Lessons Learned NASA PM Challenge 2010 29  Raytheon’s Perspective (2) Integration & Test - Early risk reduction testing is critical - Optimize test levels to reduce risk

1. Glory
Aerosol Polarimetry Sensor
Lessons Learned and Applied,
a Government and Contractor
Perspective
Bryan Fafaul Roberto Diffoot
Glory Project Manager Director, Glory APS Program
NASA Goddard Space Flight Center Raytheon Space & Airborne Systems
Glory APS Lessons Learned NASA PM Challenge 2010 1
Used with Permission

2. • Introduction / Agenda Fafaul / Diffoot
• Glory Mission Overview Fafaul
• APS Overview Diffoot
• APS Contract Background / Summary Fafaul / Diffoot
• APS Lessons Learned and Applied
• NASA’s Perspective Fafaul
• Raytheon’s Perspective Diffoot
• Summary / Closure Fafaul / Diffoot
• Questions and Answers Fafaul / Diffoot
Glory APS Lessons Learned NASA PM Challenge 2010 2

3. Glory Mission Overview
Bryan Fafaul
NASA Goddard Space Flight Center
Glory APS Lessons Learned NASA PM Challenge 2010 3

4. Mission Overview
• Mission Objectives
• Increase our understanding of aerosols as
agents of climate change by flying an
Aerosol Polarimetry Sensor (APS), and
• Continue measuring the sun’s direct and
indirect effects on climate by flying a Total
Irradiance Monitor (TIM) Instrument
• Mission Design
• 3 years (5 years of consumables)
• A-train orbit (705 km Altitude, 98.2
degrees inclination; Sun-synchronous)
• November 2010 launch readiness from
Vandenberg Air Force Base (VAFB)
Glory APS Lessons Learned NASA PM Challenge 2010 4

5. Glory Science Summary
APS will help to quantify the
role of aerosols as natural and
anthropogenic agents of climate
change with much better
accuracy than existing
instruments
TIM will continue measuring the 31-
year record of Total Solar Irradiance
(TSI) with improved accuracy and
stability to determine its direct and
indirect effects on climate
Glory APS Lessons Learned NASA PM Challenge 2010 5

6. The Road To Glory Overview
Glory APS Lessons Learned NASA PM Challenge 2010 6

7. Glory Observatory Configuration
S-Band Omni & Mast APS Instrument
TIM System Star Trackers
(x2)
Instrument TIM System
Module
Assembly
Cloud Cameras (x2)
GPS
Dual
Antenna
GLORY Body
Observatory Mounted
Front Isometric Solar
View GLORY Panel
Observatory
Rear Isometric
View
Glory APS Lessons Learned NASA PM Challenge 2010 7

8. Glory in the A-Train
• Glory will be formation flying with the Afternoon Constellation (A-Train):
• 705 km orbit altitude / 98.2° inclination (sun-synchronous)
• Ascending node Mean Local Time (MLT) crossing of ~1:41 pm
• Position relative to Aqua (based on nominal control box location)
• ~ 11 minutes behind at MLT crossing
• 215 km east offset (on Worldwide Reference System-2 grid) -- along track with CALIPSO
• Coincident science observations with CALIPSO / CALIOP and Aqua / MODIS
Glory APS Lessons Learned NASA PM Challenge 2010 8

9. Aerosol Polarimetry Sensor
(APS) Overview
Roberto Diffoot
Raytheon Space and Airborne Systems (SAS)
Glory APS Lessons Learned NASA PM Challenge 2010 9

10. Glory Aerosol Polarimetry
Sensor Overview
Earth Shield Aperture • The Aerosol Polarimetry Sensor (APS)
Assembly MLI Blankets not shown Door supports a three year NASA mission
Assembly
Polarized • Measurements of global aerosols in
Reference order to reduce the uncertainty in
Assembly
(internal) radiative forcing functions
• Initiation of continuous global
NADIR monitoring of aerosols in the
atmosphere
Unpolarized • The APS instrument description
Reference
Assembly • Size: 48 cm x 61 cm x 112 cm
(internal) • Weight: 61kgs (134.2 lbs)
Mainframe and • Operational Power: 55.0 Watts
Electronics Module
(internal) • The APS instrument scans the earth
over a nominal field-of-view of +50/-60
degrees about Nadir
Alignment Cube • The APS instrument generates along-
Assembly
Dark
track, multiple angle radiometric and
Solar Reference Reference polarimetric data with a 5.6 km (8 mrad)
Assembly Scan Motor Assembly circular IFOV
Assembly (internal)
• APS collects data simultaneously in nine
VNIR/SWIR spectral bands and four
polarization states
APS will help to quantify the role of aerosols • APS includes four on-board calibration
as natural and anthropogenic agents of sources to maintain high polarimetric
climate change and radiometric accuracy on-orbit
Glory APS Lessons Learned NASA PM Challenge 2010 10

11. Aerosol Polarimetry Sensor
Aperture Door
Assembly (ADA)
Cryo Radiator
Assy (CA) Unpolarized
Reference Assembly
(URA)
Optics & Detectors Module (ODM)
Polarized Reference
Assembly (PRA)
Electronics Scan Mirror
Module (EM) Assembly (SMA)
Solar Reference
Assembly (SRA)
Dark Reference
Assembly (DRA)
Glory APS Lessons Learned NASA PM Challenge 2010 11

12. Optics and Detector Module (ODM)
The ODM contains all
36 VNIR and SWIR
detectors and 161 of
185 optical elements
218 mm (8.6 in)
356 mm
247 mm (9.8 in)
(14.0 in)
ODM Level Test Implemented As Early Risk Mitigation Strategy
Glory APS Lessons Learned NASA PM Challenge 2010 12

13. Aerosol Polarimetry Sensor
Earth Cover
Main Aperture
Door
Earth
Shield
EM
Connector
Interface
Solar Reference
Assembly Door Scan Mirror
Assembly
Glory APS Lessons Learned NASA PM Challenge 2010 13

14. Aerosol Polarimetry Sensor
Earth Shield
Light Baffle
Assembly
Cryoradiator
Solar Reference
EM Radiator Assembly Door
EM Connector
Interface
Glory APS Lessons Learned NASA PM Challenge 2010 14

15. APS Integration and Test Flow
Mar 08 Apr May Jun Jul Aug Sep Oct Nov Dec Jan 09 Feb Mar
Test Readiness Pre-Environment Delta Consent to
Review (TRR) Review (PER) (PER) Ship Review (CTSR)
5/1 5/10 7/27 8/27
90- EM
EM EM
EM Integration
Rework Complete AI&T
and Test 9/9
E2 Cleanroom Vibe Table Weigh Verification
4/8 5/10
Final EMC/ TV
ODM/EM Int, EMI Thermal Ano- O&M
Integ &
Int and APS B/L Tests
Vib and Vacuum maly Ship,
Test & Pre TV Test Resolu- Test
Test Test tion
7/31
Integrate Cal, 8/8
4/7 Complete
PM
ODM
PM Integration
Integrated Test Facility (ITF) Top Loader Chamber
Glory APS Lessons Learned NASA PM Challenge 2010 15

16. APS Contract Background /
Summary
Bryan Fafaul
NASA GSFC
Roberto Diffoot
Raytheon Space and Airborne Systems (SAS)
Glory APS Lessons Learned NASA PM Challenge 2010 16

17. APS Contract Background (1)
• February 2004 – NASA issues letter contract to Raytheon Santa
Inconsistent NASA
Barbara Remote Sensing (SBRS) in Goleta California
Staffing Issues
Funding / RTN
• February 2005 – Preliminary Design Review (PDR) successfully
completed
• February to April 2005 – NASA redirects Glory Project to study
possibly flying APS and the Total Irradiance Monitor (TIM) on
NPOESS in lieu of stand alone Glory mission
• July 2005 – NASA authorizes Glory as a full stand alone mission
NASA Funding Improved /
• November 2005 – Glory Project Confirmed, launch readiness
RTN Staffing Issues
set at December 2008
• December 2005 – APS contract definitized with instrument
Continue
delivery set for December 2006
• January 2006 – NASA Project Manager changed
• April 2006 – APS Critical Design Review (CDR) successfully
completed 2 months late
• May 2006 - Raytheon Goleta General Manager Changed
Glory APS Lessons Learned NASA PM Challenge 2010 17

18. APS Contract Background (2)
• August 2006 – APS Integrated Baseline Review (IBR) held 2
months late and not fully successful
Raytheon Goleta Plant Closure / APS Program
• September 2006 – Rumors of possible plant closure; Raytheon
Moved to Raytheon SAS in El Segundo
Goleta workforce exodus begins
• September 2006 - Raytheon APS Program Manager changed
• October 2006 – Raytheon announces Goleta Plant is closing
and work will be transferred to Space and Airborne Systems in
El Segundo; Raytheon Goleta loosing core competency as
workforce exodus accelerates
• November 2006 – NASA decision to move APS to El Segundo
as quickly as possible; Raytheon Goleta workforce depleted and
being augmented with El Segundo personnel
• January 2007 – APS Program move to El Segundo complete
• March 2007 – Raytheon APS Program Manager and Civil
Space Director changed
• April 2007 – APS Engineering Ownership / Delta CDR
successfully completed
Glory APS Lessons Learned NASA PM Challenge 2010 18

19. APS Contract Background (3)
Control • July 2007 - Raytheon APS Program Manager Changed
Takes
Team
• July 2007 to April 2008 – Significant program replan for both
APS and Glory Mission which included implementations of
many descopes; Implementation of early risk mitigation tests
• May 2008 – Optics and Detector Module (ODM) assembly and
NASA / Raytheon Team Performing
as One / APS Integration and Test
test (w/flight electronics) completed with excellent performance
• August 2008 – Polarimeter Module (PM) and Electronics
Module (EM) assemblies completed
• September 2008 – APS integration complete
• October 2008 – APS baseline performance testing completed
with excellent results
• February 2009 – APS environmental Testing successfully
completed with excellent instrument performance
• March 2009 – APS delivered to NASA
• April 2009 – APS successfully integrated on the Spacecraft with
no issues
Glory APS Lessons Learned NASA PM Challenge 2010 19

20. APS Performance Summary
• Prior to testing the APS optics we defined expectations for the
performance parameters ranging from poor to acceptable, expected
and target. Of the 27 parameters evaluated 25 were in the desired
target range with 2 in the expected range
• The completed APS has excellent radiometric and polarimetric
performance which was maintained throughout environmental testing
at Raytheon
• The APS optical performance was tested after installation on the
spacecraft at Orbital and showed identical performance to that of a
similar pre-ship test at Raytheon
APS Scientific Performance Is Excellent!
Glory APS Lessons Learned NASA PM Challenge 2010 20

21. APS Integrated w/ Spacecraft
Glory APS Lessons Learned NASA PM Challenge 2010 21

22. APS Integrated w/ Spacecraft
Glory APS Lessons Learned NASA PM Challenge 2010 22

23. Glory Observatory Fully Integrated
Glory APS Lessons Learned NASA PM Challenge 2010 23

24. APS Development Lessons
Learned and Applied
Bryan Fafaul
NASA GSFC
Roberto Diffoot
Raytheon Space and Airborne Systems (SAS)
Glory APS Lessons Learned NASA PM Challenge 2010 24

25. NASA Perspective (1)
• Send the right message early …
• provide strong commitment to the program by definitizing the
contract quickly and providing consistent funding, no starts and
stops
• Demand “a” team…
• recognize early the strengths and weaknesses of the “team”
• make necessary personnel changes early and keep the team
together, stability is key
• senior leadership “commitment to succeed” is critical in making
good personnel assignments
• Stable requirements, strong Systems Engineering, and science
understanding yields good designs…
• Systems Engineers must challenge requirements to improve system
complexity, performance, and system reliability
• scrub requirements early and often, don’t wait to get in trouble
• understand the science, it’s critical to making good system trades;
unlike we are taught, all science requirements are not created equal
Glory APS Lessons Learned NASA PM Challenge 2010 25

26. NASA Perspective (2)
• Establish performance baseline early and keep the engineers doing
engineering…
• as soon as the contract is definitized, establish the baseline and get
the team focused on engineering and execution
• rebaseline when necessary, sticking with an obsolete baseline too
long hurts everyone
• Be prepared for anything and everything…
• open communication between Raytheon and NASA was key for
NASA to complete move risk assessment (security, loss of key
personnel, schedule, contingency plans, etc) and provide
concurrence in a timely manner
• Management anomalies occur more often and linger longer than you
think making root cause very difficult to identify and fix
• as engineers, we do a great job troubleshooting, fixing, and
regression testing hardware anomalies. The same rigor needs to be
employed with program execution or management anomalies
Glory APS Lessons Learned NASA PM Challenge 2010 26

27. NASA Perspective (3)
• System Engineering, Systems Engineering, Systems Engineering…
• understanding the system, requirements, verification is key to
remaining flexible when things go wrong
• it’s not just checking the box, characterization is key to making
trades
• Timely design closure and critical expert review is key to execution…
• designs that come together late require heroic efforts and may result
in mistakes
• follow through on post design review clean-up activities, they are
key to staying on plan
• employ the trust but verify philosophy by auditing drawings,
procurement packages, and analyses; you can’t execute if the plan
is always changing
• Don’t forget the “ilities”, they have the potential to cripple progress…
• make sure safety, reliability, and quality analyses are complete
early; failure to do so may result in not meeting requirements,
delayed procurements, and late design changes
Glory APS Lessons Learned NASA PM Challenge 2010 27

28. NASA Perspective (4)
• Planning is king, but don’t go overboard because time is money…
• know your instrument and understand how to integrate it
• make sure the paper is ready to go, practice makes perfect; the
flight hardware shouldn’t be your first time
• Ground Support Equipment is just as or more important than the
flight hardware to keep things moving
• Get ready, set, ship it and don’t forget it’s gotta fit…
• ICDs don’t guarantee anything; identify risk reduction activities early
and often
• maximize risk reduction every where you can (hardware, software,
procedures, test tools, etc)
Glory APS Lessons Learned NASA PM Challenge 2010 28

29. Raytheon’s Perspective (1)
Environmental Tests
• Hardware safety is the first priority
• Don’t over-test (know the limits of the hardware)
• Ensure the test team is ready for each test phase
• Preparation is key to success
• Customer concurrence with the test plan is a “must have”
• Rigorous review and analysis of special test equipment is as important
as with the flight hardware
• All environmental test engineering work products need an in-depth
reviewed … Specially if not validated
• Be sure “best practices” for margin management and safety factors
are implemented
• Un-validated work products are prone to discovery during test
• EMI/EMC test driven by local knowledge and tradition
• Subject matter experts (customer, contractor and consultants) not fully
aligned on test methodology and implementation
• Coordinated test planning, implementation and execution is imperative
Glory APS Lessons Learned NASA PM Challenge 2010 29

30. Raytheon’s Perspective (2)
• Implementation of Raytheon/NASA “Best Practices”
– Strong linkage between fabrication and verification activities (over
arching systems engineering approach)
– Avoid “not invented here” mentality – use the best personnel,
equipment and processes no matter where it comes from
– Optimized fabrication and assembly activities to achieve high
performance margins as a risk mitigation for unknown/unknowns
– Take advantage of “lessons learned” from other programs
• NASA programs demands both requirements liquidation and extended
characterization
– Need to have the flexibility to implement characterization testing
beyond just requirements liquidation
• Early integration test events
– Performance verification prior to starting final integration and test
sequences
– Fit checks, dry-runs of test procedures and peer reviews – accelerated
learning
Glory APS Lessons Learned NASA PM Challenge 2010 30

31. Raytheon’s Perspective (3)
• Implementation of automation tools and inventory control processes
– Benefit of electronic data bases – remote access, better archiving and
protection, electronic links
– Better suited for mature designs … Some benefits on development
programs
– Discipline of process is driven by user
– Proper personnel training is a must
• Cross integration of engineering products must be performed during the
design and development processes
– Design and hardware verification is not completed without the model
validation
– You will always have an engineering model
o The question is: Would it be Flight 1? … Plan accordingly!
Glory APS Lessons Learned NASA PM Challenge 2010 31

32. Raytheon’s Perspective (4)
• Implementation of Raytheon/NASA best practices
• NASA programs demand both requirements liquidation and extended
characterization
• Early integration test events
• Implementation of automation tools and inventory control processes
• Cross integration of engineering products must be performed during the
design and development processes
Be flexible … Early leaning mitigates risk … Careful
deployment of new tools … Cross integration of engineering
products and design is a must
Glory APS Lessons Learned NASA PM Challenge 2010 32

33. Summary / Conclusion
Bryan Fafaul
NASA GSFC
Roberto Diffoot
Raytheon Space Systems (SAS)
Glory APS Lessons Learned NASA PM Challenge 2010 33

34. Summary / Conclusions
• APS significant cost overruns and schedule delays were due to
planning and execution, not technical issues
• Raytheon and NASA teams never quit, senior leadership on both
sides were committed to completing APS
• Be prepared for anything, the decision to close the Goleta facility was
a significant impact to the program
• Loss of key personnel, loss of ownership, loss of corporate
knowledge can be over come, but it requires a good team that is
committed to success, lot’s of hard work, and then more hard work
• Don’t be afraid to work together, leverage resources wherever it
makes sense, flexibility is critical
• Badgeless teams, government furnished equipment and parts, and
streamlined processes
• Empower the people. If you have good engineers, use them, don’t
just rely of processes
• Don’t be committed to only your way of doing things, there are
many ways to get the job done
Glory APS Lessons Learned NASA PM Challenge 2010 34

35. Summary/Conclusions (2)
• Strategically selected risk reduction initiatives can bring major benefits to
the program
• Early learning at the proper level of assembly
• Maximizes reaction time to fix problems and/or provides for timely
opportunity to leverage results
• Establishing trust within the team is pivotal
• Direct, open and honest communication - Say what you are going to
do, do what you say
• The first sign of gaining control on a red program is to eliminate
surprises – Predictability is a wellness indicator
• Give the team an opportunity to succeed
• Challenging but achievable schedule and cost requirements
• Be passionate and relentless about what needs to be done
• Instill pride on everything your team does … Even during the dark
days … Hardware will make you humble … Never, never give up
Glory APS Lessons Learned NASA PM Challenge 2010 35

36. A well integrated government /
contractor team that values
communication, commitment,
and respect is the key to
SUCCESS!
Glory APS Lessons Learned NASA PM Challenge 2010 36

37. Questions and Answers
http://glory.gsfc.nasa.gov/
Glory APS Lessons Learned NASA PM Challenge 2010 37