Quality assurance must ensure all areas of the organization are meeting performance standards as set forth in internal quality assurance surveillance plans. Clear roles and responsibilities, performance objectives, and methods of measuring and monitoring quality ensure that the vision of the organization is properly implemented. How can you promote the importance and best practices of sustaining and implementing quality assurance programs? At the end of this session, participants will be able to: a. Understand how to ensure teams are aligned with values of quality assurance b. Discuss methods to improve reliability, accountability, and performance c. Explore methods of measuring and monitoring quality Learning Outcomes: Participants will increase value and improve performance by designing and or assessing current Quality Assurance strategies that impact overall team and individual accuracy, efficiency, and reliability.

1. QUALITY ASSURANCE:
INCREASING WORKFORCE
EFFECTIVENESS, EXPERTISE,
AND RELIABILITY
Presenters:
Dennis Owens, Patrice Gregory, and J. Anthony Wingate
Friday, February 8, 2013 | 8:15 a.m. – 9:30 a.m.
Washington Marriott Wardman Park | Maryland Suite A
Sandia National Laboratories is a multi-program laboratory managed and operated by Sandia Corporation, a wholly
owned subsidiary of Lockheed Martin Corporation, for the U.S. Department of Energy’s National Nuclear Security
Administration under contract DE-AC04-94AL85000.
SAND #: 2012-7175 P
Unlimited Release (UR)

2. Topic Requirements: D
Quality Assurance: Increasing Workforce Effectiveness, Expertise, and Reliability
Quality assurance must ensure all areas of the organization are meeting performance
standards as set forth in internal quality assurance surveillance plans. Clear
roles and responsibilities, performance objectives, and methods of measuring and
monitoring quality ensure that the vision of the organization is properly implemented.
How can you promote the importance and best practices of sustaining and
implementing quality assurance programs?
At the end of this session, participants will be able to:
a. Understand how to ensure teams are aligned with values of quality assurance
b. Discuss methods to improve reliability, accountability, and performance
c. Explore methods of measuring and monitoring quality
Learning Outcomes: Participants will increase value and improve performance
by designing and or assessing current Quality Assurance strategies that impact
overall team and individual accuracy, efficiency, and reliability.
QUALITY ASSURANCE: INCREASING WORKFORCE EFFECTIVENESS, EXPERTISE AND RELIABILITY

3. Objectives: D
• At the end of this session, participants will be able
to:
a. Understand how to ensure teams are aligned with
values of quality assurance (QMS design)
b. Discuss methods to improve reliability, accountability,
and performance (LSS CI event strategy)
c. Explore methods of measuring and monitoring quality
(Performance Auditing against QP and objectives)
QUALITY ASSURANCE: INCREASING WORKFORCE EFFECTIVENESS, EXPERTISE AND RELIABILITY

4. Presenter Bios: D
Dennis Owens J. Anthony Wingate
Dennis Owens manages the Defense Systems J. Anthony Wingate manages the Subsystem and
Quality Engineering department at Sandia National Component Quality Engineering I department at
Laboratories (SNL) in Albuquerque, New Mexico. Sandia National Laboratories in Albuquerque, New
His organization provides R&D Systems Engineer- Mexico. His organization provides R&D Product
ing, Quality Engineering, and Quality Assurance Qualification Engineering support to Sandia’s
support to Sandia’s Work for Others (WFO) Depart- nuclear mission programs. Anthony’s career began
ment of Defense projects/programs. Dennis’ career as a summer intern at WKM Value Company and
began at GE Aircraft Engines in 1988 in advanced Houston Lighting & Power and GE Lexan Plastics. In
composite fabrication and later turbine component 1987 he joined prestigious Manufacturing Manage-
manufacturing. In 1995 he joined Silmax, Inc., a ment Program (MMP) at GE Lexan Plastics. In 1992
semi-conductor and wafer reclamation company, as he joined Martin Marietta (now Lockheed Martin)
a Quality Engineer. At Silmax, Mr. Owens was responsible for designing and
a as a process as a process engineer.
implementing the quality management system and supplier quality program.
In 1994 he was part of a small group of engineers selected to transfer
In 1999 Mr. Owens joined Allied Signal Power Systems (ASPS) as a Supplier generator technology from GE located in Pinellas, Fl. to Sandia National
Quality Engineer. At ASPS he was responsible for designing the quality Laboratories (SNL) in Albuquerque, NM. While at SNL Mr. Wingate was
management system and supplier quality programs. In 2001 Mr. Owens responsible for quality and product engineering of generator technology. In
joined SNL as a Senior Quality Engineer. At Sandia his career has span from 2007 he took an assignment as a Lead Project Engineer at Sandia’s Integrated
generator technology and software/tester qualification to Advanced Concept Military Systems Development Center where he was responsible for design,
and Technology Development (ACTD) programs for Sandia’s Integrated fabrication and deployment of counter-measures on several test. In 2008 he
Military Development Center (IMSDC) now Integrated Military Systems (IMS). was promoted to manager of the Subsystem and Component Quality
Engineering department within the Surety Assessment, Engineering, and
Mr. Owens holds certifications in internal auditing (RABQSA), and LM21 Lean Analysis Center.
Six Sigma. From 2005-2007 he authored several articles in the American
Society for Quality (ASQ) Quality Progress magazine on quality management, Mr. Wingate holds a certification in LM21 Lean Six Sigma and has contributed
defect analysis, and continuous improvement. He has been invited to speak to several high-profile lean events for the Laboratory. He has received
at NASA and ASQ conferences and is a 2013 BEYA Trailblazer award recipient. numerous awards such as 1998 and 2005 Sandia President’s Quality Awards,
Mr. Owens received his Bachelor’s degree in Mechanical Engineering Tech- Sandia Awards for Excellence, 2005 and 2010 National Nuclear Security
nology from New Mexico State University and a Master's of Arts degree in Administration (NNSA) Defense Programs Award of Excellence, and is the
Organizational Management and Leadership from University of Phoenix. 2013 BEYA Affirmative Action Award recipient. Mr. Wingate received his
drowens@sandia.gov Bachelor’s degree in Mechanical Engineering from Lamar University in
Beaumont, TX. jawinga@sandia.gov
QUALITY ASSURANCE: INCREASING WORKFORCE EFFECTIVENESS, EXPERTISE AND RELIABILITY

5. Presenter Bios: D
Patrice Gregory
Patrice Gregory is a Senior R&D Systems Engineer in the Subsystem and Component Quality
Engineering II department at Sandia National Laboratories in Albuquerque, New Mexico. Her
organization provides R&D Product Qualification Engineering support to Sandia’s nuclear
mission programs.
Patrice’s career began as a summer intern at NASA Ames Research Center in the Rotorcraft
Division, where she was involved in the wind tunnel testing of a quarter scale V-22. She
joined Sandia in 2011, where she leads the qualification efforts of magnetics components for
various weapon systems.
Patrice holds a Bachelor of Science degree in Mathematics from Spelman College, a Bachelor
of Science degree in Mechanical Engineering from Georgia Institute of Technology, and
Masters of Science and Doctor of Philosophy in Mechanical Engineering from the University
of Maryland College Park. She performed her doctoral research on the comparison of
interconnect failures of electronic components under rapid loading conditions.
pbgrego@sandia.gov
QUALITY ASSURANCE: INCREASING WORKFORCE EFFECTIVENESS, EXPERTISE AND RELIABILITY

6. Our Goal: D
Expose you to :
 a successful approaches in quality management
system (QMS) design and assessment
 lean principles to ensure accuracy, efficiency,
and reliability in QMS performance
 An Engineer’s perspective of quality applied to
research and development
Why? To help you extract simplicity out of complexity
QUALITY ASSURANCE: INCREASING WORKFORCE EFFECTIVENESS, EXPERTISE AND RELIABILITY

7. Definitions: D
 Defect- the lack of something necessary or desirable for
completion or perfection
 Designed in Defects- a defect that which can be avoided only
through an alteration or redesign of the item.
 Blunder- a stupid, careless, or thoughtless mistake
 Waste- anything or process that does not add value to an item
 Value- what the customer is buying or willing to pay for
 Predictability ( vs. Reliability)-To state, tell about, or make
known in advance, especially on the basis of special
knowledge
 Prevention- the action of stopping something from happening
As Quality Professionals we have to first change the way we think (or see)
before we can influence others
QUALITY ASSURANCE: INCREASING WORKFORCE EFFECTIVENESS, EXPERTISE AND RELIABILITY

8. Burning Platforms: A
 No More Budget
 2001: NG Production Center and the birth of  No Schedule Relief
Lean Six Sigma (LSS) LEAN/SIX SIGMA
Tools
Business Platform
 Minimal Inventory To Meet Schedule
Demands
 Competition
 2007: Integrate Military Systems
Development Centers response to customer
(MDA) request
 2010: Surety Assessment, Engineering, and
Analysis Center vision to improve operations
Quality
What is the common theme from each of these stories?
QUALITY ASSURANCE: INCREASING WORKFORCE EFFECTIVENESS, EXPERTISE AND RELIABILITY

9. Case Study: Surety Assessment, Engineering
and Analysis ISO Project (Sample Documentation): A
Path to Excellence
Accomplishments ISO9001:2008 LSS Project Team
We will describe the case study that yielded the
following results:
• Project completion, initiation to registration, within 7 months
• Project cost at $150k
• Quality Manual 12 pages in length and completed in 1 day
• Leveraged existing product realization process
• Eliminated legacy documentation waste
• QMS tied to Center strategic planning model
• Approach designed as a cost effective model for use by other
organizations within Sandia National Labs
• Project recognized by National Nuclear Security Administration
(NNSA)
Resources to stimulate your thinking
An intuitive approach to defect prevention is key
Unclassified – Unlimited Release (UUR)
QUALITY ASSURANCE: INCREASING WORKFORCE EFFECTIVENESS, EXPERTISE AND RELIABILITY

10. Understand how to ensure teams are aligned
with values of quality assurance: A
 Management commitment,
 Manage by walk-around
 Attendance to staff meeting
 Direct support to Lean Events
 Quality Policy and Quality Management System (QMS)
 Objectives that touch the critical parts of your business
 Clear roles and responsibilities,
 Clearly defined at all levels through communication
 Unique review and planning through -out the organization
 Requirements, Plan, Implement, Validate
 Product Realization
QUALITY ASSURANCE: INCREASING WORKFORCE EFFECTIVENESS, EXPERTISE AND RELIABILITY

11. VSA Event Activities Time-Line: (Sample Documentation) A
QUALITY ASSURANCE: INCREASING WORKFORCE EFFECTIVENESS, EXPERTISE AND RELIABILITY

12. “Switching” on the Light: (Sample Documentation) A
QUALITY ASSURANCE: INCREASING WORKFORCE EFFECTIVENESS, EXPERTISE AND RELIABILITY

13. Event Cost Estimate: (Sample Documentation) A
• Total Number of Participants:
 1 Director, 2 Sr. Managers, 13 Managers, 4
Staff
• Event Cost Estimate:
 High: $$
 Low: $
QUALITY ASSURANCE: INCREASING WORKFORCE EFFECTIVENESS, EXPERTISE AND RELIABILITY

14. Discuss methods to improve reliability
(predictability), accountability, and performance: P
Operating
 Application of Lean Six Sigma Tools • Standard Operating Procedures
• SPC
• Process Metrics
Excellence
(constraints, considerations, interactions, etc.) • Process Modification
Control Process and
Sustain Performance
• Mistake Proofing
• Standard Work
• Brain Storming
• DOE
Improve & Innovate Process to Meet
Customer Needs • Kaizen
• Lean Processes
• Cause & Effect
• Brain Storming
• FMEA
• 6s Applications
• MSA Analyze Process for • Visual Controls
• Variation Improvement Areas
• Process Capability
• Sigma & DPMO Levels • Waste Identification
 Lean/Six Sigma Training
• Flow Diagrams
Measure Capability to
• Pull System Analysis
• Input-Process-Output
Meet Customer Needs • To-be Process
• Process Flow Diagram
• Quality Function Deployment
• Define CTQ’s
• Value Stream Analysis
Define Process &
• Time Value Mapping
Customer Value • Cycle Time & Efficiency
• Value Stream
• As-is Process
 Well written Charter
Champions/Roles (accountability)
Predictability (reliability)
Prevention (performance)
Sustainability (performance)
QUALITY ASSURANCE: INCREASING WORKFORCE EFFECTIVENESS, EXPERTISE AND RELIABILITY

15. EVENT / PROJECT CHARTER (Sample Documentation): A
EVENT / PROJECT CHARTER
Event/ Project Description: ISO 9001:2008 Registration Event / Project Dates:
Team 1 (July 2010), Team 2 (Aug 2010), Team 3 (Sept 2010)
Preliminary Objectives: Champion: Rick Fellerhoff
1. Define the end state of the vision for the Center Stakeholders: Mike Daily, Todd Jones
2. Use ISO9001:2008 and Lean six sigma as tools to define, measure, analyze, Team Members:
improve, and control, operational processes and procedures that are in line with Team #1: Dennis Owens- BB, Marlene Lucero-GB,
the Center vision, objectives and definitions of Center responsibility and
•Focus: Quality Manual
authority throughout NW and WFO.
3. Define the strategy to execute multiple and parallel events in order to achieve Team #2: J. Anthony Wingate-BB, Peggy Rhodes-
registration within 6 months (December 2010) GB, Ron Farmer, GB
4. Prove Center is Sandia’s resource with the technical expertise to design and •Focus: Center Related Processes
implement cost effective quality management systems in an R&D laboratory
environment Team #3: J. Anthony Wingate-BB Dennis Owens-
5. Recognition as Sandia’s “go-to” organization for QMS consultation BB, Marlene Lucero & Peggy Rhodes- GB
•Focus: Dept specific operational plans
Customer Demand & Constraints: Return on Investment:
• Perceived understanding and value of ISO and Lean within the Center • Efficient, effective , and measureable operational
protocols for the Center
• 12300 Recognized as the Center of Excellence of
NWC Quality in the area of Assessments and
Engineered solutions
Process Information: Value Stream Tie and/or Business Objective Tie:
• Industry Standard - ANSI/ISO/ASQ Q9001-2008 • Improve Center management practices
• Center: • Teach other Sandia organizations how to improve
• Website, Operational processes, Procedures, and Value propositions • Paul Hommert Vision
• Vision, Objectives, Policy and VP memos
Communication Defect
QUALITY ASSURANCE: INCREASING WORKFORCE EFFECTIVENESS, EXPERTISE AND RELIABILITY

16. Explore methods of measuring and monitoring
quality : D
• Rule #1: the audit process objectives should only focus on determining the effectiveness
of the quality policy statement and business objectives through cause and effect (e.g.
the causal analysis chain of events)
• Rule #2: the audit process should report how/if “the system” is operating as an
integrated unit
6 “M’s”
+
Manpower Materials Machine
Value/Goal
Measurement Methods Mother Nature
(Environment)
What audit
What management needs to understand in order to act
finds
Talking Hearing Barrier WIFM
Design
N/C Defect Defect Defect Defect (What’s In It For Me)
Defect
Communication Defect
QUALITY ASSURANCE: INCREASING WORKFORCE EFFECTIVENESS, EXPERTISE AND RELIABILITY

17. Summary: D
What we want you to take away (golden nuggets)
 If you are a senior level leader, design and communicate your quality
vision in terms of an integrated system. Insist your audit program
focus analyzes that vision and is based on value(or things your
willing to pay for).
 If you’re a line level leader, integrate the process of quality into your
regular routine and communicate this often at your staff meetings as
an expectation of performance.
 If you're a staff member, show your commitment to your leadership’s
vision by taking the time to understand your quality assurance
program and be vocal about its value to your job. This will help us
find the disconnects.
 Flip the Script
Quality improvement is a journey, not a destination therefore, measure
your successes and challenges with a “next steps” mentality.
QUALITY ASSURANCE: INCREASING WORKFORCE EFFECTIVENESS, EXPERTISE AND RELIABILITY

18. Objectives: D
• You should now be able to:
a. Understand how to ensure teams are aligned with
values of quality assurance (QMS design)
b. Discuss methods to improve reliability,
accountability, and performance (LSS CI event
strategy)
c. Explore methods of measuring and monitoring
quality (Performance Auditing against QP and
objectives)
QUALITY ASSURANCE: INCREASING WORKFORCE EFFECTIVENESS, EXPERTISE AND RELIABILITY

19. Questions?
QUALITY ASSURANCE: INCREASING WORKFORCE EFFECTIVENESS, EXPERTISE AND RELIABILITY