2010 tqa assessor workshop by thomas schamberger day one

Baldrige Foundation – Thailand Quality Assessor
Workshop
2010
Welcome

Workshop
2010
Enhancing the Effectiveness ofEnhancing the Effectiveness of
the TQA Assessment Processthe TQA Assessment Process
for Assessors and Seniorfor Assessors and Senior
AssessorsAssessors
Thomas Schamberger
Executive Director
Foundation for Malcolm Baldrige National Quality
Award, Inc.

Workshop
2010
Objectives:
– Identify the competencies and skills required as a assessor & senior
assessor/team leader to include some advanced skills
– Senior Assessor Roles – Team Leader and Team Member
– Review Assessment Process
– Critical Success Factors
– Tools and Techniques for Conducting a Successful Site Visit
– Feedback Report Writing Skills and Score Analysis based on findings
– Conflict management & Crisis management
– How to build teamwork
– Identify the competencies needed as a mentor and coach
– What to Do and What Not to Do – Lessons Learned

Workshop
2010
Introduction Activity—Questions

Workshop
2010
Today and Tomorrow let’s agree to
Establish a Mentoring Relationship
Therefore…….
in order for it to be the most
effective let’s agree to build on
mutual trust and openness by
both parties by asking questions
and sharing information so that
we can grow and learn

Workshop
2010
Day 1 Key Outcomes and
Learning
Assessor Skills and Competencies
Senior Assessor Skills and Competencies
Senior Assessor Roles
Advanced Assessor Skills
Independent Review and Key Factors
Process and Results Item - Comment Writing
using ADLI and LeTCI

Workshop
2010
– Oral/Written Communication
– Ethical Behavior
– Action Orientation
– Peer Relationships
– Time Management
(Individual)
– Self-Development
– Technical Learning
(Computer Skills)
General Assessor Skills are….

Workshop
2010
Assessment Knowledge & Skills
– Analytical -
•• Use of statistical methodsUse of statistical methods
•• Evaluating financial data and resultsEvaluating financial data and results
•• Ability to analyze levels, trends, comparisons,Ability to analyze levels, trends, comparisons,
segmentation, breadth of results, and linkagessegmentation, breadth of results, and linkages
between Process and Results Itemsbetween Process and Results Items
– Probing Skills
•• Isolate the real issues to uncover most appropriateIsolate the real issues to uncover most appropriate
findingsfindings
•• Guide the team on the best questions to askGuide the team on the best questions to ask
and know how you can simplify themand know how you can simplify them
– Use of technology tools and resources to
•• manage and communicate informationmanage and communicate information
•• Interpersonal skillsInterpersonal skills

Workshop
2010
Assessor Competencies
1. Writing skills (i.e. ability to produce/give clear,
accurate and useful feedback) and asking relevant
questions
2. Ability to comprehend organizational context based
on Criteria Framework
3. Field/site visit requirements
4. Better understanding of Criteria Framework and
score consistency

Workshop
2010
5. Ability to gather, assimilate and analyze evidence
6. Ability to produce and deliver clear, concise, and
accurate feedback
7. Ability to understand organization and review the
organization in terms of structure, processes,
performance and results
8. Ability to interact positively with other members of
team, organizers, and representatives of the
organization being assessed
9. Adherence to requirements of rules of conduct,
ethics, computer practices and confidentiality
considerations

Workshop
2010
10. Leadership Skills
11. Better understanding of evolving contemporary
management philosophies/methodologies (i.e.
revisions to criteria require understanding of new
terminology and how it impacts an organization’s
ability to respond)
12. Systematic process for conducting an assessment
13. Methodology for sharing knowledge in conducting
assessments more effectively

Workshop
2010
Team Leader Skills as an Assessor
1. Organizing Skills
2. Planning Skills
– Managing the Process
– Managing People
3. Time Management (Team)
– Managing Scorebook Production
4. Team Building

Workshop
2010
– General Assessor Skills
– Assessment Process
Knowledge & Skills
– Team Leader Skills
Senior Assessor Basic
Competencies

Workshop
2010
5 Considerations for a Senior Assessor5 Considerations for a Senior Assessor……....
The
Organization
must be able
to Tell their
Story
The
Organization
must be able
to Tell their
Story
The more engaged
the organization is
the more you can
fully assess them
and strengthen
confidence in them
that you are
capturing their story.
The more engaged
the organization is
the more you can
fully assess them
and strengthen
confidence in them
that you are
capturing their story.
Engage Organization
Use the strength of
your team and make
assignments. Don’t
carry the full load.
Use the strength of
your team and make
assignments. Don’t
carry the full load.
Strengths +
Quantity =
Quality
Strengths +
Quantity =
Quality
Delegate
Work to make the process Fluid vs. Rigid
Know where you’re
going so that you
know who, what,
when, where, and
how you are going to
be able to get there
Know where you’re
going so that you
know who, what,
when, where, and
how you are going to
be able to get there
Plan with the
End in Mind
Plan with the
End in Mind
Start at the End
Apply Murphy’s
Law to the
Assessment
Apply Murphy’s
Law to the
Assessment
Crisis Management
Plan for a crisis by
exploring all options
before hand. Be
prepared to be
prepared.
Plan for a crisis by
exploring all options
before hand. Be
prepared to be
prepared.
Keep the Team
together at all times
and don’t allow
individuals to work
alone. One person
can’t win it for you.
Keep the Team
together at all times
and don’t allow
individuals to work
alone. One person
can’t win it for you.
Individuals
are not as
Strong as a
Team
Individuals
are not as
Strong as a
Team
Team Strength

Workshop
2010
? ?
?
Questions ?
?
? ?
?
?

Workshop
2010
–
Team Leader
External Coach
(if required or
asked)
Senior Team Member
Internal Coach
Senior Assessor Roles

Workshop
2010
Senior Assessor as Team Member
Review of Assessor Work
Provide Guidance to Team and
Lead when required
Provide Constructive Feedback
Back-up team leader

Workshop
2010
Internal/External Coach’s responsibilities
Provide feedback to team member
– planning documents
– team assignments
– correspondence
– call and meeting agendas
Provide support to Team Leader
Mentor/coach team lead and team
members
Communicate frequently

Workshop
2010
–– EmpathyEmpathy
–– NonNon--JudgmentalJudgmental
–– BelievableBelievable
–– Reflective Instructor/TeacherReflective Instructor/Teacher
–– Supporter of the learning processSupporter of the learning process
–– Observational FeedbackObservational Feedback
–– Experiential (Shares Experience)Experiential (Shares Experience)
–– Effective in different interpersonalEffective in different interpersonal
contexts; adjusting mentoring tocontexts; adjusting mentoring to
meet the needs of the menteemeet the needs of the mentee
Coaching Competencies

Workshop
2010
“Based on some of the obstacles listed on
the next slide as a leader/member
of a Business Excellence assessment team?”
Let’s Discuss –
“What can we do to prevent/ or
minimize the impact in encountering
these obstacles for a team?
Four Groups
Group Discussion

Workshop
2010
1. Unclear expectations – Assessor (s) ask repeated questions for understanding,
restates instructions incorrectly, or describes how it was done in the state program.
2. Lack of team member knowledge, skills, and ability – Assessor misuses terms,
submits a poorly written bio (e.g., grammar or organization is poor), or prefaces
comments with “I’m new”; Assessor is quieter and participates less than other team
members.
3. Technology breakdowns (e.g.. computer, Internet) – Assessor has repeated ISP or
server shutdown for maintenance; repeated computer malfunctions.
4. Lack of responsiveness – Team members are slow to respond to e-mail or phone
messages, miss initial deadlines, misplaces team notices, assessor gets behind in
responding to e-mails or voice mails, etc.
Early Warning SignalsEarly Warning Signals……..

Workshop
2010
Senior Assessor Competencies
Former skills and competencies are
essential …
but those skills alone will notbut those skills alone will not
make you a competentmake you a competent
Senior AssessorSenior Assessor

Workshop
2010
Advanced Skill Set for a Senior Assessor
….Someone who CARESCARES
–– ability toability to CCoach Effectively
–– isis AAgile and/or flexible in
face of change
–– hashas RReading Comprehension
–– capablecapable ofof EEnergizing a Team
–– reducesreduces SStress Management

Workshop
2010
Listening Skills…
– Must be able to carefully listen to the assessment
team and people within the organization being
assessed each time they talk as this is the key to
understanding where they are coming from.
– Maintain good eye contact - show genuine interest in
hearing them out.
• Enables you to uncover more information. The
more you obtain, the easier it is for you to help
identify their strengths and areas for improvement.
Providing Effective feedback
1. Scorebook Comments
2. Consensus
3. Site Visit
Coaching includes..

Workshop
2010
Group Discussion
What would you say (or write) to check
a team member’s rationale for a
comment that you don’t agree with or
that you think could be written better?
Discuss in small group for five minutes –

Workshop
2010
Example Coaching Responses….
to maintain self-esteem of team member
“Help me understand ....”
“I see that you did a good job (specifics) .... one
additional thing to consider would be ....”
“Your OFI comment addresses the specific approach,
but do you think it would add value to add a ‘so
what’? If so, what would that sound like?
“You’ve been working hard on these comments. One
comment that I need to check with you on is …. I’m
looking for a phrase that ....”

Workshop
2010
Senior Team Member
• Review
• Guidance
• Constructive Feedback

Workshop
2010
Agility Means… Leading MoreLeading More and Managing Lessand Managing Less
byby ……
Making sure the right things are being discussed,
evaluated, prioritized and delivered.
Making sure the big picture is always kept in sight.
Understanding the importance
of communication, conflict
resolution and influence
when it comes to getting
things done.
Changing direction when
things change around them ... as they do constantly ... to
make sure they are still heading for success.

Workshop
2010
Planning Skills…
– Ability to plan ahead
– Create a blueprint for the assessment that
contains every piece of information that you
need to gather and validate/verify and all the
activities that the team needs to do to promote
easy, fun learning. Doing this, you can make the
whole thing organized.
Managing Less Requires..Requires..

Workshop
2010
Team Leader Responsibilities
Planning
Managing the Process
Managing Scorebook
Production
Managing People

Workshop
2010
Planning
Tools
1. Wave Timeline
2. Proposed Timeline for Independent &
Consensus Review (visual & link)
3. Timeline: Stakeholder Roles and
Responsibilities (visual & link)

Workshop
2010
Assessment Timeline - Example

Workshop
2010
Reading and Comprehension
1. “Cover to Cover”- not beginning to
end –(i.e. - develop a reading
technique & order that works
effectively)
2. Based on information ability to link
process to results
3. Identify cross-cutting themes
and Key Business Factor’s
4. Validate responses to
Core Values and Concepts

Workshop
2010
EEnergizing a Teamnergizing a Team
1.1. Keep your wordKeep your word
2.2. Operate with integrity and respectOperate with integrity and respect
3.3. Make appropriate team assignmentsMake appropriate team assignments
4.4. DonDon’’t mislead or misrepresent thet mislead or misrepresent the
Award processAward process
5.5. Make the experienceMake the experience
Fun!!!!Fun!!!!
6.6. Create a learningCreate a learning
environmentenvironment

Workshop
2010
Obstacles to Team SuccessObstacles to Team Success
Lack of team member knowledge, skills, and ability
Technology breakdowns
Weather-related emergencies
Personal emergencies
Interpersonal conflict
Competing priorities
Changes in requirements
Team member added
Unclear expectations
Team member drops

Workshop
2010
How to Build an Effective TeamHow to Build an Effective Team
Stage 1 Stage 2 Stage 3 Stage 4
Empowering
Empowering
LeadingLeading
TeamTeamTeam Formation
BuildingBuilding
Team Formation CollaborationCollaboration SelfSelf--DirectionDirection

Workshop
2010
Managing the Process
Organizing People and Activities
Efficient Work Flow
Measurement
Effective Decision Making
Communications

Workshop
2010
Managing the People
Resources
Shared goals
Shared language
Shared process
Shared tools
Team diversity (sector, expertise, process
experience)
Established roles and responsibilities

Workshop
2010
1. Generate possibilities (Plan B and C)
2. Make choices decisively
3. Communicate clearly and regularly to maintain
focus
4. Manage your time first
5. Solve problems promptly
6. Improve team relationships
don’t let conflict fester/grow
Stress reducers….

Workshop
2010
Group DiscussionGroup Discussion
Using examples from your experience as
Assessors, in terms of a team member not pulling
their weight, what has worked well for you as a
Team Leader, or what have you experienced as a
team member, that has worked well to prevent
conflict on the team arising from this kind of
issue?”
Discuss in small group for five minutes –

Workshop
2010
Steps Required in the
Award Process
- Independent Review
Writing Process/Result
Comments
- Identifying Key Themes,
and Key Factors

Workshop
2010

Workshop
2010
Independent Review and
Key Factors

Workshop
2010
Scorebook Development Process

Workshop
2010
Analysis
“To analyze” is to
Divide a whole into its component parts.
Identify or separate ingredients of a
substance.
State the constituents of a mixture.
Examine, Study, Investigate, Break Down

Workshop
2010
Item Evaluation Process
Step 1 – Read the Criteria Item
Step 2 – Identify the 4-6 most important key
factors for the Item
Step 3 – Read the appropriate section of the
application
Step 4 – Generate a list of summary
notations

Workshop
2010
Worksheet Guidelines
Do
– Include 4-6 key factors based on the Criteria
requirements for the Item. These will differ
depending on the Item.
– Include the 6-10 comments per Item that are
most relevant and important to the applicant
based on its key factors.
– Ensure that the Item’s score is supported by
the 6-10 comments.

Workshop
2010
Process and Result Item
Comment Writing and
Item Backup

Workshop
2010
What comments should do:
Let the applicant know what it does well—and
what it needs to improve—specific to
equitable evaluation1. Criteria requirements
2. The particular organization meaningful feedback
3. Evaluation factors (ADLI and LeTCI) insights on
organizational maturity

Workshop
2010
Comment Guidelines

Workshop
2010
Summary Notation Essentials
Subject
from the
Criteria or
Application
+
Verb &
Descriptive
Language
from the
Criteria
+
Transition:
by/through
/using
+
Example from
application
methods/
activities
+ Scoring Guidelines language
+ so-what
+ grammar and punctuation check

Workshop
2010
Content Guidelines
Do
– Address central requirements of the Criteria.
– Use a single, complete thought to specify the
strength (using specific examples from the
application) or OFI (using specific omissions or
concerns identified from the application) clearly.
– Write process comments so they contain a
subject identified from the criteria, the application,
or the scoring guidelines, verb(s) and
requirements from the Criteria, examples from
the application, and figure numbers, as
appropriate.

Workshop
2010
Content Guidelines
Do
• Write results comments so they contain a
subject identified from the results or
Criteria requirement being addressed,
from/to timeframes, from/to performance
levels, linkages to KFs, so whats, and
figure numbers, as appropriate.
• State observations in a factual manner,
e.g., “Customer satisfaction rates have
increased from 75% in 2002 to 94% in
2004 and now exceed best-in-class levels.

Workshop
2010
Content Guidelines
Do
• Draw linkages across Items or
between an Item and the applicant’s
Organizational Profile.
• Place the comment on the correct Item
Worksheet based on the Criteria, not
on where the information appears in
the application.
• Ensure that the comment does not
contradict other comments in the same
or other Items or in the Key Themes
Worksheet.

Workshop
2010
Content Guidelines
Do not
– Go beyond the requirements of the Criteria or
assert your personal opinions.
– Be prescriptive by using “could,” “should,”
and “would.”
– Be judgmental by using terms such as
“good,” “bad,” or “inadequate.”
– Comment on the applicant’s style of writing or
data

Workshop
2010
Style Guidelines
Do
– Use a polite, professional, and positive tone.
– Use active voice and present tense (e.g., ‘completes’
rather than ‘is completed’).
– Use vocabulary/phraseology from the Criteria and the
Scoring Guidelines.
– For Stage 1 and 2 scorebooks, tell what is missing if
something “is not clear.”
– For Stage 3, clarify all “not clear” statements.

Workshop
2010
Style Guidelines
Do
• Use such words as “the applicant” or
“the organization” to refer to the
applicant in the Stages 1 and 2
scorebooks.
• Use the applicant’s name in Stage 3
scorebooks.
• Use the applicant’s terminology when
appropriate.
Do not
• Use jargon or acronyms unless they
are used by the applicant.

Workshop
2010
Comments - General
Item-Specific Key Factors Should Be!
Hunting for OFI’s? Score High!
Don’t Get Stuck in “Microview” (Question by
Question)
Use Criteria and Scoring Guidelines for Comments

Workshop
2010
Process

Workshop
2010
Evaluating Process Items
Approach
Deployment
Learning
Integration
ADLI

Workshop
2010
Using ADLI to Evaluate Process
Items–Module Objectives
1. Use the approach, deployment, learning, and
integration (ADLI) evaluation factors while
assessing an applicant’s responses to Process
Items.
2. Use the ADLI evaluation factors throughout the
Independent Review phase of the Award
process.

Workshop
2010
A-D-L-I Evaluation Factors
– Approach
• The methods used to accomplish the
process
• The appropriateness of the methods to
the Item requirements
• The effectiveness of your use of the
methods
• The degree to which the approach is
repeatable and based on reliable data
and information (i.e., systematic)

Workshop
2010
– Deployment
• Your approach is applied in
addressing Item requirement relative
and important to your organization
• Your approach is applied consistently
• Your approach is used by all
appropriate work units

Workshop
2010
– Learning
• Refining your approach through cycles
of evaluation and improvement
• Encouraging breakthrough change to
your approach through innovation
• Sharing of refinements and innovation
with other relevant work units and
processes in your organization

Workshop
2010
– Integration
• Your approach is aligned with your
organizational needs identified in other
Criteria Item requirements
• Your measures, information and
improvement systems are complementary
across processes and work units
• Your plans, processes, results, analyses,
learning, and actions are harmonized
across processes and work units to support
organization-wide goals

Workshop
2010
Process Comments - General
Use single complete thought
1-3 complete sentences
Address central Criteria requirement
Relevant and important to applicant based on the Key
Factors
Non-prescriptive
Non-judgmental
Draws linkages to the Organizational Profile or Other
Items
Does not conflict with other comments

Workshop
2010
Process Comment General
Guidelines
Comments should be
– Criteria-based
– Relevant
– Clear
– Concise
– Actionable

Workshop
2010
Process Comment General
Guidelines
Comments should not
– “Parrot” the application
– Be prescriptive
– Be judgmental
– Conflict with one another

Workshop
2010
Basic Format for Process Item
Comments OFI
Intro. Statement Subject Verb and Implied
Descriptive or explicit
It is not clear from the language "so what"
how/apparent Criteria,
application, from the related to
The applicant does or scoring Criteria KFs or
not report/address guidelines Criteria
how
Adhering to content and style guidelines
+ + +

Workshop
2010
Process Comment Example
l Applicant determines the student segments and markets that its education programs
will address through an effective, systematic, 4-step Segmentation Process. That
process, managed by the Research and Knowledge Management Department,
involves: 1) gathering, analyzing, and integrating information and data using a variety
of listening/learning methods (Fig. 3.1-3); 2) validating existing segments and
identifying emerging segments and market opportunities; 3) communication final
segmentation throughout the district; and 4) organizing information, data and
knowledge by segment. The segmented information is used to meet organization,
information, and NCLB requirements and is integrated with the district’s
organizational needs identified with other Criteria categories, e.g., strategic
planning (Figure 2.1-1), relationship management, and curriculum design (Figure 6.1-
1).

Workshop
2010
Basic Format Example: Strength
Senior leaders use a variety of methods (Figure 1.1-2) to
communicate values, directions, and expectations to
faculty, staff, partners, and stakeholders. These methods,
as well as a five-step communication process, were
developed through a PDSA cycle by the district
Communication Team. In addition, as a result of Baldrige
self-assessments and benchmarking best-in-class
communication methods, the team has implemented
multiple improvements, including adding two-way
communication methods, developing a Communication
Methods Manual, integrating “communication effectiveness”
into the PMP, and, most recently, integrating
communication liaisons within the SICs.

Workshop
2010
Basic Format Example: OFI
o While the applicant has a Culture Change
Process to nurture an environment conducive to
student and district excellence, it is not clear how
this process creates an environment that fosters
and requires legal and ethical behavior. This may
be particularly important to the district given the
highly regulated environment described in its
Organization Profile.

Workshop
2010
Process Comment
To carry out its strategic planning, the organization uses a
systematic, biennial 12-step Strategic Planning Process
(Figure 2.1-1) that involves community leaders, volunteers,
member agencies, and donors. The organization determines
its strategic challenges in the “Current State” step, and it
includes inputs from the SWOT Analysis and Environmental
Scan. The SPP was initiated in 1997 and restructured in
2004 as a result of benchmarking analyses of food banks
and other nonprofit organizations.

Workshop
2010
Comment Diagnosis Activity
Step Time
1. As a triad, review the comment assigned and identify the
criteria elements contained within the comment and
determine whether there are any problems with the
comment. Ask
a. What is wrong?
b. What is the evidence?
Hint: Use the Comment Guidelines and the Criteria as
tools.
5 min.
2. Review some of your triad’s findings. 10 min.

Workshop
2010
Find the Elements of a Comment
XYZ uses the Comprehensive Complaint Management System (CCMS) (Figure 3.2-3) to
log and track the resolution of customer complaints received through personal
conversations, phone calls, e-mails, or survey responses. Data entered into the system
are analyzed for root causes to determine a solution and are communicated to the
appropriate departments, with additional training provided as necessary. These data are
reviewed monthly at corporate performance meetings and again during year-end strategic
planning. As the result of these reviews, XYZ improved the process in 2007 with the
addition of the Handle It Now process to allow all frontline customer service staff members
to resolve complaints on the spot. This process has been implemented in all 25 XYZ
branches.

Workshop
2010
XYZ uses the Comprehensive Complaint Management System (CCMS) (Figure 3.2-3)
[Approach] to log and track the resolution of customer complaints [Criteria]
received through personal conversations, phone calls, e-mails, or survey responses.
Data entered into the system are analyzed for root causes to determine a solution and
are communicated to the appropriate departments, with additional training provided as
necessary. These data are reviewed monthly at corporate performance meetings and
again during year-end strategic planning. As the result of these reviews, XYZ improved
the process in 2007 with the addition of the Handle It Now process [Learning] to
allow all frontline customer service staff members to resolve complaints on the spot.
This process has been implemented in all 25 XYZ branches. [Deployment]

Workshop
2010
Completing the Item Worksheet
Identify each comment by its Area to
Address
Identify the key factors that link to the
comment
Rate comments with a +, ++, -, or –
Review the comments as a final check

Workshop
2010
Results

Workshop
2010
Results Evaluation Essentials
Factors to consider as you evaluate results:
– Gaps – missing information
– Performance levels
– Segmentation
– Trends
– Comparisons
– Breadth of performance improvements
– Linkage of results measures to important
organizational performance results

Workshop
2010
Evaluating Results Items
Levels
Trends
Comparisons
Integration
LeTCI

Workshop
2010
Results: Performance Analysis
– Performance Levels (Le)
• Refer to numerical information that
places or positions an organization’s
results and performance on a
meaningful measurement scale.
Performance levels permit evaluation
relative to past performance,
projections, goals, and appropriate
comparisons
• Are key results missing?
• What levels are provided?
• Is the measurement scale meaningful?

Workshop
2010
Trends (T)
– Refer to numerical information that shows the direction and
rate (slope of trend data) and breadth (how widely deployed
and shared) of performance improvements. A Minimum of
three data points generally is needed to begin to ascertain
a trend. More data points are needed to define a
statistically valid trend.
– Are trends provided?
– Is the interval between measures or frequency appropriate?
– Are the trends positive, negative, or flat?
– What is the rate of change (slope of the trend)?

Workshop
2010
Comparisons
– Refer to establishing the value of results by their relationship
to similar or equivalent measures. Comparisons can be made
to results of competitors, industry averages, or best-in-class
organizations. The maturity of the organization should help
determine what comparisons are most relevant.
– Are comparisons provided?
– Are the comparisons to an industry sector average, key
competitors, or best-in-class organizations?
– How does the applicant compare against other organizations?

Workshop
2010
Linkage (Li)
– Refers to a connection to important customer, product
and service, market, process, and action plan
performance requirements identified in the
Organizational Profile and in Process Items
– To what extent do results link to key factors and Process
Items (e.g., important customers/patients/ students,
product and service, market, process, and action plan
performance requirements)?
– Are results segmented to help the applicant improve
(e.g., by customer, patient, or student segment;
employee type; process/education program or service)?

Workshop
2010
Using LeTCI to Evaluate Results
Items—Module Objectives
1. Identify the levels, trends, comparisons, and
integration (LeTCI) evaluation factors for Results
Items.
2. Use the LeTCI evaluation factors to effectively
assess an applicant’s responses to Results Item
requirements.

Workshop
2010
Results Comment: A Sample
While Figure 7.3-1 (Average Tuition and Fees)
indicates that NCON tuition costs have risen from
an average of $4,000 in 2005 to $5,800 in 2008,
the level of increase has remained below that of
peers 1 and 2 since 2005. This aligns with the
fiscal Health point of LIGHT and the principal
success factor of moderate tuition rates.

Workshop
2010
ABC provides no comparative data for workforce results relative
to workforce engagement and satisfaction. Without appropriate
comparative data, ABC may have difficulty addressing its strategic
challenges of a shortage of qualified licensed professionals and
the influx of private urgent-care centers.

Workshop
2010
ABC provides no comparative data for workforce
results relative to workforce engagement and
satisfaction [Criteria]. Without appropriate
comparative data [comparisons], ABC may have
difficulty addressing its strategic challenges of a
shortage of qualified licensed professionals and the
influx of private urgent-care centers [applicant-
specific].

Workshop
2010
Activity: Comment Writing
Practice
1 As a table, select the core idea that will be the subject of
the comment.
1 min.
2 Individually write the comment. 14 min.
3 In a round-robin, read your comments to your tablemates
and identify the elements you used.
12 min.
4 Select a comment to scribe on chart paper and share. 3 min.
30 minutes

Workshop
2010
Scoring: Process Item

Workshop
2010
Scoring
Key Scoring Terms
– Systematic approach
– Deploy/deployment
– Evaluation and improvement
– Align/alignment
– Learning
– Integration

Workshop
2010
Steps in the Scoring Process
1. Review Criteria requirements and summary notations
2. Note balance and importance of strengths and OFIs
3. Read the description of the scoring ranges in the
Scoring guidelines
4. Determine the scoring range most descriptive of the
organization’s achievement level
5. Read range descriptions above and below to
determine where, within the range, the score would fall
6. Verify that the balance and substance of strengths and
OFIs correspond to the score
7. Record the score

Workshop
2010
Issues Leading to Greatest Scoring
Variability
Scores not adequately related to the key
factors or Scoring Guidelines
Acceptance of applicant statements
Feed of scoring above 50%
Using the Areas to Address and Item Notes
as a checklist
Treatment of missing information

Workshop
2010
“Best Fit” Considerations
Not all comments will fit into one scoring band
Some comments are more important – remember
Key Factors
What is the balance of A-D-L-I comments?
Being on the border between 2 scoring bands is
typical – and okay
Select a score (5% point increments) that most
suitably reflects the applicant’s performance to the
criteria for that item ONLY

Workshop
2010
More “Best Fit” Considerations
Don’t Average
A good approach can get you to 50% but lack
of deployment could stop you from moving any
further
Approach has more influence in the lower
bands and deployment has more influence in
the mid-bands
Learning and Integration have more influence
in the upper bands

Workshop
2010
Rating Comments
Strengths
+ Baseline observation
++ Particularly important observation that has
significant impact on applicant’s performance
management system, potential benchmark, or
key theme significance.
Opportunities for Improvement
Baseline observation
Particularly important observation that has significant
impact on applicant’s performance management system or
key theme significance

Workshop
2010Baldrige National Quality Program
Comment-Writing Activity
Aligning Scoring Ranges with
Comment Language

Workshop
2010
Itches and Scratches
Performance Projections
“If you don’t know where you are
going, any road will take you there.”
Lewis Carroll

Workshop
2010
l Now Mentioned in Every Scoring Range
l Not a Significant Factor until 50–65% Range
l “Clear” Rationale in Top Two Ranges

Workshop
2010
Are They Present?
Do They Make Sense?
Are They Related to Strategic Objectives,
Competitive Performance, and Customer
and Stakeholder Needs?
We Are Not Examining “Absolute”
Goodness.
We Are Examining Relevance.

Workshop
2010
Evaluating Financial Performance
l Most Trends and Actual Performance May Be
Negative This Year.
l This Alone Does Not Result in a Low Results
Item Score.
l Review Results in Context of Overall
Economy and Competitors’ Performance.
—What Is the Organization’s Comparative
Performance?
—How Is It Adjusting?

Workshop
2010
What have we learned?
Day 1 Recap and What’s Next

Workshop
2010
Day 1 Key Outcomes and
Learning
l Assessor Skills and Competencies
l Senior Assessor Skills and Competencies
l Senior Assessor Roles
l Advanced Assessor Skills
l Independent Review and Key Factors
l Key Themes: First Look
l Comment Writing using ADLI and LeTCI

Workshop
2010
Reflection
Take a few moments to reflect on your personal learning from
today’s session, and make a few notes on your next steps.
1. What have you learned regarding your
strengths?
2. What did you discover that may be
opportunities for your development?
3. How will you address your opportunities for
development?

2010 tqa assessor workshop by thomas schamberger day one

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to 2010 tqa assessor workshop by thomas schamberger day one

Similar to 2010 tqa assessor workshop by thomas schamberger day one (20)

More from Areté Partners

More from Areté Partners (14)

Recently uploaded

Recently uploaded (10)

2010 tqa assessor workshop by thomas schamberger day one