Post-Occupancy Evaluation (POE) Methodologies for School Facilities

Post-Occupancy Evaluation (POE)
Methodologies for School Facilities

Andre A. Larroque, AIA, CSI, CCCA, CCS, REFP
Building Standards Coordinator
u d g Sta da ds Coo d ato
New Mexico Public School Facilities Authority
Marcel Harmon, PhD, PE, LEED-AP O+M
Sr. Associate / Applied Anthropologist
M.E. GROUP
M E GROUP
Nate Maniktala, LEED-AP BD+C, MBA
Principal | Vice President
M.E. GROUP

New Mexico Public School Facilities Authority

GENUINE | PROGRESSIVE | ADVOCATES

POE CASE STUDY

Overview
Methods
Recommendations

POE DEFINITION
NMPSFA POE Handbook Draft
db k f

“The systematic investigation of a Facility to determine the
success or failure of one or more design elements within the
project to be performed by a third‐party Evaluator after the User
has occupied the Facility.”

POE PURPOSE
NMPSFA POE Handbook Draft:

Key Purpose of the POE: “…to investigate, analyze, qualify,
and report on the successes and weaknesses of the school
and design for future replication or repair.”

Some Benefits:
• Improve statewide design & sustainability guidelines.
• Improve state processes for project delivery & management
• Recommend remedies for specific building deficiencies
• Recommend behavioral changes to improve M&O

V. Sue Cleveland HS
Rio Rancho Public Schools, New Mexico

2010 CEFPI James D. MacConnell Award Finalist

BUILDING DATA
418,000 s.f.
2,350 Students
$120 M Construction
$11.5M FF&E + Tech
7 Academies

PROJECT GOALS
From VSCHS Program

PLANNING GOALS LEARNING GOALS
• Strong career pathways • Safe and secure setting
• High academic standards • Commons area for whole school
• Engage every learner
Engage every learner • Variety inclusiveness and
Variety, inclusiveness, and
• Access to up‐to‐date technology flexibility
• Academy organization & concepts • Special use spaces
• Engage with community
ih i • Integrated information
Integrated information
• Utilize community resources technology
• Safe school environment • Computer access for all students
and staff
and staff
• Collaborative teaching & learning
• Personalize learning LEED Silver
• Advance potential of block
Advance potential of block
scheduling.

POE SCOPE
NMPSFA Hi‐Performance (HiP) School Pilot Projects
Analyze Overall Process and Design Approach
y g pp
Evaluate Effectiveness of Design Tools & Resources.
Evaluate Impact of Design on Owner/Users

Photo Rights: Patrick Coulie Photography

POE APPROACH
NMPSFA coordination with District & School
Kick‐off Meeting with evaluators and key participants
Lead‐Off Survey (LOS)
• On‐line Survey identifies major trends prior to field work
On line Survey identifies major trends prior to field work
On‐Site Investigation (OSI)
– Methodology flexible depending on nature of project and professional
recommendations of Evaluator.
recommendations of Evaluator
• Archival research
• Interviews
• Observations
• Energy performance measurements
• Photography
g p y
• Others as agreed to

BUILDING SIDE METHODS
Individual Space Assessments

Trended Data
• Via data loggers and BMS


Instantaneous Measurements
• Via hand‐held meters & cameras

Computer Room A2101: Space Temperature, Light Intensity & Humidity
e (Degrees F)
Temperature
T )

Date and Time (Monday , February 28 thru Saturday , March 5)


Workroom /
Conference Room:
C f R
Hours lights “ON”
when space is
p
unoccupied.
Hours Per Day when lights are ‘ON’ when space is unoccupied (Hours)
For time period measured, the lights were enabled ‘ON’ an average of 28.05% when the space was unoccupied.

Carbon Dioxide (CO2) Measurements: Classroom A2100
Unacceptable Range >1000 PPM
m)
tration (ppm
CO2 Concent

OCCUPIED CLASSROOM: 12:40 PM‐ 1:32 PM
C

Acceptable Range < 800 PPM

Time of Day
Analysis: CO2 Levels are acceptable throughout a normally occupied class time.

O&M Interviews / Assessments

How general tasks are performed
How the BMS is used
How O&M interacts with the building occupants


VSCHS Example Survey Comments
• “teachers’ restrooms not always cleaned each night;”
• “inadequately salted/sanded icy parking lots during inclement weather;”
• “chairs leaving marks on the floor (and not cleaned);”
chairs leaving marks on the floor (and not cleaned);
• “selection of exterior trash cans of poor design (and possibly more needed);”

VSCHS Policies / Procedures Issues
• Potential Knowledge Gap
‐Building operating plan
‐Custodial “best practices” manual
Custodial best practices manual
‐Common repository of documentation on‐site
• Streamline Work Order Process
• Adequate Staffing

VSCHS Training/Education – O&M and Occupant
• Recycling Policies/Procedures
• Classroom technology
• Blinds/shade use
Blinds/shade use
• LEED/sustainable educational material
• Automatic drinking fountains
• O&M reference repository in building
f b ld
• Custodians best practices manual

Utility Analysis – Energy Model Assessment

Comparing the modeled
annual energy consumption
with actual usage.

Provides a means to
evaluate:
•Modeling assumptions
•Modeling procedures
•Modeling software

Utility Analysis – Energy Model Assessment
Discrepancies
• Actual end use load break out is unknown – discrepancy could be
substantially due to process / plug loads and not HVAC, lighting or
domestic hot water systems.
• Scheduling and weather data inaccuracies
• Changing rate structures
• Inaccurate energy source allocations
Inaccurate energy source allocations
• Unforeseen/unanticipated uses (related to the first bullet point)
• Building system operational issues and/or
occupant usage issues
t i

This emphasizes the importance of installing
This emphasizes the importance of installing
adequate metering for measurement & verification

Utility Analysis – EUI Comparisons
70 66 64
59
60
50
41
40
32
30
EUI
20
10
0
Design Actual 2012‐ Target Finder Peer Group Peer Group
(energy 2011 Average Baseline Low Baseline High
model)

In this case Energy Star’s average building baseline is a close
approximation of the peer group baseline

Design / Construction Assessment

• Review documentation
• Examining the facility’s performance
• Survey project team members
• Interpret occupant side data
Interpret occupant side data


Community Involvement
• Helps identify community values
• Aides in establishing community buy‐in
• Generally successful for Sue Cleveland
Generally successful for Sue Cleveland
‐Rated just under “well” (3.95), on a scale of 1 to 5

Administrative/Teacher/Staff Involvement
• Perception is that this was more heavily weighted towards
administrative staff
• Involve teachers, staff and students who will actually be using the
school
• Solicit O&M staff involvement
• Consider ethnographic type evaluations as more the norm than
g p yp
exception

Overall Pluses and Minuses
+ High quality information provided to design team
+ Qualifications based selection process

+/‐ Commissioning Process
+/ Commissioning Process
+/‐ VE Process

‐ District’s facility standards could more
District s facility standards could more
specifically address performance
requirements
‐ Multiple funding sources
Multiple funding sources
‐ Schedule (design completed after
construction had started)

Different Perspectives of VE

Ad Hoc Solutions

Visibility Issues

OCCUPANT SIDE METHODS
Behavior & Environment

Bldg Design/Operation

Occupant Needs Drive Occupant Behaviors
Comfort
Space Heaters
Function
Extension Cords
Extension Cords
Social Value
Recycling
Meaning
Helping Others

Thermal Comfort Example

• Activity Levels
• Clothing Insulative
Properties

• Air Temperature
Air Temperature • Air Movement / Velocity
Air Movement / Velocity
• Mean Radiant • Relative Humidity
Temperature

Ethnography
Analyses of human interactions in a defined space and time

Important of Context

Influence of Perspective

Social/Cultural Psychological

Physiological

Different factors provide the lens through which we perceive
the world around us
the world around us

Perception: Designer/Occupant Gap

Nasar, J. L., A. E. Stamps III, and K. Hanyu
, , p , y
2005. Form and Function in Public Buildings. Journal of
Environmental Psychology 25:159‐165.
– R id t i T k M t l d C l b Ohi
Residents in Tokyo, Montreal and Columbus, Ohio
were shown pictures of city halls, libraries, art
museums, and live theaters in various cities in the
San Francisco region. They were asked to guess each
S F i i Th k d h
building's function.
– Correct 32% of the time, slightly better than the 25%
possible from random guessing.

It’s a what?

a d

b e
Museum,
City Hall,
Live Theater,
Library

c f

g j

h k
Museum,
City Hall,
Live Theater,
,
Library

i l

Perception: Designer/Occupant Gap

a) Alameda City Hall, g) Daly City Library,
b) El Cerrito City Hall,
El Cerrito City Hall h) Alameda Library,
Alameda Library
c) San Leandro City Hall, i) El Cerrito Library,
d) Museum of Modern Art, j) Cowell Theater,
Theater,
e) Heritage Architectural
History Museum, k) Beach Blanket Babylon
Theater,
f) Ansel Adams Museum of
Adams Museum of
Photography, l) ODC Theater

Field Methods

Photo Rights: robertoerosalesblog.com

Create a Plan
• Form a team, including as many stakeholders as possible
• Create a General Plan
– Brainstorm target groups, target systems/equipment, target locations,
target activities/tasks, questions, and assumptions
– Remove or revise items participants cannot answer
– Remove or revise “personally‐biased” questions
– Organize remaining items. Create categories and groups
Organize remaining items. Create categories and groups
• Review the General Plan with the team
• Finalize a path for your ethnographic work

Surveys
Question: Are any of the following conditions true for the lighting in your
primary classroom(s) or work area(s)? Please check all that apply.

90% (1) The light fixtures provide enough light by
themselves.
80% (2) I can sometimes work by daylight only.
(3) Reflected images of light fixtures appear on
70% my work surface, PC monitor, and/or on the
1
smart board/white board/chalk board.
60% 2
(4) Reflected images of the windows/skylights
3 appear on my work surface, PC monitor,
appear on my work surface PC monitor
50% 4 and/or on the smart board/white
board/chalk board.
5
40% (5) Light fixtures create glare in my field of
6
vision as I work.
30% 7 (6) Windows/skylights create glare in my field
8 of vision as I work.
20% 9 (7) Direct sunlight falls on my work surface, PC
monitor, and/or on the smart board/white
10% board/chalk board, washing it out and
/ , g
making it difficult to see what I’m working
0% on.
Teachers Admin. & Staff Total Teachers & Staff (8) The light fixtures produce an audible hum.
(9) The light fixtures noticeably flicker.

Focus Groups

Focus Groups
Occupants assume the role of designer
– What they would “keep”
– What they would “change”

Conducted earlier in field work
– Help focus subsequent interviews /
observations
– Help focus subsequent space
condition assessments

Focus Groups
Top “Keep” Items
No. of Participant
No. of Participant
Code Category Categories
Votes Categories
g
Represented
R t d
K1 Commons Areas 14 S, P and C 3
K2 Stadium and stadium design 12 T, S and P 3
K3 Computer labs in library 9 S, P and C 3
Smart boards and other
K4 technology
gy 9 T and S 2
K5 View of mountains 8 T and C 2
K6 Concert Hall 7 S 1
K7 Windows and Natural Light 6 T, S, SA, P and C 5
K8 Layout/openness of library 6 S and C 2
K9 Library extra rooms/classrooms 6 C 1

Top “Change” Items
No. of Participant No. of Participant
Code Category
Votes Categories Categories Represented
Long Distances to Traverse Across
C1 Campus 22 T, S, SA and C
T S SA and C 4
C2 Better control of daylight 21 T, S, P, and C 4
C3 Improve Security/Control 18 T, S, SA, P, C 5
C4 Inadequate Parking 18 T, SA, P and C 4
Revise Thermostat and Call Button
C5 Locations in Classrooms
Locations in Classrooms 13 T, SA, P and C
T SA P and C 4
White Boards ‐ Add and Revise
C6 Locations 13 T, S and P 3
Improve Temperature
C7 control/thermal comfort 11 T, S, and C 3

Observations & Interviews
Methods / Techniques
– Person‐Oriented Record
– Setting‐Oriented Record
– Task Oriented Record
– Desk Tour
– Shadowing
– Student Day in the Life
Student‐Day‐in‐the‐Life

Observations
Things to look for during formal and impromptu observations

Observations

Interior Classroom
Windows

Facilitate Impromptu
Teacher Discussions

Focus Group Exercise

INTEGRATED ANALYSIS
Thermal Comfort – V. Sue Cleveland HS
Survey Responses
• Teachers, Staff and Administrators: Spaces are only thermally
Teachers, Staff and Administrators: Spaces are only thermally
acceptable (ASHRAE definition) during the fall (ranges from 66% ‐ 75%
for other three seasons)
• Students: Not thermally acceptable during any season (ranges from
Students: Not thermally acceptable during any season (ranges from
32% ‐ 51% for all four seasons).

INTEGRATED ANALYSIS
Thermal Comfort – V. Sue Cleveland HS
Complaints centered on:
• Extremes of too hot or too cold (majority being too cold)
Extremes of too hot or too cold (majority being too cold)
• Swings between too hot and too cold over the course of the day in
the same space;
• Large temperature deltas within the same space at the same time
• Delayed system response and/or system overcorrection
• Student access to thermostats

INTEGRATED ANALYSIS
Thermal Comfort – Data Logger Correlations
Temperature deltas:
• Temperature gradients measured within12 rooms/spaces
Temperature gradients measured within12 rooms/spaces
• Ten (83%) had maximum temperature deltas of 5°F or greater
• Seven (58%) had average temperature deltas of 2°F or greater.
• Measured temperature deltas would sometimes last for several hours
at a time, spanning multiple class times.

Temperature fluctuations:
• Confirmed fluctuations in temperatures of 2+°F within a timeframe of
Confirmed fluctuations in temperatures of 2 F within a timeframe of
15 minutes or less in several spaces corresponding to occupant
complaints.

INTEGRATED ANALYSIS
Thermal Comfort – Acoustics Connection
Unoccupied Mode
• BAS was set to “unoccupied” for the first floor high ceiling
BAS was set to unoccupied for the first floor high ceiling
classrooms without a lay‐in ceiling, disabling the HVAC system due
to noise.
• Contributed to temperature measurements outside of the 68 °F
Contributed to temperature measurements outside of the 68 F‐
74°F range 41% ‐ 51% of the time in two of the spaces measured.

INTEGRATED ANALYSIS
Thermal Comfort – Current HVAC Design Standards
Limitations
• ASHRAE Standard 55 2004 and the ISO 7730 Standard for Moderate
ASHRAE Standard 55‐2004 and the ISO 7730 Standard for Moderate
Thermal Environments are based on experimental studies of adults,
not children.
• New ‘adaptive’ models of thermal comfort have not been
New adaptive models of thermal comfort have not been
incorporated into school HVAC systems standards. Student
metabolic rates vary as they move between rooms or activities.
• HVAC system design focuses almost exclusively on the thermal and
HVAC system design focuses almost exclusively on the thermal and
humidity specifications as directed by building codes. Internal
mixing, air velocities, and vertical temperature gradients are rarely
addressed.
addressed

Committee to Review and Assess the Health and
Committee to Review and Assess the Health and
Productivity Benefits of Green Schools (National
Research Council, 2006:65‐66)

INTEGRATED ANALYSIS
Thermal Comfort – Student & Adult Differences
Benefits of Personal Control
• Surveys focus groups and interviews: Expressed strong
Surveys, focus groups and interviews: Expressed strong
appreciation of and desire for local temperature control .
• Productivity increases with increasing personal control over
temperature and ventilation.
temperat re and entilation
• Having control mitigates some of the negative perceptions
associated with non‐optimal HVAC system performance
• “being able to monitor my own classroom temperature
… is GREAT!!!”

INTEGRATED ANALYSIS
Correlation Between Temperature Control and Thermal Comfort
Ratings
• Spearman’s Rho test: found a positive correlation between adult
temperature control effectiveness ratings and their thermal
comfort ratings.
comfort ratings
• Contributing factor for the more positive adult thermal comfort
ratings.

INTEGRATED ANALYSIS

INTEGRATED ANALYSIS
Clothing Variability Signage

Computers

T‐Shirt Logos

Messages using images,
graphics, text, student Flat
generated videos displaying /
/ Screens
promoting desired behavior

Spread using existing school
Spread using existing school
means or others – be
creative and relevant to your Smart Phones
audience

RECOMMENDATIONS
Lighting Controls ‐ Consistency
• Use the same type of lighting controls in the same types of rooms.
• Interviews & Observations: Inconsistent location of occupancy
Interviews & Observations: Inconsistent location of occupancy
sensors leads to lights being left on in the rooms without occupancy
sensors.
• Dataloggers indicated that lights were being left on in unoccupied
indicated that lights were being left on in unoccupied
spaces 15% of the time for the sample of spaces measured.

RECOMMENDATIONS
Improve Classroom Acoustics
• Revisit the design of high ceiling spaces to improve their acoustics.
• Will likely require combination of alternate finishes strategically
Will likely require combination of alternate finishes, strategically
located sound absorbing panels/materials, ensuring airflow rates do
not exceed diffuser sound ratings, and the input of an acoustical
engineer.
engineer
• Negative impact on intelligibility / comprehension and thermal
comfort.

RECOMMENDATIONS
Improve Value Engineering Process
Estimate the resulting impacts on a) productivity/performance and
health, b) operations and maintenance, or c) other building systems.
health b) operations and maintenance or c) other building systems
• Value engineering north side shades limited the ability to control glare
and allow dimming for presentations.
• The se of these classrooms as instr ctional spaces as compromised
The use of these classrooms as instructional spaces was compromised.
• Glare was estimated to be a) costing the school $169,457 in lost
teacher/staff productivity annually and b) negatively impacting student
math scores by ‐1.50% and reading scores by ‐0.91%.
th b 1 50% d di b 0 91%

RECOMMENDATIONS
Qualification Based Selection (QBS) Process
• Deemed a success for the selection of contractors. It focuses on
quality and value, as opposed to price.
quality and value as opposed to price
• It rewards teamwork and proactive behavior.
• It elevates construction to a service profession.
• It helps ensure that the facility meets the original design intent
and owner expectations.

IMPLEMENTATIONS
Actions Taken by NMPSFA
• Modify or re‐affirm standard planning guidelines for:
• Glare & daylighting
Glare & daylighting
• Acoustics
• Energy modeling.
• Building envelope design;
• Room occupancy sensors;
• Space recommendations (e.g., Commons areas in high schools)
Space recommendations (e.g., Commons areas in high schools)
• PSFA planning &project management improvements;
• Fine‐tune and finalize the POE Handbook draft;
• Continue to assess the value of LEED certification;
• Contribute to a new data base of statewide facility attributes;
• Potentially correct some building deficiencies at VSCHS;
y g ;
• Assist the District in making adjustments and improvements in
building operations including maintenance and user education.

MOVING FORWARD
Actions to be Taken by NMPSFA
• Complete POEs on five remaining HiP schools
• Consider doing POEs on all significant state funded school
Consider doing POEs on all significant state‐funded school
construction projects.
• Continue incorporation of lessons‐learned into design and process
guidelines.
g idelines
Hurley Elementary, Hurley, NM

Lindsey‐Steiner Elementary, Portales, NM

Post-Occupancy Evaluation (POE) Methodologies for School Facilities

Recommended

Recommended

More Related Content

What's hot

What's hot (20)

Viewers also liked

Viewers also liked (20)

Similar to Post-Occupancy Evaluation (POE) Methodologies for School Facilities

Similar to Post-Occupancy Evaluation (POE) Methodologies for School Facilities (20)

More from Marcel Harmon

More from Marcel Harmon (13)

Post-Occupancy Evaluation (POE) Methodologies for School Facilities