The goal of this study was to evaluate the usability and user-experience of a new type of standing workstation-system to be used in the context of an Educational Makerspace.

1. Leonardo A. Madariaga
MS Human Factors Engineering
Committee:
Daniel Hannon - Mechanical Engineering-Tufts University
Chris Rogers - Mechanical Engineering - Tufts University
Ethan Danahy - Computer Science- -Tufts University
of the physical environment
in Makerspaces:
exploratory study
Implications
for
the
design
and
user
evaluation
of
a
new
workstation-­‐system
April 2015

2. Discussion
Question
Background
Method
Results
Agenda
Exploratory Study of the Physical Environment in Makerspaces: Implications for the design and
user evaluation of a new workstation-system

3. Problem solving and innovationBackground
Top Down
Bottom Up
People/Places
Events
Vision
Resources
Technologies

4. Some examplesBackground
(Photo by Jonathan Dietz, used with permission)
MIT Museum Makerspace

5. Background Some examples
(Photo by Jonathan Dietz, used with permission)
Malden High School

6. Background Some examples
(Photo by Jonathan Dietz, used with permission) NUVU Studio

7. Background
(Photo by Jonathan Dietz, used with permission)
Some examples
NUVU Studio

8. Background
(Photo by Author)
Some examples
Artisans Asylum

9. “Makerspacessupportlearningin
aninformal,play-focused
environmentthatcancultivatean
interestinscience,technology,
anddesign.”
Scott Nicholson, associate professor in the
School of Information Studies, Syracuse
University, NY, 2012
“Despite a flurry of interest and activity
around designing and creating
Makerspaces, we still know little about
the content and processes of learning
in Makerspaces”
Hakverson & Sheridan
HARVARD EDUCATIONAL REVIEW, 2014
Community
Shops
Library
Makerspaces
Museum
Makerspaces
Home
Shops
Hobby
Clubs
Commercial
Makerspaces Making
Educational
Makerspaces
Machine
Wood
Shops
Science
Labs
Computer
Labs
Collaborative
spaces
Video
Conference
Design
Thinking
Educational
50´s 60´s 80´s - 90´s 00´s
A brief timelineBackground

10. Market analysis: WorkstationsBackground
Locus
Standing Desk
US$1400
Ergotron
WorkFit-D
Sit-Stand
Desk
US$670
Jesper Office
Height-adjustable
Standing Desk
US$399
Refold
Cardboard
standing desk
$250
D-School/ Stanford University
Periodic
Table
US$493

11. Market analysis: Storage & Organization systemsBackground
Really Good
Student Bin
Organization
rack
US$180
i-Rover
IPT102 7
Device
Tech Cart
US$320
D-School
Stanford
University
Mobile
Storage
Unit
US$923
Copernicus
3D PRINTING
Station
US$486
Copernicus
STEM
MAKER
Station
US$599

12. Paper/Book Author Findings Physical Environment/Usability
The Philosophy of Educational
Makerspaces Part 1 of Making an
Educational Makerspace
Kurti, Kurti and
Flemming (2014)
Flexible by providing open spaces with elements that can capture users attention the first
time they visit a space
A Design Case Examining
Learning in the d-lab
Leigh et al. (2013)
Transparency achieved through visual and acoustical “access” to activities
Enhanced interconnectivity and socialization spaces inviting collaboration
Spatial plasticity through the movability of furnishings
A Multi-disciplinary Design
Environment
Carlson and Sullivan
(2006)
Providing team storage lockers to organize tools and projects supplies. Larger and
flexible storage areas would be desirable features.
Architectural concepts, such as flexibility and visibility, should inform most of the design
decisions of design lab.
Development of a Multidisciplinary
Engineering Learning Center
Batill and Gedde (2001)
A flexible and adaptable space. (b) Work areas and benches for project fabrication tools,
test and assembly. (c) Storage space for students (d) Movable, student operated
equipment (e) Project development and set-up area
Ergonomics product development
of a mobile workstation for health
care
Toivonen et al (2011)
Two Usability tools were used: VAS (Visual Analogue Scales) and System Usability
Scale. (SUS). (N=8)
The study was able to define 19 design criteria for improvement of the product.
A usability evaluation of
workplace-related tasks on a
multi-touch tablet computer by
adults with Down syndrome
Kumin (2012)
Well-defined tasks can be used to account for user error, time to completion and rating of
difficulty performing the task with specific devices. (N=10)
Flexibility/Mobility/Display/Storage/Project focused
Literature review
Usability testing/ Well-defined Tasks
Background

13. How can a Makerspace be more effective for
managing larger amounts of projects and people at
the same time?
With limited (shared) space
With limited time (class 1/class2..)
With limited number of people to manage the space
Physical
Environment
Question
Physical
Structure
Symbolic
artifacts
Physical
stimuli
Open VS Closed Offices
Sitting Vs Standing
Formal Meeting Space Vs Casual
Natural Vs Artificial Lighting
Incoming email/notes on a wall
Security signs
Smell of coffee
Color of the walls
Style of furnishing
Logotypes / Images
(Davis,
1984)

14. Observation
Workflow
& Design
Goals
Prototype
Tufts Tisch/CEEO Jumbo´s Makerstudio
/Arts & Crafts Center/
Cambridge Friends/Malden High School
Modelling a
general sequential
worklfow
Defining design goals
and specifications
DIY-Design
Experimental unit
Contrast with Market
Usability
Testing
Simulate
Set Up
Design activity
Clean Up
Storage
Method
Classroom needs / College / School

15. • How do people use the physical environment of Makerspaces?
• How does the physical environment help Makerspace users achieve their goals?
Qualitative
research
method,
such
as
shadowing
were
taken
into
account
for
note
taking
and
understanding
actions
of
users
in
context
(McDonald,
2005)
ObservationMethod
(Photos by author)

16. Enter
Check on-going activities
Read announcements
Leave personal belongings
near work spot
Retrieve on-going project from
storage space
Set-up
Gather
supplies
/tools
Start
design
prototyping
Mobility
around
space
Clean-up
Leave
Functional
perimeter
with tools and
materials
Other
central
workstations
Clean work surface
Discard scrap
Recycle
Storage
Retrieve materials
Check availability of tools
Check availability of equipment
Leave on-going project and materials
in process in storage area
Bring personal belongings
Collaboration
with peers
Discussion of
problems
Sequential
Workflow
Makerspace
Usage
User

17. Initial concept
The on-going project is the center of the making process
Shift from “stationary tables” to a “dynamic set of surfaces”
Allow to free up space
Improve storage / organization / set up / clean up
Method
(Photos by author)

18. Product goals, specification and prototypingMethod
(Photos by author)

19. Product goals, specification and prototypingMethod
(Drawings by author)
Below Market average (US$ 642)
Cost per tray US$25
Does not consider Design/Assembly hours

20. Product goals, specification and prototypingMethod
Below Market average (US$ 350)
Does not consider Design/Assembly hours
(Drawings by author)

21. Experimental prototypeMethod
(Drawings and Photo by author)
Tray RACK
Movable Workstation

22. Researcher
Usability testing
Tray Rack
Materials rack
and 3D printer
Entrance
Disposal
Can
Workplace
Setting A: Baseline condition
Setting B: Experimental
Experimental layout
Method

23. •30 Participants (17Female/13Male)
•Age: M=24.6, SD=7.9
•18 sessions, spread over 9 days
•24 in groups of 2
•6 individually
•Performed a series of task using both
settings, two design activities
•Repeated Measures Design
(Stangor, 2004)
•System Usability Scale
(Brooke, 2996)
•User Experience Questions
(Toivonen, 2011)
Usability testingMethod
Study was approved by Tufts University Institutional Review Board Nº 1501002

24. Task workflow
Usability testingMethod

25. PredictionsMethod
Increased
System Usability Score (SUS)
Easier Storage
& Organization
Adjective
scale
SUS SD
Worst
Imaginable 12.5 13.1
Awful 20.3 11.3
Poor 35.7 12.6
OK 50.9 13.8
Good 71.4 11.6
Excellent 85.5 10.4
Best
Imaginable 90.9 13.4
Adjective Scale
(Bangor
et
Al,
2009)
Experimental (B)Standard (A)
Standard (A)
Experimental (B)
Faster Exit
from Makerspace
Time<4 min
Subjective Eval.Subjective Eval.
Set Up/Clean Up

26. Usability testingResults

27. Open ended answersResults
Worksurface was equal in both settings (986 sq inches)
Sitting still preferred for some detailed prototyping
Familiarity: Baseline condition is present in the majority of Makerspaces
(Coding according to Qualitative Data Analysis, Miles and Huberman, 1994)

28. Open ended answersResults
Lack of storage space
Shifting position
Access to power
Clean up: Male
Projector size didn´t free-up space
8%#
31%#
8%#
31%#
15%#
8%#
0%# 0%# 0%#
8%#
0%#
41%#
0%#
41%#
0%#
12%#
6%# 6%#
12%#
18%#
0%#
5%#
10%#
15%#
20%#
25%#
30%#
35%#
40%#
45%#
Lim
ited#Space#
Backpack#storage#
M
aterials#rolling#off#Shi@#posiAon#
Clean#up#
Access#to#pow
er#
Diffi
cult#collaboraAon#space#
SiI
ng#
Projector#connecAon#Projector#size#
Most%difficult%aspects%of%se0ng%A%
MALE#
FEMALE#

29. Results Open ended answers
Storage directly in the project-tray
Free up worksurface with small projector
Improved mobility: gather materials, supervise equipment
Device-centered users require easy access to power

30. Results Open ended answers
The project-tray felt “less spacious”
Middle separation was perceived as a “separator”
Standing position
The amount of devices can provoke issues with cords
15%$
8%$ 8%$ 8%$
38%$
8%$ 8%$
0%$ 0%$
8%$
0%$
35%$
12%$ 12%$
47%$
6%$
0%$
6%$ 6%$
18%$
0%$
5%$
10%$
15%$
20%$
25%$
30%$
35%$
40%$
45%$
50%$
Tangled$cords$
M
iddle$separa:on$
Needs$seat$
Tray$access$
Spaciousness$
Locking$M
echanism
$
W
eight$resistance$
SetEup$
Backpack$support$
Standing$
Most%difficult%aspects%of%se0ng%B%
MALE$
FEMALE$

31. Hypothesis 1: Increased Usability of the Experimental Setting (setting B)
Setting A can be qualified as between “OK” and
Good”, whereas setting B can be qualifies as between
“Good” and “Excellent”.
(Bangor et al, 2008)
Usability testingResults

32. Usability testingResults
Hypothesis 1: Increased Usability of the Experimental Setting (setting B)
Both groups scored setting B with significantly higher scores.
Less than 1 year: : t (15)=-3.0365, p=0.0042
More than 1 year: t (13)= -2.0812, p= 0.0289
People working individually (N=6)did not evaluate
significantly the workstation as more usable.
People working in groups (N=24) displayed
significant higher evaluation of experimental
workstation

33. Usability testingResults
Gender-wise difference in evaluation.
Male participants scored significantly higher the new system, t(12)=-4.6209 p= 0.0006.
Female participants didn´t reveal any difference with statistical significance, t (16)=-1.6753,
p=0.1133.
Hypothesis 1: Increased Usability of the Experimental Setting (setting B)

34. Usability testingResults
Nº Statement Male Female Diff.
P-value
(M diff F)
1
I think that I would like to use this system frequently. 4.54 3.94 0.597 0.0305
2
I found the system unnecessarily complex. 2.15 2.24 0.0814 0.789
3
I thought the system was easy to use. 4.31 3.76 0.5429 0.075
4
I think that I would need the support of a technical person to be able to
use this system.
1.94 1.54 0.402 0.161
5
I found the various functions in this system were well integrated 4.31 4.24 0.072 0.817
6
I thought there was too much inconsistency in this system 1.94 1.62 0.32 0.252
7
I would imagine that most people would learn to use this system very
quickly.
4.61 4.11 0.497 0.055
8
I found the system very cumbersome to use. 1.69 1.941 0.2488 0.305
9
I felt very confident using the system. 4.384 3.823 0.561 0.037
10
I needed to learn a lot of things before I could get going with this
system.
1.69 2 0.307 0.188
Hypothesis 1: Increased Usability of the Experimental Setting (setting B)

35. Participants within the average group (N=17) scored setting B with a higher
difference in terms of its Usability. (Makes sense: design for average)
Participant in the “extremes” (N=13) with either low or high standing elbow height
just reported a 1.1 points lower score in terms of Usability.
Usability testingResults
Hypothesis 1: Increased Usability of the Experimental Setting (setting B)

36. User experience: storage & organization
Hypothesis 2: Better storage and organization of projects
Expected value for the workstation system was higher than expected
(Statistically significant difference: t (29)= -16.0709, p<0.001)
No gender-wise difference in evaluation
Strong perception that the system would make it easier to store and
come back to a project
Results

37. User experience: storage & organization
Statistically significant , t(29)= -5.3866, p<0.001.
Positive evaluation of the user experience organization of projects
No gender-wise difference in evaluation
Results
Hypothesis 2: Better storage and organization of projects

38. Usability testingResults
No statistical significance in differences between timings
Set UP Clean UP

39. Usability testing
Is
it
faster
to
exit
a
Makerspace
with
Setting
B?
Significant difference between means
t(29)= -10.9958, p<0.001
Results

40. Results Selected quotes
“I
really
liked
the
storage
idea.
For
many
Makerspaces,
students
come
and
go
as
they
need
to,
the
hours
are
less
regular.
It's
so
crucial
to
have
a
place
that
is
all
my
own
that
I
can
access
very
quickly
and
easily.
Having
the
shelves
turn
into
work
stations
is
a
great
solution
to
that.
/
/
Also,
in
this
environment,
when
a
maker
is
moving
from
machine
to
machine
or
back
to
their
computer
for
adjustments
or
checking
the
various
projects
going
on,
they
are
very
active.
It
is
much
more
practical
to
be
standing
up
at
the
standing
up
at
the
workstation
so
moving
around
and
switching
tasks
is
easier”
Valued
aspects:
Storage/Mobility/Standing/Projector
size
and
loca6on

41. Usability study met expectations. Further research is needed to understand why female scores did
not display significant differences. Adoption rate of the new workstation could be more dynamic, not
depending on the level of expertise.
Set up and clean up times did not display significant differences, although subjective perception
supports the idea that exit can be faster, storage and organization easier.
Sense of spaciousness was lower in experimental setting, even though both work areas were the
same.The presence of “edges” and “central division” is probably related to this
MostValued aspects:
Mobility in the Makerspace:The project-tray decompresses the central zones allowing users to
move around using a functional perimeter.
Storage and organization of materials and projects: During the design and prototyping process
materials and/or projects won´t get mixed up, because the project-tray allows each user to
manage his/her own working surface.
Results Summary

42. Simplification of the design task due to: time constraints of the experiment, fixed set
of materials.
Results Limitations
Carryover effects: Difficult to isolate influences over users when they used one
setting and then the other
Practice and fatigue: Continuous assessment in a limited period of time
Define tasks that necessarily require the use of different sizes of materials
Usability evaluation just after testing each setting / Think aloud

43. Discussion
Developed a design for workstation-system based on current observation
of workflow and needs in educational makerspaces
Tested an experimental method comprised of repetitive tasks related to
Makerspace operation.
No previous findings related to clean up, set up and storage tasks when
using workstations.
Replicate the study with different type of seats
Compare workstation performance against other commercially available
standing workstations (adjustable and non-adjustable)
Survey: Perceived usefulness and limitations among teachers

44. Acknowledgements
Friends
Committee
Family
CEEO Administrative Staff
Graduate Students
Participants

45. Questions?
Leonardo Madariaga
leonardo.madariaga@tufts.edu
MS Human Factors Engineering
Advisor: Daniel J.Hannon
Tufts University
Department of Mechanical Engineering
April 2015
of the physical environment
in Makerspaces:exploratory study
Implications
for
the
design
and
user
evaluation
of
a
new
workstation-­‐system