HCI have long moved beyond the evaluation setting of a single user
sitting in front of a single desktop computer, yet many of our fundamentally
held viewpoints about evaluation continues to be ruled by outdated biases
derived from this legacy. We need to engage with real users in 'Living
Laboratories', in which researchers either adopt or create functioning systems
that are used in real settings. These new experimental platforms will greatly
enable researchers to conduct evaluations that span many users, places, time,
location, and social factors in ways that are unimaginable before.