The document summarizes user research conducted to design a system for querying real-time big data, as needed in fast-paced domains like Formula 1 racing. The researcher interviewed 4 subjects with relevant backgrounds, then conducted a co-design study where subjects helped build visual queries. Key findings included: (1) questions should follow a consistent format with measures, changes, and time windows; (2) sandboxes should provide guidance but not constrain options; (3) physical metaphors and modularity support flexibility and reuse. The research provides insights on visual querying that apply broadly to real-time data analysis.

1. The Internet of Fast Things:
User Research for
Realtime Big Data Query

2. About me
1988
Formative education
2006
Painting, BFA
2006-2013
Designer of
statistical reports
2015
Design Management,
M.Des

3. Project Overview

4. Storing
big data
We already know what to do with
historical (or static) big data—most
BI solutions are not much different
than Excel, just more powerful.
!
So what happens when we want to
look at real time data?

5. Unlocking
value
Realtime visual analysis is possible
from a processing standpoint (see
Apache Spark), but separating the
noise from the value is a complex
problem.
Noise
Meaning

6. The reason we use big data is to
arrive at some decision or action.
Getting to that stage starts with
asking the right question. Stop
talking about being data-driven, and
start being question driven.
From data-driven
to question-driven

7. Rules for
realtime data
query
Asking the right question, when you
can’t look at everything, drives the
cycle of learning and discovery.
!
To learn more about this, see here:
www.google.com/patents/
US20140351233
Crupi, 2014
You can’t look at
everything
Questions are
time-based
Questions are either superlative
or comparative
Two question formats:
Calculations and Patterns
1
2
3
4

8. Calculations
vs. Patterns
A calculation is a question like “show me
the average speed over the past 10
minutes”. The answer to that question is the
black line, which you’re likely familiar with.
A pattern fits on top of a calculation, and
notifies us of a certain event or behavior.
For example, “show me when a customer
enters and exits the store in 5 minutes and
doesn’t buy anything”.
Calculation
Pattern

9. Use case:
Formula 1
Formula One is a great use case for
studying real time data query,
because it’s fast-paced and
complex. If we can design for a
system like this, our findings will be
relevant to less complex systems.
Noah Joseph, 2013

10. The race engineer is our top level user. He is the one
who turns hypothetical strategy into reality.
He is supported by a team of engineers (sitting at the
pit wall, and in another room off-site), who are all
monitoring different systems. The cars each have
about 1,000 sensors on them, and there is data from
the track, the environment, other cars, and the driver.

11. The race
engineer
has a Masters, and sometimes a
Ph.D in Mechanical Engineering, or a
related field.
He’s a top expert in his field, but
he’s not a programmer, so he can't
write query.

12. We need a third
language
that bridges the gap between a natural
language question and query.
In historical data settings, we have time
to learn the ‘language’ the UI uses to
form questions (like writing a formula in
Excel). But in real time, users want to
write ad-hoc questions when time is
precious.
SELECT *
FROM (
SELECT bucketId
FROM (
SELECT FuelIncrease,
bucketId,
MAX(FuelIncrease) OVER(ORDER BY
bucketId ROWS BETWEEN 5 PRECEDING AND CURRENT ROW) AS
MaxFuelIncreaseWithin60Seconds
FROM (
SELECT AvgFuelConsumption -
LAST_VALUE(AvgFuelConsumption) OVER (ORDER BY
bucketId ROWS BETWEEN 1 PRECEDING AND CURRENT ROW) AS
FuelIncrease,
AvgPressure,
AvgTemperature,
AvgFuelConsumption,
bucketId
FROM (
SELECT AVG(pressure) AS AvgPressure,
AVG(temperature) AS AvgTemperature,
AVG(fuelConsumption) AS AvgFuelConsumption,
ROUND(timestampInMS / 10000, 0) AS bucketId
FROM source
GROUP BY ROUND(timestampInMS / 10000, 0)
) q1
) q2 USING (bucketId)
Show me when the tire
temperature drops below 30°C
and also the speed drops below
70 km/h within 5 seconds.

13. Assumptions
and
hypotheses
‣ Building questions should work like a
sandbox, not a path*
‣ Question elements should emulate Legos—
modular, configurable, playful
‣ Colors and block position should have
semantic meaning
!
Thanks to Stephen P. Anderson for this one.
Speed
RPM
Temp

14. User Research

15. Subjects
I recruited four users, to get a sense of
how actual engineers think and act. For
the first round, I interviewed them,
asking about pain points, current
solutions, goals, etc.
I searched for users with some Formula
1 background, or with experience with
data analysis in fast-paced situations.
Performance
engineer
Race
engineer
Former air force
flight instructor
Mechanical
engineer

16. Interview
strategy
3 out of 4 interviews were conducted
via Skype, due to geographical
distance.

17. Problems with
the study
Users with relevant background were
hard to recruit, due to the exclusive
nature of Formula One, and the
coincidence of the study with the height
of the season.
Two users were recruited via LinkedIn,
and two were found via personal network.

18. Initial
prototype
From the initial interviews, I developed
an initial prototype—a form of visual
syntax that I thought addressed what
my subjects discussed.
Question:
Show me when the tire
temperature drops below 30°C and
also the speed drops below 70
km/h within 5 seconds.
Visual syntax:

19. Co-design
For the second round of interviews,
I asked my subjects (3 this time) to help me
design the solution. I gave them relatively simple
questions, and asked them to ‘translate’ these
questions into a visual syntax by placing together
the building blocks I provided.
For the two remote subjects, I used a
combination of Skype (for audio) and
www.realtimeboard.com (as a collaborative
whiteboard)

20. ‣ Time window: placed ‘underneath’
the question, time unit placed in
upper right corner (see photo, right)
‣ Built each question quickly and with
relative ease
‣ I want to be able to define the time
window in a couple ways. With a
specific time period (e.g. ‘1:00:00 to
2:00:00’), or a general one (e.g. ‘over
3 minutes’)
User 1
Air Force flight instructor

21. ‣ Time window: placed over question
(see image, right)
‣ Brake pedal should have two
possibilities: 1. brakes engaged
(yes/no), 2. brake pressure
‣ I want to be able to say ‘OR’ (this
was a glaring omission)
‣ I want to be able to name parts of
statements, so I can reuse them
later
User 2
Race engineer

22. User 3
Performance engineer
‣ Very focussed on consequences:
Discussed the consequence of each
action, the effect of each variable, how
the cars work and react to changes
‣ Time window: placed to the right of the
question
‣ I prefer to build the question in order of
priority. The main source of the
‘problem’ on the first row, and the rest
of the conditions on the following rows
‣ Each part of the question should have
its own row. It’s more organized that
way.
‣ Other engineers might like to organize
in another way.

23. Design conclusions

24. 1. Format of
the question
Measure: whatever livestream
the user is choosing to look at
(e.g. SPEED)
Change: the type of question
the user wishes to ask, either
superlative or comparative (e.g.
‘more than’)
Show me when…
measure changes time
fuel consumption
page views
temperature
revenue
water pressure
increases
highest
worst
slowest
top
decreases
seconds
minutes
laps
hours
over

25. 2. Sandboxes
and simplicity
Focus action, and prevent users from
getting lost by giving relevant clues
BUT
Do not constrain options, and allow
users to build their question in the
sequence they desire.
Beginners are asked to drag a
livestream into this box, but advanced
users can start with another block.

26. 3. Context
Looking at both realtime and
historical data gives tremendous
value. Allow users to compare both,
by making some visual distinction
between the two data sets
SPEED SPEED*
previous lap
previous race
previous year
right this second
vs.

27. 3. Physical
metaphors and
affordances
Drag and drop boxes imitate
physical building blocks, allowing
users to build something, repair, and
build again. Changes while building
are reflected in the graph of the
livestream data.
SPEED
EXCEEDS
200 kph
Show me when…

28. 4. Limitation
Question types are limited to what is
relevant in realtime.
more than less than equals
increase decrease
max min

29. 5. Modularity
Questions can be saved and
archived for repeated use;
Parts of questions can be saved as
well.
“speed above 200 kph”
“tire temp issue”
Show me when…
AND

30. 6. Time
window
placement
Time window is placed to the right
of the measure and change blocks
!
Can be specific (2:00:05 - 2:25:00) or
general (6 seconds, 3 laps)
tire temp < 30°C
speed < 70 kph
AND 8 sec

31. What’s next?
This same research and analysis
process, and many of the conclusions,
can be applied to myriad industries
and requirements.

32. Thanks for reading!
Alexandra Joseph
@alexqjoseph
hello@alexandrajoseph.com