presented at 25th International Conference on Intelligent User Interfaces (canceled due to COVID-19) https://dl.acm.org/doi/abs/10.1145/3377325.3377529 Abstract: This paper proposes a visual analytics framework for formulating metrics for evaluating multi-dimensional time-series data. Multidimensional time-series data has been collected and utilized in different domains. We believe evaluation metrics play an important role in utilizing those data, such as decision making and labeling training data used in machine learning. However, it is a difficult task for even domain experts to formulate metrics. To support the process of formulating metrics, the proposed framework represents metrics as a linear combination of data attributes, and provides a means for formulating it through interactive data exploration. A prototype interface that visualizes target data as an animated scatter plot was implemented. Through this interface, several visualized objects can be directly manipulated: a node and a trajectory of an instance, and a convex hull as the group of nodes and trajectories. Linear combinations of attributes are adjusted in accordance with the manipulation of different objects' types by the user. The effectiveness of the proposed framework was demonstrated through two application examples with real-world data.

1. Rei Takami*, Hiroki Shibata, Yasufumi Takama
* takami-rei@ed.tmu.ac.jp
Tokyo Metropolitan University, JAPAN
ACM Conference on Intelligent User Interfaces (IUI' 20): Day 2 Intelligent Visualization Session 12020/3/19

2. Introduction_________
ACM Conference on Intelligent User Interfaces (IUI' 20): Day 2 Intelligent Visualization Session 22020/3/19
Multi-dimensional time-series data (e.g. medical, sports)
Evaluation metrics is required for decision making, hypothesis generation
Metrics should be formulated with trial-and-error by domain experts
“How to identify outliers in time-series data?”
“How to classify / rank time-series data?”

3. Introduction
ACM Conference on Intelligent User Interfaces (IUI' 20): Day 2 Intelligent Visualization Session 32020/3/19
Definition of Evaluation Metrics:
interpretable, attribute-based representation of user-defined criteria
Sabermetrics (baseball), well-being metrics
Metrics formulation Process
Multi-dimensional time-series data: ① more complicated than other data types
② difficult for domain experts Support with VA
① Preprocessing ② Analysis ③ Formulate
□ Data collection
□ Feature selection
□ Normalization
□ Analysis of preprocessed data
□ Understanding of dimension’s
(features) characteristics
□ Parameter adjustment of
preprocessing algorithms
□ Definition of metrics
based on knowledge
□ Metrics validation

4. Introduction
ACM Conference on Intelligent User Interfaces (IUI' 20): Day 2 Intelligent Visualization Session 42020/3/19
• Human-in-the-loop visual analytics
Semantic interaction
• Existing work:
• Analysis , interactive ranking of multi-dimensional data
Ignore time-series characteristics (e.g. seasonality)
Difficulty of visualizing multi-dimensional time-series data
DR
Model
Visualization

5. Proposed Framework: Definitions
ACM Conference on Intelligent User Interfaces (IUI' 20): Day 2 Intelligent Visualization Session 52020/3/19
Target Data: Finite time-series data
• 𝑇 time points, 𝑁 instances, 𝑀 attributes: 𝑫 = 𝑑 𝑡𝑛𝑚
• Projecting 𝑫 to 2D space by Dimensionality reduction (DR) ( PCA) at each 𝒕
• 𝑷 𝒕𝒏 : coordinates of 𝒅 𝑡𝑛 on 2D space, defined by local (𝜶) / global (𝝎) parameters
𝜶 : Displacement of each data object (i.e. user-defined bias)
𝝎 : Contribution of m-th attributes to the axis
α
α
ωω
α
t
𝑷 𝒕𝒏 = ෍
𝑚=1
𝑀
𝑑 𝑡𝑛𝑚 𝝎 𝝉𝒎 + 𝜶 𝝉𝒏
(𝜶 𝒕𝒏 = 𝛼 𝑡𝑛
X
, 𝛼 𝑡𝑛
Y
𝝎 𝒕𝒎 = (𝜔 𝑡𝑚
X
, 𝜔 𝑡𝑚
Y
))
𝝎 𝒕
𝐗
𝝎 𝒕
𝐘

6. Proposed Framework: Definitions
ACM Conference on Intelligent User Interfaces (IUI' 20): Day 2 Intelligent Visualization Session 62020/3/19
• Associate 𝝎 to evaluation metrics (expressed by linear combination)
• Directly manipulate visualized objects
Experts can reflect their knowledge to 2D projection,
Utilize parameters as starting point of formulating metrics
α
α
ω
ω
α
t
𝑷 𝒕𝒏 = ෍
𝑚=1
𝑀
𝑑 𝑡𝑛𝑚 𝝎 𝝉𝒎 + 𝜶 𝝉𝒏
(𝜶 𝒕𝒏 = 𝛼 𝑡𝑛
X
, 𝛼 𝑡𝑛
Y
𝝎 𝒕𝒎 = (𝜔 𝑡𝑚
X
, 𝜔 𝑡𝑚
Y
))
𝝎 𝒕
𝐗
𝝎 𝒕
𝐘

7. Proposed Framework: Visualized Objects
ACM Conference on Intelligent User Interfaces (IUI' 20): Day 2 Intelligent Visualization Session 72020/3/19
𝑡 𝑛
𝑡 𝑛+1 𝑡 𝑛+2
𝑡 𝑛+3
ω
α
t
𝑡 𝑛
TrajectoryNode
𝑡 𝑛
• Different parameters specified for each 𝒕
Visualized objects associated with different time/spatial range
• Supporting progressive, flexible parameter adjustment for 2D projection
Node: current time point
Path
Node
Convex hull (Polygon)
𝑜1
𝑜1
ω
α
t
𝑡 𝑛
𝑡 𝑛+3
𝑜2 𝑜3
𝑜2 𝑜3
Based on node: (at each t )
Based on trajectory (at time range)

8. Proposed Framework: Direct Manipulations
ACM Conference on Intelligent User Interfaces (IUI' 20): Day 2 Intelligent Visualization Session 82020/3/19
8
Close object
Discrete object
Projection Axis
Absolute manipulation Relative manipulation
Object manipulating strategies :
• Absolute: emphasize global placement Adjust weight of eath attribute ( 𝜔 )
• Relative: modify local relationship between objects
Adjust 𝛼 of target object(s) Can be converted to 𝜔
ω ω’

9. Prototype Interface: Implementation with PCA
ACM Conference on Intelligent User Interfaces (IUI' 20): Day 2 Intelligent Visualization Session 92020/3/19
(a) Scatter-plot view (b) Detailed View
Bar chart: parameter (ω)
Initial ω: PC (principal component)
Visualizing 𝑷 𝒕𝒏
Animation Control UI
Convex hulls
Trajectories
Navigations

10. Prototype Interface: Implementation with PCA
ACM Conference on Intelligent User Interfaces (IUI' 20): Day 2 Intelligent Visualization Session 102020/3/19
(a) Scatter-plot view (b) Detailed View
Parallel coordinates
Temporal change of coordinates
of selected objects
Navigations

11. Prototype Interface: Implementation with PCA
ACM Conference on Intelligent User Interfaces (IUI' 20): Day 2 Intelligent Visualization Session 112020/3/19
(a) Scatter-plot view (b) Detailed View
Navigations
Changing
target visual object

12. Contact hitters
Silver Sluggers
Power hitters
Example: MLB (Measure League Baseball) Data
ACM Conference on Intelligent User Interfaces (IUI' 20): Day 2 Intelligent Visualization Session 122020/3/19
Dataset:
MLB batters’ (American League) statistics (2018) :
• 𝑇 = 12, 𝑁 = 198, 𝑀 = 11, Attributes: number of home runs, etc.
Purpose: Evaluating batters’ performance/characteristics
• X axis: Performance
• Silver sluggers: right side
• Y axis: Characteristics
• Contact hitters: upper ends
• Power hitters: lower ends
Modify Parameters to emphasize object placement
Baseball-Reference.com, https://www.baseball-reference.com/

13. Example: MLB (Measure League Baseball) Data
ACM Conference on Intelligent User Interfaces (IUI' 20): Day 2 Intelligent Visualization Session 132020/3/19
Contact hitters
(Relative)
Silver Sluggers
(Absolute)
Power hitters
(Relative)
PA: in 2nd half𝜔𝑡
(X)
𝜔𝑡
′(Y)
𝜔𝑡
′(X)
α ω
SF
HBP
BB:
in 2nd half
SB
HR
3B
2B
1B
PA
𝜔𝑡
(Y)
(Parameter adjustment)

14. Example: MLB (Measure League Baseball) Data
ACM Conference on Intelligent User Interfaces (IUI' 20): Day 2 Intelligent Visualization Session 142020/3/19
Acquired knowledge through VA
• Attribute with strong effect
• X axis (performance): number of PA in 1st half
• Y axis (characteristics): BB in 1st half, 3B and HR in 2nd half
• Definition of 𝐵type
• Non-linear metrics with reference to existing metrics:
TB (Total bases), OBP (On-base percentage)
𝐵type =
1B × 2 + SB × 2 + 3B × 2 − 2B + HR × 3
# of team game × 3.1
1B × 2 + SB × 3 + 3B × 3 − BB + 2B × 2 + HR × 4
PA
(1st half)
(2nd half)
( 1B 2B, 3B: # of single, 2-base, 3-base hits, HR: # of home runs, BB: # of base on balls,
SB: # of stolen bases, and PA: # of plate appearances. )

15. Conclusion and Future Works
ACM Conference on Intelligent User Interfaces (IUI' 20): Day 2 Intelligent Visualization Session 152020/3/19
Summary
• VA framework for supporting formulation of evaluation metrics for
multi-dimensional time-series data
• Implementation of proposed framework with PCA
• Application example
Future Works
• Evaluate qualitative/quantitative effectiveness
• Case study, User study
• Extend proposed framework non-linear dimensionality reduction
• Integrated framework including preprocessing

16. References
ACM Conference on Intelligent User Interfaces (IUI' 20): Day 2 Intelligent Visualization Session 16
A. Endert, L. Bradel, C. North, Beyond control panels: Direct manipulation for visual
analytics, IEEE Transactions on Computer Graphics and Applications, Vol. 33, No. 4, pp. 6-13, 2013.
W. Aigner, S. Miksch, W. Műller, H. Schumann, C. Tominski, Visualizing time-oriented
data – a systematic view, Computers ＆ Graphics, Vol. 31, No. 3, pp. 401-409, 2007.
D. A, Keim, F. Mansmann, J. Schneidewind, J. Thomas, H. Ziegler, Visual analytics: Scope
and challenges, Visual Data Mining, Springer, pp. 76-90, 2008.
A. C. Chen, X. Fu, Data + intuition: A hybrid approach to developing product north star
metrics, Proceedings of the 26th International Conference on World Wide Web Companion, pp. 617–625,
2017.
H. Kim, J. Choo, H. Park, and A. Endert, Interaxis: Steering scatterplot axes via observation-
level interaction, IEEE Transactions on Visualization and Computer Graphics, Vol.22, No.1, pp. 131– 140,
2016.
J. Bernard, D. Sessler, T. Ruppert, J. Davey, A. Kuijper, and J. Kohlhammer,
User-based visual interactive similarity definition for mixed data objects - concept and first
implementation, Proceedings of 22nd International Conference in Central Europe on Computer
Graphics, Visualization and Computer Vision, pp.329–338, 2014.
2020/3/19

17. References
ACM Conference on Intelligent User Interfaces (IUI' 20): Day 2 Intelligent Visualization Session 17
B. Kondo and C. Collins, Dimpvis: Exploring time-varying information visualizations
by direct manipulation, IEEE Transactions on Visualization and Computer Graphics, Vol. 20, No. 12, pp.
2003–2012, 2014.
T. Schreck, C. Panse, A new metaphor for projection-based visual analysis and data
exploration, Proceedings of Visualization and Data Analysis, Vol. 6495, 2007.
E. Wall, S. Das, R. Chawla, B. Kalidindi, E. T. Brown, A. Endert, Podium: Ranking data
using mixed-initiative visual analytics, IEEE transactions on visualization and computer graphics, Vol. 24,
No. 1, pp. 288-297, 2018.
D. Sacha, L. Zhang, M. Sedlmair, J.A. Lee, J. Peltonen, D. Weiskopf, S.C. North, and D.A.
Keim,Visual interaction with dimensionality reduction: A structured literature analysis, IEEE
Transactions on Visualization and Computer Graphics, Vol.23, No.1, pp. 241–250, 2017.
2020/3/19