Finding Patterns in Temporal Data

Krist Wongsuphasawat
Krist WongsuphasawatData Visualization
FINDING PATTERNS
IN TEMPORAL DATA
                                                   osium!
                                       CI   L Symp
KRIST WONGSUPHASAWAT             27th H 010!
                                        , 2
                                 May 27
TAOWEI DAVID WANG
CATHERINE PLAISANT
BEN SHNEIDERMAN

HUMAN-COMPUTER INTERACTION LAB
UNIVERSITY OF MARYLAND
FINDING PATTERNS
IN TEMPORAL DATA
                                                   osium!
                                       CI   L Symp
KRIST WONGSUPHASAWAT             27th H 010!
                                        , 2
                                 May 27
TAOWEI DAVID WANG
CATHERINE PLAISANT
BEN SHNEIDERMAN

HUMAN-COMPUTER INTERACTION LAB
UNIVERSITY OF MARYLAND
TEMPORAL CATEGORICAL DATA
•  A type of time series
                              Event!           Category!         Event!
        Numerical!
      Stock: Microsoft                   Patient ID: 45851737
 $'#")#"$!$&!$($$   &0!B$0&            !"#$"#"$$%&!'(")    &*++,-./&
 $'#")#"$!$&!$(!;   &0!B$!&            !"#$"#"$$%&!'(0)    &123+43567&
 $'#")#"$!$&!$(0$   &0!B$"&            !"#$"#"$$%&""(''    &89:&
 $'#")#"$!$&!$(';   &0!B$%&            !"#$;#"$$%&$;($<    &=/>>+&
 $'#")#"$!$&!!($$   &0!B!)&            !"#!'#"$$%&$)(!?    &1@,A&
                                       &           Time
                                          Arrival
                                           Emergency
                                                  ICU
                                                        Floor
                                                                  Exit
TEMPORAL CATEGORICAL DATA




Electronic Health Records: symptoms, treatment, lab test"
Traffic incident logs: arrival/departure time of each unit"
Student records: course, paper, proposal, defense, etc."
"
Others: web logs, usability study logs, etc."
10+ years work on temporal visualization
      (mostly on Electronic Health Records)
LIFELINES


            D
SINGLE RECOR




                        [Plaisant et al. 1998]
                http://www.cs.umd.edu/hcil/lifelines
LifeLines – Single Patient
working with physicians at
WASHINGTON HOSPITAL CENTER
EXAMPLE DATA
•  Patient transfers

       ARRIVAL         Arrive the hospital
       EMERGENCY       Emergency room
       ICU             Intensive Care Unit
       INTERMEDIATE    Intermediate Medical Care
       FLOOR           Normal room
       EXIT-ALIVE      Leave the hospital alive
       EXIT-DEAD       Leave the hospital dead
TASKS
•  Example: Finding “Bounce backs”

              ICU      Floor      ICU



                         within 2 days
LIFELINES 2
RECORD

       D
R ECOR
    RECORD


    RECORD

  RECORD
                   [Wang et al. 2008, 2009]
             http://www.cs.umd.edu/hcil/lifelines2
Multiple Records   ARF (Align-Rank-Filter)
                                 Framework




             Temporal Summary




LifeLines2 – Search and Visualize
ALIGNMENT
•  Sentinel events as reference points

                              Time

                      June      July        August
  Patient #45851737      Arrival
                         Emergency
                               ICU
                                          Floor
                                                   Exit
  Patient #43244997                    Arrival
                                       Emergency
                                             ICU
                                                          Floor
                                                                  Exit
ALIGNMENT (2)
•  Time shifting

                               Time

                       0         1M      2M
   Patient #45851737       Admit
                           Emergency
                                ICU
                                       Floor
                                               Exit
   Patient #43244997       Admit
                           Emergency
                                ICU
                                       Floor
                                               Exit
SIMILAN
RECORD

       D
R ECOR
    RECORD


    RECORD

  RECORD
             [Wongsuphasawat & Shneiderman 2009]
               http://www.cs.umd.edu/hcil/similan
Similan – Search by Similarity
Similan – Search by Similarity
FINDING “BOUNCE BACKS”
   Before                             After




  •  Much faster to specify new query
  •  Visualizing the results gives better understanding
USER STUDIES: SEARCH
LifeLines2            Similan
    Exact!          Similarity-based!
MUST have A, B, C    SHOULD have A, B, C


      Query                Query


    Record#2              Record#2

                                           more
    Record#1              Record#1
                                           similar

    Record#3              Record#3
USER STUDIES: SEARCH
    LifeLines2            Similan
        Exact!          Similarity-based!
    MUST have A, B, C    SHOULD have A, B, C


          Query                Query



1       Record#2              Record#2

                                               more
        Record#1              Record#1
                                               similar

        Record#3              Record#3
NEW STUFF
Needs for an overview -> LifeFlow!!
TASKS
•  Example: Finding “Bounce backs”

              ICU       Floor        ICU



                            within 2 days

•  Other questions
                     Arrival
                                            ?
                      ICU

             ?                          ?
LIFEFLOW
    RECORD
       D                                                 VISUALIZE
R ECOR
           RECO                                     Display the aggregation
               RD
   RECORD


RECORD
           RECORD
RECORD

RECORD                      AGGREGATE
                      Merge multiple records into tree
AGGREGATE
•  Aggregate by prefix

     #1

     #2

     #3

     #4




               Example with 4 records
AGGREGATE
•  Aggregate by prefix

     #1

     #2

     #3

     #4
VISUALIZE
•  Inspired by the Icicle tree [Fekete 2004]




        Number of files!
VISUALIZE (2)
•  Use horizontal axis to represent time
•  Video
DEMO – LIFEFLOW
When the lines are combined into flow!
FUTURE WORK
•  Comparison
                               ICU
    ICU         Intermediate               Intermediate




          Jan-Mar 2008               April-June 2008
Floor
TAKE-AWAY MESSAGE
Information visualization is a powerful way
to explore temporal patterns.

         You can work with us
         on new case studies.
1 of 30

Recommended

Krist Wongsuphasawat's Dissertation Proposal Slides: Interactive Exploration ... by
Krist Wongsuphasawat's Dissertation Proposal Slides: Interactive Exploration ...Krist Wongsuphasawat's Dissertation Proposal Slides: Interactive Exploration ...
Krist Wongsuphasawat's Dissertation Proposal Slides: Interactive Exploration ...Krist Wongsuphasawat
1.1K views61 slides
Finding Comparable Temporal Categorical Records: A Similarity Measure with an... by
Finding Comparable Temporal Categorical Records: A Similarity Measure with an...Finding Comparable Temporal Categorical Records: A Similarity Measure with an...
Finding Comparable Temporal Categorical Records: A Similarity Measure with an...Krist Wongsuphasawat
1.2K views23 slides
Krist Wongsuphasawat's Dissertation Defense: Interactive Exploration of Tempo... by
Krist Wongsuphasawat's Dissertation Defense: Interactive Exploration of Tempo...Krist Wongsuphasawat's Dissertation Defense: Interactive Exploration of Tempo...
Krist Wongsuphasawat's Dissertation Defense: Interactive Exploration of Tempo...Krist Wongsuphasawat
1.6K views91 slides
Lifeflow: Visualizing an Overview of Event Sequences by
Lifeflow: Visualizing an Overview of Event SequencesLifeflow: Visualizing an Overview of Event Sequences
Lifeflow: Visualizing an Overview of Event SequencesKrist Wongsuphasawat
5.1K views38 slides
Information Visualization for Health Care by
Information Visualization for Health CareInformation Visualization for Health Care
Information Visualization for Health CareKrist Wongsuphasawat
2.1K views43 slides
Ops Meta-Metrics: The Currency You Pay For Change by
Ops Meta-Metrics: The Currency You Pay For ChangeOps Meta-Metrics: The Currency You Pay For Change
Ops Meta-Metrics: The Currency You Pay For ChangeJohn Allspaw
8.4K views109 slides

More Related Content

More from Krist Wongsuphasawat

Navigating the Wide World of Data Visualization Libraries by
Navigating the Wide World of Data Visualization LibrariesNavigating the Wide World of Data Visualization Libraries
Navigating the Wide World of Data Visualization LibrariesKrist Wongsuphasawat
2K views72 slides
Encodable: Configurable Grammar for Visualization Components by
Encodable: Configurable Grammar for Visualization ComponentsEncodable: Configurable Grammar for Visualization Components
Encodable: Configurable Grammar for Visualization ComponentsKrist Wongsuphasawat
451 views79 slides
6 things to expect when you are visualizing (2020 Edition) by
6 things to expect when you are visualizing (2020 Edition)6 things to expect when you are visualizing (2020 Edition)
6 things to expect when you are visualizing (2020 Edition)Krist Wongsuphasawat
475 views203 slides
Increasing the Impact of Visualization Research by
Increasing the Impact of Visualization ResearchIncreasing the Impact of Visualization Research
Increasing the Impact of Visualization ResearchKrist Wongsuphasawat
1.3K views33 slides
6 things to expect when you are visualizing by
6 things to expect when you are visualizing6 things to expect when you are visualizing
6 things to expect when you are visualizingKrist Wongsuphasawat
2.3K views214 slides
What to expect when you are visualizing (v.2) by
What to expect when you are visualizing (v.2)What to expect when you are visualizing (v.2)
What to expect when you are visualizing (v.2)Krist Wongsuphasawat
607 views182 slides

More from Krist Wongsuphasawat(20)

Navigating the Wide World of Data Visualization Libraries by Krist Wongsuphasawat
Navigating the Wide World of Data Visualization LibrariesNavigating the Wide World of Data Visualization Libraries
Navigating the Wide World of Data Visualization Libraries
Encodable: Configurable Grammar for Visualization Components by Krist Wongsuphasawat
Encodable: Configurable Grammar for Visualization ComponentsEncodable: Configurable Grammar for Visualization Components
Encodable: Configurable Grammar for Visualization Components
6 things to expect when you are visualizing (2020 Edition) by Krist Wongsuphasawat
6 things to expect when you are visualizing (2020 Edition)6 things to expect when you are visualizing (2020 Edition)
6 things to expect when you are visualizing (2020 Edition)
ร้อยเรื่องราวจากข้อมูล / Storytelling with Data by Krist Wongsuphasawat
ร้อยเรื่องราวจากข้อมูล / Storytelling with Dataร้อยเรื่องราวจากข้อมูล / Storytelling with Data
ร้อยเรื่องราวจากข้อมูล / Storytelling with Data
Reveal the talking points of every episode of Game of Thrones from fans' conv... by Krist Wongsuphasawat
Reveal the talking points of every episode of Game of Thrones from fans' conv...Reveal the talking points of every episode of Game of Thrones from fans' conv...
Reveal the talking points of every episode of Game of Thrones from fans' conv...
Adventure in Data: A tour of visualization projects at Twitter by Krist Wongsuphasawat
Adventure in Data: A tour of visualization projects at TwitterAdventure in Data: A tour of visualization projects at Twitter
Adventure in Data: A tour of visualization projects at Twitter
Data Visualization: A Quick Tour for Data Science Enthusiasts by Krist Wongsuphasawat
Data Visualization: A Quick Tour for Data Science EnthusiastsData Visualization: A Quick Tour for Data Science Enthusiasts
Data Visualization: A Quick Tour for Data Science Enthusiasts
Krist Wongsuphasawat50.6K views
Using Visualizations to Monitor Changes and Harvest Insights from a Global-sc... by Krist Wongsuphasawat
Using Visualizations to Monitor Changes and Harvest Insights from a Global-sc...Using Visualizations to Monitor Changes and Harvest Insights from a Global-sc...
Using Visualizations to Monitor Changes and Harvest Insights from a Global-sc...
Making Sense of Millions of Thoughts: Finding Patterns in the Tweets by Krist Wongsuphasawat
Making Sense of Millions of Thoughts: Finding Patterns in the TweetsMaking Sense of Millions of Thoughts: Finding Patterns in the Tweets
Making Sense of Millions of Thoughts: Finding Patterns in the Tweets
From Data to Visualization, what happens in between? by Krist Wongsuphasawat
From Data to Visualization, what happens in between?From Data to Visualization, what happens in between?
From Data to Visualization, what happens in between?

Recently uploaded

Igniting Next Level Productivity with AI-Infused Data Integration Workflows by
Igniting Next Level Productivity with AI-Infused Data Integration Workflows Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows Safe Software
225 views86 slides
The details of description: Techniques, tips, and tangents on alternative tex... by
The details of description: Techniques, tips, and tangents on alternative tex...The details of description: Techniques, tips, and tangents on alternative tex...
The details of description: Techniques, tips, and tangents on alternative tex...BookNet Canada
121 views24 slides
Throughput by
ThroughputThroughput
ThroughputMoisés Armani Ramírez
36 views11 slides
DALI Basics Course 2023 by
DALI Basics Course  2023DALI Basics Course  2023
DALI Basics Course 2023Ivory Egg
14 views12 slides
handbook for web 3 adoption.pdf by
handbook for web 3 adoption.pdfhandbook for web 3 adoption.pdf
handbook for web 3 adoption.pdfLiveplex
19 views16 slides
Spesifikasi Lengkap ASUS Vivobook Go 14 by
Spesifikasi Lengkap ASUS Vivobook Go 14Spesifikasi Lengkap ASUS Vivobook Go 14
Spesifikasi Lengkap ASUS Vivobook Go 14Dot Semarang
35 views1 slide

Recently uploaded(20)

Igniting Next Level Productivity with AI-Infused Data Integration Workflows by Safe Software
Igniting Next Level Productivity with AI-Infused Data Integration Workflows Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Igniting Next Level Productivity with AI-Infused Data Integration Workflows
Safe Software225 views
The details of description: Techniques, tips, and tangents on alternative tex... by BookNet Canada
The details of description: Techniques, tips, and tangents on alternative tex...The details of description: Techniques, tips, and tangents on alternative tex...
The details of description: Techniques, tips, and tangents on alternative tex...
BookNet Canada121 views
DALI Basics Course 2023 by Ivory Egg
DALI Basics Course  2023DALI Basics Course  2023
DALI Basics Course 2023
Ivory Egg14 views
handbook for web 3 adoption.pdf by Liveplex
handbook for web 3 adoption.pdfhandbook for web 3 adoption.pdf
handbook for web 3 adoption.pdf
Liveplex19 views
Spesifikasi Lengkap ASUS Vivobook Go 14 by Dot Semarang
Spesifikasi Lengkap ASUS Vivobook Go 14Spesifikasi Lengkap ASUS Vivobook Go 14
Spesifikasi Lengkap ASUS Vivobook Go 14
Dot Semarang35 views
Data-centric AI and the convergence of data and model engineering: opportunit... by Paolo Missier
Data-centric AI and the convergence of data and model engineering:opportunit...Data-centric AI and the convergence of data and model engineering:opportunit...
Data-centric AI and the convergence of data and model engineering: opportunit...
Paolo Missier34 views
Future of Learning - Yap Aye Wee.pdf by NUS-ISS
Future of Learning - Yap Aye Wee.pdfFuture of Learning - Yap Aye Wee.pdf
Future of Learning - Yap Aye Wee.pdf
NUS-ISS41 views
AI: mind, matter, meaning, metaphors, being, becoming, life values by Twain Liu 刘秋艳
AI: mind, matter, meaning, metaphors, being, becoming, life valuesAI: mind, matter, meaning, metaphors, being, becoming, life values
AI: mind, matter, meaning, metaphors, being, becoming, life values
Special_edition_innovator_2023.pdf by WillDavies22
Special_edition_innovator_2023.pdfSpecial_edition_innovator_2023.pdf
Special_edition_innovator_2023.pdf
WillDavies2216 views
.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV by Splunk
.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV
.conf Go 2023 - How KPN drives Customer Satisfaction on IPTV
Splunk88 views
How to reduce cold starts for Java Serverless applications in AWS at JCON Wor... by Vadym Kazulkin
How to reduce cold starts for Java Serverless applications in AWS at JCON Wor...How to reduce cold starts for Java Serverless applications in AWS at JCON Wor...
How to reduce cold starts for Java Serverless applications in AWS at JCON Wor...
Vadym Kazulkin75 views
.conf Go 2023 - Data analysis as a routine by Splunk
.conf Go 2023 - Data analysis as a routine.conf Go 2023 - Data analysis as a routine
.conf Go 2023 - Data analysis as a routine
Splunk93 views
PharoJS - Zürich Smalltalk Group Meetup November 2023 by Noury Bouraqadi
PharoJS - Zürich Smalltalk Group Meetup November 2023PharoJS - Zürich Smalltalk Group Meetup November 2023
PharoJS - Zürich Smalltalk Group Meetup November 2023
Noury Bouraqadi120 views
Architecting CX Measurement Frameworks and Ensuring CX Metrics are fit for Pu... by NUS-ISS
Architecting CX Measurement Frameworks and Ensuring CX Metrics are fit for Pu...Architecting CX Measurement Frameworks and Ensuring CX Metrics are fit for Pu...
Architecting CX Measurement Frameworks and Ensuring CX Metrics are fit for Pu...
NUS-ISS37 views
SAP Automation Using Bar Code and FIORI.pdf by Virendra Rai, PMP
SAP Automation Using Bar Code and FIORI.pdfSAP Automation Using Bar Code and FIORI.pdf
SAP Automation Using Bar Code and FIORI.pdf

Finding Patterns in Temporal Data

  • 1. FINDING PATTERNS IN TEMPORAL DATA osium! CI L Symp KRIST WONGSUPHASAWAT 27th H 010! , 2 May 27 TAOWEI DAVID WANG CATHERINE PLAISANT BEN SHNEIDERMAN HUMAN-COMPUTER INTERACTION LAB UNIVERSITY OF MARYLAND
  • 2. FINDING PATTERNS IN TEMPORAL DATA osium! CI L Symp KRIST WONGSUPHASAWAT 27th H 010! , 2 May 27 TAOWEI DAVID WANG CATHERINE PLAISANT BEN SHNEIDERMAN HUMAN-COMPUTER INTERACTION LAB UNIVERSITY OF MARYLAND
  • 3. TEMPORAL CATEGORICAL DATA •  A type of time series Event! Category! Event! Numerical! Stock: Microsoft Patient ID: 45851737 $'#")#"$!$&!$($$ &0!B$0& !"#$"#"$$%&!'(") &*++,-./& $'#")#"$!$&!$(!; &0!B$!& !"#$"#"$$%&!'(0) &123+43567& $'#")#"$!$&!$(0$ &0!B$"& !"#$"#"$$%&""('' &89:& $'#")#"$!$&!$('; &0!B$%& !"#$;#"$$%&$;($< &=/>>+& $'#")#"$!$&!!($$ &0!B!)& !"#!'#"$$%&$)(!? &1@,A& & Time Arrival Emergency ICU Floor Exit
  • 4. TEMPORAL CATEGORICAL DATA Electronic Health Records: symptoms, treatment, lab test" Traffic incident logs: arrival/departure time of each unit" Student records: course, paper, proposal, defense, etc." " Others: web logs, usability study logs, etc."
  • 5. 10+ years work on temporal visualization (mostly on Electronic Health Records)
  • 6. LIFELINES D SINGLE RECOR [Plaisant et al. 1998] http://www.cs.umd.edu/hcil/lifelines
  • 8. working with physicians at WASHINGTON HOSPITAL CENTER
  • 9. EXAMPLE DATA •  Patient transfers ARRIVAL Arrive the hospital EMERGENCY Emergency room ICU Intensive Care Unit INTERMEDIATE Intermediate Medical Care FLOOR Normal room EXIT-ALIVE Leave the hospital alive EXIT-DEAD Leave the hospital dead
  • 10. TASKS •  Example: Finding “Bounce backs” ICU Floor ICU within 2 days
  • 11. LIFELINES 2 RECORD D R ECOR RECORD RECORD RECORD [Wang et al. 2008, 2009] http://www.cs.umd.edu/hcil/lifelines2
  • 12. Multiple Records ARF (Align-Rank-Filter) Framework Temporal Summary LifeLines2 – Search and Visualize
  • 13. ALIGNMENT •  Sentinel events as reference points Time June July August Patient #45851737 Arrival Emergency ICU Floor Exit Patient #43244997 Arrival Emergency ICU Floor Exit
  • 14. ALIGNMENT (2) •  Time shifting Time 0 1M 2M Patient #45851737 Admit Emergency ICU Floor Exit Patient #43244997 Admit Emergency ICU Floor Exit
  • 15. SIMILAN RECORD D R ECOR RECORD RECORD RECORD [Wongsuphasawat & Shneiderman 2009] http://www.cs.umd.edu/hcil/similan
  • 16. Similan – Search by Similarity
  • 17. Similan – Search by Similarity
  • 18. FINDING “BOUNCE BACKS” Before After •  Much faster to specify new query •  Visualizing the results gives better understanding
  • 19. USER STUDIES: SEARCH LifeLines2 Similan Exact! Similarity-based! MUST have A, B, C SHOULD have A, B, C Query Query Record#2 Record#2 more Record#1 Record#1 similar Record#3 Record#3
  • 20. USER STUDIES: SEARCH LifeLines2 Similan Exact! Similarity-based! MUST have A, B, C SHOULD have A, B, C Query Query 1 Record#2 Record#2 more Record#1 Record#1 similar Record#3 Record#3
  • 21. NEW STUFF Needs for an overview -> LifeFlow!!
  • 22. TASKS •  Example: Finding “Bounce backs” ICU Floor ICU within 2 days •  Other questions Arrival ? ICU ? ?
  • 23. LIFEFLOW RECORD D VISUALIZE R ECOR RECO Display the aggregation RD RECORD RECORD RECORD RECORD RECORD AGGREGATE Merge multiple records into tree
  • 24. AGGREGATE •  Aggregate by prefix #1 #2 #3 #4 Example with 4 records
  • 25. AGGREGATE •  Aggregate by prefix #1 #2 #3 #4
  • 26. VISUALIZE •  Inspired by the Icicle tree [Fekete 2004] Number of files!
  • 27. VISUALIZE (2) •  Use horizontal axis to represent time •  Video
  • 28. DEMO – LIFEFLOW When the lines are combined into flow!
  • 29. FUTURE WORK •  Comparison ICU ICU Intermediate Intermediate Jan-Mar 2008 April-June 2008 Floor
  • 30. TAKE-AWAY MESSAGE Information visualization is a powerful way to explore temporal patterns. You can work with us on new case studies.