Visusalisation of Video, by Ben Kraal

Visualisations of Video. Ben Kraal People + Systems Lab QUT What I’m going to talk about today is how to turn video into data, make that data useful and get some insights from it. [fade in] In other words, how to create visualisations of video.

Hi, I’m Ben. @bjkraal

I work at the People + Systems Lab, QUT. We do Design Research.

These are some of the projects that have been associated with the P+S Lab in the past. We’re focussed on learning about context, activity, culture and people.

context activity culture people And we use those as our basis to move our focus from objects to doing design research about systems and experiences.

early design stage before design started finished artifact design and development This is an (overly simple) explanation of how design research fits into practice. On the left, you can do design research before you’ve started design. On the right, you can do DR with a finished artefact. In the middle is practice. Practice is where you’ll find techniques like user research in the early stage of design or usability testing in the development phases.

Visualisations of Video. What I’m going to talk about today is how to turn video into data, make that data useful and get some insights from it. [fade in] In other words, how to create visualisations of video.

UX is off the desktop and in the world.

Designers will do more research.

Use Video to collect data.

Video makes data. Lots of data. This is the tapes from two small studies. At P+S we will often use two cameras concurrently to capture two different views simultaneously. Depending on how we record the footage we either mix the two cameras picture-in-picture or we use the power of our analysis software to play the footage simultaneously. Purely from a logistical point of view, we generate hundreds of gigabytes of video each year, pushing into terabytes, and that’s before we do anything with it.

Visualise your video. The best way to deal with all this video is to to turn it into data and then create visualisations to describe that data.

More than you ever wanted to know about I’m going to use a case study to explain these ideas. The case study is about a project we started a few years ago. It’s about Leg Ulcers. All about Leg Ulcers: * sores * venous insufficiency (varicose veins etc) * compression therapy * nurses * assessed by instrumenting healthy legs and testing whether achieves ideal pressure

Here’s a still from some of the footage I took. Nurse on the ﬂoor. Patient on the chair. Patient has her left leg completely bandaged and the right is partially done.

Visualise your video. Back to visualising your video.

Code your video. Before you can visaulise your video, you need to code it. Coding data is a practice that comes from the social sciences. A lot of the time they can code transcripts of interviews. Coding video is quite similar.

Tag your video. Another way to consider coding data is to think about tagging it.

Tag parts of your video. Or, more fully, to think about tagging parts of it.

Turn video into data. And this tagging, or coding, is what turns video into data.

We use Noldus Observer.

Noldus Observer looks like this. Coding scheme in the centre. The way it works is to allow you to establish a coding scheme and then as the parts of the video that are described in the coding scheme occur, the software makes it very easy to track.

But you don’t have to. Now, we use Observer, but you don’t have to.

Visualisations make video “sit still”.

Represent the data in different ways. And let you represent data in different ways.

25:00 24:00 23:00 22:00 21:00 20:00 Eg, like time-series charts. Like this one that describes some bandaging activity. 19:00 18:00 17:00 16:00 15:00 14:00 13:00 12:00 11:00 10:00 09:00 08:00 07:00 06:00 05:00 04:00 03:00 02:00 01:00 00:00 doing planning reacting materials

Aggregate your data. And then you can graph it, or slice and dice it in new and interesting ways.

planning doing reacting materials 00:00 01:00 02:00 03:00 04:00 05:00 06:00 07:00 08:00 09:00 10:00 11:00 12:00 13:00 14:00 15:00 16:00 17:00 18:00 19:00 20:00 21:00 22:00 23:00 24:00 25:00 This chart describes 25 minutes of activity. So you can see the power already. 25 minutes of video on one slide that you can take in and see the shape of in 30 seconds. Now, to explain this a little better, I’m going to make up a ﬁctional example.

00:00:00 00:05:00 This shows 10 minutes of (ﬁctional!) bandaging activity. But that’s not really very exciting.

00:00:00 00:05:00 stocking undercast type 2 bandage stocking It’s much more interesting if we can describe the activity in more detail. Eg: which bandaging materials are being used and for how long. We’ll assign colours to different bandage types. Yellow can be a thin cotton stocking. Orange can be undercast. Purple can be a type 2 bandage. And then another thin cotton stocking to hold it all together. [these should fade in.]

00:00:00 00:05:00 stocking undercast type 2 bandage stocking If you wanted, you could code for other behaviours, or activities, too. You might code for taping bandages down. [fade in] Or for using scissors to cut bandage to length. (stocking comes in a roll, so you cut it before you use it.) Or tearing it to length, as you can do with undercast.

Visualisations let you make comparisons.

Compare tasks across observations.

Or within observations.

Combine levels of detail.

Ask questions about activity and task relationships.

Visualisations add dimension to analysis. Visualisations add dimension to analysis. And let you add dimensions to analysis.

00:00:00 00:05:00 stocking undercast type 2 bandage stocking Which brings us back to here. We didn’t ﬁnd that knowing where bandages were ﬁxed, taped or torn was useful, so we stopped looking for it. [fade out] And we even found that looking only at the bandaging activity wasn’t that useful (because the way that bandages are used is proscribed, so we weren’t going to learn about non-standard practices, nor did we want to), so we took a longer view.

00:00:00 00:10:00 preparing doing s uc t2 s [This should start with just the vertical bars and the 00.00.00. Zoom the 10.00 in. Taking this back to a wider view, of ten minutes, and bringing the same bandaging activity [click] in I can introduce some other codes, for activity at a higher level than bandages. First I’m introduce a code that encompasses all the bandaging activity: doing [fade in]. Which leaves all this time before the bandaging took place. Now, in the video of the nurse she spends quite a lot of time before doing the bandaging preparing the work space. So we can call that preparing. Everyone with me?

planning doing reacting materials 00:00 01:00 02:00 03:00 04:00 05:00 06:00 07:00 08:00 09:00 10:00 11:00 12:00 13:00 14:00 15:00 16:00 17:00 18:00 19:00 20:00 21:00 22:00 23:00 24:00 25:00 Now, you’ve seen this before, but now I trust you can follow it. [talk through chart] Now, there’s something on here that I haven’t explained yet, and that’s the code that’s shown in pink about 3/4 the way through. And right now it doesn’t matter, and I’ll get to it later.

Expertise. The chart I’ve just shown you was of a nurse who demonstrated high expertise in executing the application of the bandages.

Less Expertise. Now let me show you one with slightly less expertise.

00:00:00 00:10:00 p p preparing d doing d s uc t2 s [fade in from l-r and t-b] Preparing, doing bandaging, preparing next two, then doing. Then preparing last one, then doing. We say this demonstrates less expertise because pattern of preparing and doing is broken up. There is not a demonstration of the nurse being able to fully imagine the end-to-end bandaging process and so prepare everything before executing it. Instead, we see a partial sort of preparation and doing.

25:00 24:00 23:00 Now, what I showed hypothetically before is seen in this actual chart. Let me zoom it in. 22:00 21:00 20:00 19:00 18:00 17:00 16:00 15:00 14:00 13:00 12:00 11:00 10:00 09:00 08:00 07:00 06:00 05:00 04:00 03:00 02:00 01:00 00:00 doing planning reacting materials

planning doing reacting materials 00:00 01:00 02:00 03:00 04:00 05:00 06:00 07:00 08:00 09:00 10:00 11:00 12:00 13:00 14:00 Now, this is a different nurse and a different set of bandaging materials. As you can see, it’s basically the same pattern with four different types of compression therapy working together. And, during the bandaging you can see that the nurse takes some brief pauses in the middle to gather the next materials she needs. Now, because she only takes breaks after completing each bandage, we say that she demonstrates experience because she knows the individual component tasks, but she doesn’t pull them together as ﬂuidly as the ﬁrst nurse.

Less Expertise. So that was a demonstration of lower expertise.

Inexperience. Now, what inexperience looks like. I’ll show you a real chart and not to a hypothetical.

planning doing reacting materials 00:00 01:00 02:00 03:00 04:00 05:00 06:00 07:00 08:00 09:00 10:00 11:00 12:00 13:00 14:00 First, in this chart, there’s no initial preparation because the activity started before I turned on the camera. ;) But in the video it’s possible to see that the nurse has prepared most of what she needs on a small tray. She begins and then has a break in the “doing” with no corresponding turn to “preparing” -- she’s asking another nurse for help. And that’s what the pink is, not “asking for help” but “breaking away” or shifting focus from the completion of the bandaging activity to some other activity. Then she resumes the activity and there is a large break at about the 5 minute mark. She’s demonstrating the bandage shown in green to the patient. Then she begins but soon after has a large focus shift *while doing* the bandaging. She’s winding it up the patient’s leg, deciding that it’s not right and winding it back down again. Then at the end she has to stop and gather the ﬁnal material before applying it.

planning doing reacting materials 00:00 01:00 02:00 03:00 04:00 05:00 06:00 07:00 08:00 09:00 10:00 11:00 12:00 13:00 14:00 15:00 16:00 17:00 18:00 19:00 20:00 21:00 22:00 23:00 24:00 25:00 planning doing reacting materials 00:00 01:00 02:00 03:00 04:00 05:00 06:00 07:00 08:00 09:00 10:00 11:00 12:00 13:00 14:00 15:00 16:00 17:00 18:00 19:00 20:00 21:00 22:00 23:00 24:00 25:00 planning doing reacting materials 00:00 01:00 02:00 03:00 04:00 05:00 06:00 07:00 08:00 09:00 10:00 11:00 12:00 13:00 14:00 15:00 16:00 17:00 18:00 19:00 20:00 21:00 22:00 23:00 24:00 25:00 And to go back, here’s all three charts at once. The nice regular chart of the nurse who demonstrated high expertise, the slightly less regular and more “busy” chart of the demonstration of slightly less expertise and the chart of the inexperienced nurse. I should say that we did do more analysis than just these three observations. I’ve shown these three because they illustrate the story. Also, the signiﬁgance and innovation of this approach needs to be underscored. Creating these visualisations is not especially hard, but using them in this way for analysis is new and adding the extra dimensions to describe activities at a higher level while maintaining the lower-level descriptions is where the power lies.

So we did this research. Now what? From doing the research we learned three things.

Expertise is visible in the organisation of activity. First, we learned that expertise is visible in the organisation of activity. This was something that the expert nurses could see as well, but they had no way to describe it. They just “knew” if someone was good. Maybe they saw something else, too, but our new heuristic is very useful, not least because it means that there is a shortcut to seeing bandaging expertise in the ﬁeld, rather than in a lab.

Change the training? the assessment? The second thing is we were able to ask whether the nurses should change their training or assessment for compression therapy.

Develop a new treatment? And ﬁnally, we were able to start asking questions about how it would be possible to develop new treatments that were, for example, less labour-intensive, while still taking into account all the really valuable knowledge that the nurses have about what they do. Investigating these existing practices in order to develop something new isn’t something we’ve completely pursued, so I can’t describe what the new product or system would look like.

Design research with video is hard. To bring this to a close, Design research with video is hard. It takes a long time and you end up with a lot of unwieldy data.

Create visualisations to work with video. One way to work with that data is to create visualisations of the video in order to turn it into something that’s easier to work with. Creating the visualistions takes time, but if you’re going to really work the data and look for something new, taking that time is worth it. When doing design research, particularly of this type which is of an existing product or situation but with a view to develop something new, this kind of in-depth analysis is valuable and can lead to signiﬁcant insights. But you can’t have the insights without the research and analysis.

Understand and planning doing analyse interaction reacting materials 00:00 01:00 02:00 03:00 04:00 05:00 06:00 07:00 08:00 09:00 10:00 11:00 12:00 13:00 14:00 15:00 16:00 17:00 18:00 19:00 20:00 21:00 22:00 23:00 24:00 25:00 through visualisation. Ben Kraal People + Systems Lab, QUT @bjkraal This is the one thing I want you to take away. You can understand and analyse interaction through visualisation of activities.

Further Reading Popovic, Vesna and Kraal, Ben J. (2008) Focus- Shift: Interaction and Expertise Level. In: Design Research Society 2008, 16-19 July 2008, Sheffield, UK. http://eprints.qut.edu.au/14673/ Kraal, Ben J. and Popovic, Vesna (2007) Looking for Expertise in Physical Interactions. In: OZCHI 2007 - Australasian Computer-Human Interaction Conference, November 28-30, Adelaide, Australia. http://eprints.qut.edu.au/11083/

http://youngandbrilliant.net	21
http://www.uxaustralia.com.au	15
http://safe.tumblr.com	9
http://a-me.tumblr.com	1
http://benkraal.tumblr.com	1
http://translate.googleusercontent.com	1

Visusalisation of Video, by Ben Kraal

by benkraal on Aug 31, 2009

Accessibility

Categories

Tags

Upload Details

Usage Rights

6 Embeds 48

Statistics

Visusalisation of Video, by Ben Kraal — Document Transcript