The document discusses developing a data model and knowledge organization system to represent affective aspects of visitor experiences with artworks. The author conducted research involving interviews and physiological feedback with museum visitors viewing three artworks. Responses included straightforward feelings and more complex responses involving empathy and identity. The model represents experiences as relationships between visitors, artworks and contexts. Complex responses were difficult to model and indicate aesthetic experiences involve imagined links between viewers and artwork subjects beyond simple attributes.

1. Affect in information systems
a knowledge organization system approach to
documenting visitor-artwork experiences
Erin Canning
Digital Platform Administrator, Aga Khan Museum
MW19 | Boston
Boston, Massachusetts, USA, April 2-6, 2019

2. Affective metadata for object experiences in
the art museum (University of Toronto, 2018)
Affect:
a culturally,
socially, and historically
constructed category
that both encompasses
and reaches beyond
feelings and emotions
(Cifor, 2016)
How can affective properties of artworks and
artwork-visitor experiences in the art museum be
represented and structured so that they can be
integrated into a data model and corresponding
knowledge system that accurately represents
the affective elements of the artworks and
experiences, while maintaining the agency of the
object, the exhibition context, and the viewer?

6. Model Mapping
Legend
Purple lines – translate to entities
Orange lines – translate to authority records for reference
Green lines – model domain (object experience)

7. Additional entities proposed:
• AE1_AffectivePotentialAssignment
Subclass of: E13_AttributeAssignment
Superclass of: None
• AE2_Session
Subclass of: E7_Activity
Superclass of: None
Additional properties proposed:
• AF1_occurs_in_context_of (was_context_to)
Doman: E5_Event
Range: E5_Event
• AF2_captured (captured_by)
Domain: AE1_AffectivePotentialAssignment
Range: AE2_Session
• AF3_elicited (elicited_by)
Domain: AE2_Session
Range: E28_ConceptualObject
• AF4_occurred_at (happened)
Domain: E28_ConceptualObject
Range: E61_TimePrimitive

8. Model validation and acquiring a data set
The Marchesa Casati, 1919
Augustus Edwin John
oil on canvas
96.5 x 68.6 cm
Art Gallery of Ontario
Nude with Clasped Hands, 1905-06
Pablo Picasso
gouache on canvas
96.5 x 75.6 cm
Art Gallery of Ontario
Interior with Four Etchings, 1904
Vilhelm Hammershoi
oil on canvas
62.0 x 51.2 cm
Art Gallery of Ontario

9. Findings
Artwork Example Simple Response Example Complex Reaction
The Marchesa Casati, 1919
Augustus Edwin John
oil on canvas
96.5 x 68.6 cm
Art Gallery of Ontario
Interior with Four Etchings, 1904
Vilhelm Hammershoi
oil on canvas
62.0 x 51.2 cm
Art Gallery of Ontario
Nude with Clasped Hands, 1905-06
Pablo Picasso
gouache on canvas
96.5 x 75.6 cm
Art Gallery of Ontario

10. Findings
Artwork Example Simple Response Example Complex Reaction
The Marchesa Casati, 1919
Augustus Edwin John
oil on canvas
96.5 x 68.6 cm
Art Gallery of Ontario
”I felt immediately captivated. I
couldn’t stop looking at her face,
it was magnetic."
Interior with Four Etchings, 1904
Vilhelm Hammershoi
oil on canvas
62.0 x 51.2 cm
Art Gallery of Ontario
“I felt sad. I could see depression
present in it. It’s definitely not
full of energy or life… It is
emotionally heavy.”
Nude with Clasped Hands, 1905-06
Pablo Picasso
gouache on canvas
96.5 x 75.6 cm
Art Gallery of Ontario
“It made me feel calm. I could
feel the calmness in the painting
as something that reached out
to the viewer, to me.”

11. Findings
Artwork Example Simple Response Example Complex Reaction
The Marchesa Casati, 1919
Augustus Edwin John
oil on canvas
96.5 x 68.6 cm
Art Gallery of Ontario
”I felt immediately captivated. I
couldn’t stop looking at her face,
it was magnetic."
“I am awed by a woman who lived her life like
that, especially at that time... It [the artwork] is a
reminder of women who came before us, paved
the way, gave the gift of that to women – even if
they did it for themselves, it impacted all of us.”
Interior with Four Etchings, 1904
Vilhelm Hammershoi
oil on canvas
62.0 x 51.2 cm
Art Gallery of Ontario
“I felt sad. I could see depression
present in it. It’s definitely not
full of energy or life… It is
emotionally heavy.”
“I could imagine myself in that house, which
made me feel anxious. The thought of being
there is unpleasant... I felts like I was being
drawn into the place – I would not want to be in
that room.”
Nude with Clasped Hands, 1905-06
Pablo Picasso
gouache on canvas
96.5 x 75.6 cm
Art Gallery of Ontario
“It made me feel calm. I could
feel the calmness in the painting
as something that reached out
to the viewer, to me.”
“It made me think of feeling pressure to make
myself align with social views of women... It
made me really think of the weight of all that
pressure.“

12. Addressing the
question of
empathy:
responding to
entities depicted
in an artwork,
as well as the
artwork as a
whole

13. trees
me, wondering if I’ve lost sight of the forest for the trees
The project in
hindsight:
looking back and
thinking forwards
forest

14. … which brings us back to “what is this” and “why”
hello yes why did you make us read this

15. Affect and museum missions: Active Collections
One way to tell that an object is active and holds
deep meaning for museum audiences is when it
elicits deep emotion – joy, pride, shame,
heartbreak.
— Wood, E., Tisdale, R. & Jones, T. (2018). Active
Collections. http://www.activecollections.org
“

16. Affect and museum missions: Measuring impact
Te Papa Audience Impact Model
Kingston, A. (2018). Te Papa’s Audience Impact Model: Beyond foot traffic and vanity
metrics [Google Slides]. Retrieved from: http://bit.ly/NDF2018AIM
Attention à Emotion à Action
Attention Reaction Connection Insight Action
1 2 3 4 5 6 7 8 9 10
Attention
caught
Immediate
response
Personal
connection
Simple
learning
Contextual
learning
Applied
personal
learning
Applied
empathetic
learning
Personal
action
Group /
community
impact
National
impact

17. Affect and museum missions: Radical empathy
Radical empathy offers a way to engage with others’ experiences that involves
discarding the assumption that we share with them the same modal space of
belonging in the world. Our conception of empathy is radical in its openness
and its call for a willingness to be affected, to be shaped by another’s
experiences, without blurring the lines between the self and the other. […]
Practising radical empathy with users means acknowledging the deep
emotional ties users have to records.
— Cifor, M. & Caswell, M. (2016). “From Human Rights to Feminist Ethics:
Radical Empathy in the Archives.” Archivaria, [S.l.], p. 23-43.
“

18. I’m interested only in expressing basic human emotions –
tragedy, ecstasy, doom, and so on. And the fact that a lot of
people break down and cry when confronted with my pictures
shows that I can communicate those basic human emotions...
The people who weep before my pictures are having the same
religious experience I had when I painted them.
— Mark Rothko (Shimamura, 2013, p. 237)
“

19. Thank you! Questions?
twitter: @eecanning
email: erin.canning@akdn.org
Slide 14:
The Sampling Officials of the Amsterdam
Drapers’ Guild, Known as ‘The Syndics’, 1662
Rembrandt van Rijn
oil on canvas
191.5 cm × 279 cm
Rijksmuseum
Slide 13:
Big Basin Redwoods Forest, c. 1899
Andrew P. Hill
matt collodium printing-out paper
19.2 cm × 24.0 cm
Rijksmuseum

20. This talk, and the corresponding paper, is based on my Master’s thesis on affective metadata for art object experiences, in which I aimed to determine howdocumented evidence of affective aspects of visitor-artwork experiences in museums can be structured in a way that can be integrated into a data model and corresponding knowledge organization systemthat accurately represents the nature of the experience while
maintaining the agency of the object, context, and viewer. For this, I proposed a schema and controlled vocabulary with which to document affective qualities of viewer-artwork experiences, and validated it through empirical research, involving combined methods of interviews, questionnaires, observations, and heart rate variance tracking in order to produce a data set covering the affective aspects of the experiences
that visitors had with a small set of predetermined artworks.The focus of my presentation here today will be on the conceptual modeling part of this work, resulting in model development using CIDOC CRM, as well as the impact that the results of my validation process had on this model. I’malso going to focus on the ideas behind this as tech specs don’t translate super well to presentations, and while they are covered
more in the paper, what I’d like to do is get you excited about the ideas behind affective metadata, and showyou what kind of newknowledge affect can bring to your collections data.
So the first idea to talk about, is the concept of experience. In order to develop a data model, it is important to define the domain. In this case, while I amtalking about affective aspects of a visitor-artwork experience, at its heart I amtalking about that experience, and what entities it is comprised of.
So, howdo we start to make sense of an aesthetic experience? We can begin by breaking it down into the core entities involved: A person, who brings in their own background, knowledge, and history; the context of the experience, namely where and when it is taking place, as well as the nuances of that context; and the artwork that is the subject or focus of the experience.
So if these are the pieces that make up an aesthetic experience, howcan we learn about the details of that experience?
After all, what we would be documenting in what I amtalking about as the documentation of affective aspects is not just the experience on its own, a person going through an internal process, but an experience that is being researched in some way, as research and initial documentation or recording must occur in order for us to have this data in the first place.
So for this part of it we’re going to have two broad entities – a session, which is the research and evidence, the documentation of the research methods; and then what I’mgoing to refer to as an actualization, which is the translation of that research data into a structured understanding of affect, referencing a controlled vocabulary like a thesaurus to ensure a consistent point of reference. The session involves a person
and a context – as it takes a person to study, and occurs within a larger viewing context – as well as the responses that are captured. Responses can take various forms, fromself-responses in the formof interviewand questionnaire answers, to information about behaviours, facial expressions, heart rate variance, skin conductance, pupil dilation – any method through which we come to understand aesthetic experiences.
And then we have the actualization. This doesn’t belong solely to the object or the person experiencing the object, but to both, and to the experience. Therefore, affective characteristics of an experience with an artwork lie in potential in relation to the artwork itself, but require an experience in order to be actualized, to be known. The thesaurus we use can be referenced to by the actualization entity, in order to say, this
is what we mean, these are the terms that were used by the participant in the session, in order to describe the affective experience with this artwork.
This kind of informal model can then be translated into an data model, which is what you see here on the right. This model shows a streamlined profile of CIDOC CRM, showing just the entities directly used in this area, as well as a small number of additions. It expands on the major entities that I just touched on which are required to understand and document the affective attributes of an experience, and the relationships
between these entities. As CIDOC-CRMfocuses on documentation, this proposed model aims to accurately represent the elements required to structure the documentation of affective experience. I’ve highlighted the areas that the existing CRMwould need to be supplemented in order to support the integration of highly subjective and relational information such as affective response.
In this model, the actualization of an affective response is represented as a type of attribute that is assigned to an object and is tied to the session it came out of. This shows it to be similar to the assertion of other points of data about an object such as mediumand size. However, while other details may be modeled as being relatively straightforward, with properties of unit and value, actualization is modeled as the
recorded final data point in a process that involves the actor, object, and context in a sessionwhich is documented by the assertion of a point of affective metadata.
In order to gather a ground truth data set with which to validate the model, I conducted a small, 12-participant visitor response study on three previously identified artworks – which you can see here on the screen – at the Art Gallery of Ontario. My research instruments included a participant profile questionnaire, a field questionnaire, interviews, observational tracking, and physiological feedback in the formof heart rate
variance. The inclusion of physiological feedback and behavior tracking supports a mixed-methods approach to data gathering, which allowed me to explore howto incorporate the different kinds of research methods already being used in museumvisitor studies and empirical aesthetics research in the proposed model.
Before talking about findings, I would like to reiterate that I aminterested in information systems and their design, and so that was the focus of this research. As such, I won’t be able to talk about response trends or anything like that, because I simply didn’t acquire enough data for that kind of analysis, as doing so wasn’t the goal of the study. However, I amgoing to talk broadly about the kinds of responses that I
experienced, and howthat reflects on the model. So given all this disclaimer, there were some really interesting responses, and having themcoded against an affect thesaurus and stored in a database this way allows both for easy analysis and interrogation, and connects the instances as historical elements to the artwork in question. This means that not only are we preserving the records of these visitor experiences
connected to the object in the institutional source of truth for object knowledge – the collections information management system– but we are making it so that we can growa database of experiential and affective information, and later performlong-viewanalyses on the data.
Broadly speaking, I encountered two types of responses: relatively straightforward ones, where the participant had a felt response to the artwork in question, and then more complex ones that incorporated elements of the participant’s identity, and where they described felt connections to aspects of the artwork. The majority of responses were relatively straightforward. You may note that this is only about the responses
gathered fromself-response methods: this is because I found that the behavior and heart rate variance tracking largely complemented participant’s answers but weren’t useful on their own. It was only through talking with my participants that I could really understand what they were feeling and experiencing. Additionally, behavior and heart rate were easy to accommodate in the data model, whereas the complex
responses caused me some difficulty.
In the more complex responses, two main practices came up: one, is that participants responded to the figure depicted in the artwork in a similar way to howthey might respond to a real, living person, and two, often processes of empathy and imagination were engaged in as part of the participant’s experience with the artwork. Participants discussed imagining themselves in the place of the subject, what they would feel
if they were the subject, or what it would be like to be in the physical or emotional space that the artist depicted the subject as inhabiting. This then influenced their affective responses to the artwork. They would say things like “I feel bad for her”or “If I were her, I would feel annoyed”. It became clear that not only were participants experiencing elicited affects directly, but that they had affective responses that came as a
result of these cognitive connections. These kinds of responses also highlight the difference between affect and emotion– these aren’t just emotional responses, but other felt ways of connecting one’s body to the world. One participant talked about the artwork reminding themof the weight of social pressures for howa female-presenting body “should” appear, and their personal history and struggles to navigate that.
That is not emotion, but distinctly affect, that is occurring as part of their felt reaction to the artwork in question. I really struggled to accommodate these kinds of responses into the model, because they involve many steps in an internal process, and can be seen more as relationships that viewers have with artworks than responses to the artworks. This presence of empathy in aesthetic response seems indicative of
something that is more like an imagined event that links the viewer and a represented entity in an artwork than simply a property or attribute of an affective experience.
So howcan we accurately represent these kinds of complex responses in this model? he first step is creating a path for responses to be mapped to elements of the artwork, and not just the artwork as a whole. To do this, the area of the data model dedicated to the object could be expanded to explicitly reference entities depicted within the artwork. The property that links together Responses and artworks could then also
be used to link these depictions with Responses. However, this is not sufficient to address the question in its entirety. While it shows the relationship between responses to depicted entities within an artwork and responses to the artwork as a whole, it doesn’t address the difference in internal process that occurs when imagination and identity-based meaning-making occurs as part of the affective response. The question
remains of howto structure and integrate the complex information about these kinds of empathy-based affective relationships with artworks.
At the end of all this, I think the data model I tested out was a good first try, but I was left with some reservations. While this model is able to capture a wide range of common affective responses, it struggles to deal with the ones that may in fact be the most meaningful, both to the viewer experiencing themand the museuminterested in knowing about them. Mapping in the existing documentation in the formof direct
research data was not a problem, but creating an accurate representation of the results was when it came to these more complex responses. And while I amsomewhat tempted to say “okay, no problem, lets return to the CIDOC mandate of model-the-documentation” and leave it at that, that doesn’t really address the core question here. Because my goal isn’t to map in audience research data, my goal is to create a
systemto relate affective aspects to an artwork. And these complex pieces are highly informative affective aspects. To leave themout because they don’t fit the basic stimuli-response model would to create an inaccurate representation of affective response. Where I do think there is merit in simplification, however, is in the level of complexity with which I approached this problemof mapping affect. I came at this whole
endeavor as an academic, but nowhave been working in the field full time, and amgetting better, I would like to think, at understanding the level of depth and detail that would be practical for a museumto both gather this kind of data as well as make meaning fromit. I therefore have questions of, What level of granularity brings us understanding, and helps us not get too lost in the minutia?What do we learn when we
ask questions to different levels of complexity, and in different ways? And then, what about other spaces of encounter, outside of the physical gallery? Howcan we use this framework to compare social media acknowledgements of affect vs. in-gallery ones? I think answering these kinds of questions will help chart a path forward for the next iteration of this model.
So with all this in mind, I’d like to return to what a systemlike this is doing, and why this is important. This sort of system, which takes event-centricity as a starting point, incorporates contextual, subjective, and shifting information, and involves visitor experiences in documentation, suggests a way for collections management systems to be augmented so that they can bring further understanding to museumobjects by
placing themwithin the context of their affective meanings and the roles that they play for museumvisitors. And this is key to think about, because as museums, we’re not just holders of physical objects. We create stories and experiences, and visitors are key to our missions and our existence. So why should our key systems for holding institutional memory be leaving this work out of their domain, especially when it can
be tied back to objects? When I first started this project, I came across blog posts and conversations between people here at this conference, where this idea was being discussed about the dreamof museumsoftware that grewout of missions and practices, as opposed to simply inherited structures made digital. If, at a pared down level, the mission of a museumis to connect people with collections in meaningful ways,
then why is that exact information not given a place in our documentation, in our systems? What if we could have a systemthat made space for the connections between people and objects, and what these connections have been like? That information would not only tell us vastly newthings about our collections, but would also be a support to wider museumpractice. This line of “software structured for museum
missions” really stuck with me, and incorporating affect and experience in is a key part of making that a reality.
Our objects can support our missions. I think sometimes notions of experience and visitor-centricity can focus on exhibitions, or projects, or presentation, but collections are a key part of it all. And if this is the case, we can ask questions like, Does the way that we have presented and contextualized a particular object support our goals? Does a given object have the potential to be put in a particular role? Active Collections
argues for a change in conversation fromvalue of objects froma point of rarity or monetary value, to value froma point of experience; to stop talking about the size of a collection and start talking about its impact. Taking this argument, we then require information systems that are capable of documenting this kind of information about objects. If we are to consider our objects in light of their potential impact, as the
Active Collections manifesto recommends, then we need information systems that support these practices. We need information systems that are capable of relating information about experience, impact, and affect with objects.
There is also a real need to assess goals of experience and impact in museumpractices. This can be seen in projects like the Audience Impact Model fromNewZealand’s Te Papa Museum, which seeks to answer questions like “What is the value of what we offer? What is the impact of what we do on our audiences?”. In order to develop this evaluative model, they first came up with a high-level framework of Attention to
Emotion to Action, and then refined it to Attention – Reaction – Connection – Insight – Action, covering a total of 10 stages. Aspects of affect can be seen at many of these levels. This is a kind of model of evaluation that could be used to produce documented evidence that could then be incorporated into an information systemsuch as the one I amproposing. What I really like about this framework, too, is that it doesn’t
stop at the individual having the experience in the museum, but then asks: so what. What does the person do with that, what continued impact does that experience have? How, through this person, is the collection and museumliving and affecting a wider realm? And that brings me to my last point, which is the concept of radical empathy.
Whereas empathy is the “ability to understand and appreciate another person’s feelings and experiences”, radical empathy is “a willingness to be affected, to be shaped by another’s experience, without blurring the lines between the self and the other”. In essence, radical empathy says that empathy isn’t enough, that we need to use empathy to do better in our lives at micro and macro levels. It’s allowing the experiences
we have through empathy to change us, and then asks us to help better our personal communities to become more understanding and empathetic as a whole. This seems to be exactly the sort of thing that the Te Papa’s Audience Impact Model looks to assess: howmuseums can evaluate their success at instilling this in their audiences. To support these missions institutionally, we need systems that support this kind of
practice and allowus to engage in this work, not systems that at best make things difficult and at worst fight against us. We need systems that allowus to record information about our objects that connect themto our missions and the myriad ways that our audiences forge connections, both in the physical museumspace and online. We don’t need the persistence of a systembased on traditional cataloguing practices,
but one that allows us to bring in other ways of knowing and types of knowledge, and showthose as valid. An extra field on a record is not enough: this requires rethinking the model.
What it comes down to, I think, for me, is that I believe strongly in two things: I believe in infrastructure that supports needs, and I believe in the power of museums and objects. I think museums can use the experiences created within them, and by them, to help people move fromengagement through empathy, to radical empathy – empathy that inspires action and change, whether that occurs at a personal level or a
broader one. But if our infrastructure is to support this, then it needs to be able to accommodate the information that speaks to this: it needs to accommodate affect. These experiences are already happening; what we don’t have, though, is an information systemthat incorporates that information and shows it as the object knowledge that it is. And that is what I amtrying to do here, with a first attempt at modeling
affective response.
Museumcollections information management systems can be more than they currently are. They have the potential to be transformed to support a museum’s mission and practices while also serving as a repository for information – in this case, information about the artwork that goes beyond traditional paradigms. Affective attributes make up a part of what an artwork is and does, and can only be seen through viewers’
experiences with the artwork. By providing a way to structure and represent this information, we can learn more about artworks, and howand why they come to hold places of importance and meaning. With further model refinement and validation, and the consideration of complex, intersectional, and identity-based affective responses, affective metadata could become a realistic future for object documentation, in
which the boundaries of what is thought of as object information worthy of documentation within collections information management systems are expanded and redrawn.