Warren Hayes Ai Symposium - working draft

what we are/and are not doing • images themselves are our main topic, rather than as illustrations of other topics, linking information to and from them and linking the images themselves • Interested in using linked data to represent the kinds of things artists themselves say about their work (or the person doing the markup) • What we don’t do: expect people to agree on artistic interpretations or emotional content of images • ignore our paper in the proceedings – it was a working draft and we have made significant changes. © 2009 - 2010 Margaret Warren, Pat Hayes, CARMA Productions 3/23/2010

Image identity • By Image, we mean a digital image, typically one that can have associated EXIF data. • So, other kinds of "images" (drawings, paintings, non-digital photographs, stained glass, etc..) have to be classified as Works. • A work may be a physical thing with dimensions, a location, a time and place of creation, etc. that is separate from those same properties in an image; a digital image can be a work. • Basic relation between Work and Image is facsimile, a kind of 'visually same As' relation. © 2009 - 2010 Margaret Warren, Pat Hayes, CARMA Productions 3/23/2010

shows, depicts, subject, conveys Shows: trees, grass, house, window, ventilator Shows: head Depicts: particular type of car, Depicts: hands each of the people Conveys: grief, sadness Subject: group looking at a car © 2009 - 2010 Margaret Warren, Pat Hayes, CARMA Productions 3/23/2010

Pictorial elements Basic graphical 'unit' of depiction is a pictorial element. This includes 'boxes' but also lines, regions, areas, patches of color, texture, etc.. “ elements of art - The basic components used by the artist when producing works of art. Those elements are color, value, line, shape, form, texture, and space. The elements of art are among the literal qualities found in any artwork. (Artlex, 1996-2010) Pictorial elements are related by inclusion: 3/23/2010 © 2009 - 2010 Margaret Warren, Pat Hayes, CARMA Productions

Depicts vs. looksLike Pictorial Elements: Shape of thing that looks like a female torso Shape of thing that looks like a representation of a brain area that looks like the torso is sitting astride a seat of some sort Title: Surfing With Psyche Depicts: egg, a test tube, a photographic transparent layer, a piece of jewelry © 2009 - 2010 Margaret Warren, Pat Hayes, CARMA Productions 3/23/2010

Pictorial Elements can be 'empty' A reclining nude is depicted by white space defined by an outline. The white space is a pictorial element. Cataloging pictorial elements is a large task. As a beginning, we distinguish visible/implied and line/area. Here, the nude is depicted by an implied area. All the visible elements are red lines. © 2009 - 2010 Margaret Warren, Pat Hayes, CARMA Productions 3/23/2010

backgrounds hasPictorialBackground hasDepictedBackground all white, not necessarily sky, clouds, not necessary able to define with a region to define with a region © 2009 - 2010 Margaret Warren, Pat Hayes, CARMA Productions 3/23/2010

Angels in the clouds depicts: Blue Angels ,jets, planes, formation conveys: military power? subject: specific formation? hasDepictedBackground: sky, clouds depicts: angel shows: clouds region contains a shape (PictorialElement) that looksLike an Angel (iconic) depicts: clouds © 2009 - 2010 Margaret Warren, Pat Hayes, CARMA Productions 3/23/2010

Looking like an archetype • Some shapes are best described by what they look like iconically. • If we can use RDFS or OWL Full, this could be described as 'looking like' a class; but this will break OWL-DL reasoners. • Or, we could treat a SKOS concept as an archetype, but this is not (yet) a widely accepted convention. © 2009 - 2010 Margaret Warren, Pat Hayes, CARMA Productions 3/23/2010

visiblePictorialElements looksLike iconic heart hasColor Red looksLike iconic dancing figures hasColor yellow orange red hasPictorialBackground: white © 2009 - 2010 Margaret Warren, Pat Hayes, CARMA Productions 3/23/2010

What's in an image? • We conventionally say that depicted things are "in" an image. • Is this a philosophical mistake? Maybe. But rather than try to correct it, the ontology tries to avoid it. • Basic methodological rule for linked data: don't try to repair "bad" intuitions. © 2009 - 2010 Margaret Warren, Pat Hayes, CARMA Productions 3/23/2010

shows: person, depicts: face in hands shows: broken headlight, turn signal subject: crying conveys: grief or despair, etc. depicts: front fender of school bus subject: conveys: beauty in decay, decline of education © 2009 - 2010 Margaret Warren, Pat Hayes, CARMA Productions 3/23/2010

Can any Thing be depicted? Do we need a class of 'depictable things'? Things depicted include (keywords): • the car (specific type) • people (specific people) • a group of 7 people • a building Subject: group studying a car? • trees one of the people? the car itself? • brown grass © 2009 - 2010 Margaret Warren, Pat Hayes, CARMA Productions 3/23/2010

Adult, Adults Only, Agriculture, Beautiful People, Beauty, Beauty In Nature, Biting, Branch, Color Image, Concepts, Custard Apple, Desire, Enjoyment, Food, Freshness, Fruit, Green, Gripping, Growth, Harvesting, Healthy Eating, Healthy Lifestyle, Horizontal, Human Face, Human Hand, Human Mouth, Hungry, Leaf, Lifestyles, Meal, Nutrient, One Person, One Woman Only, One Young Woman Only, Only Women, Open, Opening, Organic, People, Photography, Picking, Plant, Southeast Asian Ethnicity, The Human Body, Tree, Vegetarian Food, Women, Young Adult © 2009 - 2010 Margaret Warren, Pat Hayes, CARMA Productions 3/23/2010

What can be depicted? • FOAF: People. • DBPedia: People, places, events,… • Umbel (Cyc): VisibleThings • Visual Resources Association: myths, legends, intentions, … • We provide a different property conveys for the relationship between an image and "un-depictable" things such as mood, idea, intention, emotion, etc.. © 2009 - 2010 Margaret Warren, Pat Hayes, CARMA Productions 3/23/2010

http://www.gettyimages.com/Search/Search.aspx?query=z.i.H4 sIAAAAAAAEAO29B2AcSZYlJi9tynt_SvVK1- B0oQiAYBMk2JBAEOzBiM3mkuwdaUcjKasqgcplVmVdZhZAz O2dvPfee-- 999577733ujudTif33_8_XGZkAWz2zkrayZ4hgKrIHz9- fB8_In7dfLn91etf49f4NX6PX_dskV3kvyb9mtD_f9N5ns3S9fKqz larYnnxa_7f9PwaG59f0_xM6qpqX2Z1tmjMZ78W_r- 78xvSj1_v3evr5vSdeePX9P7mltm5Bai_4- VfOzuf2g78P35tfLDD8Bete3PR_rr6-2_xa3DX9tc99-s99-t99-s- __rrUB-lhWf_Qne_Lv1RLyw24Z-_Pj7axV-_Hn8- sTA6f_Nr5yEU- ydD2bFQzjtQ3N_8Wm1e6_zJUPYslLoDxf1NBM0tDPMH_s_U 3QV1iQL5yr4f_PVr55PGfmP- 4BnJXY_e779W7pp7v__azWplP8cfBqPfyr54OXUvXk5_bf39t8 TfO_a3ffvbLn77dara4zb7F381yf2vzF94fu3Z1dL8_muaP8BSv0 6z9kYb_PVr55nDz_-Dfl_5X9g_ft0mgBb--etM83futeCvX_vds- dOGugP- 0VWe2h7f_xGzbRa5U_Wy1npUT_41AD8ccB5sHMfv_86xGpr2 zz46_8Bs9pv0zoEAAA. © 2009 - 2010 Margaret Warren, Pat Hayes, CARMA Productions 3/23/2010

Linked Image Data • Linked data… • Newest manifestation of the semantic web: use RDF to create a web of semantic links between data items. • Meanings of links are described by lightweight (small, intuitive, useful) ontologies. © 2009 - 2010 Margaret Warren, Pat Hayes, CARMA Productions 3/23/2010

Linked data is link + description • RDF descriptions use URIs as names to refer to things; linking means using those URIs to also connect to the described thing, or a source of more information about it. • This can be smoothly integrated into HTML using RDFa. <foaf:knows> <foaf:Person rdf:about="http://danbri.org/foaf.rdf#danbri"> <foaf:mbox_sha1sum>362ce75324396f0aa2d3e5f1246f40bf3bb44401</foaf:mbox_sha1sum> <foaf:name>Dan Brickley</foaf:name> <rdfs:seeAlso rdf:resource="http://danbri.org/foaf.rdf"/> </foaf:Person> </foaf:knows> <foaf:based_near rdf:resource="http://dbpedia.org/resource/Milton_Keynes"/> <PersonalProfileDocument rdf:about=""> <cc:license rdf:resource="http://creativecommons.org/licenses/by-nc/3.0/"/> <dc:title>Tim Berners-Lee's FOAF file</dc:title> <maker rdf:resource="http://www.w3.org/People/Berners-Lee/card#i"/> <primaryTopic rdf:resource="http://www.w3.org/People/Berners-Lee/card#i"/> </PersonalProfileDocument> © 2009 - 2010 Margaret Warren, Pat Hayes, CARMA Productions 3/23/2010

Linked data is link + description • Description terms defined by lightweight ontologies. • Designing lightweight ontologies is an art, not a science. • Often, the formal ontologies are quite minimal, eg FOAF refuses to say anything more than the class People is all people, and Agent is "things that make stuff happen". Still, it does specify that People is an rdfs:subClass of Agent. In practice, meaning is specified by use (in the actual links). • It is considered good form to re-use existing relations, classes and properties whenever possible. But this can set up some tension with the meaning-as-use observation. © 2009 - 2010 Margaret Warren, Pat Hayes, CARMA Productions 3/23/2010

Linked data is here, now Collectively, the data sets consist of over 13.1 billion RDF triples (dataset statistics), 3/23/2010 © which2010 Margaret Warren,around 142 million RDF links (linksset statistics) (November 2009). 2009 - are interlinked by Pat Hayes, CARMA Productions

Linked Image data? • There are many images in existing linked data, but almost always as illustrations of things (people, places, buildings..), as in FOAF. • We want to describe images as objects in their own right. Images are our topic. So we need an image-describing vocabulary with its own lightweight ontology. Earlier work on image description all focuses either on the things depicted, or on the cultural setting of the images, or on the curation and cataloging of museum collections. © 2009 - 2010 Margaret Warren, Pat Hayes, CARMA Productions 3/23/2010

Very lightweight image ontology Questions one might ask…. 1. Can any Thing be depicted? Do we need a class of 'depictable things'? 2. Do all images depict something? 3. What exactly is depiction anyway? 4. What exactly is an Image? (Is it a file type, for example? Are all images digital images?) etc… © 2009 - 2010 Margaret Warren, Pat Hayes, CARMA Productions 3/23/2010

Pictorial elements Basic graphical 'unit' of depiction is a pictorial element. This includes 'boxes' but also lines, regions, areas, patches of color, texture, etc.. “ elements of art - The basic components used by the artist when producing works of art. Those elements are color, value, line, shape, form, texture, and space. The elements of art are among the literal qualities found in any artwork. (Artlex, 1996-2010) Pictorial elements are related by inclusion: 3/23/2010 © 2009 - 2010 Margaret Warren, Pat Hayes, CARMA Productions

Ontology can be molded to fit A depiction of the torso of a female mannequin AND/OR A Pictorial Element that looks like a female torso © 2009 - 2010 Margaret Warren, Pat Hayes, CARMA Productions 3/23/2010

Depicting vs. looking like • Simple depiction (as in FOAF) means something like "is a picture of". Our depiction is wider in scope, and does not imply visual similarity. An Xray image is a depiction, for example. So we also need the notion of looking like. • This is also important for characterizing pictorial elements. looksLike Depicted female Depicted female looksLike iconic female torso, x-ray torso female torso* © 2009 - 2010 Margaret Warren, Pat Hayes, CARMA Productions torso 3/23/2010

Images are pictorial elements. If an image can depict, then a PART of an image can depict. if a whole image can depict, so can part of an image. © 2009 - 2010 Margaret Warren, Pat Hayes, CARMA Productions 3/23/2010

Defining classes of image • Can now define for example 'line drawing': an image all of whose component visible pictorial elements is a line. <owl:Class rdf:ID="LineDrawing"> <owl:intersectionOf rdf:parseType="Collection"> <owl:Class rdf:about="#Image" /> <owl:Restriction> <owl:onProperty rdf:resource="#hasVisualPart" /> <owl:allValuesFrom> <owl:unionOf rdf:parseType="Collection"> <owl:Class rdf:about="#ImplicitPictorialElement" /> <owl:Class rdf:about="#LinearPictorialElement" /> </owl:unionOf> </owl:allValuesFrom> </owl:Restriction> </owl:intersectionOf> </owl:Class> © 2009 - 2010 Margaret Warren, Pat Hayes, CARMA Productions 3/23/2010

http://carmapro.com	41
http://margaretwarren.us	22
http://www.slideshare.net	1

Warren Hayes Ai Symposium - working draft

by Margaret Warren on Mar 23, 2010

Accessibility

Categories

Tags

Upload Details

Usage Rights

3 Embeds 64

Statistics

Warren Hayes Ai Symposium - working draft — Presentation Transcript