This document describes a user-centered image management system that traces user actions to help annotate and search images. It uses a case-based reasoning model called Club ♣♣♣♣ to formalize objects, procedures, and a trace model. A prototype was created to digitize documents from the Teleki library, allowing users to upload, annotate with keywords, and search large numbers of images. As users work, their actions are traced and used to suggest helpful keywords and search results based on prior similar cases. The system aims to bridge the gap between how users interpret images and describe them with words.

1. EGYED- ISZLAI : User centered image management system DIAL 2006 LYON 1
User centered image management
system for digital libraries
EGYED-ZSIGMOND El d
LIRIS, INSA-Lyon
ISZLAI Zoltán
Sapientia EMTE, Targu-Mures

2. EGYED- ISZLAI : User centered image management system DIAL 2006 LYON 2
Outline
Addressed issue
Application
Theoretical model (Club ♣♣♣♣)
The prototype application

3. EGYED- ISZLAI : User centered image management system DIAL 2006 LYON 3
Addressed issue
Documents
pre formatting
segmentation
automatic/ assisted
annotation
search
scanner or camera
Online image annotation and management system, which
traces the actions of the users and uses the capitalized
experience to help them according to the case based
reasoning paradigm
Manual image annotation and search is a human related problem,
difficult to formalize

4. EGYED- ISZLAI : User centered image management system DIAL 2006 LYON 4
Addressed issue
Images
Many Users
Annotation and search system
Existing annotations
Lt. Tonton
Vérification
Massage
Dégagement
Homme en noir
Lt. Tonton
Homme en noir
Massage
DégagementVérification
DA: Secouriste DA: Victime
DA: Opération
Annotations
Interpretation traces
Lt. Tonton
Vérification
Massage
Dégagement
Homme en noir
Lt. Tonton
Vérification
Massage
Dégagement
Homme en noir
Lt. Tonton
Vérification
Massage
Dégagement
Homme en noir
Lt. Tonton
Vérification
Massage
Dégagement
Homme en noir
Annotations (words)Images
Semantic
gap
Interpretation

5. EGYED- ISZLAI : User centered image management system DIAL 2006 LYON 5
Outline
Addressed issue
Application
Theoretical model (Club ♣♣♣♣)
The prototype application

6. EGYED- ISZLAI : User centered image management system DIAL 2006 LYON 6
Application
Digitalization and online publication of documents from a library

7. EGYED- ISZLAI : User centered image management system DIAL 2006 LYON 7
Application
Marosvásárhely (Târgu-Mureş)
ManuscriptsManuscripts ofof thethe mathematicianmathematician
JJáános BOLYAI (1802nos BOLYAI (1802––1860)1860)
fatherfather ofof thethe nonnon euclideaneuclidean geometrygeometry
~ 15000 pages~ 15000 pages
Preserved in the TELEKI-BOLYAI library
in Roumania, Transylvania

8. EGYED- ISZLAI : User centered image management system DIAL 2006 LYON 8
Outline
Addressed issue
Application
Theoretical model (Club ♣♣♣♣)
The prototype application

9. EGYED- ISZLAI : User centered image management system DIAL 2006 LYON 9
Objets Utilisateurs
Procédés
Club ♣♣♣♣
We consider that a user manipulates objects through procedures.
Objects Users
Procedures
The trace model is based on a graph containing user, procedure and
object type nodes.
The use of the system is traced according to this graph.

10. EGYED- ISZLAI : User centered image management system DIAL 2006 LYON 10
♣♣♣♣ Phases of the assistance management
Formalize objects manipulated and the relations among them:
use model
Gather (imagine) the different object manipulation procedures and
formalize them in tasks
Methods to cut the traces in reusable and adaptable cases.
Decide which tasks to trace in order to capitalize user experience:
observation model
The use model and the observation model enables the trace
construction

11. EGYED- ISZLAI : User centered image management system DIAL 2006 LYON 11
♣♣♣♣ Case Based Reasoning
elaborate
Problem New case
Found
case
Solution
case
Adapted
case
Learned
case
search
compare
adapt
validate
Case
base and
domain
theory
learn
Pb (resolved)
Sol (resolved)
Pb (target)
Sol (target)
Comparison
Resolved case Target case
[Aamodt and Plaza1994]
[Mille1998]
Adaptation

12. EGYED- ISZLAI : User centered image management system DIAL 2006 LYON 12
Users
Procedures
Objects
♣♣♣♣ Use and observation model
Annotate images
Search Navigate
Albums
Create gallery
through images
through albums
Through usersThrough keywords
Images
Keywords
User1
User2

13. EGYED- ISZLAI : User centered image management system DIAL 2006 LYON 13
Application to image management
Images
Keywords
Users
Galleries
Use model:
Simplified model

14. EGYED- ISZLAI : User centered image management system DIAL 2006 LYON 14
Observation model
GUI transition diagramme
was not successful
search
by suggested keyword
by suggested image
by entered keywords
by related keywordsOrthographical correction
with all the wordswith any of the words
select image
by user keywords
was successful
by user image result
by user ’s gallery
by user ’s gallery result
by user gallery keywords
Enlarging image
Select keyword
by all keywords
by suggested keywords
KIDKIDKID
KID KID KID
KIDKID
GID UIDIID
KID
UID IID
GID UID
KID
IID
IID
UID KID

15. EGYED- ISZLAI : User centered image management system DIAL 2006 LYON 15
♣♣♣♣ Tracing the use of the system
Users
Procedures
Objects
Annotate images
Search Navigate
Albums
Create gallery
through images
through albums
Through usersThrough keywords
Images
Keywords
User1
User2
Users
Procedures
Objects
Users
Procedures
Objects
Annotate images
Search Navigate
Albums
Create gallery
through images
through albums
Through usersThrough keywords
Images
Keywords
User1
User2
Trace view 1
Trace view 2

16. EGYED- ISZLAI : User centered image management system DIAL 2006 LYON 16
Following the user
Once the traces defined we cut them into reusable cases
The assistance concerns :
« What to do ? »
rather than
« How to do ? »
type situations
Users
Procedures
Objects
Annotate images
Search Navigate
Albums
Create gallery
through images
through albums
Through usersThrough keywords
Images
Keywords
User1
User2
Users
Procedures
Objects
Users
Procedures
Objects
Annotate images
Search Navigate
Albums
Create gallery
through images
through albums
Through usersThrough keywords
Images
Keywords
User1
User2
Comparison Adaptation
Pb (target)
Pb (resolved)
Sol (target)
Sol (resolved)Sol (resolved)
Sol (target)

17. EGYED- ISZLAI : User centered image management system DIAL 2006 LYON 17
Traces
U:Angelo
P: Search
KID: Bolyai
P: select image
P: Enlarging image
IID:428665
KID: theoreme
KID: tulip
KID: parralel
KID: Math
KID: color
KID: petal
KID: hungarian
KID: german
KID: botanic
KID: love
KID: Teleki
KID: card
KID: latin
P: by suggested keywords
KID: manuscript
Users
Procedures
Objects
O: GID
O: IID
O: UID
U:Angelo
U:Elod
U:Sonia
P: Search
P: Orthographical correction
P: by related keywords P: by user gallery
P: by all keywords
P: by user gallery keywords
P: Enlarging image
P: by user image result
P: by image keywords P: by suggested image
P: select image
P: Select keyword
P: by user gallerys
P: by suggested keyword
O: KID
Use trace model,
trace explanation
Traces

18. EGYED- ISZLAI : User centered image management system DIAL 2006 LYON 18
Outline
Addressed issue
Application
Theoretical model (Club ♣♣♣♣)
The prototype application

19. EGYED- ISZLAI : User centered image management system DIAL 2006 LYON 19
Image management system
• Create an image management website to
– publish the documents of the Teleki library
– experiment user experience capitalization and reuse techniques
• Features :
– Online image management
– Image upload (one by one or zipped sets)
– Keyword based manual description (free and selected)
– Keyword based image search
– Basic image processing tools
– Experience, image similarity and co-annotation based keyword
recommendation
– New graphical metaphors to explore the image base and keywords
• Objectives

20. EGYED- ISZLAI : User centered image management system DIAL 2006 LYON 20
Image annotation interface

21. EGYED- ISZLAI : User centered image management system DIAL 2006 LYON 21
Search interface

22. EGYED- ISZLAI : User centered image management system DIAL 2006 LYON 22
Conclusion / directions
• User tracing online image annotation and
management system
• a case based reasoning use trace model
• non intrusive way of suggesting keywords and
results to help users describe what they wish
(dream) to find
• show the traces to the users
• Improve the user interface (new visual metaphors)
• Include more sophisticated image processing tools
• Beta version available online :
http://plexir-liris.insa-lyon.fr/myimages/
login/pass : test/test
Users
Procedures
Objects
Annotate images
Search Navigate
Albums
Create gallery
through images
through albums
Through usersThrough keywords
Images
Keywords
User1
User2
Users
Procedures
Objects
Users
Procedures
Objects
Annotate images
Search Navigate
Albums
Create gallery
through images
through albums
Through usersThrough keywords
Images
Keywords
User1
User2

23. EGYED- ISZLAI : User centered image management system DIAL 2006 LYON 23
Thank you
Questions ?