Explanation Aware Design And Computing 2009 09 11

Explanation-aware System Design and Computing Thomas Roth-Berghofer Senior researcher, trb@dfki.de German Research Center for Artiﬁcial Intelligence DFKI GmbH, Kaiserslautern, Germany Thomas Roth-Berghofer Freitag, 11. September 2009 1

Thomas Roth-Berghofer Freitag, 11. September 2009 2

Traditional view on software systems User U I Software System Thomas Roth-Berghofer Freitag, 11. September 2009 4

„Traditional“ behaviour of software systems „Trust me. I know what I am doing!“ SLEDGE HAMMER Thomas Roth-Berghofer Freitag, 11. September 2009 5

Overview • Explanation-aware view on software design: communication scenario • Aspects of explanation-aware design • Example: coTag — Code tagging and similarity-based retrieval Thomas Roth-Berghofer Freitag, 11. September 2009 6

Communication participants User Software System Explainer U I Originator Thomas Roth-Berghofer Freitag, 11. September 2009 7

The user communicates by way of a user interface Explainer U (UI) with the whole I software system and Originator is the recipient of explanations. Thomas Roth-Berghofer Freitag, 11. September 2009 8

Originator Explainer U I The originator is the tool Originator the user works with to perform tasks and solve problems. Thomas Roth-Berghofer Freitag, 11. September 2009 9

Explainer Explainer The explainer can be seen U as another tool that helps I understanding how the originator works and what knowledge the originator uses. Originator Thomas Roth-Berghofer Freitag, 11. September 2009 10

Explanation knowledge • concept explanations Explainer • templates •… Originator Thomas Roth-Berghofer Freitag, 11. September 2009 11

Problem solving knowledge Explainer • results Originator • concepts • workﬂows •… Thomas Roth-Berghofer Freitag, 11. September 2009 12

Reasoning information Explainer • intermediate results • context snapshots • … Originator Thomas Roth-Berghofer Freitag, 11. September 2009 13

What are explanations? Thomas Roth-Berghofer Freitag, 11. September 2009 14

What are explanations? Explanations are answers to questions. Thomas Roth-Berghofer Freitag, 11. September 2009 15

Cognitive aspects of explanations • „Explanations are the most common method used by humans to support decision making.“ (Roger Schank, 1986) • Main purpose: •Explain a solution. •How was the solution derived? •How does a system work? •How to handle a system •Explain failures. Thomas Roth-Berghofer Freitag, 11. September 2009 16

Computational aspects of explanations • Backward explanations: •Explain result and how it was obtained. • Forward explanations: • Explain (indirectly) by showing different ways to further optimise a given result. • Open up possibilities for exploratory use. Thomas Roth-Berghofer Freitag, 11. September 2009 17

EXAMPLE: Code-tagging and similarity- based retrieval with myCBR Roth-Berghofer, Th. and Bahls, D. (2008). Code tagging and retrieval with myCBR. In Petridis, M., Coenen, F., and Bramer, M., editors, Research and Development in Intelligent Systems XXV, London, UK. Springer Verlag. Thomas Roth-Berghofer Freitag, 11. September 2009 18

Programmer‘s dilemma Thomas Roth-Berghofer Freitag, 11. September 2009 19

Typical questions of programmers • Where is the code fragment I used to solve a similar problem with in the past? • Is this piece of code still available? • Is it worth the effort to search for it? • If so, what would be the right search term? Thomas Roth-Berghofer Freitag, 11. September 2009 20

Personalised approach • Personal vocabulary: tags • Linking tags Thomas Roth-Berghofer Freitag, 11. September 2009 21

Linking tags GridBag Similar! PatternLayout Thomas Roth-Berghofer Freitag, 11. September 2009 22

Personalised approach • Personal vocabulary: tags • Linking tags • Work context • Social dimension: tag exchange • Similarity-based retrieval Thomas Roth-Berghofer Freitag, 11. September 2009 23

Case-Based Reasoning cycle Agnar Aamodt and Enric Plaza. Case-based reasoning: Foundational issues, methodological variations, and system approaches. AI Communications, 7(1):39–59, 1994. Thomas Roth-Berghofer Freitag, 11. September 2009 24

Design decisions / constraints • Integration in IDE eclipse • Storage of code snippets and tags separately from code • Queries = Search text plus work context • Community repository for experience exchange Thomas Roth-Berghofer Freitag, 11. September 2009 25

Code snippet & context Java code snippet Work context • java.net.URL • java.net.URLConnection • java.io.InputStream • java.lang.StringBuffer • java.io.BufferedReader • java.lang.String • java.lang.Exception Thomas Roth-Berghofer Freitag, 11. September 2009 26

Case structure Attribute Value type category Tags String (multiple) Problem description Context items String (multiple) Problem description Code snippet String Solution Document type String Provenance Project name String Provenance File path String Provenance Author ID String Provenance Creation date Long Provenance Rating Float Maintenance Rating count Integer Maintenance Thomas Roth-Berghofer Freitag, 11. September 2009 27

Case structure Set by user Set by coTag Attribute Value type category Tags String (multiple) Problem description Context items String (multiple) Problem description Code snippet String Solution Document type String Provenance Project name String Provenance File path String Provenance Author ID String Provenance Creation date Long Provenance Rating Float Maintenance Rating count Integer Maintenance Thomas Roth-Berghofer Freitag, 11. September 2009 28

Acquiring case Thomas Roth-Berghofer Freitag, 11. September 2009 29

Query view • Search for tags: init, logging conﬁg • Include context => regard currently selected code Thomas Roth-Berghofer Freitag, 11. September 2009 30

Retrieval • Result for: init, logging, conﬁg • Ranked list of code snippets Thomas Roth-Berghofer Freitag, 11. September 2009 31

Presentation of cases Thomas Roth-Berghofer Freitag, 11. September 2009 32

Situations in which explanations play a role • Instructing explanations: • Novice users want to know about how tagging and (similarity-based) retrieval works. • Convincing explanations: • Regular users want to check when the retrieval does not meet their expectations. • Improving explanations • Regular users want to correct coTag‘s behaviour. Thomas Roth-Berghofer Freitag, 11. September 2009 33

Explanation of matching • Search terms: • init, logging, conﬁg • Case tags: • init, Logger Thomas Roth-Berghofer Freitag, 11. September 2009 34

Graphical explanation of trigram matching • Syntactical similarity • Typos • Stemming Thomas Roth-Berghofer Freitag, 11. September 2009 35

Similarity customisation • Tag similarities: unsimilar 0% partly similar 25% similar 50% very similar 75% identical 100% • Updates personal and community similarity measure Thomas Roth-Berghofer Freitag, 11. September 2009 36

Three levels of similarity calculation Personal Imported Trigram Thomas Roth-Berghofer Freitag, 11. September 2009 37

Customised (personal) and imported similarity Thomas Roth-Berghofer Freitag, 11. September 2009 38

Client-side architecture Thomas Roth-Berghofer Freitag, 11. September 2009 39

Tag and exchange code snippets Thomas Roth-Berghofer Freitag, 11. September 2009 42

Summary • Re-ﬁnding information is a quite typical task in knowledge-work. • Tagging is a helpful and well- known technique. • Similarity-based retrieval can improve searches. • Explanation-aware design and development of applications helps dealing with increased complexity of similarity-based retrieval. Thomas Roth-Berghofer Freitag, 11. September 2009 45

Explaining Semantic Search Results of Medical Images in MEDICO Forcher, B., Möller, M., Sintek, M., and Roth-Berghofer, Th. Explanation of semantic search results of medical images in Medico. In Th. Roth-Berghofer, N. Tintarev, and D. B. Leake, editors, Workshop 10@IJCAI-09: Explanation-aware Computing (ExaCt 2009), pages 13–24, 2009. Thomas Roth-Berghofer Freitag, 11. September 2009 46

Goal of Medico Project Development of • intelligent • robust and • scalable semantic search engine for medical images Thomas Roth-Berghofer Freitag, 11. September 2009 47

Reconstructive explanations Explainer Originator line of explanation line ofThomas Roth-Berghofer reasoning Freitag, 11. September 2009 48

RadSem • Exploration interface with concept explanations support domain understanding. • Justiﬁcation interface provides action explanations, which counteract encapsulation and information hiding. Thomas Roth-Berghofer Freitag, 11. September 2009 49

Good explanations • Relevant OPEN QUESTION: • Innovative OPERATIONALISATION • Convincing OF THOSE CRITERIA • Short and easy to overlook • Provide different perspectives and follow-up questions Kinds Explainer Goals • Concept • Transparency • Why • Justiﬁcation • How • Relevance • Action • Conceptualisation explanations Originator • Learning Thomas Roth-Berghofer Freitag, 11. September 2009 50

Kinds of explanations • Concept explanations • Action explanations • Why- and How-explanations Thomas Roth-Berghofer Freitag, 11. September 2009 51

Concept Explanations • The goal of concept explanations is to build links between unknown and known concepts. • Variations: • Deﬁnition: “What is a bicycle?” – “A bicycle is a land vehicle with two wheels in line. Bicycles are a form of human powered vehicle.” • Functional mapping: “What is a bicycle?” – “A bicycle serves as a means of transport.” • Prototypical usage of individual things or actions: “What is a bicycle?” – “The thing, that man over there just crashed with.” • … Thomas Roth-Berghofer Freitag, 11. September 2009 52

Action explanations • Action explanations explain the activities of the respective system (originator). Action explanations: “Why was this seat post selected?” – “For the given price, only one other seat post was available. But this was too short. • In RadSem: Reconstructive explanations based on search concepts and found concepts. Thomas Roth-Berghofer Freitag, 11. September 2009 53

Why-explanations • Why-explanations provide causes or justiﬁcations for facts or events. • Examples: • Justiﬁcation: “Why does the universe expand?” – “Because we can observe a red shift of the light emitted by other galaxies.” • Cause: “Because the whole matter was concentrated at one point of the universe and because the whole matter moves away from each other Thomas Roth-Berghofer Freitag, 11. September 2009 54

Explanation goals Transparency How did the system reach an answer? Justiﬁcation Why is the answer a good answer? Relevance Why is the question relevant? Conceptualisation What is the meaning of a concept? Learning Teach the user about the given domain. Sørmo, F., Cassens, J., Aamodt, A.: Explanation in Case-Based Reasoning – Perspectives and Goals, 2005. Freitag, 11. September 2009 55

When are explanations good explanations? • Short and easy to overlook • Innovative • Relevant • Convincing • Different perspectives and follow-up questions • Natural W. R. Swartout and J. D. Moore. Explanation in second generation expert systems. In J. David, J. Krivine, and R. Simmons, editors, Second Generation Expert Systems, pages 543–585, Berlin, 1993. Springer Verlag. Thomas Roth-Berghofer Freitag, 11. September 2009 56

Take home messages • The ability to explain reasoning processes and results can substantially affect the usability and acceptance of a software system. • Basic explanation scenario Explainer helps identifying communication U I partners and knowledge bases. Originator • Explanation goals and kinds further help structuring knowledge acquisition and use in software design and computing. Thomas Roth-Berghofer Freitag, 11. September 2009 57

Thank you! Explanation-aware System Design and Computing Thomas Roth-Berghofer Senior researcher, trb@dfki.de Thomas Roth-Berghofer Freitag, 11. September 2009 58

Publications of ExaCt research group 2009 [Roth-Berghofer and Bahls, 2008] Roth-Berghofer, T. R. and Bahls, D. [Roth-Berghofer, Tintarev, Leake, 2009] Roth-Berghofer, Th., Tintarev, N., (2008). Code tagging and retrieval with myCBR. In Petridis, M., Coenen, F., and Leake, D.B., editors. Proceedings of the IJCAI-09 workshop on and Bramer, M., editors, Research and Development in Intelligent Systems Explanation-aware Computing (ExaCt 2009), July 2009. XXV, London, UK. Springer Verlag. [Adrian et al., 2009] Adrian, B., Forcher, B., Roth-Berghofer, Th., and [Roth-Berghofer and Mittag, 2008] Roth-Berghofer, T. R. and Mittag, F. Dengel, A. Explaining ontology-based information extraction in the (2008). ReduxExp: A justification-based explanation-support server. NEPOMUK semantic desktop. In Thomas R. Roth-Berghofer, Nava Tintarev, Proceedings of AI-2008. the twenty-eighth SGAI international conference and David B. Leake, editors, Workshop 10@IJCAI-09: Explanation-aware on artificial intelligence. In Petridis, M., Coenen, F., and Bramer, M., editors, Computing (ExaCt 2009), pages 94–101, 2009. Research and Development in Intelligent Systems XXV, London, UK. Springer Verlag. [Forcher et al., 2009] Forcher, B., Möller, M., Sintek, M., and Roth- Berghofer, Th. Explanation of semantic search results of medical images in [Roth-Berghofer and Richter, 2008a] Roth-Berghofer, T. R. and Richter, M. medico. In Thomas R. Roth-Berghofer, Nava Tintarev, and David B. Leake, M., editors (2008a). Künstliche Intelligenz—Topic: Explanation, volume 22, editors, Workshop 10@IJCAI-09: Explanation-aware Computing (ExaCt Bremen. BöttcherIT Verlag. 2009), pages 13–24, 2009. [Roth-Berghofer and Richter, 2008b] Roth-Berghofer, T. R. and Richter, M. [Stahl, Roth-Berghofer, 2009] Stahl, A. and Roth-Berghofer, Th. Rapid M. (2008b). On explanation. Künstliche Intelligenz, 22(2):5–7. Prototyping of CBR applications with the Open Source Tool myCBR. 2007 Künstliche Intelligenz, 23(1):34–37, March 2009. [Bahls and Roth-Berghofer, 2007] Bahls, D. and Roth-Berghofer, T. (2007). 2008 Explanation support for the case-based reasoning tool myCBR. In [Roth-Berghofer et al., 2008] Roth-Berghofer, Th., Schulz, S., Bahls, D., and Proceedings of the Twenty-Second AAAI Conference on Artificial Leake, D.B., editors. Proceedings of the ECAI-08 workshop on Intelligence. July 22–26, 2007, Vancouver, British Columbia, Canada., Explanation-aware Computing ExaCt2008. University of Patras, July 2008. pages 1844–1845. The AAAI Press, Menlo Park, California. http://ceur- ws.org/Vol- 391. [Eppert, 2007] Eppert, M. (2007). Generating provenance explanations for [Forcher, Adrian, Roth-Berghofer, 2008] Forcher, B., Adrian, B., and Roth- the gnowsis rebirth machine - a first pass. Pro ject thesis, University of Berghofer, Th.. Explanation styles in iDocument. ExaCt 2008, ECAI-08 Kaiserslautern. Workshop. [Roth-Berghofer et al., 2007] Roth-Berghofer, T. R., Schulz, S., and Leake, [Bahls, 2008] Bahls, D. (2008). Explanation support for the case-based D. B., editors (2007). Proceedings of the AAAI-07 workshop on reasoning tool myCBR. Project thesis, University of Kaiserslautern. Explanation-aware Computing ExaCt2007. AAAI Press. Technical Report WS-07-06. [Forcher et al., 2008] Forcher, B., Adrian, B., and Roth-Berghofer, T. (2008). Explanations in the information extraction system iDocument. Künstliche Intelligenz, 22(2). [Mittag, 2008] Mittag, F. (2008). ReduxExp: A justification-based explanation-support server. Project thesis, University of Kaiserslautern. Freitag, 11. September 2009 59

Publications of ExaCt research group 2006 [Richter et al., 2006] Richter, M. M., Roth-Berghofer, T., and Schulz, S., editors (2006). Explanation-aware Computing, volume 25. SAP - Slovak Academic Press Ltd., Bratislava. 2005 [Roth-Berghofer et al., 2005a] Roth-Berghofer, T., Cassens, J., and Sørmo, F. (2005a). Goals and kinds of explanations in case-based reasoning. In Althoff, K.-D., Dengel, A., Bergmann, R., Nick, M., and Roth-Berghofer, T., editors, WM 2005: Professional Knowledge Management, pages 264–268, Kaiserslautern, Germany. DFKI GmbH. [Roth-Berghofer and Cassens, 2005] Roth-Berghofer, T. R. and Cassens, J. (2005). Mapping goals and kinds of explanations to the knowledge containers of case-based reasoning systems. In Muñoz-Avila, H. and Ricci, F., editors, Case-Based Reasoning Research and Developmen, pages 451– 464, Heidelberg. Springer Verlag. [Roth-Berghofer et al., 2005b] Roth-Berghofer, T. R., Schulz, S., and Woody, A., editors (2005b). Proceedings of the AAAI Fal l Symposium on Explanation-aware Computing ExaCt2005. AAAI Press. Technical Report FS-05-04. 2004 [Roth-Berghofer, 2004] Roth-Berghofer, T. R. (2004). Explanations and Case-Based Reasoning: Foundational issues. In Funk, P. and González- Calero, P. A., editors, Advances in Case-Based Reasoning, pages 389–403. Springer-Verlag. [Memmel, Roth-Berghofer, 2004] Memmel, M. and Roth-Berghofer, Th. Explanation and e-learning: A ﬁrst pass. In Klaus-Peter F¨ahnrich, Klaus P. Jantke, and Wolfgang S. Wittig, editors, Von e-Learning bis e-Payment. Das Internet als sicherer Marktplatz, pages 255–263. Akademische Verlagsgesellschaft Aka, 2004. Freitag, 11. September 2009 60

Invitation to participate HTTP://ON-EXPLANATION.NET • Manifesto • Mailing list • Workshop series • … Freitag, 11. September 2009 61

Explanation Aware Design And Computing 2009 09 11

0 comments

Notes on slide 1

Favorites, Groups & Events