Explanation-aware System
                      Design and Computing
                                Thomas Roth-Berghofer
...
Thomas Roth-Berghofer

Freitag, 11. September 2009                           2
Thomas Roth-Berghofer

Freitag, 11. September 2009                           3
Traditional view on
                               software systems
                              User


                 ...
„Traditional“ behaviour
                    of software systems
                                „Trust me. I
             ...
Overview

                     • Explanation-aware view on software
                              design: communication sc...
Communication
                                 participants
                              User       Software System

    ...
The user
            communicates by way
              of a user interface           Explainer
                           ...
Originator
                              Explainer
         U
         I




                                             ...
Explainer
                              Explainer        The explainer can be seen
         U                             ...
Explanation knowledge

                                   • concept explanations
                      Explainer    • temp...
Problem solving
                                 knowledge

                      Explainer




                          ...
Reasoning information

                      Explainer


                                   • intermediate results
       ...
What are
          explanations?



                              Thomas Roth-Berghofer

Freitag, 11. September 2009      ...
What are
          explanations?
          Explanations are
          answers to
          questions.
                    ...
Cognitive aspects of
                                 explanations
                     •        „Explanations are the mos...
Computational aspects
                      of explanations
                     •        Backward explanations:
         ...
EXAMPLE:

                    Code-tagging and similarity-
                    based retrieval with myCBR
                ...
Programmer‘s dilemma




                                     Thomas Roth-Berghofer

Freitag, 11. September 2009          ...
Typical questions of
                   programmers

             • Where is the code fragment I used to solve a
         ...
Personalised approach
                               • Personal
                                 vocabulary: tags
        ...
Linking tags
                                  GridBag



                                      Similar!



              ...
Personalised approach
                               • Personal
                                 vocabulary: tags
        ...
Case-Based
                   Reasoning cycle




                       Agnar Aamodt and Enric Plaza. Case-based reasonin...
Design decisions /
                   constraints
             • Integration in IDE eclipse
             • Storage of code...
Code snippet & context
                   Java code snippet    Work context
                                       • java....
Case structure
                              Attribute    Value type              category
                               ...
Case structure                                  Set by user
                                                              ...
Acquiring case




                                    Thomas Roth-Berghofer

Freitag, 11. September 2009                 ...
Query view

             • Search for tags: init,
                   logging config
             • Include context
        ...
Retrieval


             • Result for: init, logging,
                   config
             • Ranked list of code
        ...
Presentation of cases




                                           Thomas Roth-Berghofer

Freitag, 11. September 2009   ...
Situations in which
                   explanations play a role

             • Instructing explanations:
                ...
Explanation of matching


             • Search terms:
                   • init, logging, config
             • Case tags:...
Graphical explanation of
                   trigram matching

             • Syntactical similarity
                   • T...
Similarity customisation
             • Tag similarities:
                         unsimilar       0%
                    ...
Three levels of similarity
                   calculation

                              Personal


                      ...
Customised (personal)
                   and imported similarity




                                         Thomas Roth-...
Client-side architecture




                                          Thomas Roth-Berghofer

Freitag, 11. September 2009 ...
Client-side architecture




                                          Thomas Roth-Berghofer

Freitag, 11. September 2009 ...
Client-side architecture




                                          Thomas Roth-Berghofer

Freitag, 11. September 2009 ...
Tag and exchange code
                   snippets




                                      Thomas Roth-Berghofer

Freitag...
Thomas Roth-Berghofer

Freitag, 11. September 2009                           43
Thomas Roth-Berghofer

Freitag, 11. September 2009                           44
Summary
             • Re-finding information is a quite
                  typical task in knowledge-work.
             • T...
Explaining
                     Semantic Search Results of
                     Medical Images in MEDICO



            Fo...
Goal of Medico Project
                        Development of
                              • intelligent
                ...
Reconstructive explanations
                                 Explainer




                                 Originator
   ...
RadSem



             • Exploration interface with concept explanations
                  support domain understanding.
 ...
Good explanations
                                             •   Relevant                OPEN QUESTION:
                ...
Kinds of explanations

                     • Concept explanations
                     • Action explanations
            ...
Concept Explanations
              •       The goal of concept explanations is to build links between
                    ...
Action explanations
                  • Action explanations explain the activities of
                         the respect...
Why-explanations

              •       Why-explanations provide causes or justifications for
                      facts o...
Explanation goals
                     Transparency                How did the system reach an answer?

                  ...
When are explanations
                    good explanations?
                  • Short and easy to overlook
              ...
Take home messages
                • The ability to explain reasoning processes and results
                     can subst...
Thank
                               you!

                     Explanation-aware System
                      Design and ...
Publications of ExaCt
                   research group
             2009                                                 ...
Publications of ExaCt
                   research group
             2006
             [Richter et al., 2006] Richter, M. ...
Invitation to participate
                                  HTTP://ON-EXPLANATION.NET

             •     Manifesto
      ...
Upcoming SlideShare
Loading in …5
×

Explanation Aware Design And Computing 2009 09 11

1,734 views

Published on

The ability to explain reasoning processes and results can substantially affect the usability and acceptance of a software system. The described basic explanation scenario of user, originator and explainer helps identifying communication partners and required knowledge bases. Explanation goals and kinds further help structuring knowledge acquisition and use in software design and computing. Explanation-awareness is exemplified through coTag, a code tagging and retrieval plugin for eclipse.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,734
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
18
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Explanation Aware Design And Computing 2009 09 11

  1. 1. Explanation-aware System Design and Computing Thomas Roth-Berghofer Senior researcher, trb@dfki.de German Research Center for Artificial Intelligence DFKI GmbH, Kaiserslautern, Germany Thomas Roth-Berghofer Freitag, 11. September 2009 1
  2. 2. Thomas Roth-Berghofer Freitag, 11. September 2009 2
  3. 3. Thomas Roth-Berghofer Freitag, 11. September 2009 3
  4. 4. Traditional view on software systems User U I Software System Thomas Roth-Berghofer Freitag, 11. September 2009 4
  5. 5. „Traditional“ behaviour of software systems „Trust me. I know what I am doing!“ SLEDGE HAMMER Thomas Roth-Berghofer Freitag, 11. September 2009 5
  6. 6. Overview • Explanation-aware view on software design: communication scenario • Aspects of explanation-aware design • Example: coTag — Code tagging and similarity-based retrieval Thomas Roth-Berghofer Freitag, 11. September 2009 6
  7. 7. Communication participants User Software System Explainer U I Originator Thomas Roth-Berghofer Freitag, 11. September 2009 7
  8. 8. The user communicates by way of a user interface Explainer U (UI) with the whole I software system and Originator is the recipient of explanations. Thomas Roth-Berghofer Freitag, 11. September 2009 8
  9. 9. Originator Explainer U I The originator is the tool Originator the user works with to perform tasks and solve problems. Thomas Roth-Berghofer Freitag, 11. September 2009 9
  10. 10. Explainer Explainer The explainer can be seen U as another tool that helps I understanding how the originator works and what knowledge the originator uses.  Originator Thomas Roth-Berghofer Freitag, 11. September 2009 10
  11. 11. Explanation knowledge • concept explanations Explainer • templates •… Originator Thomas Roth-Berghofer Freitag, 11. September 2009 11
  12. 12. Problem solving knowledge Explainer • results Originator • concepts • workflows •… Thomas Roth-Berghofer Freitag, 11. September 2009 12
  13. 13. Reasoning information Explainer • intermediate results • context snapshots • … Originator Thomas Roth-Berghofer Freitag, 11. September 2009 13
  14. 14. What are explanations? Thomas Roth-Berghofer Freitag, 11. September 2009 14
  15. 15. What are explanations? Explanations are answers to questions. Thomas Roth-Berghofer Freitag, 11. September 2009 15
  16. 16. Cognitive aspects of explanations • „Explanations are the most common method used by humans to support decision making.“ (Roger Schank, 1986) • Main purpose: •Explain a solution. •How was the solution derived? •How does a system work? •How to handle a system •Explain failures. Thomas Roth-Berghofer Freitag, 11. September 2009 16
  17. 17. Computational aspects of explanations • Backward explanations: •Explain result and how it was obtained. • Forward explanations: • Explain (indirectly) by showing different ways to further optimise a given result. • Open up possibilities for exploratory use. Thomas Roth-Berghofer Freitag, 11. September 2009 17
  18. 18. EXAMPLE: Code-tagging and similarity- based retrieval with myCBR Roth-Berghofer, Th. and Bahls, D. (2008). Code tagging and retrieval with myCBR. In Petridis, M., Coenen, F., and Bramer, M., editors, Research and Development in Intelligent Systems XXV, London, UK. Springer Verlag. Thomas Roth-Berghofer Freitag, 11. September 2009 18
  19. 19. Programmer‘s dilemma Thomas Roth-Berghofer Freitag, 11. September 2009 19
  20. 20. Typical questions of programmers • Where is the code fragment I used to solve a similar problem with in the past? • Is this piece of code still available? • Is it worth the effort to search for it? • If so, what would be the right search term? Thomas Roth-Berghofer Freitag, 11. September 2009 20
  21. 21. Personalised approach • Personal vocabulary: tags • Linking tags Thomas Roth-Berghofer Freitag, 11. September 2009 21
  22. 22. Linking tags GridBag Similar! PatternLayout Thomas Roth-Berghofer Freitag, 11. September 2009 22
  23. 23. Personalised approach • Personal vocabulary: tags • Linking tags • Work context • Social dimension: tag exchange • Similarity-based retrieval Thomas Roth-Berghofer Freitag, 11. September 2009 23
  24. 24. Case-Based Reasoning cycle Agnar Aamodt and Enric Plaza. Case-based reasoning: Foundational issues, methodological variations, and system approaches. AI Communications, 7(1):39–59, 1994. Thomas Roth-Berghofer Freitag, 11. September 2009 24
  25. 25. Design decisions / constraints • Integration in IDE eclipse • Storage of code snippets and tags separately from code • Queries = Search text plus work context • Community repository for experience exchange Thomas Roth-Berghofer Freitag, 11. September 2009 25
  26. 26. Code snippet & context Java code snippet Work context • java.net.URL • java.net.URLConnection • java.io.InputStream • java.lang.StringBuffer • java.io.BufferedReader • java.lang.String • java.lang.Exception Thomas Roth-Berghofer Freitag, 11. September 2009 26
  27. 27. Case structure Attribute Value type category Tags String (multiple) Problem description Context items String (multiple) Problem description Code snippet String Solution Document type String Provenance Project name String Provenance File path String Provenance Author ID String Provenance Creation date Long Provenance Rating Float Maintenance Rating count Integer Maintenance Thomas Roth-Berghofer Freitag, 11. September 2009 27
  28. 28. Case structure Set by user Set by coTag Attribute Value type category Tags String (multiple) Problem description Context items String (multiple) Problem description Code snippet String Solution Document type String Provenance Project name String Provenance File path String Provenance Author ID String Provenance Creation date Long Provenance Rating Float Maintenance Rating count Integer Maintenance Thomas Roth-Berghofer Freitag, 11. September 2009 28
  29. 29. Acquiring case Thomas Roth-Berghofer Freitag, 11. September 2009 29
  30. 30. Query view • Search for tags: init, logging config • Include context => regard currently selected code Thomas Roth-Berghofer Freitag, 11. September 2009 30
  31. 31. Retrieval • Result for: init, logging, config • Ranked list of code snippets Thomas Roth-Berghofer Freitag, 11. September 2009 31
  32. 32. Presentation of cases Thomas Roth-Berghofer Freitag, 11. September 2009 32
  33. 33. Situations in which explanations play a role • Instructing explanations: • Novice users want to know about how tagging and (similarity-based) retrieval works. • Convincing explanations: • Regular users want to check when the retrieval does not meet their expectations. • Improving explanations • Regular users want to correct coTag‘s behaviour. Thomas Roth-Berghofer Freitag, 11. September 2009 33
  34. 34. Explanation of matching • Search terms: • init, logging, config • Case tags: • init, Logger Thomas Roth-Berghofer Freitag, 11. September 2009 34
  35. 35. Graphical explanation of trigram matching • Syntactical similarity • Typos • Stemming Thomas Roth-Berghofer Freitag, 11. September 2009 35
  36. 36. Similarity customisation • Tag similarities: unsimilar 0% partly similar 25% similar 50% very similar 75% identical 100% • Updates personal and community similarity measure Thomas Roth-Berghofer Freitag, 11. September 2009 36
  37. 37. Three levels of similarity calculation Personal Imported Trigram Thomas Roth-Berghofer Freitag, 11. September 2009 37
  38. 38. Customised (personal) and imported similarity Thomas Roth-Berghofer Freitag, 11. September 2009 38
  39. 39. Client-side architecture Thomas Roth-Berghofer Freitag, 11. September 2009 39
  40. 40. Client-side architecture Thomas Roth-Berghofer Freitag, 11. September 2009 40
  41. 41. Client-side architecture Thomas Roth-Berghofer Freitag, 11. September 2009 41
  42. 42. Tag and exchange code snippets Thomas Roth-Berghofer Freitag, 11. September 2009 42
  43. 43. Thomas Roth-Berghofer Freitag, 11. September 2009 43
  44. 44. Thomas Roth-Berghofer Freitag, 11. September 2009 44
  45. 45. Summary • Re-finding information is a quite typical task in knowledge-work. • Tagging is a helpful and well- known technique. • Similarity-based retrieval can improve searches. • Explanation-aware design and development of applications helps dealing with increased complexity of similarity-based retrieval. Thomas Roth-Berghofer Freitag, 11. September 2009 45
  46. 46. Explaining Semantic Search Results of Medical Images in MEDICO Forcher, B., Möller, M., Sintek, M., and Roth-Berghofer, Th. Explanation of semantic search results of medical images in Medico. In Th. Roth-Berghofer, N. Tintarev, and D. B. Leake, editors, Workshop 10@IJCAI-09: Explanation-aware Computing (ExaCt 2009), pages 13–24, 2009. Thomas Roth-Berghofer Freitag, 11. September 2009 46
  47. 47. Goal of Medico Project Development of • intelligent • robust and • scalable semantic search engine for medical images Thomas Roth-Berghofer Freitag, 11. September 2009 47
  48. 48. Reconstructive explanations Explainer Originator line of explanation line ofThomas Roth-Berghofer reasoning Freitag, 11. September 2009 48
  49. 49. RadSem • Exploration interface with concept explanations support domain understanding. • Justification interface provides action explanations, which counteract encapsulation and information hiding. Thomas Roth-Berghofer Freitag, 11. September 2009 49
  50. 50. Good explanations • Relevant OPEN QUESTION: • Innovative OPERATIONALISATION • Convincing OF THOSE CRITERIA • Short and easy to overlook • Provide different perspectives and follow-up questions Kinds Explainer Goals • Concept • Transparency • Why • Justification • How • Relevance • Action • Conceptualisation explanations Originator • Learning Thomas Roth-Berghofer Freitag, 11. September 2009 50
  51. 51. Kinds of explanations • Concept explanations • Action explanations • Why- and How-explanations Thomas Roth-Berghofer Freitag, 11. September 2009 51
  52. 52. Concept Explanations • The goal of concept explanations is to build links between unknown and known concepts. • Variations: • Definition: “What is a bicycle?” – “A bicycle is a land vehicle with two wheels in line. Bicycles are a form of human powered vehicle.” • Functional mapping: “What is a bicycle?” – “A bicycle serves as a means of transport.” • Prototypical usage of individual things or actions: “What is a bicycle?” – “The thing, that man over there just crashed with.” • … Thomas Roth-Berghofer Freitag, 11. September 2009 52
  53. 53. Action explanations • Action explanations explain the activities of the respective system (originator). Action explanations: “Why was this seat post selected?” – “For the given price, only one other seat post was available. But this was too short. • In RadSem: Reconstructive explanations based on search concepts and found concepts. Thomas Roth-Berghofer Freitag, 11. September 2009 53
  54. 54. Why-explanations • Why-explanations provide causes or justifications for facts or events. • Examples: • Justification: “Why does the universe expand?” – “Because we can observe a red shift of the light emitted by other galaxies.” • Cause: “Because the whole matter was concentrated at one point of the universe and because the whole matter moves away from each other Thomas Roth-Berghofer Freitag, 11. September 2009 54
  55. 55. Explanation goals Transparency How did the system reach an answer? Justification Why is the answer a good answer? Relevance Why is the question relevant? Conceptualisation What is the meaning of a concept? Learning Teach the user about the given domain. Sørmo, F., Cassens, J., Aamodt, A.: Explanation in Case-Based Reasoning – Perspectives and Goals, 2005. Freitag, 11. September 2009 55
  56. 56. When are explanations good explanations? • Short and easy to overlook • Innovative • Relevant • Convincing • Different perspectives and follow-up questions • Natural W. R. Swartout and J. D. Moore. Explanation in second generation expert systems. In J. David, J. Krivine, and R. Simmons, editors, Second Generation Expert Systems, pages 543–585, Berlin, 1993. Springer Verlag. Thomas Roth-Berghofer Freitag, 11. September 2009 56
  57. 57. Take home messages • The ability to explain reasoning processes and results can substantially affect the usability and acceptance of a software system. • Basic explanation scenario Explainer helps identifying communication U I partners and knowledge bases. Originator • Explanation goals and kinds further help structuring knowledge acquisition and use in software design and computing. Thomas Roth-Berghofer Freitag, 11. September 2009 57
  58. 58. Thank you! Explanation-aware System Design and Computing Thomas Roth-Berghofer Senior researcher, trb@dfki.de Thomas Roth-Berghofer Freitag, 11. September 2009 58
  59. 59. Publications of ExaCt research group 2009 [Roth-Berghofer and Bahls, 2008] Roth-Berghofer, T. R. and Bahls, D. [Roth-Berghofer, Tintarev, Leake, 2009] Roth-Berghofer, Th., Tintarev, N., (2008). Code tagging and retrieval with myCBR. In Petridis, M., Coenen, F., and Leake, D.B., editors. Proceedings of the IJCAI-09 workshop on and Bramer, M., editors, Research and Development in Intelligent Systems Explanation-aware Computing (ExaCt 2009), July 2009. XXV, London, UK. Springer Verlag. [Adrian et al., 2009] Adrian, B., Forcher, B., Roth-Berghofer, Th., and [Roth-Berghofer and Mittag, 2008] Roth-Berghofer, T. R. and Mittag, F. Dengel, A. Explaining ontology-based information extraction in the (2008). ReduxExp: A justification-based explanation-support server. NEPOMUK semantic desktop. In Thomas R. Roth-Berghofer, Nava Tintarev, Proceedings of AI-2008. the twenty-eighth SGAI international conference and David B. Leake, editors, Workshop 10@IJCAI-09: Explanation-aware on artificial intelligence. In Petridis, M., Coenen, F., and Bramer, M., editors, Computing (ExaCt 2009), pages 94–101, 2009. Research and Development in Intelligent Systems XXV, London, UK. Springer Verlag. [Forcher et al., 2009] Forcher, B., Möller, M., Sintek, M., and Roth- Berghofer, Th. Explanation of semantic search results of medical images in [Roth-Berghofer and Richter, 2008a] Roth-Berghofer, T. R. and Richter, M. medico. In Thomas R. Roth-Berghofer, Nava Tintarev, and David B. Leake, M., editors (2008a). Künstliche Intelligenz—Topic: Explanation, volume 22, editors, Workshop 10@IJCAI-09: Explanation-aware Computing (ExaCt Bremen. BöttcherIT Verlag. 2009), pages 13–24, 2009. [Roth-Berghofer and Richter, 2008b] Roth-Berghofer, T. R. and Richter, M. [Stahl, Roth-Berghofer, 2009] Stahl, A. and Roth-Berghofer, Th. Rapid M. (2008b). On explanation. Künstliche Intelligenz, 22(2):5–7. Prototyping of CBR applications with the Open Source Tool myCBR. 2007 Künstliche Intelligenz, 23(1):34–37, March 2009. [Bahls and Roth-Berghofer, 2007] Bahls, D. and Roth-Berghofer, T. (2007). 2008 Explanation support for the case-based reasoning tool myCBR. In [Roth-Berghofer et al., 2008] Roth-Berghofer, Th., Schulz, S., Bahls, D., and Proceedings of the Twenty-Second AAAI Conference on Artificial Leake, D.B., editors. Proceedings of the ECAI-08 workshop on Intelligence. July 22–26, 2007, Vancouver, British Columbia, Canada., Explanation-aware Computing ExaCt2008. University of Patras, July 2008. pages 1844–1845. The AAAI Press, Menlo Park, California. http://ceur- ws.org/Vol- 391. [Eppert, 2007] Eppert, M. (2007). Generating provenance explanations for [Forcher, Adrian, Roth-Berghofer, 2008] Forcher, B., Adrian, B., and Roth- the gnowsis rebirth machine - a first pass. Pro ject thesis, University of Berghofer, Th.. Explanation styles in iDocument. ExaCt 2008, ECAI-08 Kaiserslautern. Workshop. [Roth-Berghofer et al., 2007] Roth-Berghofer, T. R., Schulz, S., and Leake, [Bahls, 2008] Bahls, D. (2008). Explanation support for the case-based D. B., editors (2007). Proceedings of the AAAI-07 workshop on reasoning tool myCBR. Project thesis, University of Kaiserslautern. Explanation-aware Computing ExaCt2007. AAAI Press. Technical Report WS-07-06. [Forcher et al., 2008] Forcher, B., Adrian, B., and Roth-Berghofer, T. (2008). Explanations in the information extraction system iDocument. Künstliche Intelligenz, 22(2). [Mittag, 2008] Mittag, F. (2008). ReduxExp: A justification-based explanation-support server. Project thesis, University of Kaiserslautern. Freitag, 11. September 2009 59
  60. 60. Publications of ExaCt research group 2006 [Richter et al., 2006] Richter, M. M., Roth-Berghofer, T., and Schulz, S., editors (2006). Explanation-aware Computing, volume 25. SAP - Slovak Academic Press Ltd., Bratislava. 2005 [Roth-Berghofer et al., 2005a] Roth-Berghofer, T., Cassens, J., and Sørmo, F. (2005a). Goals and kinds of explanations in case-based reasoning. In Althoff, K.-D., Dengel, A., Bergmann, R., Nick, M., and Roth-Berghofer, T., editors, WM 2005: Professional Knowledge Management, pages 264–268, Kaiserslautern, Germany. DFKI GmbH. [Roth-Berghofer and Cassens, 2005] Roth-Berghofer, T. R. and Cassens, J. (2005). Mapping goals and kinds of explanations to the knowledge containers of case-based reasoning systems. In Muñoz-Avila, H. and Ricci, F., editors, Case-Based Reasoning Research and Developmen, pages 451– 464, Heidelberg. Springer Verlag. [Roth-Berghofer et al., 2005b] Roth-Berghofer, T. R., Schulz, S., and Woody, A., editors (2005b). Proceedings of the AAAI Fal l Symposium on Explanation-aware Computing ExaCt2005. AAAI Press. Technical Report FS-05-04. 2004 [Roth-Berghofer, 2004] Roth-Berghofer, T. R. (2004). Explanations and Case-Based Reasoning: Foundational issues. In Funk, P. and González- Calero, P. A., editors, Advances in Case-Based Reasoning, pages 389–403. Springer-Verlag. [Memmel, Roth-Berghofer, 2004] Memmel, M. and Roth-Berghofer, Th. Explanation and e-learning: A first pass. In Klaus-Peter F¨ahnrich, Klaus P. Jantke, and Wolfgang S. Wittig, editors, Von e-Learning bis e-Payment. Das Internet als sicherer Marktplatz, pages 255–263. Akademische Verlagsgesellschaft Aka, 2004. Freitag, 11. September 2009 60
  61. 61. Invitation to participate HTTP://ON-EXPLANATION.NET • Manifesto • Mailing list • Workshop series • … Freitag, 11. September 2009 61

×