0
Designing a Prosthetic
 Memory for Software
     Developers


    PhD Thesis Proposal
  In Software Engineering
          ...
Preview

• Settings: Software implementation and
  maintenance work in the IDE
• Problem: Managing, using, and sharing
  k...
Preview

• Thesis statement:
 “A prosthetic memory for software
 development…
 …inspired by human episodic memory…
 …can s...
Outline

• Knowledge of activities
     Problems, techniques, proposed support
•   Knowledge of artifacts
•   Traceabilit...
Activity knowledge




                                                                            5
Activities   Artifact...
Running Example

• Wally works on multiplayer cellphone chess
• Wally plans game loop
   Each side maintains copy of boar...
Running Example:
                           Disorientation
•    Created Model classes and UI
•    Create ChessClient class...
Running Example:
                   Disorientation
• Need to backtrack to add check for
  end-of-game, then move on to the...
Running Example:
                     Omissions
• Wally remembers he needs to implement
  the server and the client-receiv...
Keeping Track Of Reminders

• Wally could have used bug DB to
  record tasks for reminders
         Appropriate for high-...
Keeping Track of Reminders

• Wally could have used to-do
  comments for reminders [Storey ‘08]
         Cheap and easy t...
Proposed approach

• We propose building a prosthetic
  memory for software development
         “Device or software that...
Proposed Approach:
Journal of Objective Activities
• A “journal”/“memory” of development work
• Formed around “objective o...
Proposed Approach:
Journal of Objective Activities
• Chronological view
         Peripherally shows
          most recent...
Proposed approach:
Add Subjective “reminders”
• Developer can provide
  “subjective” observations
• Interleaved with objec...
Proposed approach:
Add Activity Announcements
• Visual partition if developer declares when
  starting or switching tasks ...
Proposed approach:
                 Backtracking
• Simplified backtracking by removing top
• Can further filter for leftover...
Proposed approach:
                 Context View
• Memory view shows limited portion
         Hides old reminders relevan...
Proposed approach:
             Sharing activity
Sharing activities can
  improve awareness

•   Status/Away
    announcem...
Proposition 1

             Providing discrete categorized
             subjective observations about their
             a...
Artifact knowledge




                                                                            21
Activities   Artifac...
Knowledge About Artifacts

• Developers generate important knowledge
  while creating artifacts
         Some manifested ...
Running Example:
 Need for artifact knowledge

•     Asuk has to write the server
•     Examines the client sending code
•...
Running Example:
Need for artifact knowledge
Excerpt from
ICommunicator




                              24
Running Example:
 Need for artifact knowledge
Excerpt from
CommunicatorImpl




                                          ...
Documentation Types

                                        External                                 Internal
Amount     ...
Running example:
Adequate documentation




                         27
Documentation Feasibility

• Optimally: comprehensive up-to-date
  readable documentation
• But: Costs are high
         ...
Proposed Approach:
  Observations on Artifacts
• A compromise between internal and
  external documentation
• User provide...
Proposed Approach:
      Observations on Artifacts
Creating
Observations




                                             ...
Proposed Approach:
  Observations on Artifacts

Memory
View




                              31
Proposed Approach:
      Observations on Artifacts

Context
View




                                                     ...
Proposition 2

      Providing discrete categorized
      subjective observations about
      artifacts… (What we just saw...
Decision Traceability




                                                                            34
Activities   Arti...
Decision Traceability

• Occasional need to understand
  “how and why” of current artifact state
         Infrequent and ...
Running Example

• Asuk is trying to understand Wally’s
  implementation of Communicator
         What is the purpose of ...
Proposed Approach:
                   Traceability
• Asuk selects authorization line and queries for all
  relevant events...
Running Example

• Asuk still doesn’t understand the
  game loop
• Wally did not document his intentions
         Well kn...
Episodic Memory

• Developers avoid cost/distraction
• How to get developers to provide
  knowledge?
• Humans capable of m...
Episodic vs. Semantic Memory

                                        Semantic                   Episodic
Content and     ...
Episodic Memory in
               SW Development
• Recollection of development episode in EM
         Visual context
    ...
Obtaining Subjective
                 Knowledge
• Knowledge workers reason and reinforce
  working memory with “inner voic...
Example of Think-Aloud




                         43
Obtaining Subjective
          Knowledge
• Developer “narrates” work for benefit of self
  and of future users
   Most lik...
Propositions 3+4

3. Developers can provide, with little
  effort, a continuous narrative that
  captures important tracea...
Thesis and Contributions




                                                                            46
Activities   A...
Expected Practical Benefits

• Objective record only:
         Tracking some recent navigation and changes
         Traci...
Expected Technical
                 Contributions

• Objective knowledge collection
         Infrastructure for IDE monit...
Thesis Statement

• A prosthetic memory for software
  development inspired by human
  episodic memory can successfully
  ...
Expected Research
                  Contributions
• Demonstrating that cost reductions alone
  yield more knowledge preser...
Potential impact

• Step towards addressing the
  knowledge problem in SW
  development
• Proactive knowledge preservation...
Major Technical Challenges

• Collecting adequate telemetry
• Eliciting objective observations
• Collecting subjective kno...
Completed work




                                                                            53
Activities   Artifacts  ...
Research Evolution

• Early work on interruptions
         Means for automatically deferring
          interruptions
    ...
Research Evolution

• Studies of design
         Seeking to support distributed design
         Studied collocated desig...
Research Evolution

• Knowledge preservation in eMoose
         Focused on computer-based work
              More straig...
eMoose Architecture




                                                                            57
Activities   Artifa...
What it looks like




                     58
Personal experience with tool

• Used for over 6 months
• Typed discrete observations about activities
         Effective...
Validation plans




                                                                             60
Activities   Artifact...
Prototype Deliverables

• Initial prototype ready for use
• Still much work
         Robustness
         Correctness of ...
Formative Evaluation

• Goal: General initial evaluation and
  preparation for prototype release
• Recruit testers within ...
Phase 1

• Goal: Evaluate ability to manage tasks and
  reminders and avoid disorientation (prop. 1)
• Recruit developers ...
Phase 2

• Goal: Knowledge preservation and sharing
  with discrete observations (prop. 2)
• Recruit from phase 2 particip...
Phase 3

• Goal: Study knowledge preservation and
  consumption with continuous narrative
• Recruit users for lab study
• ...
Phase 4 (optional)

• Goal: Study everything together
• Recruit users for lab study
• Sequence of hand-offs from one user
...
Timeline




                                                                            67
Activities   Artifacts   Trace...
Risks (BACKUP)




                                                                             68
Activities   Artifacts ...
Capture difficulties

• Risk: Not all objective knowledge
  can be captured
• Existing knowledge sufficient for most
  pur...
Voice Capture

• Risk: Environment not suited for verbal
  narration
• Risk: Not finding appropriate engine
• Continuous na...
Voice Capture

•     Risk: Speech to text inaccurate
•     We do not need absolute accuracy
•     Knowledge readable even ...
Insufficient contributions

• Risk: Users do not provide enough
  subjective knowledge?
• Likely to see variations between...
Users prefer traditional
                    means?
• Risk: Users prefer traditional means of
  preserving knowledge
• If ...
Discrete knowledge
                 consumption
• Risk: Users do not have space for
  memory and context views?
         ...
Discrete observations

• Risk: Developers do not associate types
  with observations
         Types can be associated aft...
Continuous Narrative

• What if it is difficult to elicit
  knowledge from the continuous
  narrative?
         Could exp...
BACKUP SLIDES




                77
Navigation disorientation

• Questions:
   How did we arrive at current position?
   How do we get to next position or b...
Change disorientation

• Questions:
   What changes took place?
   What to include in a commit?
   What to revert when ...
• Bob left several “surprises”
   He didn’t realize there are special moves
    in chess with different encodings
   Rac...
Other preserved knowledge

• Knowledge preserved in:
   Version control comments
   Bug tracking comments
   Electronic...
Others using tools

• No field deployment yet
• Brought one user to lab
   Subject showed preference for speech
     Diff...
Upcoming SlideShare
Loading in...5
×

Designing a Prosthetic Memory for Software Developers

1,675

Published on

This are the slides for my initial dissertation proposal at CMU, which covered the original eMoose project and the idea of combining objective and subjective knowledge.

Because the scope of this work was too large, I ended up focusing my dissertation on the contextual side and abandoning this line of work.

1 Comment
1 Like
Statistics
Notes
  • Dear Uri Dekel:

    I have not seen all the slides in your presentation. Best wishes for your success.

    It is 3.5 years since you started this. Hope you have succeeded.
    Even now if there is any way of using a related but structurally different framework try:
    http://www.slideshare.net/putchavn/reusable-software-artifacts-repository-re-sar-06nov12

    Feel free to reach me if you like.

    putchavn@yahoo.com
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
1,675
On Slideshare
0
From Embeds
0
Number of Embeds
0
Actions
Shares
0
Downloads
10
Comments
1
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "Designing a Prosthetic Memory for Software Developers"

  1. 1. Designing a Prosthetic Memory for Software Developers PhD Thesis Proposal In Software Engineering Uri Dekel 1 22 January 2008
  2. 2. Preview • Settings: Software implementation and maintenance work in the IDE • Problem: Managing, using, and sharing knowledge about artifacts AND activities  Reduce errors, disorientation, omissions • Proposed approach: Memory aid in IDE  Presents “journal” of development episode  Interleaves “Objective” and “Subjective” Knowledge  Strength in objective-subjective synergy  Offers unified management of knowledge on artifacts and activities 2
  3. 3. Preview • Thesis statement: “A prosthetic memory for software development… …inspired by human episodic memory… …can successfully augment the organic memories of developers… …and help them preserve, manage, and share knowledge” 3
  4. 4. Outline • Knowledge of activities  Problems, techniques, proposed support • Knowledge of artifacts • Traceability of decisions • Thesis and contributions • Completed work • Evaluation plans • Risks (Backup in response to questions) 4
  5. 5. Activity knowledge 5 Activities Artifacts Traceability Thesis Completed Plan Risks
  6. 6. Running Example • Wally works on multiplayer cellphone chess • Wally plans game loop  Each side maintains copy of board  Client gets move from UI, sends to server and waits for opponent response  Server always relays moves to other side  Client receiving opponent move updates board, waits for UI • Subtasks for implementing game loop  Communication infrastructure  Encode moves  Handle end of game 6
  7. 7. Running Example: Disorientation • Created Model classes and UI • Create ChessClient class  Implement onUIMove  Add board update call • Switch to Board class  Implement board update  Add Communicator calls • Switch to Move class  Implement move encoding  Implement move decoding  Create Communicator class • Implement all four methods  Investigate networking library docs and samples  NEED: Don’t wait if end-of-game from own move • NEED: Implement end-of-game check on Board class  NEED: Implement onOpponentMove  NEED: Implement server  NEED: Handle end-of-game 7 Activities Artifacts Traceability Thesis Completed Plan Risks
  8. 8. Running Example: Disorientation • Need to backtrack to add check for end-of-game, then move on to the server • Potential for navigation disorientation [DeAlwis ‘06]  Too many open windows  Location stack in Eclipse limited  DOI models not specific enough • Wally closes all windows to “start fresh” • Potential for task disorientation  What was I doing and what did I accomplish?  What remains to be done? 8 Activities Artifacts Traceability Thesis Completed Plan Risks
  9. 9. Running Example: Omissions • Wally remembers he needs to implement the server and the client-receiving  Begins work on server • Omission of check for end-of-game before waiting for opponent response  System will get stuck if user’s own move ends game  Problem will likely pop up while testing something else and make for difficult debug 9 Activities Artifacts Traceability Thesis Completed Plan Risks
  10. 10. Keeping Track Of Reminders • Wally could have used bug DB to record tasks for reminders  Appropriate for high-level tasks  Self-contained, well defined, preplanned  Less appropriate for personal subtasks  Transient, not-well defined, ad-hoc  Expensive to use during development  Unwanted public archival [Storey ‘08]  Does not ensure timely awareness 10 Activities Artifacts Traceability Thesis Completed Plan Risks
  11. 11. Keeping Track of Reminders • Wally could have used to-do comments for reminders [Storey ‘08]  Cheap and easy to produce  Proactive investment in phrasing  Potential for placement difficulties  Clutter the code and not private if committed  Code must be inspected to reveal to-do  No clear order of completion 11 Activities Artifacts Traceability Thesis Completed Plan Risks
  12. 12. Proposed approach • We propose building a prosthetic memory for software development  “Device or software that supplements human memory, to store maintain and access copies of relevant information”  Many existing prostheses focus on physical world  Others record knowledge without context 12 Activities Artifacts Traceability Thesis Completed Plan Risks
  13. 13. Proposed Approach: Journal of Objective Activities • A “journal”/“memory” of development work • Formed around “objective observations”  Activities with external manifestations  Obvious to layperson without interpretation  Yes: Added text X to method Y  No: Made method Y robust against null parameters  Aggregation/abstraction of low level events  e.g., Edited method Y from S1 to S2 between T1 and T2  Automated capture via instrumentation/monitors  Video recording not structured/searchable 13 Activities Artifacts Traceability Thesis Completed Plan Risks
  14. 14. Proposed Approach: Journal of Objective Activities • Chronological view  Peripherally shows most recent  Maintain Orientation  Use to find artifacts • Difficult to understand and visually search 14 Activities Artifacts Traceability Thesis Completed Plan Risks
  15. 15. Proposed approach: Add Subjective “reminders” • Developer can provide “subjective” observations • Interleaved with objective • Objective provides context to subjective  Can infer details from adjacent objective  Makes subjective cheaper to produce • Proposition: reduced costs lead to more knowledge preservation 15
  16. 16. Proposed approach: Add Activity Announcements • Visual partition if developer declares when starting or switching tasks or subtasks 16 Activities Artifacts Traceability Thesis Completed Plan Risks
  17. 17. Proposed approach: Backtracking • Simplified backtracking by removing top • Can further filter for leftover reminders 17 Activities Artifacts Traceability Thesis Completed Plan Risks
  18. 18. Proposed approach: Context View • Memory view shows limited portion  Hides old reminders relevant to current context • Separate “context view” shows all relevant subjective observations  For current resource and method  For other visible method definitions  Potentially related methods (e.g., invoked) • Allows caller to know of remaining to-dos in callee 18 Activities Artifacts Traceability Thesis Completed Plan Risks
  19. 19. Proposed approach: Sharing activity Sharing activities can improve awareness • Status/Away announcements • Change/checkout indicators (many tools: Tukan, Palantir, Lotus Jazz) • Activity indicator (using provided knowledge) 19
  20. 20. Proposition 1 Providing discrete categorized subjective observations about their actions… (What we saw so far) …will help developers avoid disorientation and omissions… …and improve their peers’ awareness of their actions 20 Activities Artifacts Traceability Thesis Completed Plan Risks
  21. 21. Artifact knowledge 21 Activities Artifacts Traceability Thesis Completed Plan Risks
  22. 22. Knowledge About Artifacts • Developers generate important knowledge while creating artifacts  Some manifested in the artifact  e.g., external interfaces, obvious implementations  Some only in developer’s mind  Usage protocols  Assumptions and limitations  Differences from expected implementation  Underlying rationale • Potential for costly errors  e.g., Not complying with usage protocol 22 Activities Artifacts Traceability Thesis Completed Plan Risks
  23. 23. Running Example: Need for artifact knowledge • Asuk has to write the server • Examines the client sending code • Wants to use Wally’s Communicator • What is the usage protocol?  e.g., Has to be initialized somewhere?  e.g., Can the same communicator handle multiple concurrent connections? • Without additional information, will have to study source code 23 Activities Artifacts Traceability Thesis Completed Plan Risks
  24. 24. Running Example: Need for artifact knowledge Excerpt from ICommunicator 24
  25. 25. Running Example: Need for artifact knowledge Excerpt from CommunicatorImpl 25 Activities Artifacts Traceability Thesis Completed Plan Risks
  26. 26. Documentation Types External Internal Amount Elaborate Limited Structure Elaborate Short text Multiple Yes (including virtual) Single construct/block artifacts Artifact Explicit (with name or content Visual to following construct connection replication) Presentation Separate from artifacts In artifact, may clutter Privacy Can be controlled Public once checked-in Awareness Only when browsing docs Only when reading source 26 Activities Artifacts Traceability Thesis Completed Plan Risks
  27. 27. Running example: Adequate documentation 27
  28. 28. Documentation Feasibility • Optimally: comprehensive up-to-date readable documentation • But: Costs are high  Ensuring grammatical/semantic clarity  Making elements self-contained  Delays from concurrent documentation • And: No guaranteed returns per element • Result: Limited documentation  Rationale and intent often lost  No exhaustive listing of assumptions/limitations 28 Activities Artifacts Traceability Thesis Completed Plan Risks
  29. 29. Proposed Approach: Observations on Artifacts • A compromise between internal and external documentation • User provides discrete subjective observations instead of comments • Reduced costs  Context and selection automatically assigned  Can be provided while coding (using voice)  Short with no proactive editing  Allows use of deictic references • Reduction in clutter and privacy issues • Presented in related contexts (e.g., callers) 29
  30. 30. Proposed Approach: Observations on Artifacts Creating Observations 30 Activities Artifacts Traceability Thesis Completed Plan Risks
  31. 31. Proposed Approach: Observations on Artifacts Memory View 31
  32. 32. Proposed Approach: Observations on Artifacts Context View 32 Activities Artifacts Traceability Thesis Completed Plan Risks
  33. 33. Proposition 2 Providing discrete categorized subjective observations about artifacts… (What we just saw) …will help developers track and share knowledge and thus avoid errors 33 Activities Artifacts Traceability Thesis Completed Plan Risks
  34. 34. Decision Traceability 34 Activities Artifacts Traceability Thesis Completed Plan Risks
  35. 35. Decision Traceability • Occasional need to understand “how and why” of current artifact state  Infrequent and after-the-fact but critical • Requires knowledge about:  Development process  Knowledge from developer’s mind  Passively absorbed information 35 Activities Artifacts Traceability Thesis Completed Plan Risks
  36. 36. Running Example • Asuk is trying to understand Wally’s implementation of Communicator  What is the purpose of Authorization?  Why is the request queue initialized here? 36 Activities Artifacts Traceability Thesis Completed Plan Risks
  37. 37. Proposed Approach: Traceability • Asuk selects authorization line and queries for all relevant events  One hit: Created and not moved or edited • Asuk asks for all observations from that time  Reveals sequence of web browsing  Android APIs, various examples  Eclipse activity just before implementing method  Opened sample Tweeter client in Eclipse  Copied section from client, pasted in method  Reedited then copied additional constructs, including queue • Asuk realizes that Wally blindly copied example  Uses referenced materials to learn 37 Activities Artifacts Traceability Thesis Completed Plan Risks
  38. 38. Running Example • Asuk still doesn’t understand the game loop • Wally did not document his intentions  Well known problem  No relevant knowledge or discrete subjective observations when work began  Needed knowledge more elaborate than possible with discrete observations 38 Activities Artifacts Traceability Thesis Completed Plan Risks
  39. 39. Episodic Memory • Developers avoid cost/distraction • How to get developers to provide knowledge? • Humans capable of memorizing details and experiences while working  Turned to literature in psychology  Encountered Tulving’s Episodic Memory model 39 Activities Artifacts Traceability Thesis Completed Plan Risks
  40. 40. Episodic vs. Semantic Memory Semantic Episodic Content and Connected facts w/o Rich chronological structure learning context episode recollections Scale Limited succinct Comprehensive but knowledge not accessible Example Road routes, Vivid recollection of vacation dates road trip vacation Learning process Multiple exposures Immediate with exposure Access Association Contextual cues 40 Activities Artifacts Traceability Thesis Completed Plan Risks
  41. 41. Episodic Memory in SW Development • Recollection of development episode in EM  Visual context  Activities  Thoughts during activities, including rationale • Details and distinguish factors degrade • Need to be externalized and persisted  Cannot tap organic memory • We mimic episodic memory  Collect objective knowledge with context  Combine with record of “thoughts” 41 Activities Artifacts Traceability Thesis Completed Plan Risks
  42. 42. Obtaining Subjective Knowledge • Knowledge workers reason and reinforce working memory with “inner voice” • Researchers glimpse into this voice by asking for continuous verbalization  “Think-aloud”, protocol analysis, etc.  No interference with end product if avoiding cognitive investment  Deictic references to shared context  Dependencies between utterances, form a continuous narrative 42 Activities Artifacts Traceability Thesis Completed Plan Risks
  43. 43. Example of Think-Aloud 43
  44. 44. Obtaining Subjective Knowledge • Developer “narrates” work for benefit of self and of future users  Most likely using speech while coding/thinking • Does not need to be exhaustive  Can apply discretion  But must avoid distracting proactive investment • Broken into uncategorized subjective observations  Visually distinct from discrete categories  Does not appear in context view  Interleaved with objective and context 44
  45. 45. Propositions 3+4 3. Developers can provide, with little effort, a continuous narrative that captures important traceability knowledge 4. Other stakeholders can cost- effectively elicit the narrated traceability knowledge from the prosthetic memory 45 Activities Artifacts Traceability Thesis Completed Plan Risks
  46. 46. Thesis and Contributions 46 Activities Artifacts Traceability Thesis Completed Plan Risks
  47. 47. Expected Practical Benefits • Objective record only:  Tracking some recent navigation and changes  Tracing passive exposure to information • Objective + Discrete subjective:  Recording reminders to reduce omissions  Recording and share artifact knowledge  Activity observations partition objective record and improve orientation and awareness • Objective + All subjective narrative  Traceability  Search 47 Activities Artifacts Traceability Thesis Completed Plan Risks
  48. 48. Expected Technical Contributions • Objective knowledge collection  Infrastructure for IDE monitoring  Technique for web application monitoring  Framework for representing traces with context from multiple sources 48 Activities Artifacts Traceability Thesis Completed Plan Risks
  49. 49. Thesis Statement • A prosthetic memory for software development inspired by human episodic memory can successfully augment the organic memories of developers and help them preserve, manage, and share knowledge 49 Activities Artifacts Traceability Thesis Completed Plan Risks
  50. 50. Expected Research Contributions • Demonstrating that cost reductions alone yield more knowledge preservation  Positive result - lead to more strategies  Negative results - highlight another factor? • Demonstrating the power of an episodic chronological representation for knowledge about software system  Existing tools rely on semantic presentation 50 Activities Artifacts Traceability Thesis Completed Plan Risks
  51. 51. Potential impact • Step towards addressing the knowledge problem in SW development • Proactive knowledge preservation reduces need for retroactive means • Support for distributed organizations • May apply to to other development phases • May apply to other knowledge work 51 Activities Artifacts Traceability Thesis Completed Plan Risks
  52. 52. Major Technical Challenges • Collecting adequate telemetry • Eliciting objective observations • Collecting subjective knowledge with minimal interruption • Processing spoken narrative • Effective peripheral presentation • Effective interactive investigation 52 Activities Artifacts Traceability Thesis Completed Plan Risks
  53. 53. Completed work 53 Activities Artifacts Traceability Thesis Completed Plan Risks
  54. 54. Research Evolution • Early work on interruptions  Means for automatically deferring interruptions  Gathering and interpreting low-level telemetry to determine activity  Similar strategy followed in prototype 54 Activities Artifacts Traceability Thesis Completed Plan Risks
  55. 55. Research Evolution • Studies of design  Seeking to support distributed design  Studied collocated design  Existing tools focus on supporting drawing  Identified difficulties in managing artifacts  Memory and spatial cues frequently used  Identified importance of knowledge preservation  Passive exposure to artifacts • Trace changes and impacts  Needed for interpretation  Effort to preserve knowledge limits creativity 55 Activities Artifacts Traceability Thesis Completed Plan Risks
  56. 56. Research Evolution • Knowledge preservation in eMoose  Focused on computer-based work  More straightforward to monitor  Tracking passive exposure  Developed technique for web access  Support for heterogeneous records with contextual details  Abstraction of telemetry  Addition of subjective support 56 Activities Artifacts Traceability Thesis Completed Plan Risks
  57. 57. eMoose Architecture 57 Activities Artifacts Traceability Thesis Completed Plan Risks
  58. 58. What it looks like 58
  59. 59. Personal experience with tool • Used for over 6 months • Typed discrete observations about activities  Effective capturing of reminders and bugs • Task/subtask switching observations help avoid disorientation and omissions • Effective for finding files and members  Recent locations  Searching by task indicator  Search by free text 59 Activities Artifacts Traceability Thesis Completed Plan Risks
  60. 60. Validation plans 60 Activities Artifacts Traceability Thesis Completed Plan Risks
  61. 61. Prototype Deliverables • Initial prototype ready for use • Still much work  Robustness  Correctness of objective observations  New features  Context view • Thesis scope:  Support for Java work within Eclipse  Support for web applications 61 Activities Artifacts Traceability Thesis Completed Plan Risks
  62. 62. Formative Evaluation • Goal: General initial evaluation and preparation for prototype release • Recruit testers within department • Train in all facets of eMoose • Freedom in whether and how to use • Frequently observe and interview • Short cycles on requests and bug fixes • Prepare for outside distribution 62 Activities Artifacts Traceability Thesis Completed Plan Risks
  63. 63. Phase 1 • Goal: Evaluate ability to manage tasks and reminders and avoid disorientation (prop. 1) • Recruit developers from OSS/Industry  Leverage existing contacts (IBM/Accenture, MSE)  Preferably recruit entire project team • Custom version focused activity knowledge • Periodic surveys on satisfaction and impact • Interviews with some participants • Collect data from memory view client • Check bug database for impact 63 Activities Artifacts Traceability Thesis Completed Plan Risks
  64. 64. Phase 2 • Goal: Knowledge preservation and sharing with discrete observations (prop. 2) • Recruit from phase 2 participants  Projects where everyone used eMoose  Add knowledge preservation/sharing features • Periodic surveys on satisfaction and impact • Collect exposure data from context view • Compare amount of documentation before/after intervention 64 Activities Artifacts Traceability Thesis Completed Plan Risks
  65. 65. Phase 3 • Goal: Study knowledge preservation and consumption with continuous narrative • Recruit users for lab study • Starting task from scratch • Asked to narrate for benefit of others • Qualitative analysis of narrative contents and impact on productivity • Select work of certain developers • Have others perform maintenance and explore decision. Analyze process and use 65 Activities Artifacts Traceability Thesis Completed Plan Risks
  66. 66. Phase 4 (optional) • Goal: Study everything together • Recruit users for lab study • Sequence of hand-offs from one user to another • Compensation incorporates group effort and preserved knowledge • Qualitatively analyze results 66 Activities Artifacts Traceability Thesis Completed Plan Risks
  67. 67. Timeline 67 Activities Artifacts Traceability Thesis Completed Plan Risks
  68. 68. Risks (BACKUP) 68 Activities Artifacts Traceability Thesis Completed Plan Risks
  69. 69. Capture difficulties • Risk: Not all objective knowledge can be captured • Existing knowledge sufficient for most purposes • May affect rare trace investigations 69 Activities Artifacts Traceability Thesis Completed Plan Risks
  70. 70. Voice Capture • Risk: Environment not suited for verbal narration • Risk: Not finding appropriate engine • Continuous narrative likely to be used less than typed discrete observations • For our lab studies, we will type in transcript between users. • Technologies and availability increasing • Organizations adjusting environment to support collaboration (e.g., pair prog) 70 Activities Artifacts Traceability Thesis Completed Plan Risks
  71. 71. Voice Capture • Risk: Speech to text inaccurate • We do not need absolute accuracy • Knowledge readable even if disrupted • May apply context to resolve names 71 Activities Artifacts Traceability Thesis Completed Plan Risks
  72. 72. Insufficient contributions • Risk: Users do not provide enough subjective knowledge? • Likely to see variations between users • Returns correlated with investment • Some organizations mandate knowledge preservation 72 Activities Artifacts Traceability Thesis Completed Plan Risks
  73. 73. Users prefer traditional means? • Risk: Users prefer traditional means of preserving knowledge • If our approach is just incorrect, not much we can do… • Developers may not trust tool with storing knowledge separately  We should offer automated means of adding the knowledge to the code or to documents • Have to improve robustness 73 Activities Artifacts Traceability Thesis Completed Plan Risks
  74. 74. Discrete knowledge consumption • Risk: Users do not have space for memory and context views?  Tool meant primarily for dual-monitor  Eclipse supports quick-view  Markers or overlay in lieu of context view • Risk: Users find them distracting?  Unlikely  Memory view changes by one line at a time  Context view changes only when switching members 74 Activities Artifacts Traceability Thesis Completed Plan Risks
  75. 75. Discrete observations • Risk: Developers do not associate types with observations  Types can be associated after-the-fact  Some can be inferred automatically • Risk: What if an observation is associated with wrong context?  Associations automatically associated with current context  User may spot mistake in memory view  We may support a degree-of-relevance context model 75 Activities Artifacts Traceability Thesis Completed Plan Risks
  76. 76. Continuous Narrative • What if it is difficult to elicit knowledge from the continuous narrative?  Could explore various techniques  Creating visual playback may offload some mental load  Collaborative filtering and editing over time 76 Activities Artifacts Traceability Thesis Completed Plan Risks
  77. 77. BACKUP SLIDES 77
  78. 78. Navigation disorientation • Questions:  How did we arrive at current position?  How do we get to next position or back? • Contributing factors:  Complex and unfamiliar code base [de Alwis ‘06]  Branching and delocalization • Manifestations:  Display thrashing  Excessive navigation 78
  79. 79. Change disorientation • Questions:  What changes took place?  What to include in a commit?  What to revert when backtracking from a failed attempt? • Contributing factors:  Branching and delocalization  Interleaving of concerns within same members • Manifestations:  Dead and irrelevant code in production  Accidental deletion 79
  80. 80. • Bob left several “surprises”  He didn’t realize there are special moves in chess with different encodings  Race conditions (assumed that opponent response takes time, no need for close connection fast)  Waiting checks a single mailbox in the communicator 80
  81. 81. Other preserved knowledge • Knowledge preserved in:  Version control comments  Bug tracking comments  Electronic communications • Limitations:  Sparse  Often written after-the-fact  Difficult to piece into readable material 81
  82. 82. Others using tools • No field deployment yet • Brought one user to lab  Subject showed preference for speech  Difficulty starting and stopping recording  Able to effectively “think-aloud”  Narrative contained rationale  Narrative too comprehensive  Must train for balance  Subject did not explicitly associate types 82
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×