The document discusses the past, present, and future of programming from a human-computer interaction perspective. It provides a historical overview of programming and discusses challenges in translating how people think and solve problems into computer terms. It also suggests that lessons may come from other fields beyond HCI and that not all problems need to be solved through programming computers.

1. Discussion: The Past, Present, and Future of Programming in HCI John C. Thomas IBM T. J. Watson Research Center Snow Mountain Ranch, Colorado February 6, 2009

2. Praiseworthy Points An excellent overview of a complex field WRT CMU work, a long-term effort Sometimes, it takes a long time to solve difficult problems Focus on a particular formulation of the problem (translation) shifting to broader concerns Focus on a stance --- “naturalness” as opposed to a particular technical approach: visualization.

3. Pickworthy Nits The phrase “everyone is computing” IT still not accessible to 5 of the 6 billion people on the planet Expanding in a useful way provides its own set of challenges: HCI4D If “good” HCI is a function of users, tasks, contexts, and technology, should we expect “an answer” to programming with good HCI?

4. A Formulation? The goal is to make it possible for people to express their ideas in the same way they think about them. By “natural,” we mean “faithfully representing nature or life,” which here implies it works in the way people expect. By “natural programming” we are aiming for the language and environment to work the way that nonprogrammers expect.

5. A Broader View Translation of Internal Thinking into Computer Terms Making Implicit Knowledge Explicit Understanding How to Make the Computer Work Effectively and Efficiently Understanding How to Solve a Problem Understanding How the Program Might Interact with People, other Programs, Systems

6. Gary Larsson

7. Translation Difficulty is not Limited to Human- Computer Dance N Funk instruction: Human to Human Interaction and common confusing comments: After doing a sequence like ABAB, CDCD, ABAB, EFEF, the instructor often says: “Let’s do that part again.” OR “Everything repeats.” “I forgot. There’s an insert in there. Like this.”

8. How do we “translate” from human intention into horse action? Some simple but highly evolved tooling helps (reins, saddle, stirrups). Humans need to be trained…but so do the horses. We do not try to get the horses to do brain surgery for us.

9. Making the implicit explicit

10. Ghost of Programming Past: QBE (early 1970’s) Query By Example allowed a one-hour training session to allow people to translate from natural language queries into QBE quickly with few errors. If we took into account “formal” complexity of resulting query, student’s IQ, time, mapping ease, and their own confidence rating, we could predict 95% of the variance in errors. In a later, more open-ended study, students were asked to generate their own relevant questions and the results were much poorer.

11. Lost Before Translation Students were given table structure of university data bases and then given “issues” such as: “Some of the younger faculty feel they are not paid enough relative to the older faculty.” Generate some questions relevant to this issue and translate them into QBE. Typical query: “Are the older faculty paid too much?”  Print the names of people whose age is greater than 55 and whose pay is “too much.” As Gary Olson pointed out Thursday morning, we may get different results if we did this study today wherein most students are Google-savvy.

12. Ghost of Programming Present High Performance Computing Project Complexity Modeling Rice Trial

13. Rice Trial Students given a problem and linear code and asked to parallelize in one of two languages. Videotaped Screen capture Real time observation/coding Blinded by paradigm

14. Ghost of Programming Future At Rice Trial, a lot of “expressive” gesturing from each participant Variations among participants Seemingly very regular within participant People spent a lot of time “looking for code” despite outline views and search facilities. Scanning even with fingers to keep place. People spent a lot of time using paper.

15. Ghost of Programming Future Instead of having the computer trying to infer the emotional state of users (fraught with issues) how about recognizing the appropriate sub-task or sub-goal? Understand and provide better search/viewing facilities. E.g., fisheye views? Provide better coordination between paper documents and notes and on-line environment.

16. Tools in Other Domains “Blank Piece of Paper” phenomenon What do we do in designing house, for instance?

17. Given a Blank Slate …

18. Look at Worked Examples But the vast majority of people buy pre-made houses or use a professional builder.

19. Providing Different Perspectives Training Perceptual Aids Cognitive Aids History Pattern Languages “Random” stimulation

20. Provide Tools to Allow New Perspectives Allowing new patterns to emerge

21. Using Physical Space to Provide Overview, Detail, Continuous Zooming, Multiple Activities Examples: Historical timeline of CHI conferences Constructed map of the world on which to perform exercises for CHI 2008 workshop on Human Computer Interaction for International Development 1929 Exposition Building, the Plaza de España The Plaza de España is one of Seville 's most easily recognised buildings and the epitome of the Moorish Revival in Spanish architecture . In 1929 Seville hosted the Spanish-American Exhibition and numerous buildings were constructed for the exhibition in Maria Luisa Park, among them the Plaza.

22. Historical Perspective Zooming, annotations

27. Viscosity? Imagine that new historical research indicates that ONE of the stories portrayed by a city tile is inaccurate? This could be remedied with almost no effect on the other displays. So…is visual programming, by necessity, highly viscous or have we just not found the right paradigm…or is it really computational paradigms themselves which are highly viscous and visual programming makes it obvious?

28. End User Programming Is EUP primarily a construction problem Or a search and sense-making problem? Wikipedia of programming possible? Finding the definition of Chinese characters.

29. Must all our problems be solved on a computer?

30. These are cats watching NOT birds. There is no doubt some intrinsic pleasure in watching… but they will not get to eat any birds

31. Conclusions There might be important lessons for HCI and programming from fields other than HCI and programming. Humans are both remarkably intelligent and remarkably flawed in their abilities to solve problems and communicate even with each other. We seem to have the capacity for “theory of mind” but do not always invoke it How to program may depend a lot on the domain and perhaps some domains are not best “done” by programming computers at all.

32. Is the glass half full or half empty? From the perspective of how much we’ve learned and how far we’ve come the glass is 90% full

33. Conclusion However, from the perspective of how much of the glass is actual “hard matter”, it is 99.999999999999 % empty. Similarly, if we consider the gap between the best that ordinary humans can do and how effectively and efficiently we can construct software, the glass is still 99.999999999999 % empty.

34. References Carroll, J., Thomas, J.C. and Malhotra, A. (1980). Presentation and representation in design problem solving. British Journal of Psychology/ , 71 (1) , pp. 143-155. Thomas, J. C. & Richards, J. T. (2008). Achieving psychological simplicity: Measures and methods to reduce cognitive complexity. In Handbook of HCI, A. Sears & J. Jacko (Eds.). Hillsdale, NJ: Erlbaum. Thomas, J.C., Lyon, D. & Miller, L. (1977). Aids for problem solving. IBM Research Report. RC-6468. Yorktown Heights, NY: IBM Corporation. Thomas, J. C. (1977). Cognitive psychology from the standpoint of wilderness survival. IBM Research Report, RC-6647. Yorktown Heights, NY: IBM Corporation. Thomas, J.C. (1989). Problem solving by human-machine interaction. In Gilhooly K.J., (Ed). Human and machine problem solving . London: Plenum Publishing. Thomas, J.C. (1988). Human factors and artificial intelligence. In H. Hartson and D. Hix (Eds.). Advances in human -computer interaction. Norwood, New Jersey: Ablex. Thomas, J.C. (1983). Psychological issues in the design of data-base query languages. In M. Sime and M. Fitter (Eds.), Designing for human-computer communication. . London: Academic Press. Thomas, J.C. and Carroll, J. (1978). The psychological study of design. Design Studies, 1 (1) , pp. 5-11. Thomas, J. C. & Gould, J. D. (1975),A psychological study of Query By Example. National Computer Conference Proceedings, 44 , 439-445. New York: AFIPS Press.