Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Implementation

679 views

Published on

Slides for HCICourse.com

Published in: Education
  • Be the first to comment

  • Be the first to like this

Implementation

  1. 1. chapter 8 (modified)implementation support
  2. 2. Introductionhow does HCI affect the programmer?levels of abstraction hardware specific → interaction-technique specificlayers of development tools – windowing systems – interaction toolkits – architecture styles and frameworks
  3. 3. windowing systems device independence resource sharing application management
  4. 4. windowing systems device independencedifferent devices:mouse, trackpad, joystickkeyboard layoutsscreen resolution
  5. 5. device independence layers of abstractionApplication: what the user really wants to do Toolkit: (e.g. Java AWT) more abstraction! Windowing System: abstract pointer (mouse) abstract display: pixels, Postscript, vector graphicsOperating System: hardware interface, low-level device drivers
  6. 6. windowing systems resource sharing (usually) many one screen one pair of eyesapplications one keyboard one set of fingers one mouse
  7. 7. resource sharingwindow system virtual terminals abstract abstract abstract abstract each app its own PC! terminal 1 term. 2 term. 3 term. 4OS: resource manager shares disk, CPU, etc. device drivers
  8. 8. types of window layout• tiled (space shared) – web sidebars, expanding widgets• single app (time shared) – phone, early Windows• overlapping (time and space) – most common for PCs
  9. 9. resource issues not just the screen!users care about power management network utilisation ... especially when it costs!
  10. 10. windowing systems application managementmany applications ... so – consistent look and feel – inter-application sharing cut & paste, drag & drop scripting and automation – UI to manage it all e.g. Mac Finder, Windows Explorer app swopping, preferences +extras: dashboard
  11. 11. Architectures of windowing systemsthree possible software architectures – all assume device driver is separate – differ where management is implemented early Mac & MS Windows1. each application manages everything assumed ‘cooperative’ apps – everyone worries about synchronization – poor portability and robustness may be used for – can be efficient for low-end devices dedicated devices2. window mgt. tightly coupled with operating system – applications tied to operating system Mac OS and MS Windows3. management role as separate application – maximum portability X Windows
  12. 12. X Windows architecture
  13. 13. X Windows architecture (ctd)pixel imaging model with some pointing mechanismX protocol defines server-client communicationseparate window manager client enforces policies for input/output: – how to change input focus – tiled vs. overlapping windows – inter-client data transfer
  14. 14. not just the PCphone – similar issues – sharing screen, keyboard, etc. – ‘full screen’ apps, not overlapping windows but Vodafone 360 had semi-open appsweb – browser takes role of window managerweb micro-apps (e.g. Facebook apps) – access shared data, screen spacededicated devices (e.g. microwave control panel) – mostly code direct to hardware but appliance-oriented Java, Windows, ...
  15. 15. programming the application toolkits paint models event models
  16. 16. toolkits what they provideinteraction objects (widgets) e.g. menus, buttons, simple text areas, file selection N.B. input and output intrinsically linkedlayout support for resizable windowsabstractions for windowing & OS services e.g. copy/paste, sound
  17. 17. toolkits higher level of abstraction than raw WMpromotes consistency – look and feel, widget interactionseasier to code – amenable to object-oriented programmingeasier to port – new versions of same platform – cross-platform • e.g. Java for desktop apps, jQuery for web
  18. 18. interfaces in JavaJava toolkit – AWT (abstract windowing toolkit)Java classes for buttons, menus, etc.Notification based – AWT 1.0 – need to subclass basic widgets – AWT 1.1 and beyond – callback objectsSwing toolkit – built on top of AWT – higher level features – uses MVC architecture (see later)
  19. 19. paint models what appears on the screen and when it happens
  20. 20. screen painting layers screen your application your code toolkit layer(s)void main() { > cat fred.txtFrame f = new Frame(); This is frd.txtf.show() AWT > ls} fred.txt tom.txt Swing harry.c dick.java other application(s) > cat fred.txt This is frd.txt > ls fred.txt tom.txt harry.c dick.java
  21. 21. paint models what do you drawdirect to the screendirect to screen via viewport – clipping, coordinate transformation e.g. raw Java AWTseparate buffer – double buffer to reduce flicker, retained bitmap option in Java Swingdisplay list / model – list of operations line, image etc. toolkit worries about showing these the screen e.g. OpenGL, ? MacOS display PDF, JS-DOM
  22. 22. buffering in toolkit and/or window managere.g. Mac OS Xthree levels:• nonretained – no buffering• retained – wm buffers hidden parts• buffered – app/toolkit writes to buffer
  23. 23. paint models when do you need to drawinternal events – data has changed – application ‘knows’external events – user / network does things – window manager ‘knows’ … and then tells toolkit
  24. 24. paint models when do you actually drawinternal control – code does things when it wants – but what about external events? works OK with display list or retained bitmapexternal control – called by the toolkit / window manager – but what about internal events? … purpose of repaint() in Javadraw once per frame (game engines) – variant of external control … but code rather like internal combined with polling model for input (check button state)
  25. 25. event models when things happen
  26. 26. event models – when things happen read-evaluation loop repeat read-event(myevent) case myevent.type type_1: do type_1 processing type_2: do type_2 processing ... type_n: do type_n processing end case end repeat
  27. 27. event models notification-basedvoid main(String[] args) { Menu menu = new Menu(); menu.setOption(“Save”); menu.setOption(“Quit”); menu.setAction(“Save”,mySave) menu.setAction(“Quit”,myQuit) ...}int mySave(Event e) { // save the current file}int myQuit(Event e) { // close down}
  28. 28. going with the grainnotification style affects the interface – modal dialogue box • easy with event-loop (just have extra read-event loop) • hard with notification (need lots of mode flags) – non-modal dialogue box • hard with event-loop (very complicated main loop) • easy with notification (just add extra handler)beware! if you don’t explicitly design, it will just happen implementation should not drive design
  29. 29. screen your code java api AWT invokes Listener repaint sets myListener(…) { user ‘dirty’ flag // update state clicks repaint(); } notices component dirty flag paint(Graphics g) { asked to g.fillRect(…) paint itself // draw things } waits for more input …
  30. 30. asynchrony ... things happen in any orderhard to test code – usually one test user at a time – low network load – fast networks and fast machinesrace conditions – unexpected event orders – sometimes may seem unlikely, but ...
  31. 31. Birthday surprise problempeople connections 1 0 2 1 3 3 4 6 5 10 6 15 … … n n(n-1)/2
  32. 32. architecture and frameworks separation and presentation abstraction: Seeheim component models: MVC and PAC
  33. 33. origins of separation User Interface Management Systems (UIMS)a level above toolkits (e.g. for non-programmers)separation: application semantics and presentation – colours & wording different from deep functionalityimproves: – portability – runs on different systems – reusability – components reused cutting costs – multiple interfaces – accessing same functionality – customizability – by designer and uservirtually extinct (!) – but the concepts live on – the Seeheim workshop ...
  34. 34. Seeheim model lexical syntactic semantic Functionality DialogueUSER USER Presentation (application APPLICATION Control interface) switch
  35. 35. conceptual vs. implementationSeeheim – arose out of implementation experience – but principal contribution is conceptual – concepts part of ‘normal’ UI language … because of Seeheim … … we think differently! e.g. the lower box, the switch • needed for implementation presentation dialogue application • but not conceptual
  36. 36. semantic feedbackdifferent kinds of feedback: – lexical – movement of mouse – syntactic – menu highlights – semantic – sum of numbers changessemantic feedback often slower – use rapid lexical/syntactic feedbackbut may need rapid semantic feedback – freehand drawing – highlight trash can or folder when file dragged
  37. 37. what’s this? lexical syntactic semantic Functionality DialogueUSER USER Presentation (application APPLICATION Control interface) switch
  38. 38. the bypass/switch lexical syntactic semantic Functionality DialogueUSER USER Presentation (application APPLICATION Control interface) switch direct communication rapid semantic between application and presentation feedback but regulated by dialogue control
  39. 39. monolithic vs. componentsSeeheim has big componentsoften easier to use smaller ones – esp. if using object-oriented toolkitsSmalltalk used MVC – model–view–controller – model – internal logical state of component – view – how it is rendered on screen – controller – processes user input
  40. 40. MVCmodel - view - controller view model controller
  41. 41. MVC issues• MVC is largely pipeline model: input → control → model → view → output• but in graphical interface – input only has meaning in relation to output e.g. mouse click – need to know what was clicked – controller has to decide what to do with click – but view knows what is shown where!• in practice controller ‘talks’ to view – separation not complete
  42. 42. PAC model• PAC model closer to Seeheim – abstraction – logical state of component – presentation – manages input and output – control – mediates between them• manages hierarchy and multiple views – control part of PAC objects communicate• PAC cleaner in many ways … but MVC used more in practice (e.g. Java Swing)
  43. 43. PACpresentation - abstraction - control A P A P C C abstraction presentation control A P C A P C
  44. 44. ... and on the web
  45. 45. on the web salami sliced Seeheim between browser and server NOT the obvious split presentation dialogue semanticsbrowser web server
  46. 46. on the web salami sliced Seeheim between browser and server data updated on page (esp. AJAX) presentation dialogue semanticsbrowser links and css and javascript browser styling updating server backend databases web server scripts templates, XLST
  47. 47. on the web salami sliced Seeheim between pages/scripts – consistency and control? presentation dialogue semanticsscript 1script 2script 3script 4 CSS and dialogue flow business logic templates maintaining state
  48. 48. SummaryLevels of programming support tools• Windowing systems – device independence, resource sharing – managing multiple user tasks and applications• Toolkits – programming interaction objects• Programming the application – event paradigms: read-evaluation vs. notification – paint paradigms• Architecture – conceptual architectures for separation – need to think about dialogue and state

×