Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Model-Based User Interface Optimization: Part III: SOLVING REAL PROBLEMS - At SICSA Summer School on Computational Interaction 2015

956 views

Published on

Tutorial on Model-Based User Interface Optimization. Part III: SOLVING REAL PROBLEMS. Presented by Antti Oulasvirta, Aalto University. Presented at SICSA Summer School on Computational Interaction 2015. Note: This one-day lecture is divided into multiple parts.

Published in: Design
  • You can ask here for a help. They helped me a lot an i`m highly satisfied with quality of work done. I can promise you 100% un-plagiarized text and good experts there. Use with pleasure! ⇒ www.HelpWriting.net ⇐
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • Hello! I can recommend a site that has helped me. It's called ⇒ www.WritePaper.info ⇐ So make sure to check it out!
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
  • The professtional essay writer are having more knowledege about the writing papers. The professional essay writer are providing the best essay writing services papers to the students. The writeersity writing company had to providing the more writing papers for the professtionalist. The papers should be very quality and possible to acedemic success. HelpWriting.net Good luck!
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here

Model-Based User Interface Optimization: Part III: SOLVING REAL PROBLEMS - At SICSA Summer School on Computational Interaction 2015

  1. 1. Optimization for User Interface Design Antti Oulasvirta, Anna Feit Aalto University SICSA SUMMER SCHOOL ON COMPUTATIONAL INTERACTION GLASGOW
 JUNE 23, 2015 SOLVING REAL PROBLEMS
  2. 2. <YOUR FAVORITE UID PROBLEM>
  3. 3. GOALS • Learn to formulate your user interface design problems as an optimization task • Get more acquainted with assignment problems:A common problem type in interface design • Appreciate the nature of common design problems
  4. 4. CONTENTS • Assignment problems • Linear assignment, quadratic assignment • Linear menu, hierarchical menu, 
 menus with groups • Other problem types in HCI • Transfer • Functionality selection • Notification scheduling 5 “mini exercises”
  5. 5. YOU WILL NEED • Pen & paper • Your group
  6. 6. WHY PROBLEM FORMULATION
 IS VITAL • If you don’t do it,“optimization” does not make sense • There’s no way you can avoid it: Your code does it implicitly • You can find links to known problems and exploit their solutions • You can find efficient re-formulations • Currently, this is the single most important obstacle for optimization approaches to user interfaces • It challenges present notions of what user interface design is
  7. 7. ASSIGNMENT PROBLEMS
  8. 8. Pay attention and take notes!You will apply these definitions in a minute
  9. 9. LINEAR ASSIGNMENT • Assign n elements to m slots • Often n = m …Elements …Slots
  10. 10. DEFINITION • Assign agents to tasks • Find a bijection with minimum total cost • Assuming real-valued cost matrix C: min X a2A Ca,f(a) f : A ! T
  11. 11. AS A LINEAR PROGRAM An agent can only be assigned once Each task can hold only one agent min X i2A X j2T C(i, j)xij s.t. X j2T xij = 1 for i 2 A, X i2A xij = 1 for j 2 T
  12. 12. QUADRATIC ASSIGNMENT • Assign n elements to m slots on a rectangle-sized surface. Often n = m …Elements …Slots
  13. 13. DEFINITION • Given P facilities and L locations, a weight function w, and a distance function d, find the bijection
 that minimizes 
 
 
 
 
 Intuition:We minimize average distance among facilities.We try to “pack” together the facilities that are commonly used with each other f : P ! L X a,b2P wa,bdf(a),f(b) w weight d distance
  14. 14. FORMULATION WITH BINARY DECISION VARIABLES • Identify a permulation matrix X of dimensions 
 n x n such that ! • Observations: • Large problems (n > 30) may be impossible • Generalization of TSP and therefore NP-complete min nX i,j=1 nX h,k=1 dihfjkxijxhk f frequency d distance
  15. 15. IN-CLASS DISCUSSION Which of the following involve an assignment problem? Graphical user interfaces Consumer electronics Automotive interfaces Keyboards Web user interfaces Gestural interaction Mobile interfaces Dialogue interfaces
  16. 16. APPLICATION: MENU SYSTEMS “Rectangles assigned to locations on a larger rectangle”
  17. 17. SOLVABLE MENU TYPES
  18. 18. THE SIMPLEST MENU slot 1! slot 2! …! slot n One-dimensional (linear) menu 
 a.k.a. array / row facility layout problem commands
  19. 19. DEFINE THIS DESIGN TASK Given a frequency distribution for n commands, find an assignment of commands to menu such that it 
 minimizes expected search+selection time Command frequencies command! command! …! command Menu Command Frequency You have to make an assumption about search+selection time
  20. 20. ONE SOLUTION • Can be solved with the linear program min X i2A X j2T C(i, j)xij s.t. X j2T xij = 1 for i 2 A, X i2A xij = 1 for j 2 T Probability x Search time
 Search time from the SDP model 
 Cockburn et al. CHI’07] election times of each item i weighted by their probability T = n  i=1 piTi Selection time Ti of item i is given by Ti = Tdsi + Tpi Pointing time Tpi is governed by the Mackenzie varian aw and is Tpi = ap + bplog2(Ai/Wi + 1) where Ai is movement amplitude and Wi is height of item ically organized menus). ap and bp are empirically deter i=1 Selection time Ti of item i is given by Ti = Tdsi + Tpi Pointing time Tpi is governed by the Mackenzie law and is Tpi = ap + bplog2(Ai/Wi + 1) where Ai is movement amplitude and Wi is height tically organized menus). ap and bp are empirically constants. Decision/Search time Tdsi is linear interpolation and visual search: law and is Tpi = ap + bplog2(Ai/Wi + 1) where Ai is movement amplitude and Wi is height of tically organized menus). ap and bp are empirically d constants. Decision/Search time Tdsi is linear interpolation b and visual search: Tdsi = (1 ei)Tvsi + eiThhi where ei[0..1] is an expertise factor ei = 1 1/t i w number of repetitions with i. As the user becomes m with a menu, there is a shift from serial visual search among competing elements Thhi. and visual search: Tdsi = (1 ei)Tvsi + eiThhi where ei[0..1] is an expertise factor ei = 1 1/t i where number of repetitions with i. As the user becomes more ex with a menu, there is a shift from serial visual search Tvsi t among competing elements Thhi. Visual search time Tvsi is assumed to be linear function number of n: Tvsi = bvsn + avs Hick-Hyman decision time Thhi is dependent on the entrop item Hi = log2(1/pi). Here, pi is the probability of the ite the history of selections, pi = ti/tt where tt is the number selections in the menu. Now, Thhi is given by: where ei[0..1] is an expertise factor ei = 1 1/t i where ti is t number of repetitions with i. As the user becomes more experie with a menu, there is a shift from serial visual search Tvsi to dec among competing elements Thhi. Visual search time Tvsi is assumed to be linear function of th number of n: Tvsi = bvsn + avs Hick-Hyman decision time Thhi is dependent on the entropy of t item Hi = log2(1/pi). Here, pi is the probability of the item in the history of selections, pi = ti/tt where tt is the number of to selections in the menu. Now, Thhi is given by: Thhi = bhhHi + ahh where ahh and bhh are empirically determined constaints. The model has been validated for a few different types of me including linear, morphing, split, and rectangular menus. Figur Fast to solve but trivial: Organizes the elements
 in decreasing order of frequency
  21. 21. T = f(L,T, H, P) Selection time Menu length Target Position Practice The model is based on Fitt’s law and Hick-Hyman law Entropy THE SEARCH-DECIDE-POINT MODEL
  22. 22. HIERARCHICAL MENU command! command! command! command command! command! command command! command Title Title Title command! command! command! command! command command! command! command
  23. 23. DEFINE THIS DESIGN TASK Given a bigram distribution for n commands, find an assignment of commands to a menu hierarchy such that it minimizes expected search+selection time Bigrams Menu system Command Command You have to make an assumption about the cost of searching submenus
  24. 24. ONE SOLUTION • k submenus and n items can be organized in 
 ways into a tree • A QAP, but with much larger matrices for cost (here:A) and permutations P(n, m) ! • Practically impossible to solve to optimality when 
 n >20 (n + k)n+k+2 = [csr] is an m x m matrix and P = [pij] is an n x n P(n,m) is the set of all 1–1 (injective) mappings …, n} to {1, …, m}, corresponding to the place- the letters in these locations; and, !(!) is the loca- etter i in the current solution ! ∈ !(!, !). Now, (!) is the cost contribution of assigning letter i to !(!) and letter j to !(!). This problem can be for- with a quadratic objective function [33]: Let xij be variable that is 1 if letter i is assigned to location j, herwise. Now, Eq. 4 is equivalent to min !!"!!"!!"!!" ! !!! ! !!! ! !!! ! !!! ,!!!!!!!!!!!!!!!!!!!!(5)! " is the cost of using keyslot l after keyslot k. gn of menu systems is a QAP, too: n commands are to slots in k submenus that form a tree. Three dif- must be addressed for treating menu system design The Ant Althoug rapidly ( search g particula (ACO) colony f produce Impleme To our k We use ensures upper an pheromo add pher Bailly et al. UIST’13 p probability a cost (search time)
  25. 25. MENU WITH GROUPS command! command! command! command! command
  26. 26. DEFINE A DESIGN TASK Given a bigram distribution for n commands, find an assignment of commands to groups in a menu such that it minimizes expected search+selection time Relatedness distribution Grouped menu Command Command You have to make an assumption about how groups affect search+selection time
  27. 27. SOLUTION • Add another objective to the QAP: min nX i,j=1 nX h,k=1 dihfjkxijxhk + nX i,j=1 nX h,k=1 dihrjkxijxhk Minimizes distance
 among related 
 commands Alternative: You can formulate this as an additional constraint To decide where to put a divider, you need some heuristic to do that
  28. 28. SUMMARY • Assignment tasks are of two kinds • One-shot cost (linear) • Transaction cost (quadratic) • Powerful enough for many spatial design problems • We considered only single objective: performance. We will return to the multi-objective case
  29. 29. HTTP://GILLESBAILLY.FR/MENUOPTIMIZER.HTML VIDEO FROM BAILLY ET AL. PROC. UIST’13
  30. 30. INTERACTIVE LAYOUTS
  31. 31. FACILITY LAYOUT PROBLEM Fig. 4. Multi-floor layout. Reviews in Control 31 (2007) 255–267 Drira et al. 2007
  32. 32. DISCUSSION: EXTENSION TO LAYOUTS • New design considerations • Size of elements • Coloring • Grid layouts • Objective function • Clutter perception • Visual search time • Target acquisition time • Grid quality perception
  33. 33. BEYOND ASSIGNMENT PROBLEMS Goal:To sensitize you to different problem types in HCI
  34. 34. SKILL TRANSFER • Intuition: Exploit “familiar patterns” in a new user interface Old activities Familiar patterns
  35. 35. The Hughes Printing Telegraph (1855) PianoText [Feit DIS’14]
  36. 36. MPC-VCC Review 2014 VIDEO FROM FEIT & OULASVIRTA PROC. DIS’14
  37. 37. DEFINITION AS AN ASSIGNMENT TASK • The objective minimizes distances among assigned elements according to a given relatedness distribution (r) ! ! • O-oh!This does not work! min nX i,j=1 nX h,k=1 dihrjkxijxhk
  38. 38. ANOTHER TRY • Have to consider both source and target domain probabilities to make this work max X a2A X b2A X s2T X t2T pa,bps,txa,sxb,t subject to X s2t xa,s  1 8a 2 A X a2A xa,s = 1 8s 2 T
  39. 39. FUNCTIONALITY SELECTION
  40. 40. EXAMPLE FUNCTIONALITIES File Creation and organization Close Close the current file Create notebook Create notebooks to organize the notes Delete file Delete a file Delete notebook Delete notebook * Exit Exit the application Export Export the current file, e.g. to pdf Move note to notebook Move a selected note to one of the notebooks. New Create a new file Open choose a previously created file to be opened Open containing folder Open folder that contains the document. Recently opened choose a recently opened file to be opened Reload Reload the current file Rename Rename file. Rename notebook Save Save the current file Structural Elements Bullet size Change size of bullets. Bullet type Change bullet type. Bullets Start bullet list. Change number size Change start number Insert blank page Insert date and time Insert date and time into the text. Insert horizontal line Number style Change style of numbering, e.g. 1. or 2) or iii or IV, etc. Numbering Start numbered list. Page break Page layout Line number color Customize color of line numbers Line number continuation Continue line numbers or restart, e.g. at new page Line number font Customize font of line numbers Line number size Customize size of line numbers Line numbers Show or hide numbering of lines Margins Define the margins of the page (top, bottom, right, left) Orientation Change the orientation of the page (landscape or portrait) Page size Change the page size, e.g. to predefined templates (A4, A5, ...) or to a custom size. Text columns Define the number of columns of the layout (e.g. 2-column text) Link and share Link to calendar entry Link note to event in calendar Publish online Publish note online, e.g. on blog, facebook, etc. Put on homescreen Put the note in form of a widget on desktop or homescreen Share share notes via another application, e.g. Email, messenger, etc. View Close all windows In case of multiple open documents. Compare documents Compare two documents and show differences. Full screen Show in full screen mode, hide controls. Line wrap Wrap text at the end of a line Move document to other view In case of multiple documents per window (e.g. tabs) or split view. Save all windows Save all open document. Show grid Show ruler Split view Split view to show two documents at once. Synchronize scrolling Synchronize scrolling of two open documents for better comparison. Zoom Zoom in or out the document Printing Choose printer Choose the printer for printing the document Number of copies Change the number of copies to be printed. Print current page Print only the currently viewed page Print custom range Define custom range of pages to be printed. Print document Print multiple pages per sheet Print selection Print only selected text Printer settings Change setting of printer (e.g. color, quality, duplex etc.) as given by operating system. Search printer Search for a connected printer not displayed. Controls Copy Cut Delete element Delete selection Find Find a certain text in the document Mark all text Mark text Paste Paste copied or cut text. Redo Redo last action. Replace Replace selection by text. Undo Undo last action. Input Change encoding Change encoding of text (e.g. UTF-8) Draw freehand Draw shape e.g. rectangle, line, triangle, etc. Fill color Change fill color of shape Handwriting recognition Enter text by recognition of handwriting Insert link Insert picture Insert video Line color Change line color of drawing. Line style e.g. pen, pencil, crayon etc. Record audio Sort elements Sort selected elements alphabetically or numerically Spell check Check for correct spelling Type text Entering text via keyboard Voice recognition Enter text by voice input. Text style options Align text Change text alignment, e.g. left, right, middle, etc Character spacing Change space between characters Font color Change color of font Font size Change font size Indent Change text indent. Line spacing Change space between text lines Paragraph spacing Change space between paragraphs. Text background color Change color of text background. Text font Change font of text Text style Change style of text, e.g. bold, italic, underline, etc. Table Add column to table Add row to table Cell color Custom color of table cell Cell margins Customize text margins of a cell in the table Column size Change size of each column Create table Insert a table with defined number of rows and columns Merge cells Merge two or more cells into one. Remove column from table Remove row from table Row size Change size of each row Split cell Split a cell in the table into two. Table border Customize table border lines Text alignment in table Notes application (106 functionalities)
  41. 41. TRADE-OFFS Few LotsUsefulness Learnability Usability User satisfaction Costs Profitability
  42. 42. DEFINE A DESIGN TASK Given a set of functionalities, as well as their estimated usefulness values & usability costs, how to select the best subset of functionalities?
  43. 43. THE NEXT RELEASE PROBLEM • A software engineering problem the next release of software. ormulated as 0/1 knapsack problem: Let x1, ...xn be 0/1 va ence/presence of requirements in software. The goal is cted features to customers wi while keeping the total c max nX i=1 wixi s.t. nX i=1 cixi  B xi 2 {0, 1} at address a set of customers with different requiremen xpressing dependencies in the implementation of featur mplemented without feature A) [Bagnall et al. 2001]. Knapsack formulation A burglar breaks into a jewelry store He carries a backpack to store the jewels Backpack has limited capacity, jewels have a monet Goal: put in the backpack the combination of items th max total value Let’s design a heuristic WS 2011 But: Inadequate for UI designers
  44. 44. FORMULATING AN OBJECTIVE FUNCTION WITH EXPERTS Participants “How do you design functionalities in a real and ideal case?”
  45. 45. INTERVIEW: RESULTS Functionality selection is •Embedded in user-centered design process •iterative, corrective, exploratory •cooperative and collaborative, multiple stakeholders •Informed by multiple types of data and outputs •user profiles, scenarios, sketches, prototypes, wireframes •but designers often focus mainly on one scenario & user profile •Addressing multiple design objectives •usability, usefulness, user experience, profitability, implementability, learnability, …
  46. 46. OBJECTIVE FUNCTION erms of four objectives called USEP: usefulness U, satisfa profitability P. These terms form the core of the objectiv ut our work, supplementing it with case-specific weights, o hen needed. maxG = !U U + !SS + !EE + !P P denote a weight given to an objective, P !2⌦ ! = 1.0. wing, we define each term. Throughout, we denote a function and the full set of selected functionality . By ”normaliza al bar, we refer to scaling a variable to range [0, 1]. s e usefulness of a design U as the sum of usefulness u and ch selected functionality: ¯U = 1 2 ( X 2 ¯u + D ) P. These terms form the core of the objective function we upplementing it with case-specific weights, objectives, and axG = !U U + !SS + !EE + !P P (4) t given to an objective, P !2⌦ ! = 1.0. each term. Throughout, we denote a functionality selected set of selected functionality . By ”normalization”, marked r to scaling a variable to range [0, 1]. a design U as the sum of usefulness u and dependency ctionality: ¯U = 1 2 ( X 2 ¯u + D ) (5) 2 [0, 1] and D = P 2 P ⌧2 ,⌧ P i P j i,j (6) j2[0,1] are directional dependency scores defined by the de- Usefulness Dependency the set of functionality that maximizes ”goodness” G. We jectives called USEP: usefulness U, satisfaction S, ease- . These terms form the core of the objective function we pplementing it with case-specific weights, objectives, and xG = !U U + !SS + !EE + !P P (4) t given to an objective, P !2⌦ ! = 1.0. each term. Throughout, we denote a functionality selected et of selected functionality . By ”normalization”, marked to scaling a variable to range [0, 1]. a design U as the sum of usefulness u and dependency ctionality: ¯U = 1 2 ( X 2 ¯u + D ) (5) 2 [0, 1] and D = P 2 P ⌧2 ,⌧ P i P j i,j (6) 2[0,1] are directional dependency scores defined by the de- Satisfaction 3. Users value the loss of an important function more negatively than its inclusion in a design. tisfaction fine user satisfaction as a nonlinear function that maps the presence/absence of tionality to a satisfaction score. This score refers to subjective experience of the onality, as opposed to the usefulness score. We offer two formulations. first definition uses a linear sum: S = X 2 ¯s (7) mitation is that the absence of a functionality is not accounted for in the score. second definition builds on prospect theory that predicts that the absence of an tant functionality is appraised more critically than its presence. This is captured following mathematical function for prospect ⇡ of item : ⇡i = ⇢ p si ifxi = 1 2 p si ifxi = 0 (8) we reformulate as Ease-of-use Note that satisfaction can be negative if a design misses functionali to have. This yields a new S⇡: S = nX i=1 ¯⇡(i) S is used in the objective function after normalization. 4.3. Ease-of-use We provide two definitions for the ease-of-use of a design E. The quicker computation but simplifies the effect of increasing function is a decreasing function of the number of selected functionality. Our first formulation is a E = X 62 c where c is the estimated complexity of . ACM Transactions on Computer-Human Interaction, Vol. 1, No. 1, Article 11, Publication date: Ja Profitability Fig. 4. We assume the ease-of-use of an interface to decrease non-linearly with Here, y = 1/(x 1). Our second formulation captures the fact that ease-of-use is c ing more and more functionalities. Including very few triviall use. For example, designers group semantically to decrease sel chical menu [Bailly et al. 2014]. Interface techniques for comma functionality, toolbars, autocompletion, panels, and tool palette effect. Ease-of-use would increase linearly only in the case the i randomly. We also note that some interface techniques allow co number of features. For example, hotkeys can make access t but users typically learn only few hotkeys. While we use the re following, we recognize that the exact shape of this function including the skills of the design and the interface type. The re shown in Figure 4 and given as E = 1 | | + 1 We can also formulate this by introducing a quadratic constrai 4.4. Profitability We define the profitability of a functionality p in terms of its and costs c 2 R: p = v c . After normalization of p, we can ¯P = X 2 p = X 2 (v c ) Note that if c > v, profitability is negative. Sometimes inclu negative in profitability is justifiable in the light of other object
  47. 47. QUESTIONNAIRE Excerpt
  48. 48. BINARY QUADRATIC PROGRAMMING ! • Find a binary vector of length n to maximize quantity q • Global optimum can be computed • Fast! LP file generator Gurobi Survey data (.csv) Designers Design exploration ! Result
  49. 49. EXPLORATIVE OPTIMIZATION 100,000 design task variations 100,000 optimum designs A Usefulness Satisfaction Ease of use Profitability 24 functionalities Add_column_to_table Add_row_to_table Align_text Bullet_size Bullet_type Bullets Cell_color Cell_margins Change_encoding Change_number_size Change_start_number Character_spacing Choose_printer Close Close_all_windows Column_size Compare_documents Copy Create_notebook Create_table Cut Delete Delete_notebook Draw_freehand Draw_shape Exit Export Fill_color Find Font_color Font_size Full_screen Handwriting_recognition Indent Insert_blank_page Insert_date_and_time Insert_horizontal_line Insert_link Insert_picture Insert_video Line_color Line_number_color Line_number_continuation Line_number_font Line_number_size Line_numbers Line_spacing Line_style Line_wrap Link_to_calendar_entry Margins Mark_all_text Mark_text Merge_cells Move_document_to_other_view Move_note_to_notebook New Number_of_copies Number_style Numbering Open Open_containing_folder Orientation Page_break Page_size Paragraph_spacing Paste Print_current_page Print_custom_range Print_document Print_multiple_pages_per_sheet Print_selection Printer_settings Publish_online Put_on_homescreen Recently_opened Record_audio Redo Reload Remove_column_from_table Remove_row_from_table Rename Rename_notebook Replace Row_size Save Save_all_windows Search_printer Share Show_grid Show_ruler Sort_elements Spell_check Split_cell Split_view Synchronize_scrolling Table_border Text_alignment_in_table Text_background_color Text_columns Text_font Text_style Type_text Undo Voice_recognition Add_column_to_table Add_row_to_table Align_text Bullet_size Bullet_type Bullets Cell_color Cell_margins Change_encoding Change_number_size Change_start_number Character_spacing Choose_printer Close Close_all_windows Column_size Compare_documents Copy Create_notebook Create_table Cut Delete Delete_notebook Draw_freehand Draw_shape Exit Export Fill_colorFind Font_color Font_size Full_screen Handwriting_recognition Indent Insert_blank_page Insert_date_and_time Insert_horizontal_line Insert_link Insert_picture Insert_video Line_color Line_number_color Line_number_continuation Line_number_font Line_number_size Line_numbers Line_spacing Line_style Line_wrap Link_to_calendar_entry Margins Mark_all_text Mark_text Merge_cells Move_document_to_other_view Move_note_to_notebook New Number_of_copies Number_style Numbering Open Open_containing_folder Orientation Page_break Page_size Paragraph_spacing Paste Print_current_page Print_custom_range Print_document Print_multiple_pages_per_sheet Print_selection Printer_settings Publish_online Recently_opened Record_audio Redo Reload Remove_column_from_table Remove_row_from_table Rename Rename_notebook Replace Row_size Save Save_all_windows Search_printer Share Show_grid Show_ruler Sort_elements Spell_check Split_cell Split_view Synchronize_scrolling Table_border Text_alignment_in_table Text_background_color Text_columns Text_font Text_style Type_text Undo Voice_recognition File creation and organization Link and share Printing Table Structural elements Input Text style options Page layout Controls View F Usefulness Satisfaction Ease of use Profitability 82 functionalities Add_column_to_table Add_row_to_table Align_text Bullet_size Bullet_type Bullets Cell_color Cell_margins Change_encoding Change_number_size Change_start_number Character_spacing Choose_printer Close Close_all_windows Column_size Compare_documents Copy Create_notebook Create_table Cut Delete Delete_notebook Draw_freehand Draw_shape Exit Export Fill_color Find Font_color Font_size Full_screen Handwriting_recognition Indent Insert_blank_page Insert_date_and_time Insert_horizontal_line Insert_link Insert_picture Insert_video Line_color Line_number_color Line_number_continuation Line_number_font Line_number_size Line_numbers Line_spacing Line_style Line_wrap Link_to_calendar_entry Margins Mark_all_text Mark_text Merge_cells Move_document_to_other_view Move_note_to_notebook New Number_of_copies Number_style Numbering Open Open_containing_folder Orientation Page_break Page_size Paragraph_spacing Paste Print_current_page Print_custom_range Print_document Print_multiple_pages_per_sheet Print_selection Printer_settings Publish_online Put_on_homescreen Recently_opened Record_audio Redo Reload Remove_column_from_table Remove_row_from_table Rename Rename_notebook Replace Row_size Save Save_all_windows Search_printer Share Show_grid Show_ruler Sort_elements Spell_check Split_cell Split_view Synchronize_scrolling Table_border Text_alignment_in_table Text_background_color Text_columns Text_font Text_style Type_text Undo Voice_recognition Add_column_to_table Add_row_to_table Align_text Bullet_size Bullet_type Bullets Cell_color Cell_margins Change_encoding Change_number_size Change_start_number Character_spacing Choose_printer Close Close_all_windows Column_size Compare_documents Copy Create_notebook Create_table Cut Delete Delete_notebook Draw_freehand Draw_shape Exit Export Fill_colorFind Font_color Font_size Full_screen Handwriting_recognition Indent Insert_blank_page Insert_date_and_time Insert_horizontal_line Insert_link Insert_picture Insert_video Line_color Line_number_color Line_number_continuation Line_number_font Line_number_size Line_numbers Line_spacing Line_style Line_wrap Link_to_calendar_entry Margins Mark_all_text Mark_text Merge_cells Move_document_to_other_view Move_note_to_notebook New Number_of_copies Number_style Numbering Open Open_containing_folder Orientation Page_break Page_size Paragraph_spacing Paste Print_current_page Print_custom_range Print_document Print_multiple_pages_per_sheet Print_selection Printer_settings Publish_online Recently_opened Record_audio Redo Reload Remove_column_from_table Remove_row_from_table Rename Rename_notebook Replace Row_size Save Save_all_windows Search_printer Share Show_grid Show_ruler Sort_elements Spell_check Split_cell Split_view Synchronize_scrolling Table_border Text_alignment_in_table Text_background_color Text_columns Text_font Text_style Type_text Undo Voice_recognition File creation and organization Link and share Printing Table Structural elements Input Text style options Page layout Controls View … k Diverse designs R Usefulness Satisfaction Ease of use Profitability 106 functionalities Add_column_to_table Add_row_to_table Align_text Bullet_size Bullet_type Bullets Cell_color Cell_margins Change_encoding Change_number_size Change_start_number Character_spacing Choose_printer Close Close_all_windows Column_size Compare_documents Copy Create_notebook Create_table Cut Delete_element Delete_file Delete_notebook Draw_freehand Draw_shape Exit Export Fill_color Find Font_color Font_size Full_screen Handwriting_recognition Indent Insert_blank_page Insert_date_and_time Insert_horizontal_line Insert_link Insert_picture Insert_video Line_color Line_number_color Line_number_continuation Line_number_font Line_number_size Line_numbers Line_spacing Line_style Line_wrap Link_to_calendar_entry Margins Mark_all_text Mark_text Merge_cells Move_document_to_other_view Move_note_to_notebook New Number_of_copies Number_style Numbering Open Open_containing_folder Orientation Page_break Page_size Paragraph_spacing Paste Print_current_page Print_custom_range Print_document Print_multiple_pages_per_sheet Print_selection Printer_settings Publish_online Put_on_homescreen Recently_opened Record_audio Redo Reload Remove_column_from_table Remove_row_from_table Rename Rename_notebook Replace Row_size Save Save_all_windows Search_printer Share Show_grid Show_ruler Sort_elements Spell_check Split_cell Split_view Synchronize_scrolling Table_border Text_alignment_in_table Text_background_color Text_columns Text_font Text_style Type_text Undo Voice_recognition Zoom Add_column_to_table Add_row_to_table Align_text Bullet_size Bullet_type Bullets Cell_color Cell_margins Change_encoding Change_number_size Change_start_number Character_spacing Choose_printer Close Close_all_windows Column_size Compare_documents Copy Create_notebook Create_table Cut Delete_element Delete_file Delete_notebook Draw_freehand Draw_shape Exit Export Fill_colorFind Font_color Font_size Full_screen Handwriting_recognition Indent Insert_blank_page Insert_date_and_time Insert_horizontal_line Insert_link Insert_picture Insert_video Line_color Line_number_color Line_number_continuation Line_number_font Line_number_size Line_numbers Line_spacing Line_style Line_wrap Link_to_calendar_entry Margins Mark_all_text Mark_text Merge_cells Move_document_to_other_view Move_note_to_notebook New Number_of_copies Number_style Numbering Open Open_containing_folder Orientation Page_break Page_size Paragraph_spacing Paste Print_current_page Print_custom_range Print_document Print_multiple_pages_per_sheet Print_selection Printer_settings Publish_online Recently_opened Record_audio Redo Reload Remove_column_from_table Remove_row_from_table Rename Rename_notebook Replace Row_size Save Save_all_windows Search_printer Share Show_grid Show_ruler Sort_elements Spell_check Split_cell Split_view Synchronize_scrolling Table_border Text_alignment_in_table Text_background_color Text_columns Text_font Text_style Type_text Undo Voice_recognition Zoom File creation and organization Link and share Printing Table Structural elements Input Text style options Page layout Controls View Robust design
  50. 50. NOTIFICATION SCHEDULING • Assign incoming messages to a computer to one of the statuses: • Show immediately with a pop-up • Push to list of notifications • Don’t show
  51. 51. JOB-SHOP SCHEDULING TASK n jobs m machines Find a schedule that
 minimizes total processing
 time
  52. 52. CHARACTERIZATION • n jobs, each consisting of a chain of operations • m machines, each handles at most one operation at a time • Each operation must be processed during an uninterrupted period of a given length on a given machine • The objective is to find a schedule (an allocation of the jobs to time intervals to machines) with minimum duration • In static JSS, n jobs arrive at an idle shop • In dynamic JSS, jobs arrive intermittently
  53. 53. JOB-SHOP SCHEDULING
  54. 54. TASK: A NOTIFICATION BROKER A set of incoming messages is given, decide to which modality to channel them and when to present them. ! Tip:What is being minimized/maximed here?
  55. 55. SUMMARY
  56. 56. SUMMARY • Design tasks can described in terms of objectives and constraints • Assignment tasks are common in user interface design • Many optimization tasks in operations research have applications in HCI
  57. 57. SICSA SUMMER SCHOOL ON COMPUTATIONAL INTERACTION DAY 2: OPTIMIZATION EXAMPLE: MID-AIR GESTURE OPTIMIZATION 
 [SRIDHAR ET AL. PROC CHI’15]

×