Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

ICSE, SEIP 2018 - Echoes From Space: Grouping Commands with Large-Scale Telemetry Data

40 views

Published on

Background: As evolving desktop applications continuously accrue new features and grow more complex with denser user interfaces and deeply-nested commands, it becomes inefficient to use simple heuristic processes for grouping GUI commands in multi-level menus. Existing search-based software engineering studies on user performance prediction and command grouping optimization lack evidence-based answers on choosing a systematic grouping method.

Research Questions: We investigate the scope of command grouping optimization methods to reduce a user’s average task completion time and improve their relative performance, as well as the benefit of using detailed interaction logs compared to sampling.

Method: We introduce seven grouping methods and compare their performance based on extensive telemetry data, collected from program runs of a CAD application.

Results: We find that methods using global frequencies, user-specific frequencies, deterministic and stochastic optimization, and clustering perform the best.

Conclusions: We reduce the average user task completion time by more than 17%, by running a Knapsack Problem algorithm on clustered users, training only on a small sample of the available data. We show that with most methods using just a 1% sample of the data is enough to obtain nearly the same results as those obtained from all the data. Additionally, we map the methods to specific problems and applications where they would perform better. Overall, we provide a guide on how practitioners can use search-based software engineering techniques when grouping commands in menus and interfaces, to maximize users’ task execution efficiency.

Published in: Engineering
  • Be the first to comment

  • Be the first to like this

ICSE, SEIP 2018 - Echoes From Space: Grouping Commands with Large-Scale Telemetry Data

  1. 1. Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Command Grouping Model & Data Optimization Algorithms Results Echoes From Space: Grouping Commands with Large-Scale Telemetry Data Alexander Lattas Diomidis Spinellis
  2. 2. Command Grouping Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Command Grouping Model & Data Optimization Algorithms Results
  3. 3. A Galaxy of Commands Command Grouping Problem Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Command Grouping Model & Data Optimization Algorithms Results
  4. 4. Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Command Grouping Model & Data Optimization Algorithms Results A Galaxy of Commands Command Grouping Problem
  5. 5. Command Grouping Model & Data Optimization Algorithms Results Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Command Grouping Model & Data Optimization Algorithms Results A Galaxy of Commands Command Grouping Problem
  6. 6. Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Command Grouping Model & Data Optimization Algorithms Results A Galaxy of Commands Command Grouping Problem
  7. 7. Command Grouping Model & Data Optimization Algorithms Results Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Command Grouping Model & Data Optimization Algorithms Results A Galaxy of Commands Command Grouping Problem
  8. 8. Command Grouping Model & Data Optimization Algorithms Results Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Command Grouping Model & Data Optimization Algorithms Results A Galaxy of Commands Command Grouping Problem
  9. 9. Systematic generic methods for grouping commands into multi-level menus and toolbars User usability and speed Improved GUI design process Command Grouping Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Command Grouping Model & Data Optimization Algorithms Results A Galaxy of Commands Command Grouping Problem
  10. 10. Model & Data Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Command Grouping Model & Data Optimization Algorithms Results
  11. 11. Command Grouping Model & Data Optimization Algorithms Results 𝑇𝑐𝑜𝑚𝑚𝑎𝑛𝑑 = 𝑇 𝑀𝑒𝑛𝑡𝑎𝑙 + 𝑇 𝑀𝑜𝑣𝑒 + 𝑇𝑃𝑜𝑖𝑛𝑡 + 𝑇 𝐾𝑒𝑦 + 𝑇𝑆𝑦𝑠𝑡𝑒𝑚 Keystroke-Level Model (KLM) Card et al., 1980 Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Navigation Overhead Telemetry Data Experiment Data Command Grouping Model & Data Optimization Algorithms Results
  12. 12. Navigation Overhead Telemetry Data 𝑇𝑐𝑜𝑚𝑚𝑎𝑛𝑑 = 𝑻 𝑴𝒆𝒏𝒕𝒂𝒍 + 𝑇 𝑀𝑜𝑣𝑒 + 𝑇𝑃𝑜𝑖𝑛𝑡 + 𝑇 𝐾𝑒𝑦 + 𝑇𝑆𝑦𝑠𝑡𝑒𝑚 Keystroke-Level Model (KLM) Card et al., 1980 Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Navigation Overhead Telemetry Data Experiment Data Command Grouping Model & Data Optimization Algorithms Results
  13. 13. Command Grouping Model & Data Optimization Algorithms Results 𝑇𝑐𝑜𝑚𝑚𝑎𝑛𝑑 = 𝑇 𝑀𝑒𝑛𝑡𝑎𝑙 + 𝑻 𝑴𝒐𝒗𝒆 + 𝑇𝑃𝑜𝑖𝑛𝑡 + 𝑇 𝐾𝑒𝑦 + 𝑇𝑆𝑦𝑠𝑡𝑒𝑚 Keystroke-Level Model (KLM) Card et al., 1980 Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Navigation Overhead Telemetry Data Experiment Data Command Grouping Model & Data Optimization Algorithms Results
  14. 14. Navigation Overhead Telemetry Data Command Grouping Model & Data Optimization Algorithms Results 𝑇𝑐𝑜𝑚𝑚𝑎𝑛𝑑 = 𝑇 𝑀𝑒𝑛𝑡𝑎𝑙 + 𝑻 𝑴𝒐𝒗𝒆 + 𝑇𝑃𝑜𝑖𝑛𝑡 + 𝑇 𝐾𝑒𝑦 + 𝑇𝑆𝑦𝑠𝑡𝑒𝑚 Keystroke-Level Model (KLM) Card et al., 1980 Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Navigation Overhead Telemetry Data Experiment Data Command Grouping Model & Data Optimization Algorithms Results
  15. 15. Navigation Overhead Telemetry Data Command Grouping Model & Data Optimization Algorithms Results 𝑇𝑐𝑜𝑚𝑚𝑎𝑛𝑑 = 𝑇 𝑀𝑒𝑛𝑡𝑎𝑙 + 𝑻 𝑴𝒐𝒗𝒆 + 𝑇𝑃𝑜𝑖𝑛𝑡 + 𝑇 𝐾𝑒𝑦 + 𝑇𝑆𝑦𝑠𝑡𝑒𝑚 Keystroke-Level Model (KLM) Card et al., 1980 Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Navigation Overhead Telemetry Data Command Grouping Model & Data Optimization Algorithms Results
  16. 16. Navigation Overhead Telemetry Data Command Grouping Model & Data Optimization Algorithms Results 𝑇𝑐𝑜𝑚𝑚𝑎𝑛𝑑 = 𝑇 𝑀𝑒𝑛𝑡𝑎𝑙 + 𝑇 𝑀𝑜𝑣𝑒 + 𝑻 𝑷𝒐𝒊𝒏𝒕 + 𝑇 𝐾𝑒𝑦 + 𝑇𝑆𝑦𝑠𝑡𝑒𝑚 Keystroke-Level Model (KLM) Card et al., 1980 Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Navigation Overhead Telemetry Data Experiment Data Command Grouping Model & Data Optimization Algorithms Results
  17. 17. Navigation Overhead Telemetry Data Command Grouping Model & Data Optimization Algorithms Results 𝑇𝑐𝑜𝑚𝑚𝑎𝑛𝑑 = 𝑇 𝑀𝑒𝑛𝑡𝑎𝑙 + 𝑇 𝑀𝑜𝑣𝑒 + 𝑻 𝑷𝒐𝒊𝒏𝒕 + 𝑇 𝐾𝑒𝑦 + 𝑇𝑆𝑦𝑠𝑡𝑒𝑚 Keystroke-Level Model (KLM) Card et al., 1980 Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Navigation Overhead Telemetry Data Experiment Data Command Grouping Model & Data Optimization Algorithms Results
  18. 18. Command Grouping Model & Data Optimization Algorithms Results 𝑇𝑐𝑜𝑚𝑚𝑎𝑛𝑑 = 𝑇 𝑀𝑒𝑛𝑡𝑎𝑙 + 𝑇 𝑀𝑜𝑣𝑒 + 𝑻 𝑷𝒐𝒊𝒏𝒕 + 𝑇 𝐾𝑒𝑦 + 𝑇𝑆𝑦𝑠𝑡𝑒𝑚 Keystroke-Level Model (KLM) Card et al., 1980 Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Navigation Overhead Telemetry Data Experiment Data Command Grouping Model & Data Optimization Algorithms Results
  19. 19. Navigation Overhead Telemetry Data Command Grouping Model & Data Optimization Algorithms Results 𝑇𝑐𝑜𝑚𝑚𝑎𝑛𝑑 = 𝑇 𝑀𝑒𝑛𝑡𝑎𝑙 + 𝑇 𝑀𝑜𝑣𝑒 + 𝑇𝑃𝑜𝑖𝑛𝑡 + 𝑻 𝑲𝒆𝒚 + 𝑇𝑆𝑦𝑠𝑡𝑒𝑚 Keystroke-Level Model (KLM) Card et al., 1980 01/06/2018 Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and BusinessAlexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Navigation Overhead Telemetry Data Experiment Data Command Grouping Model & Data Optimization Algorithms Results
  20. 20. Navigation Overhead Telemetry Data Command Grouping Model & Data Optimization Algorithms Results 𝑇𝑐𝑜𝑚𝑚𝑎𝑛𝑑 = 𝑇 𝑀𝑒𝑛𝑡𝑎𝑙 + 𝑇 𝑀𝑜𝑣𝑒 + 𝑇𝑃𝑜𝑖𝑛𝑡 + 𝑇 𝐾𝑒𝑦 + 𝑻 𝑺𝒚𝒔𝒕𝒆𝒎 Keystroke-Level Model (KLM) Card et al., 1980 Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Navigation Overhead Telemetry Data Experiment Data Command Grouping Model & Data Optimization Algorithms Results
  21. 21. Command Grouping Model & Data Optimization Algorithms Results 𝑇𝑆𝑎𝑚𝑒 Navigation Overhead Telemetry Data Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Navigation Overhead Telemetry Data Experiment Data Command Grouping Model & Data Optimization Algorithms Results
  22. 22. Command Grouping Model & Data Optimization Algorithms Results 𝑇𝑆𝑎𝑚𝑒 Navigation Overhead Telemetry Data Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Navigation Overhead Telemetry Data Experiment Data Command Grouping Model & Data Optimization Algorithms Results
  23. 23. Command Grouping Model & Data Optimization Algorithms Results 𝑇𝑑𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑡 Navigation Overhead Telemetry Data Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Navigation Overhead Telemetry Data Experiment Data Command Grouping Model & Data Optimization Algorithms Results
  24. 24. Command Grouping Model & Data Optimization Algorithms Results 𝑇𝑑𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑡 Navigation Overhead Telemetry Data Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Navigation Overhead Telemetry Data Experiment Data Command Grouping Model & Data Optimization Algorithms Results
  25. 25. Command Grouping Model & Data Optimization Algorithms Results 𝑇𝑑𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑡 Navigation Overhead Telemetry Data Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Navigation Overhead Telemetry Data Experiment Data Command Grouping Model & Data Optimization Algorithms Results
  26. 26. Command Grouping Model & Data Optimization Algorithms Results 𝑇𝑑𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑡 Navigation Overhead Telemetry Data Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Navigation Overhead Telemetry Data Experiment Data Command Grouping Model & Data Optimization Algorithms Results
  27. 27. Command Grouping Model & Data Optimization Algorithms Results Eliminate 𝑇𝑆𝑦𝑠𝑡𝑒𝑚 Combine rest to: 𝑻 𝑺𝒂𝒎𝒆 : previous command in same entity 𝑻 𝑫𝒊𝒇𝒇𝒆𝒓𝒆𝒏𝒕 : previous command in different entity Count commands: 𝑵 𝑺𝒂𝒎𝒆 : previous command in same entity 𝑵 𝑫𝒊𝒇𝒇𝒆𝒓𝒆𝒏𝒕 : previous command in different entity 𝑵 𝑻𝒐𝒐𝒍𝒃𝒂𝒓 : command always available in a toolbar 𝑻 𝑻𝒐𝒕𝒂𝒍 = 𝑇𝑆𝑎𝑚𝑒 × 𝑁𝑆𝑎𝑚𝑒 + 𝑁 𝑇𝑜𝑜𝑙𝑏𝑎𝑟 + 𝑇 𝐷𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑡 × 𝑁 𝐷𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑡 Our Model Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Command Grouping Model & Data Optimization Algorithms Results Navigation Overhead Telemetry Data Experiment Data
  28. 28. Command Grouping Model & Data Optimization Algorithms Results Navigation Overhead Telemetry Data Experiment Data 32 M 200 k 2000 Commands Sessions Users Database Scenarios Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Navigation Overhead Telemetry Data Experiment Data Command Grouping Model & Data Optimization Algorithms Results
  29. 29. Command Grouping Model & Data Optimization Algorithms Results Navigation Overhead Telemetry Data Experiment Data Commands Sessions Users 𝑇𝑑𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑡 𝑇𝑆𝑎𝑚𝑒 Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Navigation Overhead Telemetry Data Experiment Data Command Grouping Model & Data Optimization Algorithms Results
  30. 30. Command Grouping Model & Data Optimization Algorithms Results Optimization Algorithms Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Command Grouping Model & Data Optimization Algorithms Results
  31. 31. Current All Group Naive Personalized OPT(Genetic) OPT(Knapsack) Cluster 𝑇 𝐷𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑡𝑇𝑆𝑎𝑚𝑒 𝑁𝑆𝑎𝑚𝑒 𝑁 𝑇𝑜𝑜𝑙𝑏𝑎𝑟 𝑁 𝐷𝑖𝑓𝑓𝑒𝑟𝑒𝑛𝑡 Measurement with the GUI at the current setup. CURRENT Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Command Grouping Model & Data Optimization Algorithms Results
  32. 32. 18% All commands available directly on the screen. ALL Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Current All Group Naive Personalized OPT(Genetic) OPT(Knapsack) Cluster Command Grouping Model & Data Optimization Algorithms Results
  33. 33. Group together commands between entities. GROUP 0.5% Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Current All Group Naive Personalized OPT(Genetic) OPT(Knapsack) Cluster Command Grouping Model & Data Optimization Algorithms Results
  34. 34. Current All Group Naive Personalized OPT(Knapsack) OPT(Genetic) Cluster 𝑁 most frequent commands always available on a toolbar. NAIVE 13% Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Current All Group Naive Personalized OPT(Genetic) OPT(Knapsack) Cluster Command Grouping Model & Data Optimization Algorithms Results
  35. 35. 𝑁 most frequent commands always available on a toolbar. For each user, from a small batch. MRU-B 10% Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Command Grouping Model & Data Optimization Algorithms Results Current All Group Naive Personalized OPT(Genetic) OPT(Knapsack) Cluster
  36. 36. 𝑁 most frequent commands always available on a toolbar. For each user, online. MRU-O 13% Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Current All Group Naive Personalized OPT(Genetic) OPT(Knapsack) Cluster Command Grouping Model & Data Optimization Algorithms Results
  37. 37. 𝑁 best commands always available on a toolbar, using a stochastic Genetic algorithm. OPT(GA) Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Current All Group Naive Personalized OPT(Genetic) OPT(Knapsack) Cluster Command Grouping Model & Data Optimization Algorithms Results
  38. 38. Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Command Grouping Model & Data Optimization Algorithms Results 𝑁 best commands always available on a toolbar, using a stochastic Genetic algorithm. OPT(GA) Current All Group Naive Personalized OPT(Genetic) OPT(Knapsack) Cluster
  39. 39. Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Command Grouping Model & Data Optimization Algorithms Results 𝑁 best commands always available on a toolbar, using a stochastic Genetic algorithm. OPT(GA) Current All Group Naive Personalized OPT(Genetic) OPT(Knapsack) Cluster
  40. 40. Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Command Grouping Model & Data Optimization Algorithms Results 𝑁 best commands always available on a toolbar, using a stochastic Genetic algorithm. OPT(GA) Current All Group Naive Personalized OPT(Genetic) OPT(Knapsack) Cluster
  41. 41. 17.40% Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Command Grouping Model & Data Optimization Algorithms Results 𝑁 best commands always available on a toolbar, using a stochastic Genetic algorithm. OPT(GA) Current All Group Naive Personalized OPT(Genetic) OPT(Knapsack) Cluster
  42. 42. 𝑁 best commands always available on a toolbar, using a heuristic Knapsack algorithm. OPT(KS) N Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Current All Group Naive Personalized OPT(Genetic) OPT(Knapsack) Cluster Command Grouping Model & Data Optimization Algorithms Results
  43. 43. 𝑁 best commands always available on a toolbar, using a heuristic Knapsack algorithm. OPT(KS) 17.43% N Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Command Grouping Model & Data Optimization Algorithms Results Current All Group Naive Personalized OPT(Genetic) OPT(Knapsack) Cluster
  44. 44. Make clusters (𝑘𝑚𝑒𝑎𝑛𝑠) of users Run OPT(KS) for each cluster. Show best 𝑁 commands to each. CLUSTER Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Current All Group Naive Personalized OPT(Genetic) OPT(Knapsack) Cluster Command Grouping Model & Data Optimization Algorithms Results
  45. 45. Make clusters (𝑘𝑚𝑒𝑎𝑛𝑠) of users Run OPT(KS) for each cluster. Show best 𝑁 commands to each. CLUSTER Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Current All Group Naive Personalized OPT(Genetic) OPT(Knapsack) Cluster Command Grouping Model & Data Optimization Algorithms Results
  46. 46. Make clusters (𝑘𝑚𝑒𝑎𝑛𝑠) of users Run OPT(KS) for each cluster. Show best 𝑁 commands to each. CLUSTER 17.43% Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Current All Group Naive Personalized OPT(Genetic) OPT(Knapsack) Cluster Command Grouping Model & Data Optimization Algorithms Results
  47. 47. Results Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Command Grouping Model & Data Optimization Algorithms Results
  48. 48. good optimal bad Overview Toolbar Size Sample Size Conclusions Method Optimization Training Time (s) ALL 18.5% - GROUP 0.6% - NAIVE 13.2% 7.8 MRU-B 10.4% 74.2 MRU-O 13.5% - * OPT(GA) 17.40% 19,749 OPT(KS) 17.43% 1,946 CLUSTER 17.43% 2035 * Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Command Grouping Model & Data Optimization Algorithms Results * user data
  49. 49. novices experts Executiontime(minutesperscenario) Toolbar Size Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Overview Toolbar Size Sample Size Conclusions Command Grouping Model & Data Optimization Algorithms Results
  50. 50. Sample size (No. of sessions) Executiontime(minutesperscenario) average execution time CLUSTER optimal CURRENT Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Overview Toolbar Size Sample Size Conclusions Command Grouping Model & Data Optimization Algorithms Results
  51. 51. Command Grouping GUI optimization systemization Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Overview Toolbar Size Sample Size Conclusions Command Grouping Model & Data Optimization Algorithms Results
  52. 52. Command Grouping CLUSTER GUI optimization systemization Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Overview Toolbar Size Sample Size Conclusions Command Grouping Model & Data Optimization Algorithms Results
  53. 53. Command Grouping CLUSTER GUI optimization systemization 15 Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Overview Toolbar Size Sample Size Conclusions Command Grouping Model & Data Optimization Algorithms Results
  54. 54. Command Grouping CLUSTER GUI optimization systemization × 500 15 Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Overview Toolbar Size Sample Size Conclusions Command Grouping Model & Data Optimization Algorithms Results
  55. 55. Command Grouping Model & Data Optimization Algorithms Results Thank you! Alexander Lattas Diomidis Spinellis alexandros.lattas17@imperial.ac.uk dds@aueb.gr efs.lattas.eu @CoolSWEng@AlexanderLattas Alexander Lattas, Imperial College London | Diomidis Spinellis, Athens University of Economics and Business01/06/2018 Command Grouping Model & Data Optimization Algorithms Results

×