A Preliminary Field Study of Game Programming on Mobile Devices

479 views
366 views

Published on

Eric Anderson, Sihan Li, and Tao Xie. A Preliminary Field Study of Game Programming on Mobile Devices. Presented in Workshop on Programming for Mobile and Touch (PROMOTO 2013), Indianapolis, IN, October 2013.

Published in: Technology
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
479
On SlideShare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
4
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

A Preliminary Field Study of Game Programming on Mobile Devices

  1. 1. A Preliminary Field Study of Game Programming on Mobile Devices Eric Anderson Sihan Li Tao Xie North Carolina State University University of Illinois at Urbana-Champaign http://arxiv.org/abs/1310.3308 1
  2. 2. Mining App Stores: Mobile App Markets Apple App Store Google Play Microsoft Windows (Phone) Apps Microsoft Research TouchDevelop
  3. 3. Mining App Stores: Program Analysis + Text Analytics o Previous approaches look at permissions  code (runtime behaviors) o What does the users expect? App Description App Code Pandita et al. WHYPER: Towards Automating Risk Assessment of Mobile Applications. USENIX Security 2013 http://web.engr.illinois.edu/~taoxie/p ublications/usenixsec13-whyper.pdf App Permissions 3
  4. 4. Mining App Stores: Program Analysis + Text Analytics o Previous approaches look at permissions  code (runtime behaviors) o What does the users expect? App Description App Code Pandita et al. WHYPER: Towards Automating Risk Assessment of Mobile Applications. USENIX Security 2013 http://web.engr.illinois.edu/~taoxie/p ublications/usenixsec13-whyper.pdf App Permissions 3
  5. 5. Mining App Stores: Program Analysis + Text Analytics o Previous approaches look at permissions  code (runtime behaviors) o What does the users expect? App Description App Code Pandita et al. WHYPER: Towards Automating Risk Assessment of Mobile Applications. USENIX Security 2013 http://web.engr.illinois.edu/~taoxie/p ublications/usenixsec13-whyper.pdf App Permissions 3
  6. 6. Mining App Stores: Program Analysis + Text Analytics o Previous approaches look at permissions  code (runtime behaviors) o What does the users expect? App Description App Code Pandita et al. WHYPER: Towards Automating Risk Assessment of Mobile Applications. USENIX Security 2013 http://web.engr.illinois.edu/~taoxie/p ublications/usenixsec13-whyper.pdf App Permissions 3
  7. 7. Mining App Stores: Program Analysis + Text Analytics o Previous approaches look at permissions  code (runtime behaviors) o What does the users expect? App Description App Code Pandita et al. WHYPER: Towards Perhaps it's my ignorance, but would Automating Risk Assessment of anything related to storage, system Mobile Applications. USENIX Security tools, your location, camera, 2013 development tools, phone calls, and http://web.engr.illinois.edu/~taoxie/p ublications/usenixsec13-whyper.pdf network communication be ridiculously more accessible to the App smartphone capabilities than what a Permissions 3 flashlight app would require?
  8. 8. Mining App Stores: Open Source Mobile App Repo https://f-droid.org/ http://research.microsoft.com/en-us/projects/touchdevelop/
  9. 9. Rapid Increase of TouchDevelop Popularity vs. 17322 Feb this year 5
  10. 10. Mining App Stores: Educational/Learning Analytics TouchDevelop Team@MSR TouchDevelop Users SE/HCI/Education … Researchers 6
  11. 11. Mining App Stores: Educational/Learning Analytics TouchDevelop Team@MSR TouchDevelop Users SE/HCI/Education … Researchers 6
  12. 12. Mining App Stores: Educational/Learning Analytics TouchDevelop Team@MSR TouchDevelop Users Data-Driven Decision Making (not just gut feeling) SE/HCI/Education … Researchers 6
  13. 13. Mining App Stores: Educational/Learning Analytics TouchDevelop Team@MSR TouchDevelop Users Data-Driven Decision Making (not just gut feeling) SE/HCI/Education … Researchers 6
  14. 14. Our Previous Study on TouchDevelop Scripts and Users • Characteristic and longitudinal study – 17,322 scripts and 4,275 users (July 2011-Feb 2013) • Scripts – Structural features – Code reuse – Change of features over time • Users – Expertise – Publishing and learning activities over time Sihan Li, Tao Xie, and Nikolai Tillmann. A Comprehensive Field Study of End-User Programming on Mobile Devices. VL/HCC 2013. 7 http://web.engr.illinois.edu/~taoxie/publications/vlhcc13-study.pdf
  15. 15. Previous Characteristic Study What do TouchDevelop scripts look like? What is the code-reuse ratio of TouchDevelop scripts? What kinds of TouchDevelop users are there? How are TouchDevelop scripts changing over time? What is the users’ progress of developing TouchDevelop scripts? 8
  16. 16. What do TouchDevelop scripts look like? Structural Features: Findings EMC: # external method call sites 9
  17. 17. What do TouchDevelop scripts look like? Structural Features: Findings Small in size 73% less than 100 LOC, avg. 133 LOC EMC: # external method call sites 9
  18. 18. What do TouchDevelop scripts look like? Structural Features: Findings Small in size 73% less than 100 LOC, avg. 133 LOC Heavily use of external code 106 EMC out of 114 MC per script Every two lines contain an EMC EMC: # external method call sites 9
  19. 19. What do TouchDevelop scripts look like? Structural Features: Findings Small in size 73% less than 100 LOC, avg. 133 LOC Heavily use of external code 106 EMC out of 114 MC per script Every two lines contain an EMC Implications: Heavily rely on external methods to achieve functionalities. Enriching the built-in APIs and library scripts. EMC: # external method call sites 9
  20. 20. Types of Code Reuse • • • • Inherit code from a parent script Use the existing library scripts Use TouchDevelop built-in APIs Copy and paste 10
  21. 21. Types of Code Reuse • • • • Inherit code from a parent script Use the existing library scripts Use TouchDevelop built-in APIs Copy and paste 10
  22. 22. What is the code-reuse ratio of TouchDevelop scripts? Code Reuse: Findings 11
  23. 23. What is the code-reuse ratio of TouchDevelop scripts? Code Reuse: Findings High code-reuse ratio 58% scripts have a parent 11
  24. 24. What is the code-reuse ratio of TouchDevelop scripts? Code Reuse: Findings High code-reuse ratio 58% scripts have a parent Reuse their own code 74% parent-child pairs are written by the same user 11
  25. 25. What is the code-reuse ratio of TouchDevelop scripts? Code Reuse: Findings High code-reuse ratio 58% scripts have a parent Reuse their own code 74% parent-child pairs are written by the same user Small modifications Quite a lot less than 10 LOC 11
  26. 26. What is the code-reuse ratio of TouchDevelop scripts? Code Reuse: Findings High code-reuse ratio 58% scripts have a parent Reuse their own code 74% parent-child pairs are written by the same user Small modifications Quite a lot less than 10 LOC Implications: Users tend to reuse code. Lack of code reuse across different users. Improve tool support for code reuse. 11
  27. 27. Expertise of Users: Metrics • Activity of script-publishing – # scripts published • Quality of scripts and comments – # positive reviews received • Grasp of the TouchDevelop language – # TouchDevelop language features used (e.g., programming concepts, built-in APIs) 12
  28. 28. Clusters of Users 74.2% 21.5% 4.2% 13
  29. 29. Clusters of Users 74.2% 21.5% 4.2% 13
  30. 30. What kinds of TouchDevelop users are there? Expertise of Users: Findings 14
  31. 31. What kinds of TouchDevelop users are there? Expertise of Users: Findings Most users are novices 74% novices, 22% ordinary users, 4% experts Big differences between users’ expertise 14
  32. 32. What kinds of TouchDevelop users are there? Expertise of Users: Findings Most users are novices 74% novices, 22% ordinary users, 4% experts Big differences between users’ expertise Implications: Give novices high priority when designing tools and services. Provide customized services (e.g., UI) to different kinds of users. 14
  33. 33. How are TouchDevelop scripts changing over time? Changes of Scripts: Findings 15
  34. 34. How are TouchDevelop scripts changing over time? Changes of Scripts: Findings Important increasing trends # scripts # library , # large scripts , , code-reuse ratio . 15
  35. 35. How are TouchDevelop scripts changing over time? Changes of Scripts: Findings Important increasing trends # scripts # library , # large scripts , , code-reuse ratio . Implications: Enriching TouchDevelop language and Improving the management of libraries to better accommodate these trends. 15
  36. 36. What is users’ progress of developing TouchDevelop scripts? Evolution of Users: Findings 16
  37. 37. What is users’ progress of developing TouchDevelop scripts? Evolution of Users: Findings Three patterns of publishing scripts 22.1% users active initially, less active later 9.6% users not active initially, more active later 68.3% users publish one or two scripts, and then leave 16
  38. 38. What is users’ progress of developing TouchDevelop scripts? Evolution of Users: Findings Three patterns of publishing scripts 22.1% users active initially, less active later 9.6% users not active initially, more active later 68.3% users publish one or two scripts, and then leave Implications: Useful to know why users leave or become less active and why they become more active. Design better strategies to retain users. 16
  39. 39. Evolution of Users: Findings 17
  40. 40. Evolution of Users: Findings Three patterns of learning language features 16.3% users learn most initially, and only a few later 12.4% users learn some initially, and quite a lot at certain point 71.3% users learn a few initially, and stop learning 17
  41. 41. Evolution of Users: Findings Three patterns of learning language features 16.3% users learn most initially, and only a few later 12.4% users learn some initially, and quite a lot at certain point 71.3% users learn a few initially, and stop learning Implications: Users have different learning behaviors. Provide an adaptive tutoring system based on the history of users’ language-feature usage. 17
  42. 42. Overview of This Study • Motivation – As mobile gaming grows, it is important to analyze game applications and how it compares to other applications. • Characteristic study – 31,699 scripts, including 539 game scripts • Goal – To discover how TouchDevelop games compare to non-games – To discover what makes some games more popular 18
  43. 43. Characteristic study on Games • RQ1: What do TouchDevelop game scripts look like when compared to non-game scripts? • RQ2: How do games compare to non-games in terms of popularity? • RQ3: What features do more popular games have that less popular games lack? 19
  44. 44. Research Question 1 • RQ1: What do TouchDevelop game scripts look like when compared to non-game scripts? 20
  45. 45. Structural Comparison Mean #LOC Median #LOC Game 369 272 Non-Game 210 50 • Finding: Game scripts are longer than nongame scripts. • Implication: Expanding IDE support could enhance the creation of large scripts. 21
  46. 46. Structural Comparison cont. Mean # action Game Non-Game Mean # event 11.3 3.2 6.1 1.6 • Finding: Game scripts typically have more actions and events than non-game scripts • Implication: IDE expansion could be highly beneficial. 22
  47. 47. Structural Comparison cont. Mean # MC Mean MC density Game 347 0.94 Non-Game 165 0.79 • Finding: Game scripts invoke more methods than non-game scripts. • Implication: Games include more features than non-games. Emphasis on games in marketing could be beneficial. 23
  48. 48. Research Question 2 • RQ2: How do games compare to non-games in terms of popularity? 24
  49. 49. Comparison of Popularity Mean # run Game Non-Game Mean # install Mean # likes 216 35 1.80 23 3 0.32 • Finding: Game scripts are more popular than nongame scripts • Implication: TouchDevelop should market games as important types of scripts. 25
  50. 50. Research Question 3 • RQ3: What features do more popular games have that less popular games lack? 26
  51. 51. Game Popularity Comparison Games Mean # LOC Median # LOC Popular 462 384 Unpopular 363 255 • Metric: Top 25% of frequently run games as popular • Finding: Popular games are longer than unpopular games • Implication: Expansion of the IDE could encourage larger scripts 27
  52. 52. Game Popularity Comparison cont. Games Mean # action Mean # event Popular 14.1 3.6 Unpopular 11.1 3.2 • Finding: Popular and unpopular game scripts have similar numbers of actions and events • Implication: Large numbers of actions and events may not contribute to popularity 28
  53. 53. Comparison of Actions and Events 16 # action and events 14 12 10 8 6 4 2 0 Mean Actions Non-Games Mean Events Games Popular Games Unpopular Games 29
  54. 54. Comparison of Mean and Median Length 500 # LOC 450 400 350 300 250 200 150 100 50 0 Mean Length Non-Games Median Length Games Popular Games Unpopular Games 30
  55. 55. Conclusion • Previous study [VL/HCC 13] reports a number of valuable findings • This study focuses on game vs. non-games popular vs. unpopular games – Games are typically longer and more feature-rich than non-games – Popular games are typically longer but not necessarily more feature rich than unpopular games 31
  56. 56. Thank you! ACKNOWLEDGMENTS. This work is supported in part by NSF grants CCF-0845272, CCF0915400, CNS-0958235, CNS-1160603, CCF-1349666, CNS-1318419, a Microsoft Research award, and NSF of China No. 61228203. We thank Nikolai Tillmann for his help and feedback in conducting the study described in this paper. 32
  57. 57. Conclusion • Previous study [VL/HCC 13] reports a number of valuable findings • This study focuses on game vs. non-games popular vs. unpopular games – Games are typically longer and more feature-rich than non-games – Popular games are typically longer but not necessarily more feature rich than unpopular games 33

×