MSR populations talk v2.key


Published on

Overview of population approach, with more tech detail of core analytics, done for MSR Cambridge

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

MSR populations talk v2.key

  1. 1. FanBook 60 5 5 30 ZippyA Population Approach to System Design Matthew Chalmers
  2. 2. FanBook 60 5 5 30 ZippyA Population Approach to Ubicomp System Design Matthew Chalmers
  3. 3. Plus ça change... “Users’ control of [ubicomp] systems should be maximised and, given the heterogeneity of likely patterns of use, such systems must also be tailorable by the user. It is not possible to predict the social conventions governing the modes of use of ubiquitous computing that will come into being, so empirical studies of new applications [are] essential.”Perceptions and Expectations of Ubiquitous Computing:Experiments with BirdDog, a Prototype Person Locator,O’Shea et al., Proc. IEE ITaP, 1991.
  4. 4. Seamful Design: TreasureLimits, variation and dynamics in wifi infrastructure made into resources for designers and users
  5. 5. Seamful Design: Feeding YoshiLimits, variation and dynamics in wifi infrastructure made into resources for designers and users
  6. 6. Seamful Design: Castles Limits, variation and dynamics in software moduleinfrastructure made into resources for designers and users
  7. 7. Analysis Tools: Replayer
  8. 8. Analysis Tools: Replayer
  9. 9. Analysis Tools: Replayer
  10. 10. Analysis Tools: Replayer
  11. 11. Shifting to iPhones
  12. 12. Shifting to iPhones
  13. 13. Shifting to iPhones
  14. 14. Yoshi (2009)
  15. 15. 28GB of quantitative log data representing 18,000hours of use by 182,000 users, and qualitative data from 8,000 users
  16. 16. Analysis tools: SGVis
  17. 17. SGVis: Categorising users
  18. 18. SGVis: Categorising users
  19. 19. SGVis: Categorising usersCalculate centroids of each identified cluster4 categories of users: top players, commuters,static players, beginners
  20. 20. Seamful Design: FlexBook Mobile social networking app Dynamic integration of software modules at runtime Shared via our “module store” server, or ad hoc peer-to-peer Reviewing and user feedback Implicit logging of use Explicit comments and ratings shown in app and Facebook
  21. 21. Post Factory
  22. 22. A software engineering revolutionConventional models of the software development processall claim that “at some point, a product is delivered to auser community, at which point, for that revision of thesoftware, the design process is over.”Paul Dourish
  23. 23. A software engineering revolution“When we look at an interactive system as an evolvingartifact in use, it follows that the process of design doesnot end with the delivery of the system to somecommunity of users. Instead, it continues as they use andadapt the system.”
  24. 24. In structural terms, what is Firefox?2011: 85% of users had extended v4 with at least 1 add-on (or module or plug-in or add-in or component or...)No single structural definition—or class—defines it, andno one pattern of use eitherSimilar situation in IE, Photoshop, Outlook and beyond...
  25. 25. Practice ahead of theory and engineeringEstablished approaches are based on program structurebeing the same wherever and whenever it is usedHow to deal with the scale of human and softwareinteraction?How to reason predictively about structure and use?How to make tools for designers, evaluators and usersthat support adaptation and appropriation?
  26. 26. A population approach to system designClass as a varied population of software structures, beingchanged in the real world of useClass as population: Dynamic instead of static Stochastic instead of deterministic and discrete Coupled with programs in use, instead of standing apart from them Complementary to (and to be used along with) existing class concepts
  27. 27. A simple example: EdinburghFestivalGuideModules FestivalEvents & FestivalVenues in 100% of instancesModule NextBusToMyNextVenue in 50% of instances and themodule QuickestWalkingRoute in 40%All recorded instances have a variable meanLocationOfUsethat shows 90% of its values within the city of Edinburgh
  28. 28. The population approachMultiple representations of a software class (or type)Tools to use, create or adapt these representationsSocial interactions and practices involving those tools
  29. 29. The traditional representationA class is a structure, and all instances conform to it Data structure: variables or state Code structure: methods, procedures or functions These form a structural description of potential behaviour and use Assumption: exact match between the structure of the class definition, and the structure of each instance
  30. 30. The traditional representationTests and changes need only be done on one thing at one time:the class structureWhat’s true for the class is true for the deployed instancesMethodologically convenient but not consistently true
  31. 31. Borrowing from biology: population thinkingA species is a varied and changing population of individuals Assumed to be undergoing continual change and evolvution Each individual member may differ from others, and variation among individuals is natural and vital to adaptation Described by family resemblances among observable attributes The species can be described by the set of current (or recently recorded) members: the aggregate of each individual’s genes, size, colour, shape, etc.
  32. 32. A new representation: class as populationA class as a varied and changing population of instances Assumed to be undergoing continual change and evolvution Each individual instance may differ from others, and variation among individuals is natural and vital to adaptation Described by family resemblances of structure and use A class can be described by the set of current (or recently recorded) members: the aggregate of each instance’s structure, context, use, etc.
  33. 33. Populations: variation and dynamismWhat’s true for one instance may not be for another 90% of unmodified instances crash, but those with module A added don’t crash. Overall, 75% of instances crashAnalysis results may be different at different times We advertise A and the percentages; a week later only 50% crashResults for different contexts may differ News of module A spreads faster in one country than another, so in Spain 25% crash while in the US 60% crash
  34. 34. New and old representationsVariation/dynamism in the match between class & instancesProbabilistic statements/models, coupled to ongoing use, atthe core of designs, analyses and interactionsPopulation serves as overarching and connecting concept
  35. 35. Populations with family resemblancesFit with how ‘classes’ are used in everyday languageMoving away from narrow reliance on necessary andsufficient membership propertiesApplication to software structures demands a holisticapproach
  36. 36. Why ubiquitous computing?It’s about more than just mobile devices...Systems that fit with context, interaction and activity ...which are varied and dynamic in ways difficult to predictThe system is part of the user’s context System interactions, and how users are modelled and presented to others in them, are resources for new user activity
  37. 37. Ubiquitous computingA holistic view spanning technology, use and users Mark Weiser: The unit of design should be social people, in their environment, plus your device Robin Milner: Here we look for synergy between the societal vision on the one hand, and the development of scientific models and engineering principles on the other.
  38. 38. New tools and new interactionsAnalysis of modules, configurations and contexts Which configurations and contexts are popular? Which are problematic? How are they used? Are there clusters that should be separately managed and designed for? How does my setup compare to my friends’? Are there configurations like my own that doesn’t crash so much? How are they used?Access to modules and usage data May support free dissemination (P2P) or retain a degree of control ‘Lead users’ keen to try out new or specially instrumented versions
  39. 39. A third representation: ostensionOne or more population members are pointed out asexamplesMany potential ways to make an ostension e.g. a category or cluster selected from a population of instances
  40. 40. From observed use patterns to softwarestructureMapTool is usually run along with DGPSLocationStreamUKTrainSchedule is frequently logged as being used in UKlocations tagged as train stationFestivalEvents and FestivalVenues modules are frequently usedwith an unofficial module AlternativeGuideToTheFringe whenthe user is in Edinburgh Fringe Festival venues
  41. 41. SGVis: Categorising users
  42. 42. Girolami: statistical analysis of populationsInferring significant family resemblances amongdisparate features of large dynamic populationsReducing a chosen set of instances to representativecode and usage data
  43. 43. Girolami: statistical analysis of populationsExtending RJMCMC and TI approaches to onlineenvironment via Dirichlet process approachHeterogeneous data => non-parametric hierarchicalmixture modelStochastic structures abstracted into formal models e.g. CTMCs, reliant on MCMC posterior inference
  44. 44. Calder: formal modelling of populationsAn online framework set within end-user appsReal-time, critical feedback to usersLonger-term feedback to evaluators and developers
  45. 45. Calder: role-based interface automataMethods are classified as Master, Servant or Valets, andas Input or Output : Master (M) actions must be performed Servant (S) actions are offered Valet (V) actions are either servant or hidden Input (?) actions receive data Output (!) actions send data 45
  46. 46. Example: a simple GPS system (c1,c4).(M,S)loc 0,0,0,0 1,0,0,0 c1.M!display (c1,c8).(M,S)req (c1,c8).(S,M)map (c8,c5).(M,S)log 3,0,0,0 2,0,2,0 2,0,1,0 46
  47. 47. Calder: component replacementWhen can we replace/substitute one component by another? C1 ⊗ C2 ⊗ C3 ⊗... ⊗ Cn ⤳ C1’ ⊗ C2 ⊗ C3 ⊗... ⊗ CnC1’ has to offer the environment C2 ⊗ C3 ⊗... ⊗ Cn whateverthe environment needs to workFormalise work by services specified in a temporal logic C1 ⊗ Env ⊨ Σ : the current system works C1’ ⊗ Env ⊨ Σ : replace C1 by C1’ and the system still worksNot aiming to check C1’ ⊗ Env ⊨ Σ, but derive it...
  48. 48. Deploy app at a large scale, and collect log dataInfer statistical patterns of module structure and useMake event-based formal models of module configurationsMake probabilistic judgements about failures/exceptionsFeed patterns and judgements into role-specific tools forawareness and analysisSupport people in these roles in adapting the app and its use
  49. 49. Scenario: FanPhotoFanPhoto: two core components,for text and photo sharingWe give it out to 100 people and they start using itWe start developing newmodules one for sharing data via Facebook one for fast compression and sharing of text, photos and video via MANETs
  50. 50. Scenario: FanPhotoA few participants areinterested in new modules 97 download them and show off 3 that they are trying them outAnalysts keep track via logdata streaming back start to see a new subcluster forming
  51. 51. Scenario: FanPhoto ▽FanPhoto ▷TextSharingIn the IDE, developers can ▷PhotoSharingsee the FanPhoto class ▷FacebookSharing Modules used by a minority of ▷MANETmodule users are shown in grey
  52. 52. Scenario: FanPhoto 97After some time, 3developers and evaluatorsmeet to play back severalweeks’ use
  53. 53. Scenario: FanPhoto 91After some time, 9developers and evaluatorsmeet to play back severalweeks’ use
  54. 54. Scenario: FanPhoto 26 51After some time, 23developers and evaluatorsmeet to play back severalweeks’ use
  55. 55. Scenario: FanPhoto FanBook 60 5 5After some time,developers and evaluators 30meet to play back severalweeks’ use
  56. 56. Scenario: FanPhoto FanBook 60 5 5After some time,developers and evaluators 30meet to play back severalweeks’ use Zippy
  57. 57. Scenario: FanPhotoA developer sets hisIDE to do automaticupdatesCode defining each FanPhotonew subclass appears labelled with users’ and analysts’ names FanBook Zippy linked to tools to analyse the ostensions that made them
  58. 58. Ontology as structure“An ontology is a formal specification of a sharedconceptualization” Gruber, via SemanticWeb.orgTraditionally, the core of software design is aboutfinding the perfect hierarchical structure One structure for all known uses, contexts and people Adaptation is the problem of changing it
  59. 59. Ontology as processPopulation approach couples process and structure Design process in which structure is a resource for use, and in which use creates or adapts structure Giddens’ duality of structure, Heidegger’s hermeneutic circle Part of the process is adaptation of a structure to new contexts and usesA design requirement for ubicomp... and CompSci?
  60. 60. Designing for duality of structureMoving from structure to use is commonplace That’s what happens when you compile code and users run itThe trick is how to move back again Making code for a class from a set of instances and usage historiesA process that allows for gradual adaptation of a class Moving between complementary forms of class definition Each move is (or should be) a social interaction
  61. 61. Code as intension A class’ code, compiled to create an executable shared among usersPopulation as extension The deployed instances, that are adapted by users and which create log data shared with evaluators and developers (and other users)Sub-population as ostension A subset of instances selected by one of these people as particularly interesting or useful, and shared among them for comment and use Reduction to a code signature reflecting family resemblances among the configurations and logs of that subset
  62. 62. Ostension revisitedMost forms of ‘analysis’ afford ostensive definition A way to focus on a subset of structures, contexts, uses and users e.g. an evaluator’s video of particular users from a system trial, a user’s list of friends on Facebook, a developer choosing a name of a city to customise design forLog data allow us to link to the software used Extend previous work that links from video to log data
  63. 63. A population approachA combination of theory, design, evaluation and use Advances in all four needed for successA circular process designed for duality of structure Change and human agency as essential elements of the process Multiple ways of representing a software class Tools to use, create or adapt these representations Users’, evaluators’ and developers’ social interactions and practices involving those tools... that feed back into low-level representations
  64. 64. FanBook 60 5 5 30