Literature Review
Upcoming SlideShare
Loading in...5
×
 

Literature Review

on

  • 762 views

 

Statistics

Views

Total Views
762
Views on SlideShare
762
Embed Views
0

Actions

Likes
0
Downloads
11
Comments
0

0 Embeds 0

No embeds

Accessibility

Categories

Upload Details

Uploaded via as Microsoft Word

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Literature Review Literature Review Document Transcript

  • LOCATION MODELLING AND MACHINE LEARNING IN SMART ENVIRONMENTS University of Sydney School of Information Technologies Literature Review Supervisor: Associate Professor Judy Kay Associate Professor Bob Kummerfeld Robert Whitaker 0111139
  • Table of Contents 1. INTRODUCTION................................................................................................................................................3 2. UBIQUITOUS COMPUTING SENSORS.........................................................................................................4 2.2. COMMUNICATION FACILITIES................................................................................................................................5 2.3. CONNECTIVITY....................................................................................................................................................5 2.4. INTERACTION......................................................................................................................................................5 3. LOCATION AND ACTIVITY MODELLING.................................................................................................6 3.1. USER MODELS...................................................................................................................................................6 3.2. USER MODELLING SYSTEMS.................................................................................................................................6 4. MACHINE LEARNING FOR PREDICTING USER LOCATION AND ACTIVITY................................7 4.1. LEARNING TECHNIQUE.........................................................................................................................................7 4.2. LEARNER...........................................................................................................................................................8 4.3. SOURCE.............................................................................................................................................................8 5. PREVIOUS WORK.............................................................................................................................................9 5.1. DETERMINING LOCATION.....................................................................................................................................9 5.2. DETERMINING AND PREDICTING LOCATION...........................................................................................................11 5.3. DETERMINING AND PREDICTING ACTIVITIES..........................................................................................................13 5.4. DETERMINING LOCATION AND ACTIVITIES............................................................................................................16 5.5. MODELLING ENVIRONMENTS...............................................................................................................................17 6. RESEARCH PROBLEM..................................................................................................................................19 7. RESEARCH CONTRIBUTIONS.....................................................................................................................21 8. RESEARCH PLAN............................................................................................................................................21 9. SUMMARY.........................................................................................................................................................22 APPENDIX A – LANCASTER GUIDE PROJECT...........................................................................................23 ..................................................................................................................................................................................23 APPENDIX B – ASSISTED COGNITION PROJECT......................................................................................24 ..................................................................................................................................................................................24 ..................................................................................................................................................................................25 APPENDIX C – DOPPELGANGER PROJECT................................................................................................26 ..................................................................................................................................................................................26 APPENDIX D – ACTIVE BADGE AND BAT PROJECT................................................................................28 APPENDIX E – WEARABLE SENSORS...........................................................................................................29 APPENDIX F – BSPY............................................................................................................................................30 REFERENCES.......................................................................................................................................................31 -2-
  • 1. Introduction Ways of determining the locations and activities of people has been a strong focus of research into ubiquitous computing. The establishment of smart environments through the development of various sensors has allowed vast amounts of data to be collected on human behaviour and their use of such environments. Through this data it is now possible to model these environments using a range of user models, and from these perform predictions using various machine learning techniques. Location and Activity Modelling Machine Learning Sensors/ Devices Location and Applications Activity Prediction User A User B Figure 1: High level view of modelling and prediction systems, showing the relationships between the various components The components of such systems can best be summarised in figure 1 and include • Sensors/Devices – responsible for the acquisition of data from the user and the devices in surrounding environment; such as mobile phones, PDAs, laptops, motion sensors, and pressure pads. • Location and Activity Modelling – The storing of acquired data to build models of the user’s use of the environment. For example, the modelling of the users movements within the environment using the data acquired from location sensors. -3-
  • • Prediction – Uses the data acquired and stored in user models to formulate some conclusion using machine learning techniques. For example predicting where a user will be in two hours time based on their current location and past movements over the last two hours. • Applications – The programs that use the generated models and prediction results to personalise themselves to the users behaviour. A scenario application is a smart meeting system, whereby User B interacts with an application that allows a meeting to be booked with User A based on User A’s passed movements and activities modelled in the system. This review will focus on each of the components in figure 1: ubiquitous computing sensors; location and activity modelling; and machine learning for predicting user location and activity. In addition to each component, the previous work in the research domain, research problem, research contributions, and research plan will also be discussed. 2. Ubiquitous Computing Sensors Ubiquitous computing is being considered the third wave of computing. It is focused around the concept that each person has many devices and that many more devices will be embedded into the surrounding environment. The concept of ubiquitous computing was introduced by (Weiser 1991) where the ideas of a computer knowing its location and adapting its behaviour were first discussed. A key component of ubiquitous computing is the devices that make up the surrounding environment. These devices can range in size, cost, power consumption, application and connectivity. They can range from a person’s laptop computer to their mobile phone to a -4-
  • security lock on a door. The list of the different devices found in these environments is endless, however the following are observed common characteristics. 2.1. Positioning Systems Many devices today have the ability to determine their location based on a predefined set of rules. There is a variety of positioning technologies; such as Bluetooth and GPS. Devices implementing these technologies provide the ability of determining its location based on a predefined coordinate grid; for example, GPS uses longitude and latitude. 2.2. Communication Facilities Many ubiquitous devices are built on top of personal communication devices such as mobile phones and PDAs. These all provide ways of the device or environment communicating with its users. But the communication between the environment and its users is not limited to these types of devices. Communication can take place in many other forms. For example the key locks in the Madsen Building, use sound to communicate with users who are permitted to access the respective room or not. 2.3. Connectivity Devices in a ubiquitous environment require some mechanism to communicate with one another; there are a range of connectivity measures available, including local area network and wireless connections. 2.4. Interaction Ubiquitous devices can be categorised as either interactive or passive devices. For interactive devices users are continuously supplying information, such as in the example of a laptop computer or PDA, while passive devices are those that acquire data without any human interaction, such as a motion sensor. -5-
  • 3. Location and Activity Modelling Location and activity modelling is concerned with modelling peoples’ locations and actions. It is a subset or specialised form of user modelling and will be used interchangeably for the remainder of this review. (Csinger 1995) defined user modelling as “the acquisition or exploitation of explicit, consultable models of the human users of systems”; similar definitions were made by (Pohl 1997) and (Orwant 1992) in their discussion on the implementation of various user modelling systems. There are two aspects of user modelling. The first is the actual user model and the second is the system responsible for the acquisition and exploitation of the data collected. Neither can exist without the other. 3.1. User Models A user model is the structure that acquired data can be stored in to allow for personalisation of systems and prediction of user behaviour. (Ashbrook, et al. 2003) considered a user model to be “a collection of data on some particular aspect of a human user’s behaviour, when associated with a limited set of contextual clues, yields predictions on what behaviour the human will engage in next”. 3.2. User Modelling Systems A user modelling system is the application system that acquires and stores new evidence about a person, identifies consistencies and inconsistencies held about a user and answers queries of the application concerning the currently held assumptions on the user (Kobsa 2000). These systems are often referred to as user modelling shells as defined in (Kobsa 1990). -6-
  • User modelling systems have seen many developments, both in the commercial and academic fields. (Kobsa 2000) identified and summarised some the key systems including BGP-MS, Doppelganer and um. From this work, Kobsa formulated a list of common characteristics, including: • The representation of assumptions about user characteristics in individual user models; • Representation of common characteristics between user models leading to the creation of stereotypes; • Classifying a user as belonging to a given stereotype; • Methods of recording a user’s behaviour and interaction with the system; • Ability to make assumptions about a user based on information recorded in their user model; • Methods of justifying assumptions made based on a user model; • Evaluation of user models. 4. Machine Learning for Predicting User Location and Activity Machine Learning has three core concepts the learning technique, the learner and the source, each are discussed below. 4.1. Learning Technique Learning is the process of acquiring new knowledge and gaining a greater understanding of knowledge previously held. (Shavlik, et al. 1990) discussed the two main ways in which a system can learn: 1. The system can acquire new knowledge from external sources; 2. The system can modify itself to exploit its current knowledge more effectively. -7-
  • When applying learning algorithms to location/activity modelling and ubiquitous computing, these systems use the first of these techniques, as they are continually acquiring data from the surrounding environment. The actual algorithms these systems use to perform learning can vary and may use more than one. Some of the more common algorithms used by systems in this area include Linear Prediction, Markov models, Bayesian Techniques and Beta Distribution. 4.2. Learner A machine learning system is an application that acquires data on the use of another machine and performs calculations on that data to return some result. There are several key characteristics to any machine learning system. (Orwant 1992) highlighted five of these characteristics and discussed their importance to user modelling, summarised in table 1. Characteristic Description Learner The system which is actually performing the learning Domain Refers to the types of data that learning algorithms will be applied to Information Sources Refers to the array of sensors data is acquired from Prior Knowledge The data already known by the system that the learning algorithms can be applied to Performance Criteria Analysing the results of the algorithm against those which are expected, this is normally not applied in user modelling as a user modelling system never stops learning. However, many papers still do perform some performance evaluations. Table 1: Characteristics of a system which learns and their application to user modelling (Orwant 1992) 4.3. Source When discussing a system in this context, it is referred to in broad terms as the object we are learning information about. This could be anything from a physical machine to an individual or group of people. The tools used are dependent on the system or object learning is being performed on. The common data sources for systems in this area are the ubiquitous devices embedded in the surrounding environment. -8-
  • 5. Previous Work The fields of ubiquitous computing, location modelling and machine learning are very active research areas that have seen the development of many projects. When examining work in this area, the relevant projects can be broken into five categories: determining location; determining and predicting location; determining and predicting activities; determining location and activity; and modelling environments. Table 2 provides a summary of these projects with details on what categories they satisfied as well as any learning algorithms they may have used. Categories Satisfied DetectionLocation Detection and Activity Detection Location and Modelling Prediction and Prediction Prediction Location Activity Project Learning Algorithm Active Badge X N/A Active Bat X N/A Lancaster Guide X N/A Assisted Cognition Bayesian, Relational Markov X X X Models Multiple User Markov Models, Clustering X X Detection MyPlace X X N/A Doppelganger Beta Distribution, Linear Prediction, X X Markov Models LaboUr X X Markov Models Wearable Sensors X X X No explicit algorithm Activity Zones X X X No explicit algorithm Table 2: Overview of the relevant projects in the research area, shows the categories each project addresses as well as any learning algorithms they may use The combining of these five properties assist in the development of a location authority (Shafer 2003) system. This section will examine important projects from these five categories in greater detail. 5.1. Determining Location -9-
  • The first step in performing any location prediction is first calculating where the user or object currently is. Projects in this category use devices to emit or query its current location. The Active Badge Project (Davies, et al. 2002, Wang, et al. 2003, Want, et al. 1992) was one of the first indoor positioning system. The system required users to wear badges that emitted signals to sensors that determined the location of the user. The project was driven by the notion of locating people in an office environment and could be applied to any problem needing to determine a person’s location. An application that implemented this system was teleporting (Richardson, et al. 1994), which involved determining the current location of the user and having their desktop settings sent to the nearest terminal. An extension on the Active Badge Project is Active Bat (Addlesee, et al. 2001, Harter, et al. 2002), which is a 3D indoor position system. In these systems small units called Bats (see Appendix D) are attached to objects in the space, for example bats would be attached to users, desks, chairs, and computers. To determine the location of a bat, base stations periodically transmit a radio message causing the corresponding bat to emit a short ultrasound pulse. If the distance from three base stations can be determined, the exact position of the bat in the 3D space can be calculated. Active Badge and Active Bat both perform well for indoor environments. However different technologies are required for outdoors. Most outdoor positioning systems use GPS to determine location. An example of such system is the Lancaster Guide Project (Davies, et al. 2002, Distributed Multimedia Research Group 2004). The aim of this project was to provide - 10 -
  • information normally found in a tour guide but also to give the user the information based on their interests, current context and movements. The application was implemented using tablet PC’s as end-systems connected to information servers through an 802.11 network deployed around the city; Appendix A contains images of the system interface. The system determined a user’s location by querying the base station the user was currently connected to and displaying the information associated with that base station. A field trial was performed on the system in 1999 for a period of four weeks. This highlighted several problems (Davies, et al. 2002), with a conclusion that such systems were restricted by the infrastructure available to them. 5.2. Determining and Predicting Location Determining an object’s current location can be performed by the methods discussed in the previous section. However, the challenge is using these techniques to perform some prediction of where an object’s location will be in the future. The Assisted Cognition Project (Kautz, et al. 2003, Kautz, et al. 2002) uses ubiquitous computing environments to enhance human capabilities with a particular emphasis on assisting people with cognitive limitations. The project uses various sensors, handheld and wireless devices to assist building models of patient behaviour and routines (See Appendix B for the system architecture). An important application is the Activity Compass (Patterson, et al. 2002). Shown in Appendix B, it uses GPS readings to generate models of user behaviour. From these, it attempts to direct the user to the location which best suits their current movement patterns. The Activity Compass incorporates learning algorithms to make its predictions. (Kautz, et al. 2003) reported the use of two learning techniques, Bayesian User Modelling (Fox, et al. 2003) and Relational Markov Models (Anderson, et al. 2002). - 11 -
  • Another product of the Assisted Cognition Project has been the development of prediction algorithms based on the relationship a person’s position on a map. (Patterson, et al. 2003) and (Hightower 2003) discuss the ideas of moving away from the traditional mathematical techniques to turning a street map into a graph. Predictions of locations are made from the graph, using past historical data, measurements of recent movements and some common sense (eg. Buses normally stop at bus stops and traffic lights, cars drive along roads). The Assisted Cognition Project focuses on developing solutions for a single user. Another similar project (Ashbrook, et al. 2003) extends this, predicting multiple user locations and movements. The system collects data using GPS hardware and storing the results in a location model. It uses clustering and Markov Models to analyse and predict the location and movements of multiple users from the models. This work provides foundation principles for developing a location prediction system for multiple users. An important aspect of the prediction process is the generation of models to formulate patterns in the acquired data. The MyPlace Project (Kay, et al. 2004) explores techniques for developing models of users, sensors and locations using accretion user modelling techniques and developing various applications to use these models. The project explores two main scenarios, the Locator Scenario and the Invisibility Scenario which attempt to use the models to perform some prediction and personalisation of the user’s environment. On the basis of these systems, we can identify three main components - 12 -
  • • Training data and a data store of historical data on the use of the application; • Pre-processing the raw data so it can be modelled; • Learning Algorithms to analysis the historical information and perform predictions. These observed components would be the foundation of any prediction system, be it location or something else. One negative aspect of these projects is the lack of evaluation currently performed; this is most likely due to the immaturity of these projects. 5.3. Determining and Predicting Activities The concepts previously described on location could also be applied to this category as well. Two projects which explore predicting of activities are DOPPELGANER and LaboUr. The DOPPELGANGER (Orwant 1991) system was a generalized tool for gathering and processing information about users. The developed system had three main components: (See Appendix C for architecture diagram) • Sensors – the devices that acquired data for the system, these could be hardware or software devices. Examples of suggested sensors included pressure pads and Unix commands like finger (provides information about the user contained in the system password file) and lastcomm (returns the name of the last command executed by the user). • Learning Toolbox (Orwant 1995) – implemented various learning algorithms each used under different circumstances and used these to form results that were returned to various applications. These techniques are summarised in table 3. - 13 -
  • • Applications – the programs which use the models created to customize themselves to the user’s interests and needs. Examples include Dopmail (Orwant 1995) and an electronic newspaper. Technique Description Reasons for Use Beta Distribution Used when determining a user’s When calculating a person’s interest in a topic it preference for some feature or can be either yes or no, using this technique then interest on a topic allows the cumulation of these opinions. Linear Prediction Used for projecting a data set into The occurrences of past events are known. This the future based on past recordings technique allowed the knowledge of these events made to predict when that event will reoccur next. Markov Models Used for determining the future Can treat each location being modelled as a physical or working state of the different state. This technique then allows the user prediction of movements between different states. Table 3: Summary of the learning techniques used in Doppelganger The architecture of this system is similar to that shown in figure 1. Both architectures acquire data from a range of sensors and are used by many different applications. The main differences exist in the modelling and prediction processes. Under the architecture in figure 1 there has been a separation of the learning and modelling processes which was not present in the DOPPELGANGER architecture. One feature implemented in DOPPELGANGER, not present in similar systems, is the notion of confidence. In addition to making predictions based on the historical data, the system also provides a confidence percentage providing the user feedback on how correct DOPPELGANGER thinks the prediction is, for example in addition to returning a result is will say that there is a 0.9 certainty the result is correct. This confidence feature is very useful in the interpretation of results and confidence in the system and still remains a notable omission in many systems in the field. - 14 -
  • Another novel concept DOPPELGANGER also introduced was the notion of user communities. User communities extend the concept of stereotypes (Kobsa 1993), to categorise people against the stereotype they best match, which may not be the one their assigned to, at the same time as updating the model of their assigned stereotype with the acquired data. The benefit of this approach is that the system might better represent those users that do not match the patterns of others in their group, but as evaluation is yet to be performed these benefits are yet to be realised. Appendix C shows the interfaces for the DOPPELGANGER System. One important aspect to highlight with these interfaces is that they provide srcutability in the presentation of the results. Scrutability enhances the confidence information provided, by allowing the user to examine how the results were calculated, view what data was used to determine the prediction, and view the prediction at varying levels of granularity. LaboUr (Pohl 1997, Pohl, et al.) concentrated on developing a centralised user modelling system and treating user modelling as an open process. The components of this architecture were similar to those in the DOPPELGANGER System, except it introduced filter mechanisms which if necessary performed transformations on acquired data to ensure the input was suitable for the learning algorithm being used. An example scenario on how these filters could be applied to such systems is shown in scenario 1. Scenario 1: Device A provides information using 3 data fields, while device B provides the same information across 4 data fields, the filtering would allow the conversion of these inputs into the same format - 15 -
  • 5.4. Determining Location and Activities This category of application integrates the concepts of the previous three sections. These Applications have the ability to both determine an object’s location and the context in which it is being used. One such application is (Lee, et al. 2002) which uses wearable sensors (See Appendix E) to detect the location and movements of a person. This application uses a range of sensors to determine if a person is walking, running, or standing still at the same time determining their location. Such motion or context is measured through the rate of change in the person’s displacement from their initial start position. Applications like these could be extended to include additional sensors to determine different contexts; an example could include a sensor detecting a person’s speech, and this sensor could then add richer information to many of the motion cases. An alternative or additional mechanism of determining activity is the association of activity with location. (Koile, et al. 2003) discusses the idea of an activity zone, an activity zone is a location in a building or room that has a specific activity associated with it. An example is a meeting room that would have the activity “people meeting” associated with it. However, as (Koile, et al. 2003) discussed there can be many different activity zones in a single location/room, all dependent on the arrangement of the objects in that location. The use of activity zones is the simplest way of introducing activity tracking into an already existing location prediction system, through the association of an activity for each location readings are taken for. - 16 -
  • 5.5. Modelling Environments Most methods of prediction use some form of modelling technique. A common approach builds up models of the entire surrounding environment and uses those models in someway. Such models are usually built using sensor-rich environments like those systems discussed in (Brumitt, et al. 2000) and (Kidd, et al. 1999). Issue Description Need for large data sets A common problem with machine learning algorithm is that they can’t build a model with significant accuracy until it has seen a relatively large number of events to base its decision on Need for labelled data Some machine learning approaches require data to be labelled as part of its calculation process, unfortunately this labelling may not be apparent from the observations stored in the user model. Concept Drift The behaviour of the user changes over time, a machine learning system must be able to adjust quickly to these changes and reflect this in the results it returns. Computational Complexity The various algorithms can require large amounts of computational power to execute. Therefore the performance of such algorithms needs to be considered in their implementation and use. Table 4: Summary of issues associated with applying machine learning to user modelling (Webb, et al. 2001) All location and activity prediction requires a data store for recording historical information. In most systems today, this is done through user models. These can be used both to develop behavioural models for a single user and/or to model the use of the environment as a whole. There are many ways modelling can take place. (Glassey, et al. 2003) identified three types of location models: • Geometric – allows the points of the physical objects to be modelled, for example the coordinates of an object on a grid • Symbolic – describes a location or object using a predefined label, for example the name of the room such as G62. • Hybrid – uses a combination of the previous two techniques - 17 -
  • Similar definitions were also provided in (Domnitcheva 2001). The way the environment can be represented using these techniques can be linked with the work on LaboUr. Since it applied filters to the incoming data from sensors; similar transformations could be applied to convert the geometric representation received from sensors into a symbolic representation used in the models. For example a system could convert the coordinate “A2” into the room label “G62”. Once a user model has been developed the analysis of the model is a complex process. (Webb, et al. 2001) discuss several issues associated with applying machine learning to user models, these have been summarised in table 4. Technique Description Linear Models Take weighted sums of known values to product a value of an unknown quantity. Markov Models Based on the Markov Assumption, where the occurrence of the next event depends on a fixed number of previous events. Given a number of observed events, the next event is predicted from the probability distribution of the events which have followed the observed events in the past. Neural Networks Performed through a structure of networks, non-linear thresholds and the weights of the edges between each node. The nodes represent the events with the weights on each path being the probability of it occurring. Classification Given an n-dimensional space that corresponds to the attributes under consideration, the generated clusters or classes contain items that are close to each other in this space and far from other clusters Rule Induction Consists of learning sets of rules that predict the class of an observation and its attributes. Bayesian Networks Are directed acyclic graphs where nodes correspond to random variables, the nodes are connected by directed arcs with each node associated with a conditional probability distribution which assigns a probability to each possible value of this node for each combination of its parents nodes. Table 5: Summarising the more common techniques used in the field (Zukerman, et al. 2001) Although the task of applying machine learning to user models is complex, there are many techniques available, as seen by the summary of techniques used in current projects summarised in table 1 and those summarised in table 5. - 18 -
  • From the analysis of the previous work and this discussion, is can be observed that user models are the logical choice for storing and analysing user behaviour in any location and activity prediction system. 6. Research Problem There has been considerable research in the areas of location prediction, activity detection and location detection. However, little work has occurred integrating all these issues into one system. The aim of this research is to explore ways of determining the current location and activity of a person and predicting future locations/activities using machine learning and user modelling techniques on data collected from the surrounding “smart” environment. Machine Use PDA User Environment Conversion into Logical Database of Representation Logical Data User Model Data store of model info Creator - personis Filters/Conversion Modelling Build Result Set Where is X most using Markov likely to be? Model Applications User Figure 2: High Level View of intended system Based on previous work described about, we can identify four main components of such environments, summarised in figure 2. These components include • Environment – this includes the various sensors used to acquire data about the environment and its users; - 19 -
  • • Filters or Conversions – the data received by the sensors may not be in an acceptable format for the system or may not be in a user friendly form eg. MAC address of device could be replaced with the name of the device; • Modelling – based on the acquired data models, are generated for both the environment and its users; • Applications – this is the software which uses the generated models. Examples include a user prediction application or a smart meeting system application. Aspects that will not be covered in the solution will be issues of security and scalability of the system, both at the device end and the application end of the system. In order to implement the various aspects of the system a variety of already available packages will be used to provide some of the functionality; summarised in table 6. Package Description/Use BSpy Indoor location detection system developed by the University of Sydney, see Appendix F for a sensor map and data view. Elvin Messaging (Distributed A messaging system that will be used to send the acquired data from the Systems Technology Centre devices to a central system. 2003, Mantara Software 2003) Personis Software (Kay, et al. user modelling software to be used to generate the various environment 2002) and user models Visualisation Software to provide meaningful feedback and srcutability of results, tools such as dot (Ganser, et al. 2002) will be used to provide a graphical representation of the results Markov Modelling Packages Markov Models will be used in the prediction algorithms, packages such as (Young, et al. 1999) will be used in the construction of these models from the user models. Table 6: Summary of possible packages to be used in the solution Overall the aim of the research is to create a location authority that addressed some of the functional requirements discussed in (Shafer 2003) and to provide evaluations on this type of system. - 20 -
  • 7. Research Contributions The intended contributions of this research include • The exploration of Markov Models and Linear Prediction to predict the location and activity with regards to time • The integration of activity and location detection and prediction into a single system • The visualisation of prediction models to provide srcutability to predictions. • Exploration of extensions to previous systems including the use of confidence in predictions and the use of various user modelling techniques. • The exploration of predictions and user models using a combination of varying granularities of data, for example the combining of daily and weekly models of a user. Mar 2004 Apr 2004 May 2004 Jun 2004 Jul 2004 Aug 2004 Sep 2004 Oct 2004 ID Task Name 22/2 29/2 7/3 14/3 21/3 28/3 4/4 11/4 18/4 25/4 2/5 9/5 16/5 23/5 30/5 6/6 13/6 20/6 27/6 4/7 11/7 18/7 25/7 1/8 8/8 15/8 22/8 29/8 5/9 12/9 19/9 26/9 3/10 10/10 17/10 24/10 1 Definition of Research Problem Review of Literature in the Field and 2 possible packages for prototype 3 Experimental and Evaluation Design Prototype Design and Development - 4 Version 1 Experimental and Evaluation 5 Execution Prototype Design and Development - 6 Version 2 Prototype Design and Development - 7 Version 3 8 Evaluation of Final Prototype 9 Writing Thesis Figure 3: Research Plan - timeline of tasks to be completed in the research process 8. Research Plan The timeline of events and the entire process has been summarised in figure 3. From this it can be seen that the process has been broken down into nine components summarised in table 7. Task Description Definition of Research involved determining the research field and specific aspects of the field to Problem be investigated Review of Literature in the Gain background knowledge of previous work in the area as well as gain - 21 -
  • field and available packages experience using various packages that may be useful in the development of a prototype system. Experimental Design Design of the various evaluations to be performed on the prototype as well as the development of an activity sensor to be used in the experiments and evaluation. Prototype Design and Development of a prototype that makes the use of one location and one Development Version 1 activity sensor, provides the current location and activity in addition to basic predictions, and constructs user models to be used in future versions. Experimental Execution Running the evaluations previously designed on a range of location and activity sensors and basic testing/use of version 1 of the prototype. Prototype Design and Make refinements to version 1 based on evaluation results, adjustments Development Version 2 made to version 1 to allow input from various activity and location sensors and the introduction of confidence into the prediction algorithms. Test and evaluate this version based on the data collected from the previous experiment. Prototype Design and Includes the development of sample applications to use the system Development Version 3 examples may include the smart meeting system. In addition improvements to the interface and visualisation of results should be performed. Execution of Final Evaluate the final prototype against the original experimental designs. Experiments Writing of Thesis Perform analysis of evaluation and write up of each stage and the final system prototype. Table 7: Description of the various tasks shown in figure 3 9. Summary Overall this review examined the key concepts associated with user prediction systems and associated previous work. It identified various aspects of the research area that requires further work including the performing of further evaluations on various systems, the integration of ideas between systems and enhancements to systems such as adding srcutability and confidence to predictions. The research problem being investigates aims to address some of these aspects through its planned contributions to the research field and the implementation of various prototypes to evaluate these contributions. - 22 -
  • Appendix A – Lancaster Guide Project Figure 4: Lancaster Guide Interface - giving user information about the current location (Distributed Multimedia Research Group 2004) Figure 5: Lancaster Guide Interface - providing the user with directions and view of the surrounding area (Distributed Multimedia Research Group 2004) - 23 -
  • Appendix B – Assisted Cognition Project Figure 6: Activity Compass device developed (Patterson, et al. 2002). The devices uses a GPS transmitter connected to a PDA for the interface. The current image is giving the user directions to their home. Figure 7: Architecture of the Assisted Cognition Project. There are three components to the system, the sensors that acquire the data from the environment, the server which performs the modelling and prediction, and the client which interacts with the models and prediction results. - 24 -
  • Figure 8: The Green line shows the users movements as tracked through the Activity Compass. Based on the map layout, predictions can be made using past movements and the direction of paths that can be followed on the map. - 25 -
  • Appendix C – Doppelganger Project Figure 9: Structure of the Doppelganger System. The diagram shows the three key components of the system; the sensors, learning toolbox and the applications (Orwant 1996) Figure 10: Screenshot of Doppelganger system. This image shows the results from a linear prediction algorithm being performed to calculate when a person is most likely to log into the computer system next and how long they will use it for (Orwant 1996) - 26 -
  • Figure 11: Visualisation of results from Doppelganger using Markov Models for predicting the location state of the user. The thicker the line connecting two nodes the more occurrences have been detected of the movement between those states (Orwant 1996). - 27 -
  • Appendix D – Active Badge and Bat Project Figure 12: An Active Bat device used to emit the signal of the users current location (Harter, et al. 2002) Figure 13: Operation of Bat location sensor system. The base stations send signals to the beacons which emit a ultrasound pulse. Readings from three base stations allows the exact position to be determined (Addlesee, et al. 2001). - 28 -
  • Appendix E – Wearable Sensors Figure 14: Sensors used to detected movement and movement type of a user (Lee, et al. 2002). The sensors in the thigh and waste determine whether the person is walking forwards or backwards. The speed at which the movements are detected determines the speed they are walking/running. Using this movement information and an initial start position the current location can be determined using the displacement of the current position from the initial position in addition to the context in which the person is currently moving. - 29 -
  • Appendix F – Bspy Figure 15: Output from BSpy System. This screenshot shows the data stored by the system currently (BSpy System 2004) Figure 16: Sensor Layout of the BSpy System in the Madsen Building. Not shown here is a sensor in the Seminar room which is located in another part of the building. (Kay, et al. 2004) - 30 -
  • References 1. Addlesee, M., Curwen, R., Hodges, S., Newman, J., Steggles, P., Ward, A. and Hopper, A. Implementing a Sentient Computing System. IEEE Computer (Auguest 2001). 2-9. 2. Anderson, C.R., Domingos, P. and Weld, D.S., Relational markov models and their application to adaptive web navigation. in Eigth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, (2002). 3. Ashbrook, D. and Starner, T. Using GPS to learn significant locations and predict movement across multiple users. Personal and Ubiquitous Computing, 7. 275-286. 4. Brumitt, B., Meyers, B., Krumm, J., Kern, A. and Shafer, S., EasyLiving: Technologies for Intelligent Environments. in 2nd International Symposium on Handheld and Ubiquitous Computing, (2000). 5. BSpy System. Search Logs, Personal Communications, 2004. 6. Csinger, A. User Models for Intent Based Authoring Computer Science, The University of British Columbia, Vancouver, 1995, 202. 7. Davies, N. and Gellersen, H.-W. Beyond Prototypes: Challenges in Deploying Ubiquitous Systems. IEEE Pervasive Computing Journal, 1 (January-March 2002). 26-35. 8. Distributed Multimedia Research Group. The Guide Project, 2004. 9. Distributed Systems Technology Centre. Elvin - Content Based Messaging Systems, 2003. 10. Domnitcheva, S., Location Modeling - State of the Art Challenges. in Ubiquitous Computing 2001, (2001). 11. Fox, D., Hightower, J., Kautz, H., Liao, L. and Patterson, D.J., Bayesian Techniques for Location Estimation. in 2003 Workshop on Location-Aware Computing, (Seattle, Washington, 2003). 12. Ganser, E., Koutsofios, E. and North, S. Drawing Graphs with dot, 2002. - 31 -
  • 13. Glassey, R.J. and Ferguson, R.I., SpaceSemantics: An Architecture for Modeling Environments. in 2003 Workshop on Location-Aware Computing, (Seattle, Washington, 2003). 14. Harter, A., Hopper, A., Steggles, P., Ward, A. and Webster, P. The Anatomy of a Context-Aware Application. Wireless Networks, 8. 187-197. 15. Hightower, J., From Position to Place. in 2003 Workshop on Location-Aware Computing, (Seattle, Washington, 2003), 10-13. 16. Kautz, H., Etzioni, O., Fox, D. and Weld, D. Foundations of Assisted Cognition Systems, UW CSE Technical Report, 2003. 17. Kautz, H., Fox, D., Etzioni, O., Borriello, G. and Arnstein, L., An Overview of the Assisted Cognition Project. in AAAI-2002 Workshop on Automation as Caregiver: The Role of Intelligent Technology in Elder Care, (2002). 18. Kay, J., Kummerfeld, B. and Carmichael, D. Consistent Modelling of users, devices and environments in a ubiquitous computing environment, University of Sydney, 2004. 19. Kay, J., Kummerfeld, B. and Lauder, P., Personis: A Server for User Models. in Adaptive Hypertext 2002, (2002), Springer, 202-212. 20. Kidd, C.D., Orr, R., Abowd, G.D., Atkeson, C.G., Essa, I.A., MacIntyre, B., Mynatt, E., Starnet, T.E. and Newstetter, W., The Aware Home: A Living Laboratory for Ubiquitous Computing Research. in Proceedings of the Second International Workshop on Cooperative Buildings, Integrating Information, Organization, and Architecture, (1999), Springer-Verlag Lecture Notes in Computer Science, 191-198. 21. Kobsa, A. Modeling the User's Conceptual Knowledge in BGP-MS, a User Modeling Shell System. Computational Intelligence, 6. 193-208. 22. Kobsa, A. User Modeling: Recent Works, Prospects and Hazards, University of Konstanz, 1993. 23. Kobsa, A. Generic User Modeling Systems. User Modeling and User-Adapted Interaction, 11. 49-63. 24. Koile, K., Tollmar, K., Demirdjian, D., Shrobe, H. and Darrell, T., Activity Zones for Context-Aware Computing. in Ubicomp, (2003). 25. Lee, S.-W. and Mase, K. Activity and Location Recognition Using Wearable Sensors. IEEE Pervasive Computing Journal, 1 (July-September 2002). 24-32. - 32 -
  • 26. Mantara Software Elvin Administrator's Guide, 2003. 27. Orwant, J. Doppelganger Goes to School: Machine Learning for User Modeling Computer Science and Engineering, Massachusetts Institute of Technology, 1991, 89. 28. Orwant, J. Machine Learning for User Modeling, Unpublished, 1992. 29. Orwant, J., Heterogeneous Learning in the Doppelganger User Modeling System. in User Modeling and User-Adapted Interaction, (1995), 107-130. 30. Orwant, J. For want of a bit the user was lost: Cheap user modeling. IBM Systems Journal, 35. 398-416. 31. Patterson, D.J., Etzioni, O. and Kautz, H., The Activity Compass. in First International Workshop on Ubiquitous Computing for Cognitive Aids, (Gothenberg, Sweden, 2002). 32. Patterson, D.J., Liao, L., Fox, D. and Kautz, H., Inferring High-Level Behavior from Low-Level Sensors. in Fifth International Conference on Ubiquitous Computing, (2003). 33. Pohl, W., Machine Learning for User Modeling. in Design of Computing Systems: Social and Ergonomic Considerations (Proceedings of the Seventh International Conference on Human Computer Interaction), (Elsevier, Amsterdam, 1997), 27 - 30. 34. Pohl, W. and Nick, A. Machine Learning and Knowledge Representation in the LaboUr Approach to User Modeling. 35. Richardson, T., Bennett, F., Mapp, G. and Hopper, A. Teleporting in an X Window System Environment. IEEE Pervasive Compunications Magazine, 1 (3). 6-12. 36. Shafer, S.A.N., Location Authorities for Ubiquitous Computing. in 2003 Workshop on Location-Aware Computing, (Seattle, Washington, 2003). 37. Shavlik, J.W. and Dietterich, T.G., General aspects of machine learning. in Readings in Machine Learning, (San Mateo, California, 1990), Morgan Kaufmann Publishers Inc, 1-10. 38. Wang, J. and Mankoff, J., Theoretical and Architectural Support for Input Device Adaption. in ACM Conference on Universal Usability, (Vancouver, British Columbia, Canada, 2003). - 33 -
  • 39. Want, R. and Hopper, A. Active Badges and Personal Interactive Computing Objects. IEEE Transactions on Consumer Electronics (February 1992). 40. Webb, G.I., Pazzani, M.J. and Billsus, D. Machine Learning for User Modeling. User Modeling and User-Adapted Interaction, 11. 19-29. 41. Weiser, M. The Computing for the 21st Century. Scientific American, 265 (3). 94-104. 42. Young, S., Evermann, G., Hain, T., Kershaw, D., Moore, G., Odell, J., Ollason, D., Povey, D., Valtchev, V. and Woodland, P. The HTK Book, 1999. 43. Zukerman, I. and Albrecht, D.W. Predictive Statistical Models for User Modeling. User Modeling and User-Adapted INteraction, 11. 5-18. - 34 -