Download to read offline
thesis_background.ppt
- 1. Location Modeling andMachine Learning in Smart Environments Robert Whitaker Supervisor: A/Prof Judy Kay A/Prof Bob Kummerfeld
- 2. Overview Problem PreviousWork Possible Data Sources Tools Available Issues
- 3. Thesis Topic Exploreways of determining a persons current location and activity Explore ways of predicting a persons location/activity using Location Modeling and Machine Learning The results returned must be scrutable
- 4. Possible Situation Where’sBoris Scenario Wish to organize a meeting with another person where the time suits both parties
- 5. Possible Steps Contactthe person you wish to meet Both people would look at their schedules and negotiate a time Both parties agree on the time they are to meet
- 6. Possible Problems Oneof the persons schedule may be incomplete When you arrive at the meeting time the person is not there. Should you wait? Where is the person? What if you can’t connect the person to organise the meeting
- 7.
- 8. Previous Work ActiveBadge Project Lancaster Guide Project Doppelganger Activity Compass Project
- 9. Active Badge ProjectFirst Indoor positioning system Users wear badges to emit their location Applied to teleporting Active Bat project extended the basic concepts developed Source: Nigel Davies and Hans-Werner Gellersen Beyond Prototypes: Challenges in Deploying Ubiquitous Systems. IEEE Pervasive Computing , Volume 1 (Jan-March 2002). 26-35.
- 10. Lancaster Guide ProjectA tourist guide for the city of Lancaster Used tablet PC’s connected to a 802.11 network Limited by the infrastructure capabilities. Source: 1. Nigel Davies and Hans-Werner Gellersen Beyond Prototypes: Challenges in Deploying Ubiquitous Systems. IEEE Pervasive Computing , Volume 1 (Jan-March 2002). 26-35. 2. The Guide Project, http://www.guide.lancs.ac.uk
- 11. Lancaster Guide InterfaceSource: The Guide Project, http://www.guide.lancs.ac.uk
- 12. Doppelganger Generalized toolfor gathering, processing and providing information about users Learning Techniques Beta Distribution Linear Prediction Markov Models DopMail Source: Orwant, J., Heterogeneous Learning in the Doppelganger User Modeling System. in User Modeling and User-Adapted Interaction , (1995), 107-130.
- 13. Doppelganger Source: Orwant, J., Heterogeneous Learning in the Doppelganger User Modeling System. in User Modeling and User-Adapted Interaction , (1995), 107-130. Applications Beta Distribution Linear Prediction Markov Models Learning Toolbox Sensors
- 14. Activity Compass ProjectLocation Modeling to help disabled PDA device application developed to assist with location tracking Tracking movements and comparing them to a map Prediction algorithms used Relational Markov Models Source: Patterson, D.J., Etzioni, O. and Kautz, H. The Activity Compass, University of Washington, 2003.
- 15. Prototype of ActivityCompass Source: Patterson, D.J., Etzioni, O. and Kautz, H. The Activity Compass, University of Washington, 2003.
- 16. Possible Data SourcesBluetooth Devices Machine Learning Windows Based Unix Based
- 17. Tools Personis ElvinMessaging Bspy Markov Modeling Toolkits Manual Logs for Evaluation Purposes
- 18. Personis User modelingsoftware Accretion representation Consists of components which model aspects of the user Allows the user model to be scruntised Source: Kay, J., Kummerfeld, B. and Lauder, P., Managing private user models and shared personas. in Workshop on User Modelling for Ubiquitous Computing , (Pittsburgh, USA, 2003).
- 19. Example of UserModel Output from Personis: Modeling the locations where the user has been
- 20. Elvin Messaging Publish/SubscribeMessaging System Messages routed by content Application: sending messages between sensors and modeling software Source: Mantara Software Elvin Administrator's Guide , 2003. Elvin Router Client Client Client Client
- 21. Bspy Bluetooth positioningsystem Detects Bluetooth devices and logs them to a database Uses Elvin messages to send information from sensor to database
- 22.
- 23. Markov Modeling ToolkitsHidden Markov Modeling Package – Python Matlab Hidden Markov Package Markov Chain Algorithm Cambridge Markov Modeling Toolkit
- 24. Manual Logs Recordsactivity and location in 15 min blocks Provides some example data to develop the algorithms off Used for the evaluation of the learning algorithm
- 25.
- 26.
- 27. Research Issues Representationof location and activity Creation of data sets Modeling Time
- 28.
Editor's Notes
- #4 Three main aspects I wish to explore in my thesis
- #5 Boris carries a bluetooth enabled pda, Natasha is a student who comes to his office wanting to discuss a problem about her assignment in Boris’s Course. She consults the locator interface at the door to determine where he is, a voice message is recorded on what the meeting is about and the locator will give a response based on where Boris is, who is asking for him and what he might be doing. Examples busy at a computer, talking with pgrads in the room next door etc. In order to understand the problem a little better I have come up with a number of possible scenarios where it could be used one of which is organizing a meeting between several parties.
- #6 To analysis this scenario further It could be broken down into a number of steps which you would normally be expected to be performed You must make contact with the person who you wish to organize the meeting The meeting must work for both people therefore there must be some consultation by each party to some data source of what they will be doing in the future Both parties then need to agree on the time This is a very simple analysis of organizing a meeting, but in doing so there are a number of possible issues that could arise
- #7 By looking at these issues it can be seen how my work could be applied to the situation If people’s schedules are incomplete – you could use the ability of querying past movements and activities to work out times you are most likely to be available When you arrive at a meeting and the person is not there, should you wait – you could use the solution to find out where the person is currently, and furthermore you could ask it how long it thinks you should wait based on the persons movements and activities around the location at the given time What if there is no time to connect the person, or the person is on holidays or a junket and not contactable, well you could use the data stored in the system to find out when the person is most likely to be available and use this information to try and organize some possible meeting times without the other persons interaction (of course there would have to be eventually some confirmation by the person eventually) There are of course many more problems that could be thought up, just simply think how many meetings you have been to that get started late, cancelled etc. and then think of the reasons why this happened.
- #8 To summaries this in a diagram at a high level. We have out environment of smart devices sending all this raw data about the user (where they are, what they are doing etc) This raw data then must be sent to a central data store and saved for historical purposes Because raw data is not very meaningful we would like to perform some simple conversions on the data, for example we would like to convert the MAC address of bobs phone to the text “bobs phone” as it is much easier to understand from the human perspective Once these conversions have taken place we use all the data we have collected to build up models of the users location and activity, there is also the option of building up models of the sensors or rooms activities and users to gain another perspective on things Once the models have been created we pretty much have our data which can be used to start answering the many questions I proposed in the earlier scenario Then we have another user come along, he then asks questions of the system like “Where is X most likely to be in 1 hrs time?”, the system would then be required to query the user models and build up a result set, working out the answer to the users questions
- #9 There has been large amounts of work performed in the area, I have brought this down to four main projects which have made significant contributions to the area The first two projects are those which are purely about where the user currently is and how can user models as such be used to enhance the users experience. The second two projects are concerned not only on where the user currently is but also have the ability to predict where a user might be. All of these systems use user modeling to some degree, but the use of machine learning in these projects is very limited and thus will be a major focus of my work
- #10 Developed at AT&T Labs Cambridge and was the first indoor positioning system. It worked by having users wear badges which emit signals which could be used to determine a persons current location. The project was driven by the notion of how to locate people in their office. One application which this was applied to was the concept of teleporting – transporting the interfaces of a uses remote applications to the nearest terminal Further work was also carried out in the area in the form of the Active Bat project which is a 3D positioning system which built on the early work from the Active Badge results I am also looking for some pictures of this work if someone might have them it would be great
- #11 Designed the system to provide visitors to the city with the type of information normally found in a tour guide but based on the users interests and movement around the city. The project involved a large filed trial in the real world It used tablet PC as the end system connected to information servers through an 802.11 network deployed around the city It was found during the trials that the interaction was very much governed by the infrastructure capabilities as it was found that tourists not always went to areas supported by the system or wished to visit parts of the city which the system did not cover. The following is a screen shot of the interface which they developed
- #12 This is the interface of the system There are a lot more examples at the website at the bottom, including a demonstration on how it works
- #13 This was probably the first most significant project done in the field which incorporate user prediction, it dates back to Jon Orwant’s work in 1995 at MIT. It is a generalized tool for gathering, processing and providing information about users. It producing its results to the applications which use it, doppelganger uses a combination of learning techniques to determine the results Beta Distribution – When determining a users preferences for some feature or interest of a topic. Accumulates the responses over time and then uses beta distribution to analysis it. Linear Predictions – used for projecting a data set into the future based on past recordings. Eg. Finding out when a person is most likely to logon again based on when they logon and how often they are on for Markov Models – used for determining the physical location and working state of the user There were several applications developed for this research one was the customizing of electronic newspapers, but another was what was called DopMail which filtered and sorted electronic email based on the data collected from the user model
- #14 There were three parts to the doppelganger system Sensors – this is where data about the users was provided, this could have been hardware or software. Examples of sensors he used were UNIX commands like finger, and peoples use of an electronic newspaper, the sensor information collected was fed into the learning toolbox Learning Toolbox – this implemented the different learning techniques I just mentioned and basically formed the result sets which could be passed over to any application using it Applications which used the system such as the already mentioned examples like DopMail and Electronic newspapers.
- #15 This project is currently active at the University of Washington and deals with the application of user modeling and user predication to assist the disabled. The main purpose was to develop this device called the Activity Compass which could help alzamirs patients remember where they are going and what they are doing. The application pretty much worked out what activity the person was most likely to be doing, a scenario they gave was walking to the shops, if the person went left at an intersection instead of right the device would alert them and point them in the correct direction. Another aspect of this research incorporated some GPS and basically working out the current location of a person and referencing it back to a street map, a graph of this map was then created to assist in prediction and basically from a users movements and past historical data it could work out what path the user was most likely to take through the map, plus some logical common sense (eg. Cars drive on roads, buses stop at bus stops only) The prediction algorithms used were Relational Markov Models, this is also at University of Washington in the same research group, basically a relational markov model groups objects of the same type into relations with each relation described by its own set of variables.
- #16 This is basically a picture of a prototype of the activity compass From this you can see that it is telling the user that the way home is straight ahead
- #17 There will be pretty much two main source for data Bluetooth devices will be used to provide the location information about the users, where they currently are in the building Machine learning software on the various platforms will be used to gain a sense of what users are doing at their machines which could be used to further enhance the location information obtained through the Bluetooth devices Eg. If the location readings are picking Bob up near the fridge but really he is at his desk typing an email, the machine learning data could help resolve this location and activity better than if we just had the readings from the Bluetooth sensors
- #18 Basically so far I have been looking at a variety of tools which could be used to assist in the solution to the problem Personis – for user modeling BSpy – for working out the location of the user Markov Modeling – for the prediction of where users will be in the future Elvin Messages – for passing the raw data from the machine learning software or activity tracking software, just like is done in BSpy Manual Logs – these will be used to evaluate the accuracy of the other sections such as the learning algorithms etc.
- #19 Is a user modeling tool, uses the accretion representation by this it means it consists of components each of which models an arbitrary aspect of the user eg. Their personal details like name and date of birth. Each of these has a type but this representation makes no assumptions for any of these types. Evidence is collected on each component of the user model The model is continually added to, as new evidence comes in about the user it will be added to the evidence list for that user. By implementing it this way it allows a user to scruntise their user model by having a historical record of data collected and they can see what the system has held about them.
- #20 I have played around with the software a bit and have started coming up with some basic user models on the location data about a user. This is an example of the sought of models I will be creating about the location of a user. Similar models will have to be developed to model what activity the user is doing. Highlight: Value is the location the person was detected near Source: just put in praxis for now, but this could be the device which the reading was detected eg. Persons phone, pda, etc. The time at which the user was at the location
- #21 Transports simple structure messages from produces to consumers which have subscribed to receive messages that match a condition Clients establish sessions with an Elvin server process and are able to send notifications for delivery or register to receive notifications sent by others. Application programs use the Elvin client libraries to send and receive notifications from the Elvin router. The router process maintains subscriptions on behalf of the consumers and routes matching notifications to the consumer applications. The server will deliver the information based on the information they contain rather than the direction they are sent Application: can use Elvin messages to communicate between sensors and central servers. Example is like in the Bspy system which could easily be applied to machine learning system. Example when a sensor detects something it will create a fire off an Elvin message which is received by all those who have subscribed to it
- #22 Basically it is a system in the school which lets you work out where someone is who currently has a Bluetooth device on them. Each sensor simply listens for devices to enter its area then sends a message off to a database where the entry is logged. To send information between sensors and database it uses Elvin messages, this gives the benefit that anyone can subscribe to this information and get the location information, thus you can get the messages in real time rather than querying the database all the time
- #23 This is a screen shot of getting data out of the database Action 1.Present – means it was there last time the scan was done and is still there 2. Found – wasn’t there last time a scan was done but is there now 3. Lost – was there last time a scan was done but is not there now
- #24 This is the area I am currently looking into but basically so far I have found 4 different packages or tools I am reviewing A hidden markov modeling package for python – The benefit this package would give is its integration with the personis software, but however from what I have seen so far the documentation is pretty poor and looks a pretty hard package to understand Matlab package – there are several packages available, they offer simply solutions to the problem from the materials I have read Markov Chain Algorithm – very simple, very quick to implement, 20 lines of code, problem is it may not provide as accurate prediction benefits as the hidden markov packages Cambridge Toolkit – got more information on this after last weeks talk, but basically it is implemented in perl, comes with lots of documentation as well as tutorial examples to help you build up your own models. From talking to Richard last week he said it is a very easy package to learn and the example scripts usually give a foundation for any models being built
- #26 There are basically two sheets to this, one the sheet giving you codes or personalized abbreviations for various locations, the second is a log showing when there activities are to be performed. This is an example of the personalized code
- #27 This is an example of logging what activities are performed when. The codes in the location column match the previous page. This sheet is somewhat difficult to use, I am actually looking at writing a small standalone application for a PDA which will simply require the user to select the activity they are doing and this will be time stamped and stored in a file, the information could then be compared to the data from bspy to find out where the activity took place. I will also be needing volunteers towards the end of semester to log their activity with the system if anyone is also interested.
- #28 Represent locations as symbols rather than the raw data eg. Instead of saving the mac address of bob’s phone we store the text bob’s phone. Same goes for a room, instead of coordinates we could say g92, but you can also add extra levels based on who is asking such as if a student is looking for their lecturer it might say madsen building while if a pgrad is looking for their supurvisor (same person) it might say g92. The issue of privacy is the main problem here when collecting data Markov models model state moving from one state to another, need to associate state with time and moving from one state at a given time to a different state at a given time.