Where DSP meets UbiComp

674 views

Published on

A summary of ongoing research we have been working on at the University of Washington. I gave this in August 2011 at Georgia Tech. My take on how signal processing shapes the projects of UbiComp, turning a novel idea into a robust reality.

Published in: Technology
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
674
On SlideShare
0
From Embeds
0
Number of Embeds
14
Actions
Shares
0
Downloads
7
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Although energy usage and measurement has lately received a great deal of attention in the HCI and UbiComp communities, water has not. And yet, the United Nations predicts that water will be the dominating issue over the next 20 years. And, as you can tell from this map, this is a problem that affects every continent on earth including the US.----By 2025, more than 2.8 billion people living in 48 countries will face water shortages. Environmental water stress is assessed as the percentage of available water resources currently abstracted compared to environmental water requirements. Environmental water requirements are defined as the volume of water needed by a river to maintain key ecosystemfunctions and biodiversity. This is expressed as a % of the naturalised flow of a river, and can vary under different conditions, notably between arid and non-arid river systems. In Figure 2 the red depicts the systems under the highest environmental water stress.
  • hydrosense is a single screw on sensor that identifies water usage down to the fixture leveland provides estimates of water flow from each fixture
  • To understand how HydroSense works, its useful to go over a brief primer of home water/pipe infrastructure.Most households obtain water from a public water supply.Public water is distributed by local utilities, relying on gravity and pumping stations to push water through major distribution pipes
  • Cold water enters the home through a service line, typically at 40-100 pounds per square inch (psi) depending on such factors as the elevation and proximity to a water tower or pumping station.Pressure is important to the proper functioning of HydroSense because it’s a pressure-based sensing solution.------The pound per square inch or, more accurately, pound-force per square inch (symbol: psi or lbf/in² or lbf/in²) is a unit of pressure or of stress based on avoirdupois units. It is the pressure resulting from a force of one pound-force applied to an area of one square inch:1 psi (6.894757 kPa) : pascal (Pa) is the SI unit of pressure.40 psi is 275.79 kilopascals100 psi is 689.47 kilopascals
  • Cold water enters the home through a service line, typically at 40-100 pounds per square inch (psi) depending on such factors as the elevation and proximity to a water tower or pumping station.Pressure is important to the proper functioning of HydroSense because it’s a pressure-based sensing solution.------The pound per square inch or, more accurately, pound-force per square inch (symbol: psi or lbf/in² or lbf/in²) is a unit of pressure or of stress based on avoirdupois units. It is the pressure resulting from a force of one pound-force applied to an area of one square inch:1 psi (6.894757 kPa) : pascal (Pa) is the SI unit of pressure.40 psi is 275.79 kilopascals100 psi is 689.47 kilopascals
  • Many homes have a pressure regulator that stabilizes the water pressure and also reduces the incoming water pressure to a safe level for household fixtures.From the regulator, most homes contain a combination of series plumbed and branched piping.
  • The cold water supply branches to the individual water fixtures (e.g., toilets/sinks/showers) and into the water heater.
  • The plumbing system forms a closed loop pressure system with water held at a relatively stable pressure throughout the piping. This is why, when you open a faucet, water immediately flows out.
  • The hot water tank connects the cold water pipes to the hot water pipes.So when you open a hot or cold water valve, the pressure signal is transmitted through the hot water heater thus allowing for a single-point pressure sensing solution.Every hot water tank has a drain valve, which is of interest to us because it provides an easy potential installation point for our hydrosensor, particularly for locations like apartments which don’t have outdoor hose bibs.
  • The cold water supply branches to the individual water fixtures (e.g., toilets/sinks/showers) and into the water heater.The plumbing system forms a closed loop pressure system with water held at a relatively stable pressure throughout the piping. This is why, when you open a faucet, water immediately flows out.
  • The cold water supply branches to the individual water fixtures (e.g., toilets/sinks/showers) and into the water heater.The plumbing system forms a closed loop pressure system with water held at a relatively stable pressure throughout the piping. This is why, when you open a faucet, water immediately flows out.
  • The cold water supply branches to the individual water fixtures (e.g., toilets/sinks/showers) and into the water heater.The plumbing system forms a closed loop pressure system with water held at a relatively stable pressure throughout the piping. This is why, when you open a faucet, water immediately flows out.
  • The cold water supply branches to the individual water fixtures (e.g., toilets/sinks/showers) and into the water heater.The plumbing system forms a closed loop pressure system with water held at a relatively stable pressure throughout the piping. This is why, when you open a faucet, water immediately flows out.
  • The cold water supply branches to the individual water fixtures (e.g., toilets/sinks/showers) and into the water heater.The plumbing system forms a closed loop pressure system with water held at a relatively stable pressure throughout the piping. This is why, when you open a faucet, water immediately flows out.
  • Poiseuille’s relates flow rate to pressure drop in a pipe when the water flow is laminar. Laminar flow means smooth flow and is found at lower flow rates.
  • Blasuis’ Equation relates pressure drop to volumetric flow rate in the case when the water is turbulant. Turbulant flow occurs at higher flow rate. The relationship between flow rate and pressure drop is no longer linear. However it is still clear that there is a relationship between the two. Therefore change in pressure is good feature that relates to flow rate and is directly measureable from a pressure signal. I however selected stabilized pressure as the feature.
  • bath tubs, showers, kitchen sinks and bathroom sinks10 – 20 samples from each fixturevaried flow rate and temperature for each sample taken
  • Not all houses were nice and linear however. Here is an example of a house with very nonlinear response between stabilized pressure and flow rate. However it was noticed that if you take the log of the flow rate and plot that vs stabilized pressure…
  • Here is another house plotting the stabilized pressure vs the flow rate. Again it is fairly linear (however not as linear as the previous house) here I am showing all the data collected from the house not discimenated by fixture.
  • Now when separate the data by which water line the event occurred on you can see that hot water events are separate from cold water events are separate from both.
  • but also notice that there is a high frequency resonance in this wave.We can isolate that by subtracting the low pass filtered wave from the raw waveform and then taking the derivative. We save this template which isolates the falloff of the second resonance.
  • Here is the Max amplitude plotted vs flow rate from the same house as before. Those with high amplitudes tend to be cold events and those with small amplitudes tend to be hot events.
  • Looking at a representative house’s feature space it can be seen there is some separation among the different fixtures there fore given an unknown test sample when using E-kNN it is likely that many of the selected neighbors will actually come from test points fixture without actually needing to know which fixture that is. Therefore one gets the benefit of the within fixture model but doesn’t need to depend on a classifier which may not actually operate at 100%, in which case prediction could be made using completely wrong data.
  • assumes data is locally linearSimilar to kNN, want to select training data in the feature space near the test point Previous kNN methods focused on selecting the correct number of neighborsE-kNN selects the minimum number of neighbors that creates a convex hull about the test pointDistance is calculate using l2-normLinear Regression performed on selected training dataDue to low neighborhood size there is low bias on the dataThe variance is bounded because the training data surrounds the test pointE-kNN shown to work well in low dimensional problems
  • To show how these rates compare I will show versus the graph. The black line is the curve associated with the water meter. The red line shows the volume error for the E-kNN within Fixture model, the green lines show the error for the minimal calibration and the yellow bars show the error for typical flow rate analysis. The flow trace meter typically adds a +/- 1% error to the error associated with the inline meter. The Within model is comparable to the flow trace error while the minimal calibration is larger but still within 5% for flow rates greater then 0.5gpm.
  • To show how these rates compare I will show versus the graph. The black line is the curve associated with the water meter. The red line shows the volume error for the E-kNN within Fixture model, the green lines show the error for the minimal calibration and the yellow bars show the error for typical flow rate analysis. The flow trace meter typically adds a +/- 1% error to the error associated with the inline meter. The Within model is comparable to the flow trace error while the minimal calibration is larger but still within 5% for flow rates greater then 0.5gpm.
  • To motivate these transformations, we need to return to the plumbing system. When we activate a fixture in the plumbing system, the entire system responds instantaneously, like letting the air out of a balloon. Depending on where we activate the water in the system, different resonances will be excited – so it will be important which resonances are activated and, as you can see from these examples, how quickly those resonances die out over time.
  • how to know ground truth?
  • so in addition installing the hydrosense system, we need to install a wireless network of sensors at every fixture that uses water in the home so we can label the water usage we see from the hydrosense system.
  • and in the end the deployment sites looked like this, here is a subset of the sinks we instrumented
  • toilets
  • and showers. Notice that we also had to instrument the diverter valve to know whether the bath or shower was running, in addition to secondary shower handles.
  • here is a clothes washer. We also tied a thermistor on the drain valve of the washer in order to know whether the homeowner used a hot/cold or cold/cold cycle.
  • and even instrumenting things like the refidgerator water dispenser.
  • and at the end of the labeling process we ended up with 156 days of water use spread out among 5 deployment sites and almost 15000 water usage labels to evaluate our hydrosense system. One thing that was quite surpirsing was that 22% off all the pressure waves we collected were compound events, that is, more than one water source was on at the same time.
  • So we really had to go back to the drawing board and rethink our algorithmic approach from previous work. This giant equation is just a multiplication of a large number of probability scores. Our new algorithm borrows from Bayesian inference in speech recognition, instead of classifying vocal utterances as words, we classify pressure transients as openings and closings. And just like in speech recognition, which may use a grammar and a language model to help with classification, so do we. The grammar and language model enables us to look at additional features, which we call paired valve priors. I’ll go over each in turn.The first term here deals primarily with signal level characteristics, and the last three terms deal mostly with leveraging human behavior around water usage. We will start with the first term.
  • which makes up a portion of term I the other portion of term 1 is made from features of the signal, not templates.
  • the other portion of term 1 is made from features of the signal, not templates.the first feature we compare is the pressure drop of the event shown here for different events n the library. Pressure drop is basically an uncalibrated measure of flow rate. Since different valves have different flow rates, we can compare the unknown valve’s pressure drop to those we have in our database.
  • In the same way, we can explicitly model the frequency and magnitude of the excited resonances in the pressure wave and compare them across the library of events. We chose an autoregressive model, but others are possible.
  • which makes up a portion of term I the other portion of term 1 is made from features of the signal, not templates.
  • the first term uses a template matching approach, where we compare an unknown event to events that we have saved in a library.
  • in particular we make use of signal transformations And we compare the percent correlation between each transformation of the unknown event and the library of events. We use 8 transformations in all, but I will briefly mention four here.
  • in particular we make use of signal transformations And we compare the percent correlation between each transformation of the unknown event and the library of events. We use 8 transformations in all, but I will briefly mention four here.
  • so the signal transformations we chose are designed to highlight the fall off of different resonances.the first template we save is the raw waveform, detrended. next we calculate the derivative of the low frequency resonance using a low pass filter followed by a derivative operation.we save this template which highlights the falloff of the low frequency resonance
  • but also notice that there is a high frequency resonance in this wave.We can isolate that by subtracting the low pass filtered wave from the raw waveform and then taking the derivative. We save this template which isolates the falloff of the second resonance.
  • lastly we employ a frequency based transform called the cepstrum, which uses a set of filter banks and a log operation. The values of the cepstrum are largely influenced by where water is activated in the plumbing system, so we can expect it to stay more or less constant regardless of the flow rate at which the fixture is activated.
  • so we compare theses signal transforms of the unknown wave to saved transformations in our library. In this example, if we want to know the probability that the kitchen sink was the origin of this unknown wave, we can multiply all the transformations together to get a single probability.
  • so we compare theses signal transforms of the unknown wave to saved transformations in our library. In this example, if we want to know the probability that the kitchen sink was the origin of this unknown wave, we can multiply all the transformations together to get a single probability.
  • Finally, we can then combine the signal features and template features by multiplication to form term (i)
  • so an unknown pressure transient comes along, we can use term one to find the probability that the originator of this valve was the kitchen sink hot valve opening. We just combine all the signal features we have from the kitchen sink. And we can do this for every example valve in our library.
  • or we can be a little smarter when more than one pressure wave is present, and incorporate our second term, a language model, with the first term. so we first build a trellis of all the fixture probabilities using term 1. (shown here as blue dots) But then we use the language model to combine the terms using transition probabilities. That is, we know the probability of the each pressure wave going to the next. I will show a subset of those transition probabilities here. So we navigate the trellis using these transition probabilities.
  • or we can be a little smarter when more than one pressure wave is present, and incorporate our second term, a language model, with the first term. so we first build a trellis of all the fixture probabilities using term 1. (shown here as blue dots) But then we use the language model to combine the terms using transition probabilities. That is, we know the probability of the each pressure wave going to the next. I will show a subset of those transition probabilities here. So we navigate the trellis using these transition probabilities.
  • and we can choose the most likely “sequence” of pressure waves, not just the most likely pressure wave. And if we wanted we could stop there.
  • Or we could find the likelihood of the N best sequences through the trellis. Shown here for the entire sequence. Maybe the best probabilities are 4.6, 4.3 , and 4.1 % likelihood. But the trellis view makes it harder to see the n best paths, so instead let’s list out each sequence.
  • Or we could find the likelihood of the N best sequences through the trellis. Shown here for the entire sequence. Maybe the best probabilities are 4.6, 4.3 , and 4.1 % likelihood. But the trellis view makes it harder to see the n best paths, so instead let’s list out each sequence.
  • Or we could find the likelihood of the N best sequences through the trellis. Shown here for the entire sequence. Maybe the best probabilities are 4.6, 4.3 , and 4.1 % likelihood. But the trellis view makes it harder to see the n best paths, so instead let’s list out each sequence.
  • Now we can add our third term, since we have a language for valve usage, we can also have a grammar. Similar to the grammar in speech recognition, there is a grammar for the way pressure waves should be combined.
  • the grammar we created is simple:if we come across a fixture open event in the sequence, like a kitchen sink, we should see a kitchen sink close later on the sequence.if we come across a fixture close event, we should have already seen an open event in the sequence.and lastly the open and close should have consistent temperatures, so if the hot valve is opened in the kitchen, it should be closed by a hot valve. we apply this grammar with a soft penalty -> we do not eliminate sequences that contain impossible valve impairings, instead we impose a penalty on those sequencesfor each grammatical mistake.
  • so we identify errors in the sequences, apply a penalty to the probability of the sequence and reorder the sequences. based on their new probabilities.now everything that was not penalized by the grammar can be paired together. So the bathroom sink can be paired the shower, and son on. This is a vital piece of knowledge that allows us to incorporate our last term which we call paired valve priors.
  • so we identify errors in the sequences, apply a penalty to the probability of the sequence and reorder the sequences. based on their new probabilities.now everything that was not penalized by the grammar can be paired together. So the bathroom sink can be paired the shower, and son on. This is a vital piece of knowledge that allows us to incorporate our last term which we call paired valve priors.
  • so we identify errors in the sequences, apply a penalty to the probability of the sequence and reorder the sequences. based on their new probabilities.now everything that was not penalized by the grammar can be paired together. So the bathroom sink can be paired the shower, and son on. This is a vital piece of knowledge that allows us to incorporate our last term which we call paired valve priors.
  • so we identify errors in the sequences, apply a penalty to the probability of the sequence and reorder the sequences. based on their new probabilities.now everything that was not penalized by the grammar can be paired together. So the bathroom sink can be paired the shower, and son on. This is a vital piece of knowledge that allows us to incorporate our last term which we call paired valve priors.
  • so we no longer need to think just about pressure waves, but instead, because waves are paired together, we can think about fixture usage. .for example, duration of use. If we look at histograms for the duration of a faucet use, its different than a toilet which is different than a bath.in the same way estimated volume of water used should be different for each fixture. Pairing pressure waves allows us to incorporate these probabilities
  • and reorder the list for each paired event.and finally choose the most likely event from the list.
  • and reorder the list for each paired event.and finally choose the most likely event from the list.
  • and reorder the list for each paired event.and finally choose the most likely event from the list.And we see that this is the example we saw from before.
  • but we can break the results down by granularity of the sensing. we can talk about results at the valve level, so knowing the exact bathroom sink hot valve that was activated.or at the fixture level, so knowing that the bathroom faucet was activated, but we don’t care about the temperature state. If you do activity inference, this might be your main interest because it gives you location.and finally we can talk about the fixture category level – so knowing only that a faucet was activated, but not where. For sustainability you might be interested in one or all of these levels of accuracy.
  • here are the results shown only for using all terms. Recall that we installed two pressure sensors
  • The addition of a second sensor resulted in a marginal to medium (but significant) increase in classification accuracies across the board, surpassing 80% at the valve level all the way up to almost 98% at the category level.But lastly let’s delve a little further into the 90% accuracy at the fixture level.
  • and reorder the list for each paired event.and finally choose the most likely event from the list.
  • Where DSP meets UbiComp

    1. 1. Where signal processing meets ubiquitous computing: the role of DSP in the vision of UbiComp;<br />Sustainability, Health, and User Interfaces<br />Eric Larson<br />
    2. 2. 2006<br />2011<br />2007<br />2008<br />2009<br />2010<br />Graduated <br />Oklahoma State University<br />Signals, Systems, Controls<br />1st Paper: on image texture analysis<br />3rd Paper: on predicting image quality<br />(shift in interests…)<br />5th Paper: on evolutionary computation / facial analysis<br />MS Thesis: Subjective Image Quality Prediction<br />Started at University of Washington<br />time series analysis…<br />HydroSense<br />User Interface and Touch Sensing<br />Mobile Health Sensing<br />
    3. 3. the DSP student<br />mathematics<br />Religion:<br />Language:<br />MATLAB, C++<br />Hobbies:<br />-filtering<br />-statistics<br />-doing FFTs by hand<br /> -finding applications…<br />
    4. 4. the UbiComp student<br />passion for UbiComp<br />Religion:<br />Language:<br />everything but MATLAB<br />Hobbies:<br />-writing code in five<br /> languages<br />-soldering<br />-machine learning<br />-user studies <br />-picking an app…<br />toaster?<br />
    5. 5.
    6. 6. water fixture<br />classification<br />TED <br />for water<br />health sensing<br />from phone<br />user interface sensing<br />
    7. 7. TED <br />for water<br />water fixture<br />classification<br />health sensing<br />from phone<br />user interface sensing<br />
    8. 8. water scarcity<br />
    9. 9.
    10. 10. water<br />
    11. 11. hydrosense<br /><ul><li>single, screw-on sensor
    12. 12. senses pressure in real time</li></ul>Froehlich et al., UbiComp2009; Larson et al., PMC2010<br />
    13. 13. brief plumbing tutorial<br />
    14. 14. water tower<br />water tower<br />plumbing primer<br />
    15. 15. water tower<br />water tower<br />plumbing primer<br />incoming cold water from supply line<br />
    16. 16. water tower<br />water tower<br />pressure regulator<br />incoming cold water from supply line<br />pressure regulator<br />utility water meter<br />
    17. 17. water tower<br />water tower<br />pressure regulator<br />incoming cold water from supply line<br />pressure regulator<br />utility water meter<br />
    18. 18. water tower<br />water tower<br />plumbing layout<br />incoming cold water from supply line<br />pressure regulator<br />utility water meter<br />
    19. 19. water tower<br />water tower<br />closed pressure system<br />bathroom 1<br />hose<br />spigot<br />kitchen<br />incoming cold water from supply line<br />dishwasher<br />thermal <br />expansion <br />tank<br />pressure regulator<br />utility water meter<br />hot <br />water heater<br />bathroom 2<br />laundry<br />
    20. 20. water tower<br />water tower<br />closed pressure system<br />bathroom 1<br />hose<br />spigot<br />kitchen<br />incoming cold water from supply line<br />dishwasher<br />thermal <br />expansion <br />tank<br />pressure regulator<br />utility water meter<br />hot <br />water heater<br />hot <br />water heater<br />bathroom 2<br />laundry<br />
    21. 21. water tower<br />water tower<br />toilet<br />toilet flushed<br />bathroom 1<br />hose<br />spigot<br />kitchen<br />incoming cold water from supply line<br />dishwasher<br />thermal <br />expansion <br />tank<br />pressure regulator<br />utility water meter<br />hot <br />water heater<br />bathroom 2<br />laundry<br />
    22. 22. water tower<br />water tower<br />toilet<br />bathroom 1<br />kitchen sink cold<br />hose<br />spigot<br />kitchen sink cold open<br />kitchen<br />incoming cold water from supply line<br />dishwasher<br />thermal <br />expansion <br />tank<br />pressure regulator<br />utility water meter<br />hot <br />water heater<br />bathroom 2<br />laundry<br />
    23. 23. water tower<br />water tower<br />toilet<br />bathroom 1<br />kitchen sink cold<br />hose<br />spigot<br />kitchen sink hot open<br />kitchen sink hot<br />kitchen<br />incoming cold water from supply line<br />dishwasher<br />thermal <br />expansion <br />tank<br />pressure regulator<br />utility water meter<br />hot <br />water heater<br />bathroom 2<br />laundry<br />
    24. 24. water tower<br />water tower<br />bathroom 1<br />hose<br />spigot<br />kitchen sink hot open<br />kitchen sink hot<br />kitchen<br />incoming cold water from supply line<br />dishwasher<br />thermal <br />expansion <br />tank<br />pressure regulator<br />utility water meter<br />hot <br />water heater<br />bathroom 2<br />laundry<br />
    25. 25. water tower<br />water tower<br />bathroom 1<br />hose<br />spigot<br />kitchen<br />incoming cold water from supply line<br />dishwasher<br />pressure regulator<br />utility water meter<br />hot <br />water heater<br />bathroom 2<br />laundry<br />
    26. 26. bathroom sink pressure signal<br />psi<br />open<br />close<br />0<br />4.5<br />9<br />time (s)<br />
    27. 27. bathroom sink pressure signal<br />psi<br />flow volume<br />open<br />close<br />0<br />4.5<br />9<br />time (s)<br />
    28. 28. Poiseuille’s Equation<br />pressure drop of laminar water flow in a pipe:<br />Q = Volumetric Flow Rate<br />ΔP = Pressure Drop<br />r = Pipe Radius<br />l = Pipe Length<br />µ = Fluid Viscosity<br />
    29. 29. Blasius’ Equation<br />Q = Volumetric Flow Rate<br />ΔP = Pressure Drop<br />r = Pipe Radius<br />L = Pipe Length<br />Re = Reynolds Number<br />ρ = Fluid Density<br />pressure drop of turbulent water flow in a pipe:<br />
    30. 30. Bucket Trials<br /><ul><li>9 Homes
    31. 31. 41 Fixtures
    32. 32. 541 trials</li></li></ul><li>Stabilized Pressure vs Flow Rate Residence 6<br />Pressure (psi)<br />Flow Rate (gpm)<br />
    33. 33. Stabilized Pressure vs Flow Rate Residence 7<br />Stabilized Pressure vs Flow Rate Residence 7<br />
    34. 34. Stabilized Pressure vs Flow Rate Residence 7<br />
    35. 35. a new feature<br />max resonance<br />raw pressure (psi)<br />smoothed pressure<br />(psi)<br />max amplitude<br />bandpass<br />derivative<br />(psi/s)<br />time (s)<br />
    36. 36. Max Amplitude vs Flow Rate Residence 7<br />
    37. 37. Kitchen Sink<br />Basement Sink<br />max amplitude<br />Basement Shower<br /> Upstairs Shower<br />Bathroom Sink<br />Upstairs Bath<br />stable pressure (psi)<br />
    38. 38. e-kNN<br />convex hull<br />resonance<br />feature 2<br />test point<br />linear regression<br />feature 1<br />pressure<br />*Garcia, Gupta, and Chen (2008)<br />
    39. 39. e-kNN results<br />median error: within 0.15 gpm<br />80% within 0.5 gpm<br />
    40. 40. volume prediction<br />% of actual volume<br />gpm<br />Neptune T-10 Water meter<br />error for e-kNN(10-fold cross val.)<br />error E-kNN only kitchen + bath<br />
    41. 41. volume prediction<br />% of actual volume<br />gpm<br /> Meter response of Neptune T-10 Water meter<br />Error for e-kNN for each fixture<br />Error E-kNN only kitchen + bath<br />Error of Typical Flow Trace Meter<br />
    42. 42. Time Invariance of Training Data<br />
    43. 43.
    44. 44. hotvscold<br /><ul><li>max resonance only
    45. 45. ICA
    46. 46. median error: 0.2 gpm
    47. 47. 75th percentile: 0.42 gpm</li></li></ul><li>TED <br />for water<br />water fixture<br />classification<br />health sensing<br />from phone<br />user interface sensing<br />
    48. 48. today‘s usage<br />shower62.4 gallons<br />bathroom sink 14.2 gallons<br />bathroom sink 20.8 gallons<br />bath<br />6.5 gallons<br />toilet<br />78.4 gallons<br />
    49. 49. today‘s usage: hotvscold<br />shower52.4 gallons<br />shower10.4 gallons<br />bathroom sink 13.2 gallons<br />bathroom sink 11.2 gallons<br />bathroom sink 20.8 gallons<br />bath<br />0.0 gallons<br />bath<br />6.5 gallons<br />bathroom sink 22.4 gallons<br />toilet<br />78.4 gallons<br />
    50. 50. hydrosense<br />bath open<br />example pressure waves<br />upstairs toilet flush<br />downstairs toilet flush<br />kitchen sink <br />hot open<br />downstairs shower open<br />dishwasher open<br />kitchen sink <br />cold open<br />kitchen sink <br />hot open<br />
    51. 51. natural water use<br />70<br />50<br />pressure (psi)<br />30<br />toilet<br />kitchen sink<br />bathroom sink<br />kitchen sink<br />
    52. 52. data collection<br />water tower<br />bathroom 1<br />hose<br />spigot<br />kitchen<br />incoming cold water from supply line<br />dishwasher<br />thermal <br />expansion <br />tank<br />pressure regulator<br />utility water meter<br />hot <br />water heater<br />bathroom 2<br />laundry<br />
    53. 53.
    54. 54.
    55. 55.
    56. 56.
    57. 57.
    58. 58. 5-week dataset<br />
    59. 59.
    60. 60. algorithm: bayesianinference<br />signal<br />behavior<br />
    61. 61. term(i) signal features<br />shower<br />event library<br />unclassified event<br />toilet<br />bath<br />
    62. 62. term(i) signal features<br />pressure drop=flow rate<br />shower<br />unclassified event<br />toilet<br />toilet<br />bath<br />
    63. 63. term(i) signal features<br />resonance tracking<br />shower<br />unclassified event<br />toilet<br />toilet<br />bath<br />
    64. 64. unclassified event<br />event library<br />kitchen faucet<br />dishwasher<br />toilet<br />unclassified event<br />signal feature comparisons<br />detrended<br />detrended<br />55%<br />derivative<br />derivative<br />85%<br />bandpass derivative<br />bandpass derivative<br />signal transforms<br />65%<br />cepstrum<br />cepstrum<br />90%<br />
    65. 65. kitchen faucet<br />term(i) template matching<br />dishwasher<br />toilet<br />shower<br />unclassified event<br />event library<br />compare across multiple signal transformations<br />
    66. 66. detrendedshower<br />kitchen faucet<br />unclassified event<br />event library<br />derivativeshower<br />dishwasher<br />toilet<br />shower<br />cepstrumshower<br />unclassified event<br />signal transforms<br />
    67. 67. kitchen faucet<br />unclassified event<br />event library<br />dishwasher<br />toilet<br />shower<br />unclassified event<br />assess similarity<br />15%<br />55%<br />signal transforms<br />18%<br />9%<br />
    68. 68. signal transforms<br />
    69. 69. term(i) template matching<br />filter transforms<br />raw pressure (psi)<br />detrended<br />derivative<br />smoothed pressure<br />(psi)<br />signal transforms<br />derivative<br />(psi/s)<br />time (s)<br />
    70. 70. term(i) template matching<br />filter transforms<br />raw pressure (psi)<br />detrended<br />derivative<br />smoothed pressure<br />(psi)<br />bandpass derivative<br />signal transforms<br />bandpass<br />derivative<br />(psi/s)<br />time (s)<br />
    71. 71. term(i) template matching<br />filter transforms<br />
    72. 72. term(i) template matching<br />transforms: const-Q<br />
    73. 73. term(i) template matching<br />filter transforms<br />raw pressure (psi)<br />detrended<br />derivative<br />frequency<br />bandpass derivative<br />signal transforms<br />cepstral<br />amplitude<br />const-Q cepstrum<br />index<br />
    74. 74. unclassified event<br />event library<br />kitchen faucet<br />dishwasher<br />shower<br />toilet<br />unclassified event<br />detrended<br />derivative<br />bandpass derivative<br />signal transforms<br />const-Q cepstrum<br />
    75. 75. unclassified event<br />event library<br />kitchen faucet<br />dishwasher<br />toilet<br />unclassified event<br />detrended<br />detrended<br />55%<br />derivative<br />derivative<br />85%<br />bandpass derivative<br />bandpass derivative<br />signal transforms<br />65%<br />const-Q cepstrum<br />const-Q cepstrum<br />90%<br />
    76. 76. term(i) signal features<br />resonance tracking<br />shower<br />unclassified event<br />signal feature comparisons<br />toilet<br />toilet<br />template comparisons<br />bath<br />
    77. 77. term (i): templates and signal features <br />70<br />pressure<br />50<br />P(kitchen sink hot open)<br />P(kitchen sink cold open)<br />30<br />P(toilet open)<br />14%<br />3%<br />15%<br />2%<br />6%<br />1%<br />P(kitchen hot/cold close)<br />P(kitchen hot close)<br />P(toilet close)<br />
    78. 78. term (i): templates and signal features <br />70<br />pressure<br />50<br />P(kitchen sink hot open)<br />P(kitchen sink cold open)<br />30<br />P(toilet open)<br />14%<br />3%<br />15%<br />2%<br />6%<br />1%<br />P(kitchen hot/cold close)<br />P(kitchen hot close)<br />P(toilet close)<br />
    79. 79. term (ii): bigram language model<br />70<br />pressure<br />50<br />P(kitchen sink hot open)<br />P(kitchen sink cold open)<br />30<br />P(toilet open)<br />14%<br />3%<br />15%<br />2%<br />6%<br />1%<br />P(kitchen hot/cold close)<br />P(kitchen hot close)<br />P(toilet close)<br />
    80. 80. term (ii): bigram language model<br />70<br />pressure<br />50<br />P(kitchen sink hot open)<br />P(kitchen sink cold open)<br />30<br />P(toilet open)<br />14%<br />3%<br />15%<br />2%<br />6%<br />1%<br />P(kitchen hot/cold close)<br />P(kitchen hot close)<br />P(toilet close)<br />
    81. 81. term (ii): bigram language model<br />70<br />pressure<br />50<br />P(kitchen sink hot open)<br />P(kitchen sink cold open)<br />30<br />P(toilet open)<br />14%<br />3%<br />15%<br />2%<br />6%<br />1%<br />P(kitchen hot/cold close)<br />P(kitchen hot close)<br />P(toilet close)<br />
    82. 82. term (ii): bigram language model<br />70<br />pressure<br />50<br />P(kitchen sink hot open)<br />P(kitchen sink cold open)<br />30<br />P(toilet open)<br />14%<br />3%<br />15%<br />2%<br />6%<br />1%<br />P(kitchen hot/cold close)<br />P(kitchen hot close)<br />P(toilet close)<br />
    83. 83. term (ii): bigram language model<br />70<br />pressure<br />50<br />P(kitchen sink hot open)<br />P(kitchen sink cold open)<br />4.6%<br />30<br />P(toilet open)<br />14%<br />3%<br />15%<br />2%<br />6%<br />1%<br />sequence 1<br />4.3%<br />sequence 2<br />P(kitchen hot/cold close)<br />4.1%<br />P(kitchen hot close)<br />sequence 3<br />P(toilet close)<br />
    84. 84. term (ii): bigram language model<br />70<br />pressure<br />50<br />kitchen sink <br />hot open<br />dishwasher <br />open<br />bathroom sink<br />hot close<br />kitchen sink<br />hot close<br />kitchen sink<br />hot close<br />kitchen sink <br />cold open<br />toilet<br />close<br />toilet<br />open<br />sequence 1<br />kitchen sink <br />hot open<br />toilet<br />open<br />bathroom sink<br />hot close<br />kitchen sink<br />hot close<br />shower <br />cold open<br />kitchen sink<br />hot close<br />kitchen sink<br />hot close<br />30<br />kitchen sink<br />hot open<br />sequence 2<br />kitchen sink <br />hot open<br />bathroom sink<br />hot open<br />bathroom sink<br />hot close<br />kitchen sink<br />hot close<br />toilet<br />open<br />kitchen sink<br />hot close<br />kitchen sink<br />hot open<br />toilet <br />close<br />sequence 3<br />
    85. 85. term(iii): grammar<br />opened closed<br />openedclosed<br />1<br />2<br />temperature consistency<br />3<br />soft penalty<br />
    86. 86. term(iii): grammar<br />70<br />pressure<br />50<br />kitchen sink <br />hot open<br />dishwasher <br />open<br />bathroom sink<br />hot close<br />kitchen sink<br />hot close<br />kitchen sink<br />hot close<br />kitchen sink <br />cold open<br />toilet<br />close<br />toilet<br />open<br />sequence 1<br />bathroom sink <br />hot open<br />toilet<br />open<br />bathroom sink<br />hot close<br />shower<br />cold close<br />shower <br />cold open<br />kitchen sink<br />hot close<br />toilet<br />close<br />30<br />kitchen sink<br />hot open<br />sequence 2<br />kitchen sink <br />hot open<br />bathroom sink<br />hot open<br />bathroom sink<br />hot close<br />kitchen sink<br />hot close<br />toilet<br />open<br />kitchen sink<br />hot close<br />kitchen sink<br />hot open<br />toilet <br />close<br />sequence 3<br />
    87. 87. term(iii): grammar<br />70<br />pressure<br />50<br />kitchen sink <br />hot open<br />dishwasher <br />open<br />bathroom sink<br />hot close<br />kitchen sink<br />hot close<br />kitchen sink<br />hot close<br />kitchen sink <br />cold open<br />toilet<br />close<br />toilet<br />open<br />bathroom sink <br />hot open<br />toilet<br />open<br />bathroom sink<br />hot close<br />shower<br />cold close<br />shower <br />cold open<br />kitchen sink<br />hot close<br />toilet<br />close<br />30<br />kitchen sink<br />hot open<br />kitchen sink <br />hot open<br />bathroom sink<br />hot open<br />bathroom sink<br />hot close<br />kitchen sink<br />hot close<br />toilet<br />open<br />kitchen sink<br />hot close<br />kitchen sink<br />hot open<br />toilet <br />close<br />
    88. 88. term(iii): grammar<br />70<br />pressure<br />50<br />kitchen sink <br />hot open<br />dishwasher <br />open<br />bathroom sink<br />hot close<br />kitchen sink<br />hot close<br />kitchen sink<br />hot close<br />kitchen sink <br />cold open<br />toilet<br />close<br />toilet<br />open<br />bathroom sink <br />hot open<br />toilet<br />open<br />bathroom sink<br />hot close<br />shower<br />cold close<br />shower <br />cold open<br />kitchen sink<br />hot close<br />toilet<br />close<br />30<br />kitchen sink<br />hot open<br />kitchen sink <br />hot open<br />bathroom sink<br />hot open<br />bathroom sink<br />hot close<br />kitchen sink<br />hot close<br />toilet<br />open<br />kitchen sink<br />hot close<br />kitchen sink<br />hot open<br />toilet <br />close<br />
    89. 89. term(iii): grammar<br />70<br />pressure<br />50<br />kitchen sink <br />hot open<br />dishwasher <br />open<br />bathroom sink<br />hot close<br />kitchen sink<br />hot close<br />kitchen sink<br />hot close<br />kitchen sink <br />cold open<br />toilet<br />close<br />toilet<br />open<br />bathroom sink <br />hot open<br />toilet<br />open<br />bathroom sink<br />hot close<br />shower<br />cold close<br />shower <br />cold open<br />kitchen sink<br />hot close<br />toilet<br />close<br />30<br />kitchen sink<br />hot open<br />kitchen sink <br />hot open<br />bathroom sink<br />hot open<br />bathroom sink<br />hot close<br />kitchen sink<br />hot close<br />toilet<br />open<br />kitchen sink<br />hot close<br />kitchen sink<br />hot open<br />toilet <br />close<br />
    90. 90. term(iv): paired valve priors<br />paired<br />estimated flow volume<br />close<br />open<br />toilet<br />bin frequency<br />toilet<br />bath<br />faucet<br />bath<br />faucet<br />120<br />90<br />60<br />6<br />30<br />150<br />9<br />12<br />3<br />1<br />seconds<br />estimated gallons<br />fixture usage duration<br />flow volume <br />
    91. 91. term(iv): paired valve priors<br />70<br />pressure<br />50<br />kitchen sink <br />hot open<br />dishwasher <br />open<br />bathroom sink<br />hot close<br />kitchen sink<br />hot close<br />kitchen sink<br />hot close<br />kitchen sink <br />cold open<br />toilet<br />close<br />toilet<br />open<br />bathroom sink <br />hot open<br />toilet<br />open<br />bathroom sink<br />hot close<br />shower<br />cold close<br />shower <br />cold open<br />kitchen sink<br />hot close<br />toilet<br />close<br />30<br />kitchen sink<br />hot open<br />kitchen sink <br />hot open<br />bathroom sink<br />hot open<br />bathroom sink<br />hot close<br />kitchen sink<br />hot close<br />toilet<br />open<br />kitchen sink<br />hot close<br />kitchen sink<br />hot open<br />toilet <br />close<br />
    92. 92. term(iv): paired valve priors<br />70<br />pressure<br />50<br />kitchen sink <br />hot open<br />dishwasher <br />open<br />bathroom sink<br />hot close<br />kitchen sink<br />hot close<br />kitchen sink<br />hot close<br />kitchen sink <br />cold open<br />toilet<br />close<br />toilet<br />open<br />bathroom sink <br />hot open<br />toilet<br />open<br />bathroom sink<br />hot close<br />shower<br />cold close<br />shower <br />cold open<br />kitchen sink<br />hot close<br />toilet<br />close<br />30<br />kitchen sink<br />hot open<br />kitchen sink <br />hot open<br />bathroom sink<br />hot open<br />bathroom sink<br />hot close<br />kitchen sink<br />hot close<br />toilet<br />open<br />kitchen sink<br />hot close<br />kitchen sink<br />hot open<br />toilet <br />close<br />
    93. 93. term(iv): paired valve priors<br />70<br />pressure<br />50<br />30<br />kitchen sink<br />kitchen sink<br />bathroom sink<br />toilet<br />kitchen sink <br />hot open<br />bathroom sink<br />hot open<br />bathroom sink<br />hot close<br />kitchen sink<br />hot close<br />toilet<br />open<br />kitchen sink<br />hot close<br />kitchen sink<br />hot open<br />toilet <br />close<br />
    94. 94. three levels of granularity<br />valve level<br />e.g., upstairs bathroom faucet hot water activated<br />1<br />fixture level<br />e.g., upstairs bathroom faucet activated<br />2<br />fixture categorylevel<br />e.g., faucet activated<br />3<br />
    95. 95. hydrosense classification results<br />real-world water usage data<br />one sensor, terms(i)-(iv)<br />fixture category<br />fixture<br />valve<br />*10-fold cross validation, 15000 events<br />*error bars = std error<br />
    96. 96. hydrosense classification results<br />real-world water usage data<br />two sensors, terms(i)-(iv)<br />one sensor, terms(i)-(iv)<br />fixture category<br />fixture<br />valve<br />*10-fold cross validation, 15000 events<br />*error bars = std error<br />
    97. 97. …<br />brushing teeth<br />shaving<br />70<br />…<br />hand washing<br />washing dishes<br />50<br />kitchen sink <br />hot open<br />dishwasher <br />open<br />bathroom sink<br />hot close<br />kitchen sink<br />hot close<br />kitchen sink<br />hot close<br />kitchen sink <br />cold open<br />toilet<br />close<br />toilet<br />open<br />bathroom sink <br />hot open<br />toilet<br />open<br />bathroom sink<br />hot close<br />shower<br />cold close<br />shower <br />cold open<br />kitchen sink<br />hot close<br />toilet<br />close<br />30<br />kitchen sink<br />hot open<br />kitchen sink <br />hot open<br />bathroom sink<br />hot open<br />bathroom sink<br />hot close<br />kitchen sink<br />hot close<br />toilet<br />open<br />kitchen sink<br />hot close<br />kitchen sink<br />hot open<br />toilet <br />close<br />pressure<br />
    98. 98.
    99. 99.
    100. 100. TED <br />for water<br />water fixture<br />classification<br />health sensing<br />from phone<br />user interface sensing<br />
    101. 101. counting coughs<br />
    102. 102. Sensing<br />Self Report <br />
    103. 103. data collection<br />6 linguistic students<br />annotate each sound type<br />Go back to daily routine for 3-7 hours<br />Coughing?<br />4 weeks<br />pay attention to your cough frequency<br />Come back and self-report cough frequency<br />Come to Lab<br />One week pilot and set up guideline and shared wiki<br />data annotation<br />
    104. 104.
    105. 105. throat-clearing<br />coughs<br />noise<br />speech + laughter<br />sniff<br />speech <br />20<br />15<br />10<br />5<br />20<br />0<br />-20<br />-40<br />-60<br />-80<br />-100<br />Frequency (kHz)<br />Mag. (dB)<br />0 2 4 6 8 10 12<br />Time (s)<br />
    106. 106. <ul><li>five-step process:
    107. 107. initial deep inspiration
    108. 108. glottal closure
    109. 109. contraction of muscles against the glottis
    110. 110. sudden glottis opening, explosive expiration
    111. 111. wheeze or “personal” sound</li></li></ul><li>PCA<br />tree bagging<br />decision<br />48000<br />features<br />13<br />features<br />
    112. 112.
    113. 113. future work<br />mySpirometer App, 2009<br /><ul><li>spirometry from a mobile phone
    114. 114. working with children’s hospital in seattle</li></li></ul><li>
    115. 115. TED <br />for water<br />water fixture<br />classification<br />health sensing<br />from phone<br />user interface sensing<br />
    116. 116.
    117. 117. Solar Reflection<br />Thermal Radiation<br />Infrared<br />UV<br />Visible<br />Thermal<br />RGB<br />Depth<br />Source <br />Intensity<br />Near<br />Far Infrared<br />0.1um<br />1um<br />10 um<br />100 um<br />Wavelength<br />
    118. 118. hand prints<br />strokes<br />chording<br />multi-user<br />pressure<br />curves<br />
    119. 119.
    120. 120. Jon Froehlich<br />Tim Campbell<br />Sean Liu<br />Sidhant Gupta<br />Eric Swanson<br />Tien-Jui Lee<br />Elliot Saba<br />Gabe Cohn<br />
    121. 121. Where signal processing meets ubiquitous computing: the role of DSP in the vision of UbiComp;<br />Sustainability, Health, and User Interfaces<br />Eric Larson<br />eclarson@uw.edu<br />ubicomplab.cs.wahington.edu<br />@ericcooplarson<br />

    ×