2004-09-23 PM Event Detection from Time Series


Published on

Published in: Technology, Business
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

2004-09-23 PM Event Detection from Time Series

  1. 1. PM Event Detection from Time Series <ul><li>Contributed by the FASNET Community, Sep. 2004 </li></ul><ul><li>Correspondence to R Husar , R Poirot </li></ul><ul><li>Coordination Support by </li></ul><ul><li>Inter-RPO WG Fast Aerosol Sensing Tools for Natural Event Tracking, FASTNET </li></ul><ul><li>NSF Collaboration Support for Aerosol Event Analysis </li></ul><ul><li>NASA REASON Coop </li></ul><ul><li>EPA -OAQPS </li></ul>Event : Deviation > x*percentile
  2. 2. Temporal Analysis <ul><li>The time series for typical monitoring data are ‘messy’; the signal variation occurs at various scales and the time pattern at each scale is different </li></ul><ul><li>Inherently, aerosol events are spikes in the time series of monitoring data but extracting the spikes from the noisy data is a challenging endeavor </li></ul><ul><li>The temporal signal can be meaningfully decomposed into a </li></ul><ul><ul><li>Seasonal component with stable periodic pattern </li></ul></ul><ul><ul><li>Random variation with ‘white noise’ pattern </li></ul></ul><ul><ul><li>Spikes or events that are more random in frequency and magnitude </li></ul></ul><ul><ul><li>Each signal component is caused by different combination of the key processes: emission, transport, transformations and removal </li></ul></ul>Typical time series of daily AIRNOW PM25 over the Northeastern US
  3. 3. Temporal Signal Decomposition and Event Detection <ul><li>First, the median and average is obtained over a region for each hour/day (thin blue line) </li></ul><ul><li>Next, the data are temporally smoothed by a 30 day moving window (spatial median - red line; spatial mean – heavy blue line). These determine the seasonal pattern. </li></ul>EUS Daily Average 50%-ile, 30 day 50%-ile smoothing Deviation from %-ile Event : Deviation > x*percentile Median Seasonal Conc. Mean Seasonal Conc. Average Median <ul><li>Finally, the hourly/daily deviation from the the smooth median is used to determine the noise (blue) and event (red) components </li></ul>
  4. 4. Seasonal PM25 by Region <ul><li>The 30-day smoothing average shows the seasonality by region </li></ul><ul><li>The Feb/Mar PM25 peak is evident for the Northeast, Great Lakes and Great Plains </li></ul><ul><li>This secondary peak is absent in the South and West </li></ul>
  5. 5. Northeast – Southeast Comparison <ul><li>Northeast and Southeast differ in the pattern of seasonal and event variation </li></ul><ul><li>Northeast has two seasonal peaks and more events–values well above the median </li></ul><ul><li>Southeast peaks in September and has few values much above the noise </li></ul>Northeast Southeast
  6. 6. Causes of Temporal Variation by Region <ul><li>The temporal signal variation is decomposable into seasonal, meteorological noise and events </li></ul><ul><li>Assuming statistical independence, the three components are additive: </li></ul><ul><li>V 2 Total = V 2 Season + V 2 MetNoise + V 2 Event </li></ul><ul><li>The signal components have been determined for each region to assess the differences </li></ul>Northeast exhibits the largest coeff. variation (56%); seasonal, noise and events each at 30% Southeast is the least variable region (35%), with virtually no contribution from events Southwest, Northwest, S. Cal. and Great Lakes/Plains show 40-50% coeff. variation mostly, due to seasonal and meteorological noise. Interestingly, the noise is about 30% in all regions, while the events vary much more, 5-30%
  7. 7. ‘ Composition’ of Eastern US Events <ul><li>The bar-graph shows the various combinations of species-events that produce Reconstructed Fine Mass (RCFM) events </li></ul><ul><li>‘ Composition’ is defined in terms of co-occurrence of multi-species events (not by average mass composition) </li></ul><ul><li>The largest EUS RCFM events are simultaneously ‘events’ (spikes) in sulfate, organics and soil! </li></ul><ul><li>Some EUS RCFM events are events in single species, e.g. 7-Jul-97 (OC), 21-Jun-97 (Soil) </li></ul>Based on VIEWS data
  8. 8. Northeast
  9. 9. Great Lakes
  10. 10. Great Lakes-Plains
  11. 11. Northeast
  12. 12. Great Plains
  13. 13. NorthWest
  14. 14. S. California
  15. 15. Southeast
  16. 16. Southwest
  17. 17. Event Definition: Time Series Approach <ul><li>Eastern US aggregate time series </li></ul>
  18. 18. Sulfate EUS Daily Average 50%-ile, 30 day 50%-ile smoothing Deviation from %-ile Event – Deviation > percentile value Median Seasonal Conc. Mean Seasonal Conc.
  19. 19. Reconstructed Fine Mass RCFM
  20. 20. Organic Carbon
  21. 21. Eelemental Carbon
  22. 22. SOIL
  23. 23. Nitrate
  24. 24. Temporal Pattern Regional Speciated Analysis - VIEWS <ul><li>Aerosol species time series: </li></ul><ul><ul><li>ammSO4f </li></ul></ul><ul><ul><li>OCf </li></ul></ul><ul><ul><li>ECf </li></ul></ul><ul><ul><li>SOILf </li></ul></ul><ul><ul><li>ammNO3f </li></ul></ul><ul><ul><li>RCFM </li></ul></ul>Regions of Aggregation
  25. 25. Dust <ul><li>Seasonal + spikes </li></ul><ul><li>East – west events are independent </li></ul><ul><li>East events occur several times a year, mostly in summer </li></ul><ul><li>West events are lest frequent, mostly in spring </li></ul>US West East
  26. 26. Dust <ul><li>asgasgasfg </li></ul>Northeast Southwest Southeast
  27. 27. Dust <ul><li>dfjdjdfjetyj </li></ul>Northwest S. California Great Plaines
  28. 28. Amm. Sulfate <ul><li>wdthehreherh </li></ul>US West East
  29. 29. Amm. Sulfate <ul><li>stheherheyju </li></ul>Northeast Southwest Southeast
  30. 30. Amm. Sulfate <ul><li>shheherh </li></ul>Northwest S. California Great Plaines
  31. 31. Organic Carbon <ul><li>sdhdfhefheryj </li></ul>US West East
  32. 32. Organic Carbon <ul><li>sdheherh </li></ul>Northeast Southwest Southeast
  33. 33. Organic Carbon <ul><li>erheryeyj </li></ul>Northwest S. California Great Plaines
  34. 34. Reconstructed Fine Mass <ul><li>estrhertheryu </li></ul>US West East
  35. 35. Reconstructed Fine Mass <ul><li>werty3rueru </li></ul>Northeast Southwest Southeast
  36. 36. Reconstructed Fine Mass <ul><li>wthwrthwerhtr </li></ul>Northwest S. California Great Plaines