"Indexing and Searching in Wireless Sensor Networks"


Published on

1 Like
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • There are many problems ranging from: Scheduling algorithms for putting sensors in sleep mode, to efficient mac prototocols, security protocols, operating systems, programming languages, databases, etc
  • A Circuit board equipped with a low frequency processor, main memory, microphone, buzzer, on-board and off-board flash memory
  • Seismic and Structural Monitoring: Harvard conducted studies at various volcanoes in Ecuador Princeton in Kenya, Berkeley on the Great Duck Island and the Golden Gate Bridge, etc.
  • Bird Nests
  • Bird Nests
  • There are many problems ranging from: Scheduling algorithms for putting sensors in sleep mode, to efficient mac prototocols, security protocols, operating systems, programming languages, databases, etc
  • 2KM at 130dB, low frequency, 155Kbps
  • Advancements in networks and multimedia
  • Advancements in networks and multimedia
  • Advancements in networks and multimedia
  • Advancements in networks and multimedia
  • Declarative: if it describes what something is like, rather than how to create it. … versus imperative languages where we state what we want and how we want it.
  • Advancements in networks and multimedia
  • NAND, NOR : logic gate that holds a single bit Seek Time (rotational latency): It is the amount of time between when the CPU requests a file and when the first byte of the file is sent to CPU. Times between 10 to 20 milliseconds are common. Magnetic Disks, Spindle RPM Average latency (ms) 4200:7.14ms, 5400:5.55ms, 7200:4.17ms, 10000:3ms, 15000 2ms.
  • !
  • All these in constant or logarithmic time
  • C: Defines page that has to be split ,S: defines page that has to be moved to flash media.
  • ScaleSearch: Given than we have 1500 pages (timestamps in range: 0..1000) and we look for tq=500 instead of going to page e.g. 300, go
  • "Indexing and Searching in Wireless Sensor Networks"

    1. 1. Indexing and Searching in Wireless Sensor Networks Demetris Zeinalipour [ zeinalipour@ouc.ac.cy ] School of Pure and Applied Sciences Open University of Cyprus http://is.ouc.ac.cy/~zeinalipour/ HPCL, University of Cyprus, February 14 th , 2008
    2. 2. Presentation Goals <ul><li>To provide an overview of recent developments in Wireless Sensor Network Technology </li></ul><ul><li>To highlight some important indexing and searching challenges that arise in this context </li></ul>
    3. 3. <ul><li>This is a joint work with my collaborators at the University of California – Riverside. </li></ul><ul><li>Our results were presented in the following papers: </li></ul><ul><ul><li>&quot;MicroHash: An Efficient Index Structure for Flash-Based Sensor Devices&quot;, </li></ul></ul><ul><ul><li>D. Zeinalipour-Yazti, S. Lin, V. Kalogeraki, D. Gunopulos and W. Najjar, The 4th USENIX Conference on File and Storage Technologies (FAST’05), San Francisco, USA, December, 2005. </li></ul></ul><ul><ul><li>&quot; Efficient Indexing Data Structures for Flash-Based Sensor Devices &quot;, </li></ul></ul><ul><ul><li> S. Lin, D. Zeinalipour-Yazti, V. Kalogeraki, D. Gunopulos, W. Najjar, ACM Transactions on Storage (TOS), ACM Press, Vol.2, No. 4, pp. 468-503, November 2006. </li></ul></ul>Acknowledgements
    4. 4. <ul><li>Capsule: an energy-optimized object storage system for memory-constrained sensor devices, ACM Sensys'06 ( UMass, Amherst ) </li></ul><ul><li>Ultra-low power data storage for sensor networks, IEEE IPSN'06 ( UMass, Amherst ) </li></ul><ul><li>TINX: a tiny index design for flash memory on wireless sensor devices, ACM Sensys'06 ( Stanford ) </li></ul><ul><li>Re-thinking data management for storage-centric sensor networks, IEEE CIDR'07 (UMass, Amherst ) </li></ul><ul><li>FlashDB: dynamic self-tuning database for NAND flash, IEEE IPSN'07, (Microsoft Research, Redmond ) </li></ul><ul><li>MStore: Enabling Storage-Centric Sensornet Research, IEEE IPSN'07 (Stanford) </li></ul><ul><li>EnviroMic: Towards Cooperative Storage and Retrieval in Audio Sensor Networks, IEEE ICDCS'07 ( Univ. of Illinois at Urbana-Champaign ) </li></ul><ul><li>DALi: A Communication-Centric Data Abstraction Layer for Energy-Constrained Devices in Mobile Sensor Networks, ACM Mobisys'07 (Princeton University) </li></ul>Main Citations
    5. 5. Presentation Outline <ul><li>Overview of Wireless Sensor Networks </li></ul><ul><li>Overview of Data Acquisition Models </li></ul><ul><li>The MicroHash Index Structure. </li></ul><ul><li>MicroHash Experimental Evaluation </li></ul><ul><li>Conclusions and Future Work </li></ul>
    6. 6. Wireless Sensor Networks <ul><li>Resource constrained devices utilized for monitoring and studying the physical world at a high fidelity. </li></ul>
    7. 7. Wireless Sensor Device
    8. 8. Wireless Sensor Network
    9. 9. Wireless Sensor Networks <ul><li>Applications have already emerged in: </li></ul><ul><ul><li>Environmental and habitant monitoring </li></ul></ul><ul><ul><li>Seismic and Structural monitoring </li></ul></ul><ul><ul><li>Understanding Animal Migrations & Interactions between species </li></ul></ul><ul><ul><li>Automation, Tracking, Hazard Monitoring Scenarios, Urban Monitoring etc </li></ul></ul>Great Duck Island – Maine (Temperature, Humidity etc). Golden Gate – SF, Vibration and Displacement of the bridge structure Zebranet (Kenya) GPS trajectory
    10. 10. Wireless Sensor Networks The Great Duck Island Study (Maine, USA) <ul><li>Large-Scale deployment by Intel Research, Berkeley in 2002-2003 (Maine USA). </li></ul><ul><li>Focuses on monitoring microclimate in and around the nests of endangered species which are sensitive to disturbance . </li></ul><ul><li>They deployed more than 166 motes installed in remote locations (such as 1000 feets in the forest) </li></ul>
    11. 11. Wireless Sensor Networks WebServer
    12. 12. Wireless Sensor Networks The James Reserve Project, CA, USA Available at: http://dms.jamesreserve.edu/
    13. 13. The Anatomy of a Sensor Device <ul><li>Processor, in various (sleep, idle, active) modes </li></ul><ul><li>Power source AA or Coin batteries, Solar Panels </li></ul><ul><li>SRAM used for the program code and for in-memory buffering. </li></ul><ul><li>LEDs used for debugging </li></ul><ul><li>Radio, used for transmitting the acquired data to some storage site (SINK) (9.6Kbps-250Kbps) </li></ul><ul><li>Sensors: Numeric readings in a limited range (e.g. temperature -40F..+250F with one decimal point precision) at a high frequency (2-2000Hz) </li></ul>Storage
    14. 14. Sensor Devices & Capabilities Sensing Capabilities <ul><li>Light </li></ul><ul><li>Temperature </li></ul><ul><li>Humidity </li></ul><ul><li>Pressure </li></ul><ul><li>Detect Tone </li></ul><ul><li>Wind Speed </li></ul><ul><li>Soil Moisture </li></ul><ul><li>Location (GPS) </li></ul><ul><li>etc… </li></ul>Xbow’s i-mote2 UC-Riverside RISE Xbow’s Telos UC-Berkeley mica2dot Xbow’s Mica
    15. 15. Characteristics <ul><li>The Energy Source is limited. </li></ul><ul><li>Energy source: AA batteries, Solar Panels </li></ul><ul><li>2. Local Processing is cheaper than transmitting over the radio . </li></ul><ul><li>Transmitting 1 Byte over the Radio consumes as much energy as ~1200 CPU instructions. </li></ul><ul><li>3. Local Storage is cheaper than transmitting over the radio . </li></ul><ul><li>Transmitting 512B over a single-hop 9.6Kbps (915MHz) radio requires 82,000 μ J, while writing to local flash only 760 μ J. </li></ul>
    16. 16. The Programming Cycle <ul><li>The Operating System </li></ul><ul><li>TinyOS (UC-Berkeley): Component-based architecture that allows programmers to wire together the minimum required components in order to minimize code size and energy consumption </li></ul><ul><li>(The operating system is really a number of libraries that can be statically linked to the sensor binary at compile time) </li></ul><ul><li>The Programming Language </li></ul><ul><li>nesC (Intel Research, Berkeley): an event-based C-variant optimized for programming sensor devices </li></ul>event result_t Clock.fire() { state = !state; if (state) call Leds.redOn(); else call Leds.redOff(); } “ Hello World”: Blinking the red LED!
    17. 17. The Programming Cycle <ul><li>The Testing Environment </li></ul><ul><li>Debugging code directly on a sensor device is a tedious procedure </li></ul><ul><li>nesC allows programmers to compile their code to </li></ul><ul><ul><ul><li>A Binary File that is installed on the sensor </li></ul></ul></ul><ul><ul><ul><li>A Binary File that runs on a PC </li></ul></ul></ul><ul><li>TOSSIM (TinyOS Simulation) is the environment which allows programmers to simulate the mote binary directly on a PC. </li></ul><ul><li>This enables accurate simulations, fine grained energy </li></ul><ul><li>modeling (with PowerTOSSIM ) and visualization ( TinyViz ) </li></ul>
    18. 18. The Programming Cycle <ul><li>The Pre-deployment Environment </li></ul><ul><li>Once you have created and debugged you code you can perform a deployment in a laboratory environment. </li></ul><ul><li>Harvard’s MoteLab uses 190 sensors, powered from wall power interconnected with an Ethernet connection. </li></ul><ul><li>The Ethernet is just for debugging and reprogramming, while the Radio for actual communication between motes </li></ul><ul><li>Motes can be reprogrammed through a web interface. </li></ul>Available at: http://motelab.eecs.harvard.edu/
    19. 19. Presentation Outline <ul><li>Overview of Wireless Sensor Networks (WSN) </li></ul><ul><li>Overview of Data Acquisition Models </li></ul><ul><li>The MicroHash Index Structure </li></ul><ul><li>MicroHash Experimental Evaluation </li></ul><ul><li>Conclusions and Future Work </li></ul>
    20. 20. The Centralized Storage Model <ul><li>A Database that collects readings from many Sensors. </li></ul><ul><li>Centralized: Storage, Indexing, Query Processing, Triggers, etc. </li></ul>
    21. 21. Centralized Storage I Available at: http://www.xbow.com/ <ul><li>Crossbow’s MoteView software </li></ul><ul><li>No in-network Aggregation </li></ul><ul><li>No in-Network Filtering </li></ul>
    22. 22. Centralized Storage II Available at: http:// telegraph.cs.berkeley.edu/tinydb / <ul><li>TinyDB - A Declarative Interface for Data Acquisition in Sensor Networks . </li></ul><ul><li>In-Network Aggregation </li></ul><ul><li>In-Network Filtering (i.e., WHERE clause) </li></ul>e.g., SELECT MAX(temp) FROM sensors
    23. 23. Centralized Storage: Conclusions <ul><li>Frameworks such as TinyDB: </li></ul><ul><ul><li>- Are suitable for continuous queries. </li></ul></ul><ul><ul><li>- Push aggregation in the network but keep much of the processing at the sink. </li></ul></ul><ul><li>New Challenges: </li></ul><ul><ul><li>Many applications do not require the query to be evaluated continuously (e.g., Average temperature in the last 6 months?) </li></ul></ul><ul><ul><li>In many applications there is no sink (e.g., remote deployments and mobile sensor nets) </li></ul></ul><ul><ul><li>- Local Storage on sensors keeps increasing (e.g., RISE and more recently imote2) </li></ul></ul>
    24. 24. Our Model: In-Situ Data Storage <ul><li>The data remains In-situ (at the generating site) in a sliding window fashion. </li></ul><ul><li>When required, users conduct on-demand queries to retrieve information of interest. </li></ul>A Network of Sensor Databases
    25. 25. <ul><li>Soil-Organism Monitoring </li></ul><ul><li>(Center for Conservation Biology, UCR) </li></ul><ul><ul><li>A set of sensors monitor the CO 2 levels in the soil over a large window of time. </li></ul></ul><ul><ul><li>Not a real-time application. </li></ul></ul><ul><ul><li>Many values may not be very interesting. </li></ul></ul>In-Situ Data Storage: Motivation <ul><ul><li>D. Zeinalipour-Yazti, S. Neema, D. Gunopulos, V. Kalogeraki and W. Najjar, </li></ul></ul><ul><ul><li>&quot;Data Acquision in Sensor Networks with Large Memories&quot;, IEEE Intl. Workshop on Networking Meets Databases NetDB ( ICDE'2005 ), Tokyo, Japan, 2005. </li></ul></ul>
    26. 26. Presentation Outline <ul><li>Overview of Wireless Sensor Networks </li></ul><ul><li>Overview of Data Acquisition Models </li></ul><ul><li>The MicroHash Index Structure </li></ul><ul><li>MicroHash Experimental Evaluation </li></ul><ul><li>Conclusions and Future Work </li></ul>
    27. 27. Flash Memory at a Glance <ul><li>The most prevalent storage medium used for Sensor Devices is Flash Memory (NAND Flash) </li></ul><ul><li>Fastest growing memory market (‘05 $8.7B, ‘06:$11B) </li></ul><ul><li>(NAND) Flash Advantages </li></ul><ul><li>Simple Cell Architecture (high capacity in a small surface) </li></ul><ul><li>Fast Random Access (50-80 μ s) compared to 10-20ms in Disks </li></ul><ul><li>Economical Reproduction </li></ul><ul><li>Shock Resistant </li></ul><ul><li>Power Efficient </li></ul>Surface mount NAND flash Removable NAND Devices
    28. 28. Flash Memory at a Glance <ul><li>Write-Constraint: Writing can only be performed at a page granularity (256B~512B), after the respective page (and its respective 8KB~64KB block!) has been deleted </li></ul><ul><li>Delete-Constraint: Erasure of a page can only be performed at a block granularity (i.e. 8KB~64KB) </li></ul><ul><li>Wear-Constraint: Each page can only be written a limited number of times (typically 10,000-100,000) </li></ul>Asymmetric Read/Write Energy Cost : Measurements using RISE Energy (Page Size = 512 B) Read = 24 μ J Write =763 μ J Block Erase = 425 μ J
    29. 29. MicroHash Objectives <ul><li>General Objectives </li></ul><ul><li>Provide efficient access to any record stored on flash by timestamp or value </li></ul><ul><li>Execute a wide spectrum of queries based on our index, similarly to generic DB indexes. </li></ul><ul><li>Design Objectives: </li></ul><ul><li>Avoid wearing out specific pages. </li></ul><ul><li>Minimize random access deletions of pages. </li></ul><ul><li>Minimize SRAM structures </li></ul><ul><ul><li>SRAM is extremely limited (8KB-64KB). </li></ul></ul><ul><ul><li>Small memory-footprint => quick initialization. </li></ul></ul>
    30. 30. Main Structures <ul><li>4 Page Types: a) Root Page, b) Directory Page, c) Index Page and d) Data Page </li></ul><ul><li>4 Phases of Operation: a) Initialization, b) Growing, c) Repartition and d) Garbage Collect. </li></ul>
    31. 31. Growing the MicroHash Index <ul><li>Collect data in an SRAM buffer page P write </li></ul><ul><li>When P write is full flush it out to flash media </li></ul><ul><li>Next create index records for each data record in P write </li></ul><ul><li>If SRAM gets full, Index pages are forced out to flash media by an LRU policy . </li></ul>(ts, 74F) Index Pages Buffer P write Buffer P write 60 80 x 70 50 90 40 Directory Index
    32. 32. Growing the MicroHash Index Flash Media A populated Flash Media idx: next empty page
    33. 33. Garbage Collection in MicroHash <ul><li>When the media gets full some pages need to be deleted => delete the oldest pages. </li></ul><ul><li>Oldest Block? The next block following the idx pointer. </li></ul><ul><li>Note: </li></ul><ul><li>This might create invalid index records. </li></ul><ul><li>This will be handled by our search algorithm </li></ul>
    34. 34. Directory Repartition in MicroHash <ul><li>MicroHash starts out with a directory that is segmented into equiwidth buckets </li></ul><ul><ul><li>e.g., divide the temperature range [-40,250] into c buckets) </li></ul></ul><ul><li>Not efficient as certain buckets will not be utilized </li></ul><ul><ul><li>Consider the first few or last few buckets below. </li></ul></ul>
    35. 35. Directory Repartition in MicroHash <ul><li>If bucket A links to more than τ index pages, evict the least used bucket B and segment bucket A into A and A’ </li></ul><ul><li>We want to avoid bucket reassignments of old records as this would be very expensive </li></ul>Example: τ=2 C: #entries since last split S: timestamp of last addition
    36. 36. Searching in MicroHash <ul><li>Searching by value </li></ul><ul><li>“ Find the timestamp (s) on which the temperature was 100F” </li></ul><ul><ul><li>Simple operation in MicroHash </li></ul></ul><ul><ul><li>We simply find the right Directory Bucket, from there the respective index page and then data record (page-by-page) </li></ul></ul><ul><li>Searching by timestamp </li></ul><ul><li>“ Find the temperature of some sensor on a given timestamp tq ” </li></ul><ul><ul><li>Problem: Index pages are mixed together with data pages. </li></ul></ul><ul><ul><li>Solutions: </li></ul></ul><ul><ul><ul><li>1. Binary Search (O(logn), 18 pages for a 128MB media) </li></ul></ul></ul><ul><ul><ul><li>2. LBSearch (less than 10 pages for a 128MB media) </li></ul></ul></ul><ul><ul><ul><li>3. ScaleSearch (better than LBSearch, ~4.5 pages for a 128MB media) </li></ul></ul></ul>
    37. 37. LBSearch and ScaleSearch <ul><li>Solutions to the Search-by-timestamp problem: </li></ul><ul><li>LBSearch: We recursively create a lower bound on the position of tq until the given timestamp is located. </li></ul><ul><li>ScaleSearch: Quite similar to LBSearch, however in the first step we proceed more aggressively (by exploiting data distribution) </li></ul>Query tq=500 tq=300 tq=350 tq=420 tq=490 tq=500
    38. 38. Presentation Outline <ul><li>Overview of Wireless Sensor Networks </li></ul><ul><li>Overview of Data Acquisition Models </li></ul><ul><li>The MicroHash Index Structure </li></ul><ul><li>MicroHash Experimental Evaluation </li></ul><ul><li>Conclusions and Future Work </li></ul>
    39. 39. Experimental Evaluation <ul><li>Implemented MicroHash in nesC. </li></ul><ul><li>We tested it using TinyOS along with a trace-driven experimental methodology. </li></ul><ul><li>Datasets : </li></ul><ul><ul><li>Washington State Climate </li></ul></ul><ul><ul><ul><li>268MB dataset contains readings in 2000-2005. </li></ul></ul></ul><ul><ul><li>Great Duck Island </li></ul></ul><ul><ul><ul><li>97,000 readings between October and November 2002. </li></ul></ul></ul><ul><li>Evaluation Parameters : i) Space Overhead, ii) Energy Overhead, iii) Search Performance </li></ul>
    40. 40. Space Overhead of Index <ul><li>Index page overhead Φ = IndexPages/(DataPages+IndexPages) </li></ul><ul><li>Two Index page layouts </li></ul><ul><ul><li>Offset , an index record has the following form {datapageid,offset} </li></ul></ul><ul><ul><li>NoOffset , in which an index record has the form {datapageid} </li></ul></ul><ul><li>128 MB flash media (256,000 pages) </li></ul>
    41. 41. Space Overhead of Index Black denotes the index pages Increasing the Buffer Decreases the Index Overhead
    42. 42. Search Performance <ul><li>128 MB flash media (256,000 pages), varied SRAM (buffer) size </li></ul><ul><li>2 Index page layouts </li></ul><ul><ul><li>Anchor: Index Pages store the last known timestamp </li></ul></ul><ul><ul><li>No Anchor: Timestamp is only stored in Data Pages </li></ul></ul>
    43. 43. Indexing the Great Duck Island Trace <ul><li>Used 3KB index buffer and a 4MB flash card to store all the 97,000 20-byte data readings. </li></ul><ul><ul><li>The index pages never require more than 28% additional space </li></ul></ul><ul><ul><li>Indexing the records has only a small increase in energy demand: the energy cost of storing the records on flash without an index is 3042mJ </li></ul></ul><ul><ul><li>We were able to find any record by its timestamp with 4.75 page reads on average </li></ul></ul>
    44. 44. Presentation Outline <ul><li>Overview of Wireless Sensor Networks </li></ul><ul><li>Overview of Data Acquisition Models </li></ul><ul><li>The MicroHash Index Structure </li></ul><ul><li>MicroHash Experimental Evaluation </li></ul><ul><li>Conclusions and Future Work </li></ul>
    45. 45. Conclusions & Future Work <ul><li>We proposed the MicroHash index , which is an efficient external memory hash index that addresses the distinct characteristics of flash memory </li></ul><ul><li>Our experimental evaluation shows that the structure we propose is both efficient and practical </li></ul><ul><li>Future work: </li></ul><ul><ul><li>Develop a complete library of indexes and data structures (stacks, queues, b+trees, etc.) </li></ul></ul><ul><ul><li>Buffer optimizations and Online Compression </li></ul></ul><ul><ul><li>Support Range Queries </li></ul></ul>
    46. 46. Indexing and Searching in Wireless Sensor Networks <ul><li>Demetris Zeinalipour </li></ul><ul><li>Thank you! </li></ul><ul><li>Questions? </li></ul><ul><li>Related Publications </li></ul><ul><li>&quot;MicroHash: An Efficient Index Structure for Flash-Based Sensor Devices&quot;, D. Zeinalipour,S. Lin, V. Kalogeraki, D. Gunopulos, W. Najjar, In USENIX FAST’05. </li></ul><ul><li>&quot; Efficient Indexing Data Structures for Flash-Based Sensor Devices &quot;, </li></ul><ul><li>ACM Transactions on Storage (TOS), November 2006. </li></ul>Presentation and publications available at: http://is.ouc.ac.cy/~zeinalipour/
    47. 47. Page Types in MicroHash <ul><li>Root Page </li></ul><ul><ul><li>contains information related to the state of the flash media, e.g. it contains the position of the last write (idx), the current write cycle (cycle) and meta information about the various indexes stored on the flash media </li></ul></ul><ul><li>Directory Page (the hash table) </li></ul><ul><ul><li>contains a number of directory records (buckets) each of which contains the address of the last known index page mapped to this bucket. </li></ul></ul><ul><li>Index Page </li></ul><ul><ul><li>contains a fixed number of index records and the 8 byte timestamp of the last known data record. The latter field, denoted as anchor is exploited by timestamp searches. </li></ul></ul><ul><li>Data Page </li></ul><ul><ul><li>contains a fixed number of data records </li></ul></ul>
    48. 48. Searching Bottlenecks <ul><li>Index Pages written on flash might not be fully occupied </li></ul><ul><li>When we access these pages we transfer a lot of empty bytes (padding) between the flash media and SRAM. </li></ul><ul><li>Proposed Solutions: </li></ul><ul><ul><li>Solution 1: Two-Phase Page Reads </li></ul></ul><ul><ul><li>Solution 2: ELF-like Chaining of Index Pages </li></ul></ul>
    49. 49. Improving Search Performance <ul><li>Solution 1: Utilize Two-Phase Page Reads. </li></ul><ul><ul><li>Reads the 8B header from the flash media. </li></ul></ul><ul><ul><li>Then read the correct payload in the next phase. </li></ul></ul>
    50. 50. Improving Search Performance <ul><li>Solution 2: Avoid non-full index pages using ELF*. </li></ul><ul><ul><li>ELF: a linked list in which each page, other than the last page, is completely full. </li></ul></ul><ul><ul><li>keeps copying the last non-full page into a newer page, when new records are requested to be added. </li></ul></ul>*Dai et. al., Efficient Log Structured Flash File System, SenSys 2004
    51. 51. Search Performance <ul><li>We compared MicroHash vs. ELF Index Page Chaining by searching all values in the range [20,100] </li></ul><ul><li>Keeping full index pages increases search performance but decreases insertion performance . </li></ul>Decreasing indexing performance using ELF (15% more writes) Increasing search performance using ELF (10% less reads)