SlideShare a Scribd company logo
1 of 4
Download to read offline
A Hierarchical Tensor-Based Approach to Compressing, Updating and
Querying Geospatial Data
Abstract:
With the rapid development of data observation and model simulation in
geoscience, spatial-temporal data have become increasingly
multidimensional, massive and are consistently being updated. As a result,
the integrated maintenance of these data is becoming a challenge. This
paper presents a blocked hierarchical tensor representation within the split-
and-merge paradigm for the compressed storage, continuously updating
and data querying of multidimensional geospatial field data. The original
multidimensional geospatial field data are split into small blocks according
to their spatial-temporal references. These blocks are represented and
compressed hierarchically, and then combined into a single hierarchical
tree as the representation of original data. With a buffered binary tree data
structure and corresponding optimized operation algorithms, the original
multidimensional geospatial field data can be continuously compressed,
appended, and queried. Data from the 20th Century Reanalysis Monthly
Mean Composites are used to evaluate the performance of this approach.
Compared to traditional methods, the new approach is shown to retain the
quality of the original data with much lower storage costs and faster
computational performance. The result suggests that the blocked
hierarchical tensor representation provides an effective structure for
integrated storage, presentation and computation of multidimensional
geospatial field data.
Existing System:
Large amounts of observation of exiting attributes/variables are
uninterruptedly generated by global observation systems. These data are
often compressed for storage. The newly arrived data should also be
continuously compressed and appended to the existing data in such a way
that the newly added should be integrated to the existing data and made as
a whole with the existing data.
In addition, this updating process should be completed in a short time and
can be repeatedly applied for the next patch of new data. The compression
and storage should maintain the consistency of the spatial-temporal
reference (STR) of this data. Balance among the data accuracy, data
compression performance and convenience for indexing, querying and
analysis is required.
Proposed System:
Multidimensional data are stored and accessed linearly in memory and on
hard disk but should be randomly queried and updated from any
dimension. Classical methods use data indexes (e.g., Pyramid, RTree, VA-
Files) to accelerate the query and storage.
These data indexes split data into segments and then map the segments to
the linear ordered data I/O sequence. When the dimension grows, both the
data segmentation and the data structure are becoming complex and
inefficient.
Big data or data-intensive computing solutions use parallel data I/O and
computation to accelerate the data accessing and updating. However, large
number of computers and complicated computation architectures are
required to provide the I/O bandwidth and computation power needed.
Hardware Requirements:
• System : Pentium IV 2.4 GHz.
• Hard Disk : 40 GB.
• Floppy Drive : 1.44 Mb.
• Monitor : 15 VGA Colour.
• Mouse : Logitech.
• RAM : 256 Mb.
Software Requirements:
• Operating system : - Windows XP.
• Front End : - JSP
• Back End : - SQL Server
Software Requirements:
• Operating system : - Windows XP.
• Front End : - .Net
• Back End : - SQL Server

More Related Content

What's hot

A BRIEF REVIEW ALONG WITH A NEW PROPOSED APPROACH OF DATA DE DUPLICATION
A BRIEF REVIEW ALONG WITH A NEW PROPOSED APPROACH OF DATA DE DUPLICATIONA BRIEF REVIEW ALONG WITH A NEW PROPOSED APPROACH OF DATA DE DUPLICATION
A BRIEF REVIEW ALONG WITH A NEW PROPOSED APPROACH OF DATA DE DUPLICATIONcscpconf
 
Spatial data analysis 1
Spatial data analysis 1Spatial data analysis 1
Spatial data analysis 1Johan Blomme
 
TYBSC IT PGIS Unit II Chapter I Data Management and Processing Systems
TYBSC IT PGIS Unit II Chapter I Data Management and Processing SystemsTYBSC IT PGIS Unit II Chapter I Data Management and Processing Systems
TYBSC IT PGIS Unit II Chapter I Data Management and Processing SystemsArti Parab Academics
 
Data indexing presentation part 2
Data indexing presentation part 2Data indexing presentation part 2
Data indexing presentation part 2Vivek Kantariya
 
B036407011
B036407011B036407011
B036407011theijes
 
Comp10 unit3d lecture_slides
Comp10 unit3d lecture_slidesComp10 unit3d lecture_slides
Comp10 unit3d lecture_slidesCMDLMS
 

What's hot (11)

Iccsa stankuteha180611
Iccsa stankuteha180611Iccsa stankuteha180611
Iccsa stankuteha180611
 
A BRIEF REVIEW ALONG WITH A NEW PROPOSED APPROACH OF DATA DE DUPLICATION
A BRIEF REVIEW ALONG WITH A NEW PROPOSED APPROACH OF DATA DE DUPLICATIONA BRIEF REVIEW ALONG WITH A NEW PROPOSED APPROACH OF DATA DE DUPLICATION
A BRIEF REVIEW ALONG WITH A NEW PROPOSED APPROACH OF DATA DE DUPLICATION
 
Spatial data analysis 1
Spatial data analysis 1Spatial data analysis 1
Spatial data analysis 1
 
Vector data model
Vector data modelVector data model
Vector data model
 
TYBSC IT PGIS Unit II Chapter I Data Management and Processing Systems
TYBSC IT PGIS Unit II Chapter I Data Management and Processing SystemsTYBSC IT PGIS Unit II Chapter I Data Management and Processing Systems
TYBSC IT PGIS Unit II Chapter I Data Management and Processing Systems
 
Spatial Data Models
Spatial Data Models Spatial Data Models
Spatial Data Models
 
Data indexing presentation part 2
Data indexing presentation part 2Data indexing presentation part 2
Data indexing presentation part 2
 
Reaerch data management
Reaerch data managementReaerch data management
Reaerch data management
 
B036407011
B036407011B036407011
B036407011
 
Comp10 unit3d lecture_slides
Comp10 unit3d lecture_slidesComp10 unit3d lecture_slides
Comp10 unit3d lecture_slides
 
Data Applied:Tree Maps
Data Applied:Tree MapsData Applied:Tree Maps
Data Applied:Tree Maps
 

Viewers also liked

Viewers also liked (7)

Politics, power and resistance
Politics, power and resistancePolitics, power and resistance
Politics, power and resistance
 
POWER and EDUCATION
POWER and EDUCATIONPOWER and EDUCATION
POWER and EDUCATION
 
Foucault by farij
Foucault by farijFoucault by farij
Foucault by farij
 
Michel Foucault Panopticon
Michel Foucault PanopticonMichel Foucault Panopticon
Michel Foucault Panopticon
 
Foucauldian discourse analysis.
Foucauldian discourse analysis.Foucauldian discourse analysis.
Foucauldian discourse analysis.
 
Postmodernism (Foucault and Baudrillard)
Postmodernism (Foucault and Baudrillard)Postmodernism (Foucault and Baudrillard)
Postmodernism (Foucault and Baudrillard)
 
Foucault
FoucaultFoucault
Foucault
 

Similar to Hierarchical Tensor Approach to Compressing Geospatial Data

Drsp dimension reduction for similarity matching and pruning of time series ...
Drsp  dimension reduction for similarity matching and pruning of time series ...Drsp  dimension reduction for similarity matching and pruning of time series ...
Drsp dimension reduction for similarity matching and pruning of time series ...IJDKP
 
Elimination of data redundancy before persisting into dbms using svm classifi...
Elimination of data redundancy before persisting into dbms using svm classifi...Elimination of data redundancy before persisting into dbms using svm classifi...
Elimination of data redundancy before persisting into dbms using svm classifi...nalini manogaran
 
Power Management in Micro grid Using Hybrid Energy Storage System
Power Management in Micro grid Using Hybrid Energy Storage SystemPower Management in Micro grid Using Hybrid Energy Storage System
Power Management in Micro grid Using Hybrid Energy Storage Systemijcnes
 
International Journal of Engineering and Science Invention (IJESI)
International Journal of Engineering and Science Invention (IJESI)International Journal of Engineering and Science Invention (IJESI)
International Journal of Engineering and Science Invention (IJESI)inventionjournals
 
Spectral Clustering and Vantage Point Indexing for Efficient Data Retrieval
Spectral Clustering and Vantage Point Indexing for Efficient Data Retrieval Spectral Clustering and Vantage Point Indexing for Efficient Data Retrieval
Spectral Clustering and Vantage Point Indexing for Efficient Data Retrieval IJECEIAES
 
A flexible phasor data concentrator design
A flexible phasor data concentrator designA flexible phasor data concentrator design
A flexible phasor data concentrator designsunder fou
 
A flexible phasor data concentrator design
A flexible phasor data concentrator designA flexible phasor data concentrator design
A flexible phasor data concentrator designsunder fou
 
Granularity analysis of classification and estimation for complex datasets wi...
Granularity analysis of classification and estimation for complex datasets wi...Granularity analysis of classification and estimation for complex datasets wi...
Granularity analysis of classification and estimation for complex datasets wi...IJECEIAES
 
Combining Data Lake and Data Wrangling for Ensuring Data Quality in CRIS
Combining Data Lake and Data Wrangling for Ensuring Data Quality in CRISCombining Data Lake and Data Wrangling for Ensuring Data Quality in CRIS
Combining Data Lake and Data Wrangling for Ensuring Data Quality in CRISAnastasija Nikiforova
 
IRJET- Deduplication Detection for Similarity in Document Analysis Via Vector...
IRJET- Deduplication Detection for Similarity in Document Analysis Via Vector...IRJET- Deduplication Detection for Similarity in Document Analysis Via Vector...
IRJET- Deduplication Detection for Similarity in Document Analysis Via Vector...IRJET Journal
 
IRJET- Deduplication Detection for Similarity in Document Analysis Via Vector...
IRJET- Deduplication Detection for Similarity in Document Analysis Via Vector...IRJET- Deduplication Detection for Similarity in Document Analysis Via Vector...
IRJET- Deduplication Detection for Similarity in Document Analysis Via Vector...IRJET Journal
 
EVALUATE DATABASE COMPRESSION PERFORMANCE AND PARALLEL BACKUP
EVALUATE DATABASE COMPRESSION PERFORMANCE AND PARALLEL BACKUPEVALUATE DATABASE COMPRESSION PERFORMANCE AND PARALLEL BACKUP
EVALUATE DATABASE COMPRESSION PERFORMANCE AND PARALLEL BACKUPijdms
 
Novel Ensemble Tree for Fast Prediction on Data Streams
Novel Ensemble Tree for Fast Prediction on Data StreamsNovel Ensemble Tree for Fast Prediction on Data Streams
Novel Ensemble Tree for Fast Prediction on Data StreamsIJERA Editor
 
Exascale Computing and Experimental Sensor Data
Exascale Computing and Experimental Sensor DataExascale Computing and Experimental Sensor Data
Exascale Computing and Experimental Sensor DataJoel Saltz
 
IEEE Big data 2016 Title and Abstract
IEEE Big data  2016 Title and AbstractIEEE Big data  2016 Title and Abstract
IEEE Big data 2016 Title and Abstracttsysglobalsolutions
 
Secure and Efficient Client and Server Side Data Deduplication to Reduce Stor...
Secure and Efficient Client and Server Side Data Deduplication to Reduce Stor...Secure and Efficient Client and Server Side Data Deduplication to Reduce Stor...
Secure and Efficient Client and Server Side Data Deduplication to Reduce Stor...dbpublications
 
Managing Multidimensional Historical
Managing Multidimensional HistoricalManaging Multidimensional Historical
Managing Multidimensional HistoricalArul Suju
 
11.challenging issues of spatio temporal data mining
11.challenging issues of spatio temporal data mining11.challenging issues of spatio temporal data mining
11.challenging issues of spatio temporal data miningAlexander Decker
 

Similar to Hierarchical Tensor Approach to Compressing Geospatial Data (20)

Drsp dimension reduction for similarity matching and pruning of time series ...
Drsp  dimension reduction for similarity matching and pruning of time series ...Drsp  dimension reduction for similarity matching and pruning of time series ...
Drsp dimension reduction for similarity matching and pruning of time series ...
 
ADAPTER
ADAPTERADAPTER
ADAPTER
 
Elimination of data redundancy before persisting into dbms using svm classifi...
Elimination of data redundancy before persisting into dbms using svm classifi...Elimination of data redundancy before persisting into dbms using svm classifi...
Elimination of data redundancy before persisting into dbms using svm classifi...
 
Power Management in Micro grid Using Hybrid Energy Storage System
Power Management in Micro grid Using Hybrid Energy Storage SystemPower Management in Micro grid Using Hybrid Energy Storage System
Power Management in Micro grid Using Hybrid Energy Storage System
 
International Journal of Engineering and Science Invention (IJESI)
International Journal of Engineering and Science Invention (IJESI)International Journal of Engineering and Science Invention (IJESI)
International Journal of Engineering and Science Invention (IJESI)
 
IJCSIT
IJCSITIJCSIT
IJCSIT
 
Spectral Clustering and Vantage Point Indexing for Efficient Data Retrieval
Spectral Clustering and Vantage Point Indexing for Efficient Data Retrieval Spectral Clustering and Vantage Point Indexing for Efficient Data Retrieval
Spectral Clustering and Vantage Point Indexing for Efficient Data Retrieval
 
A flexible phasor data concentrator design
A flexible phasor data concentrator designA flexible phasor data concentrator design
A flexible phasor data concentrator design
 
A flexible phasor data concentrator design
A flexible phasor data concentrator designA flexible phasor data concentrator design
A flexible phasor data concentrator design
 
Granularity analysis of classification and estimation for complex datasets wi...
Granularity analysis of classification and estimation for complex datasets wi...Granularity analysis of classification and estimation for complex datasets wi...
Granularity analysis of classification and estimation for complex datasets wi...
 
Combining Data Lake and Data Wrangling for Ensuring Data Quality in CRIS
Combining Data Lake and Data Wrangling for Ensuring Data Quality in CRISCombining Data Lake and Data Wrangling for Ensuring Data Quality in CRIS
Combining Data Lake and Data Wrangling for Ensuring Data Quality in CRIS
 
IRJET- Deduplication Detection for Similarity in Document Analysis Via Vector...
IRJET- Deduplication Detection for Similarity in Document Analysis Via Vector...IRJET- Deduplication Detection for Similarity in Document Analysis Via Vector...
IRJET- Deduplication Detection for Similarity in Document Analysis Via Vector...
 
IRJET- Deduplication Detection for Similarity in Document Analysis Via Vector...
IRJET- Deduplication Detection for Similarity in Document Analysis Via Vector...IRJET- Deduplication Detection for Similarity in Document Analysis Via Vector...
IRJET- Deduplication Detection for Similarity in Document Analysis Via Vector...
 
EVALUATE DATABASE COMPRESSION PERFORMANCE AND PARALLEL BACKUP
EVALUATE DATABASE COMPRESSION PERFORMANCE AND PARALLEL BACKUPEVALUATE DATABASE COMPRESSION PERFORMANCE AND PARALLEL BACKUP
EVALUATE DATABASE COMPRESSION PERFORMANCE AND PARALLEL BACKUP
 
Novel Ensemble Tree for Fast Prediction on Data Streams
Novel Ensemble Tree for Fast Prediction on Data StreamsNovel Ensemble Tree for Fast Prediction on Data Streams
Novel Ensemble Tree for Fast Prediction on Data Streams
 
Exascale Computing and Experimental Sensor Data
Exascale Computing and Experimental Sensor DataExascale Computing and Experimental Sensor Data
Exascale Computing and Experimental Sensor Data
 
IEEE Big data 2016 Title and Abstract
IEEE Big data  2016 Title and AbstractIEEE Big data  2016 Title and Abstract
IEEE Big data 2016 Title and Abstract
 
Secure and Efficient Client and Server Side Data Deduplication to Reduce Stor...
Secure and Efficient Client and Server Side Data Deduplication to Reduce Stor...Secure and Efficient Client and Server Side Data Deduplication to Reduce Stor...
Secure and Efficient Client and Server Side Data Deduplication to Reduce Stor...
 
Managing Multidimensional Historical
Managing Multidimensional HistoricalManaging Multidimensional Historical
Managing Multidimensional Historical
 
11.challenging issues of spatio temporal data mining
11.challenging issues of spatio temporal data mining11.challenging issues of spatio temporal data mining
11.challenging issues of spatio temporal data mining
 

More from ieeepondy

Demand aware network function placement
Demand aware network function placementDemand aware network function placement
Demand aware network function placementieeepondy
 
Service description in the nfv revolution trends, challenges and a way forward
Service description in the nfv revolution trends, challenges and a way forwardService description in the nfv revolution trends, challenges and a way forward
Service description in the nfv revolution trends, challenges and a way forwardieeepondy
 
Secure optimization computation outsourcing in cloud computing a case study o...
Secure optimization computation outsourcing in cloud computing a case study o...Secure optimization computation outsourcing in cloud computing a case study o...
Secure optimization computation outsourcing in cloud computing a case study o...ieeepondy
 
Spatial related traffic sign inspection for inventory purposes using mobile l...
Spatial related traffic sign inspection for inventory purposes using mobile l...Spatial related traffic sign inspection for inventory purposes using mobile l...
Spatial related traffic sign inspection for inventory purposes using mobile l...ieeepondy
 
Standards for hybrid clouds
Standards for hybrid cloudsStandards for hybrid clouds
Standards for hybrid cloudsieeepondy
 
Rfhoc a random forest approach to auto-tuning hadoop's configuration
Rfhoc a random forest approach to auto-tuning hadoop's configurationRfhoc a random forest approach to auto-tuning hadoop's configuration
Rfhoc a random forest approach to auto-tuning hadoop's configurationieeepondy
 
Resource and instance hour minimization for deadline constrained dag applicat...
Resource and instance hour minimization for deadline constrained dag applicat...Resource and instance hour minimization for deadline constrained dag applicat...
Resource and instance hour minimization for deadline constrained dag applicat...ieeepondy
 
Reliable and confidential cloud storage with efficient data forwarding functi...
Reliable and confidential cloud storage with efficient data forwarding functi...Reliable and confidential cloud storage with efficient data forwarding functi...
Reliable and confidential cloud storage with efficient data forwarding functi...ieeepondy
 
Rebuttal to “comments on ‘control cloud data access privilege and anonymity w...
Rebuttal to “comments on ‘control cloud data access privilege and anonymity w...Rebuttal to “comments on ‘control cloud data access privilege and anonymity w...
Rebuttal to “comments on ‘control cloud data access privilege and anonymity w...ieeepondy
 
Scalable cloud–sensor architecture for the internet of things
Scalable cloud–sensor architecture for the internet of thingsScalable cloud–sensor architecture for the internet of things
Scalable cloud–sensor architecture for the internet of thingsieeepondy
 
Scalable algorithms for nearest neighbor joins on big trajectory data
Scalable algorithms for nearest neighbor joins on big trajectory dataScalable algorithms for nearest neighbor joins on big trajectory data
Scalable algorithms for nearest neighbor joins on big trajectory dataieeepondy
 
Robust workload and energy management for sustainable data centers
Robust workload and energy management for sustainable data centersRobust workload and energy management for sustainable data centers
Robust workload and energy management for sustainable data centersieeepondy
 
Privacy preserving deep computation model on cloud for big data feature learning
Privacy preserving deep computation model on cloud for big data feature learningPrivacy preserving deep computation model on cloud for big data feature learning
Privacy preserving deep computation model on cloud for big data feature learningieeepondy
 
Pricing the cloud ieee projects, ieee projects chennai, ieee projects 2016,ie...
Pricing the cloud ieee projects, ieee projects chennai, ieee projects 2016,ie...Pricing the cloud ieee projects, ieee projects chennai, ieee projects 2016,ie...
Pricing the cloud ieee projects, ieee projects chennai, ieee projects 2016,ie...ieeepondy
 
Protection of big data privacy
Protection of big data privacyProtection of big data privacy
Protection of big data privacyieeepondy
 
Power optimization with bler constraint for wireless fronthauls in c ran
Power optimization with bler constraint for wireless fronthauls in c ranPower optimization with bler constraint for wireless fronthauls in c ran
Power optimization with bler constraint for wireless fronthauls in c ranieeepondy
 
Performance aware cloud resource allocation via fitness-enabled auction
Performance aware cloud resource allocation via fitness-enabled auctionPerformance aware cloud resource allocation via fitness-enabled auction
Performance aware cloud resource allocation via fitness-enabled auctionieeepondy
 
Performance limitations of a text search application running in cloud instances
Performance limitations of a text search application running in cloud instancesPerformance limitations of a text search application running in cloud instances
Performance limitations of a text search application running in cloud instancesieeepondy
 
Performance analysis and optimal cooperative cluster size for randomly distri...
Performance analysis and optimal cooperative cluster size for randomly distri...Performance analysis and optimal cooperative cluster size for randomly distri...
Performance analysis and optimal cooperative cluster size for randomly distri...ieeepondy
 
Predictive control for energy aware consolidation in cloud datacenters
Predictive control for energy aware consolidation in cloud datacentersPredictive control for energy aware consolidation in cloud datacenters
Predictive control for energy aware consolidation in cloud datacentersieeepondy
 

More from ieeepondy (20)

Demand aware network function placement
Demand aware network function placementDemand aware network function placement
Demand aware network function placement
 
Service description in the nfv revolution trends, challenges and a way forward
Service description in the nfv revolution trends, challenges and a way forwardService description in the nfv revolution trends, challenges and a way forward
Service description in the nfv revolution trends, challenges and a way forward
 
Secure optimization computation outsourcing in cloud computing a case study o...
Secure optimization computation outsourcing in cloud computing a case study o...Secure optimization computation outsourcing in cloud computing a case study o...
Secure optimization computation outsourcing in cloud computing a case study o...
 
Spatial related traffic sign inspection for inventory purposes using mobile l...
Spatial related traffic sign inspection for inventory purposes using mobile l...Spatial related traffic sign inspection for inventory purposes using mobile l...
Spatial related traffic sign inspection for inventory purposes using mobile l...
 
Standards for hybrid clouds
Standards for hybrid cloudsStandards for hybrid clouds
Standards for hybrid clouds
 
Rfhoc a random forest approach to auto-tuning hadoop's configuration
Rfhoc a random forest approach to auto-tuning hadoop's configurationRfhoc a random forest approach to auto-tuning hadoop's configuration
Rfhoc a random forest approach to auto-tuning hadoop's configuration
 
Resource and instance hour minimization for deadline constrained dag applicat...
Resource and instance hour minimization for deadline constrained dag applicat...Resource and instance hour minimization for deadline constrained dag applicat...
Resource and instance hour minimization for deadline constrained dag applicat...
 
Reliable and confidential cloud storage with efficient data forwarding functi...
Reliable and confidential cloud storage with efficient data forwarding functi...Reliable and confidential cloud storage with efficient data forwarding functi...
Reliable and confidential cloud storage with efficient data forwarding functi...
 
Rebuttal to “comments on ‘control cloud data access privilege and anonymity w...
Rebuttal to “comments on ‘control cloud data access privilege and anonymity w...Rebuttal to “comments on ‘control cloud data access privilege and anonymity w...
Rebuttal to “comments on ‘control cloud data access privilege and anonymity w...
 
Scalable cloud–sensor architecture for the internet of things
Scalable cloud–sensor architecture for the internet of thingsScalable cloud–sensor architecture for the internet of things
Scalable cloud–sensor architecture for the internet of things
 
Scalable algorithms for nearest neighbor joins on big trajectory data
Scalable algorithms for nearest neighbor joins on big trajectory dataScalable algorithms for nearest neighbor joins on big trajectory data
Scalable algorithms for nearest neighbor joins on big trajectory data
 
Robust workload and energy management for sustainable data centers
Robust workload and energy management for sustainable data centersRobust workload and energy management for sustainable data centers
Robust workload and energy management for sustainable data centers
 
Privacy preserving deep computation model on cloud for big data feature learning
Privacy preserving deep computation model on cloud for big data feature learningPrivacy preserving deep computation model on cloud for big data feature learning
Privacy preserving deep computation model on cloud for big data feature learning
 
Pricing the cloud ieee projects, ieee projects chennai, ieee projects 2016,ie...
Pricing the cloud ieee projects, ieee projects chennai, ieee projects 2016,ie...Pricing the cloud ieee projects, ieee projects chennai, ieee projects 2016,ie...
Pricing the cloud ieee projects, ieee projects chennai, ieee projects 2016,ie...
 
Protection of big data privacy
Protection of big data privacyProtection of big data privacy
Protection of big data privacy
 
Power optimization with bler constraint for wireless fronthauls in c ran
Power optimization with bler constraint for wireless fronthauls in c ranPower optimization with bler constraint for wireless fronthauls in c ran
Power optimization with bler constraint for wireless fronthauls in c ran
 
Performance aware cloud resource allocation via fitness-enabled auction
Performance aware cloud resource allocation via fitness-enabled auctionPerformance aware cloud resource allocation via fitness-enabled auction
Performance aware cloud resource allocation via fitness-enabled auction
 
Performance limitations of a text search application running in cloud instances
Performance limitations of a text search application running in cloud instancesPerformance limitations of a text search application running in cloud instances
Performance limitations of a text search application running in cloud instances
 
Performance analysis and optimal cooperative cluster size for randomly distri...
Performance analysis and optimal cooperative cluster size for randomly distri...Performance analysis and optimal cooperative cluster size for randomly distri...
Performance analysis and optimal cooperative cluster size for randomly distri...
 
Predictive control for energy aware consolidation in cloud datacenters
Predictive control for energy aware consolidation in cloud datacentersPredictive control for energy aware consolidation in cloud datacenters
Predictive control for energy aware consolidation in cloud datacenters
 

Hierarchical Tensor Approach to Compressing Geospatial Data

  • 1. A Hierarchical Tensor-Based Approach to Compressing, Updating and Querying Geospatial Data Abstract: With the rapid development of data observation and model simulation in geoscience, spatial-temporal data have become increasingly multidimensional, massive and are consistently being updated. As a result, the integrated maintenance of these data is becoming a challenge. This paper presents a blocked hierarchical tensor representation within the split- and-merge paradigm for the compressed storage, continuously updating and data querying of multidimensional geospatial field data. The original multidimensional geospatial field data are split into small blocks according to their spatial-temporal references. These blocks are represented and compressed hierarchically, and then combined into a single hierarchical tree as the representation of original data. With a buffered binary tree data structure and corresponding optimized operation algorithms, the original multidimensional geospatial field data can be continuously compressed, appended, and queried. Data from the 20th Century Reanalysis Monthly Mean Composites are used to evaluate the performance of this approach. Compared to traditional methods, the new approach is shown to retain the quality of the original data with much lower storage costs and faster computational performance. The result suggests that the blocked
  • 2. hierarchical tensor representation provides an effective structure for integrated storage, presentation and computation of multidimensional geospatial field data. Existing System: Large amounts of observation of exiting attributes/variables are uninterruptedly generated by global observation systems. These data are often compressed for storage. The newly arrived data should also be continuously compressed and appended to the existing data in such a way that the newly added should be integrated to the existing data and made as a whole with the existing data. In addition, this updating process should be completed in a short time and can be repeatedly applied for the next patch of new data. The compression and storage should maintain the consistency of the spatial-temporal reference (STR) of this data. Balance among the data accuracy, data compression performance and convenience for indexing, querying and analysis is required. Proposed System: Multidimensional data are stored and accessed linearly in memory and on hard disk but should be randomly queried and updated from any dimension. Classical methods use data indexes (e.g., Pyramid, RTree, VA- Files) to accelerate the query and storage.
  • 3. These data indexes split data into segments and then map the segments to the linear ordered data I/O sequence. When the dimension grows, both the data segmentation and the data structure are becoming complex and inefficient. Big data or data-intensive computing solutions use parallel data I/O and computation to accelerate the data accessing and updating. However, large number of computers and complicated computation architectures are required to provide the I/O bandwidth and computation power needed. Hardware Requirements: • System : Pentium IV 2.4 GHz. • Hard Disk : 40 GB. • Floppy Drive : 1.44 Mb. • Monitor : 15 VGA Colour. • Mouse : Logitech. • RAM : 256 Mb. Software Requirements: • Operating system : - Windows XP. • Front End : - JSP • Back End : - SQL Server Software Requirements:
  • 4. • Operating system : - Windows XP. • Front End : - .Net • Back End : - SQL Server