Published on

Published in: Education, Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide
  • In fact, the amount of data to be transferred from end users to nodes and vice-versa is quite large. The data movement, considering that it can be a bottle neck, impacts on the global performance of the architecture.

    1. 1. Grid Computing in Remote Sensing Data Processing Vishnu-131858
    2. 2. Contents  Introduction to Grid computing Requirements for remote sensing data processing on grid environment Advantages for remote sensing systems to employ a grid computing The structure and character of system based on grid Grid middleware for remote sensing image processing Literature review Case study1 Case study2 Summery References
    3. 3. Introduction A huge amount of remotely sensed data is acquired every day. These Remote sensing images need to be processed and stored in real time or near-real time. Processing the huge amount of data with complicated algorithms needs great computational power beyond a single machine because it is not only data intensive, but also computation intensive. Grid computing is a promising technology that seeks to easily and efficiently coordinates the sharing of geographically distributed computing resources, thereby bring supercomputing power to its users. Some grid projects have been carried out, such as Globus, Condor and Unicore.
    4. 4. Requirements for remote sensing data processing on grid environment Main objective is to design and build a Grid-Based Problem Solving Environment to allow sharing of software and computational resources for remote sensing data processing among different organizations, through high speed networks. The following requirements are identified for this grid environment: Sharing of computational resources Sharing of software resources Management of resources. Secure usage of the services available on the grid in agreement with the access policies.  Efficient usage of the network links. Composing and compiling of new processing applications upon the cooperation of existing ones. Supporting different data formats.
    5. 5. ATYPICAL VIEW OF GRID ENVIRONMENT User Resource Broker Grid Resources Grid Information Service A User sends computation or data intensive application to Global Grids in order to speed up the execution of the application. A Resource Broker distribute the jobs in an application to the Grid resources based on user’s QoS requirements and details of available Grid resources for further executions. Grid Resources (Cluster, PC, Supercomputer, data base, instruments, etc.) in the Global Grid execute the user jobs. Grid Information Service system collects the details of the available Grid resources and passes the information to the resource broker. Computation result Grid application Computational jobs Details of Grid resources Processed jobs
    6. 6. Advantages for remote sensing systems to employ a grid computing • Cost savings through the sharing of resources • Scalability to meet variations in resource demands and balance work loads • Shorter time to results • Enable collaboration across organizations and among widely distributed users, by sharing resources (data, software, and hardware). • More efficient use of available resources • Increased productivity through standardization and access to additional resources.
    7. 7. The Structure of Remote sensing processingsystem based on Grid The overall architecture of remote sensing processing system based on Grid and service is composed of a structure which includes the following four layers Data layer it can offer various raw products to processing service unified system upon users’ request for further and deeper processing results. Processing layer is formed by various services for remote sensing processing whose package mode are Grid services based on high-performance environment . Figure shows the layered system architecture and its relationship
    8. 8. Grid layer, the main role it plays is managing both the data and service resources so that to achieve the goal of sharing the resources, providing a unified system operating environment of remote sensing application and coordinating the tasks of remote sensing processing. Application support layer.  Its role is the connectivity between the remote sensing systems facing end-users and various sources of remote sensing processing in Grid. It is formed by middleware supporting Grid, Tools such as automatic generation tools of remote sensing application, user interfaces, application deployment tools, etc.
    9. 9. Grid Middleware for remote sensing image Processing Grid middleware for remote sensing image processing to meet the needs of real time or near-real time processing. The middleware is an advanced high throughput Condor-based computing grid middleware that supports The execution of remote sensing image Processing over a geographically distributed, and Heterogeneous collection of resources. Condor system provides the technologies for resource discovery, dynamic task-resource matching, communication, and result collection, etc.
    10. 10. The scheme of grid middleware for image processing
    11. 11. Functions of each sub-modules: Tasks partition --user can input the number of tasks partitioned, and then the tasks will be auto-partitioned. Task submission --user can submit all the partitioned tasks to the grid manager. Task execution --include remote sensing image processing programs, when the tasks are submitted, relative programs will be remotely called by the grid nodes for the task execution. Task monitoring --used to monitor the image processing progress, Result collection --all the execution results on each grid node will be auto-collected, Result piecing --all the returned results will be merged into one final whole image, which is the final result of image processing just like that obtained on a single computer.
    12. 12. Literature review • Chaolin Wu analysed a way to solving the large-scale or complex problems through grid computing. • To apply the Grid computing to remote sensing information, a Remote Sensing Information Grid analysis and Service Node (RSIGN) has been established in Tele-geoprocessing Group, Institute of Remote sensing Applications, Chinese Academy of Sciences. • The management and distribution of the tasks are key problems in RSIGN node. How to distribute and manage the tasks has significant influence on the efficiency of the whole Grid system. • Also discussed on two main strategies for RSIGN node: geometric parallel and algorithm parallel. and • How to choose the best task managing strategy for different problems in remote sensing information processing and analysis.
    13. 13. Yong Xue, Tong Yu et al.(2006) proposed, A New Approach to Generate the Look-Up Table for Aerosol Remote Sensing on Grid Platform. LUT is popular used in aerosol remote sensing retrieval which is a matrix of dependency variable corresponding a set of combinations of independent variable values. They used 6S module to generate LUT. And Also discuss approaches to parameterization, task partitioning, generated methodology, and the collection of result. Jing Dong (2012) designed The cluster as a hierarchical one, and achieved a Global Task Scheduler for the workflow process, improved the system performance. It is One of the most effective solutions is Grid Computing platform built on the basis of network, which is a scalable virtual unified platform with unlimited computing power and storage capacity. The RSSN (Remote Sensing Information Service Grid Node) is a PC cluster for Remote Sensing Quantitative Retrieval. It was used for producing aerosol optical depth (AOD) production covered the land area of Asia. The Computing pool was built with common commercial PCs instead of Servers or Workstations because of their price advantage. The PCs were connected as a loosely coupled LAN via Network Switches.
    14. 14. Casestudy1 Analysis of image de-blurringdistributedprocessing • To test the performance of distributed image deblurring on a grid, the first experiment was conducted on a grid testbed of 4 commodity PCs nodes connected using 100Mbps Ethernet switch have the same performance. • The image data used for their experiments is 128Mb satellite remotely sensed image. • The geometric parallel strategy is taken in our distributed processing experiments.
    15. 15. • Figure shows The overlap strategy of image division. (Grey area is the neighbourhood pixels for sub-image1) • The whole image was partitioned into 4 sub-images as equally as possible, which is same to the number of grid nodes. • To check the performance of our grid middleware and grid computing test bed, a comparison is made between our work and a traditional computational test on a single computer conducted with only one grid node of the test bed.
    16. 16.  Fig shows The comparison between the original image (upper) and de-blurred image (bottom) on grid test bed  Table shows the performance for 4 tasks partitioned and 4 available grid nodes of grid computing testbed-1 in their preliminary experiment.
    17. 17. • The execution time on testbed-1 Tg is much less than that on a single computer Ts, which demonstrates the good performance of thier grid middleware for improving the efficiency of image deblurring processing. • Conclusion: • By means of image processing grid middleware, grid nodes can be aggregated to meet the needs of intensive remote sensing image processing computations. Using grid computing platform, real time or near real time remote sensing image processing may be realized
    18. 18. Casestudy2 ANewApproachto Generate the Look-UpTable forAerosolRemote Sensing on Grid Platform • This case study focuses on realization of the compute-intensive look- up table generation on GCP-ARS (Grid Computation Platform for Aerosol Remote Sensing), which was one grid middleware that are developing based on Condor system. • GCP-ARS architecture mainly consists of three entities: clients, a resource broke, and producers. • Aerosol retrieval is launched though a client GUI for execution at producers that share its idle cycles through the resource broker. • The resource broke manages tasks application execution, resource management and result collection by means of Condor system.
    19. 19. Methodology For aerosol retrieval, they refer to apparent reflectance matrix corresponding to Sensor-Target-Solar geometric conditions, ground surface reflectance, aerosol and atmosphere conditions. 6S methodology is selected for LUT generation, which is Radiative Transfer Models (RTM) for atmospheric radiative calculation. 6S codes can simulate the radiance and reflectance in the solar spectral region (0.25- 4.0um) considering the scattering and absorbing atmospheric effects due to gases and aerosol, as well as the surface effects. • The generation of LUT is actually time-consumed. • Building one LUT of 91125 input combinations sets needs about 2 days on one PC. • Figure 4.2.1 show the flow chart to generate the LUT.
    20. 20.  Parameterization include Radiative Transfer Model parameters input, LUT structure parameters such as maximum, minimum value and of each independent variable, which decide the structure and size of the LUT, tasks partition parameters, execution program (default is 6S codes) selection.  Execution phase refer to the computation of dependency variable by execution program.  The final LUT will be saved at termination
    21. 21. EXPERIMENTSANDANALYSIS Experiments were carried out on a grid computing pool consists of 5 low commodity PCs nodes connected using 100Mbps Ethernet switch. Their test is to generate one Apparent Reflectance LUT corresponding to independent variables: solar zenith, sensor zenith, relative zenith, ground surface reflectance and aerosol optical thickness (AOT). Table shows the structure information of LUT generated
    22. 22.  Table shows the performance for varying number of tasks partitioned and producers.  The sequential execution time(Ts) was from test on a single computer. The execution time (Tn) decreases with the number of tasks partitioned increasing when the number of available producers is more than number of tasks partitioned due to more producers contribute to the execution at a time.
    23. 23. CONCLUSION: • LUT could be quickly set up through grid computing technique, which, by pooling and aggregating the idle CPU circle together, has high throughput computing to meet the needs of intensive computations. • The LUT generated by 6S module could be used for GCP-ARS to retrieve the aerosol optical thickness (AOT).
    24. 24. Summery • Grid platform for Distributed Remote sensing Image processing are very encouraging, which show the good potential of grid computing technology for satellite image processing. • By means of image processing grid middleware, grid nodes can be aggregated to meet the needs of intensive remote sensing image processing computations. • Using grid computing platform, real time or near real time remote sensing image processing may be realized.
    25. 25. REFERENCES • Giovanni Aloisio, Massimo Cafaro, Italo Epicoco, Gianvito Quarta; “A Problem Solving Environment for Remote Sensing Data Processing”, Proceedings of the International Conference on Information Technology: Coding and Computing (2004) • Jiakui Tang, Aijun Zhang, Shengyang Li “Preliminary Research on Grid- based Remote Sensing Image distributed Processing” IFIP International Conference on Network and Parallel Computing – Workshops,2007. • L. Fusco, R. Cossu, and C. Retscher. “Open Grid Services for Envisat and Earth observation applications,” in High Performance Computing in Remote Sensing, A. Plaza and C. Chang (Eds.), Chapman & Hall, Taylor & Francis Group, 2008 • Liang zhonga & hongchao maa ” research of fast processing and distribution remote sensing image based on the grid technique” the international archives of the photogrammetry, remote sensing and spatial information sciences. vol. xxxvii. part b4. beijing 2008.
    26. 26. • Tang, J.K., Xue, Y., Guan,”A New Approach to Generate the Look-up Table Using Grid Computing Platform for Aerosol Remote Sensing”, 2004 IEEE International Geoscience and Remote Sensing Symposium. • Tianhe “Eco-environmental Information Grid Platform and Its Implement” Ministry of Science & Technology of Zhejiang, 2010. • Wan wei1, yong xue “A high performance remote sensing retrieval application on an institutional desktop grid” China Center for Resource Satellite Data and Application, Beijing 100830, China. 2009
    27. 27. •Any queries…
    28. 28. Thank you…