Cloud-based Storage, Processing and Rendering for Gegabytes 3D Biomedical Images
•Download as PPTX, PDF•
0 likes•719 views
Cloud computing course project: Develop an cloud-based processing framework for large-volume 3D images with map-reduce on Hadoop platform.
Recommended
Recommended
More Related Content
What's hot
What's hot (18)
Viewers also liked
Viewers also liked (20)
Similar to Cloud-based Storage, Processing and Rendering for Gegabytes 3D Biomedical Images
Similar to Cloud-based Storage, Processing and Rendering for Gegabytes 3D Biomedical Images (20)
More from Yan Xu
More from Yan Xu (20)
Recently uploaded
Recently uploaded (20)
Cloud-based Storage, Processing and Rendering for Gegabytes 3D Biomedical Images
- 1. Team XYZ: Yan Xu Yu Zhang
- 2. Imaging Processing Communications Group A at place A Group B at place B (thousands of miles away) Long distance, Low efficiency
- 3. Commands Results Group A Group B YX YZ Hadoop
- 4. • 3D Image Storage: 2D Z slices in jpg • Libraries: HIPI (FloatImage, JPEGFileInputFormat). • Pipeline I (projection, non-overlapping patches): (xy index, FloatImage) 2D Z slices Projected Image non-overlap XY patches projection along z merge to one projected jpg Jpg seq same xy index
- 5. • Pipeline II (3-D filtering, overlapping patches): (xy index, FloatImage) overlap XY patches 3-D image filtering non-overlap XY patches (z index, FloatImage) merge XY patches Jpg seq named by xy_z.jpg Jpg seq Jpg seq same xy index same Z index 2D Z slices of original 3D image 2D Z slices of processed 3D image
- 6. 190 jpg sequences for one 3-D image (54.6MB, 1000 × 1000) Cropped into 5 × 5 patches (200 × 200) ) localhost:50030 Max Projection Mean Projection Test 3-D image:
- 7. Web Terminal Image Window Control Panel Display original 3D image Explore 3D image by slices Zoom in and out Hand move Display processing result
- 8. Tools O Map-reduce Java Javascript
- 9. Compatibility Compatible with ImageJ (open-source Java image processing tool) Extendibility Upload and run the script by users Attractiveness More user-friendly interface
- 10. Team XYZ XY YZ
Editor's Notes
- Our project is motivated from the cooperation conducted among multiple institutions for the biomedical image analysis . Group A is collecting the images using this cool microscope, another group located far away has enough computing power to process the images. Usually the collected images can reach hundred gigabytes or even more and it may takes several days to physically transfer from one place to another using a hard drive. And The imaging group don’t have direct access to view the result and give timely feedbacks to group B. We hope that we can help with this kind of cooperation, and shorten the project cycle.
- This is a prototype of our project. Images and stored and processed in cloud. And they can be viewed through a web terminal. We divide the work into two, image processing using map-reduce and web terminal to view the images. We set up single-node hadoop on our local machine and test our implementation there.
- First, I’m going to talk about image processing using map-reduce. The 3D images are stored in 2D slices, indexed by z. We developped our implementation based on HIPI, which is a hadoop image processing interface. The two most important classes used is FloatImage and JPEGFileInputFormat. FloatImage is a writable class, and JPEGFileInputFormat is inherited from FileInputFormat to read the jpg sequences. We mainly developped two processing pipelines. The first one is for the kind of image processing that does not require any neighborhood information of pixels. So we crop non-overlap patches from the original slices. For testing the pipeline, we do an image projection along the z direction by taking the max or mean of all pixels. The output of reduce is patches of projected image indexed by xy. Then we read the reduce output file and covert it to jpg, which is the projected image.
- The second pipeline is for a type of more general image processing that require the neighborhood information, such as 3-D median filtering and deconvolution. The strategy is to crop the overlap XY patches and keep the correct regions after processing by discarding the borders. The image patch is now the slashed region with the target region in the middle. The first map-reduce is to process the 3-D image patches. We restore the 3-D image patch in the first reduce function and can apply any 3-D image processing to the 3-D images. The second map-reduce is to discard the borders and reorganize the patches to generate 2D z slices of processed images.
- Here shows some running result. Our test data is 190 jpg sequences for one 3-D image. Each jpg is 1000 by 1000. We cropped each into 25 200 by 200 patches. We can view the status of the running jobs on hadoop on the webpage and here shows the projection result.
- We set up a http server to run our web page. It now mainly has two parts, one is the image window to display the original and processed images and the control panel. Since it’s 3D image, one need to explore it slice after slice using the previous and next. We can zoom in and out and hand move the image. Once push the max button here, it would show up the max-projection result.
- These are the tools used. For processing, we develop the map reduce codes in Java with the help of HIPI. For the web terminal, we run apache http server and tried multiple languages.
- The project is very interesting and more work can be developped within the current framework. We can consider to be compatible with ImageJ, which is an open-source image processing tool in Java. So that many image processing functions can be easily transferred into clouds. We also hope to extend the application in such a way that the users can upload image processing script themselves. And the last but not the least, the web interface can be more friendly and attractive.