Downloaded 178 times
![SUMMIT EAST
Synchrotron-based X-Ray Tomographic Microscopy
The only technique which can do all
peer deep into large samples
achieve isotropic spatial
resolution
with 1.8mm field of view
achieve >10 Hz temporal resolution
8GB/s of images
[1] Mokso et al., J. Phys. D, 46(49),2013
< 1μm
Courtesy of M. Pistone at U. Bristol](https://image.slidesharecdn.com/spark-east-2015-151026164211-lva1-app6891/75/Interactive-Scientific-Image-Analysis-using-Spark-3-2048.jpg)
![SUMMIT EAST
Flexibility through Types
Developing in Scala brings additional
flexibility through types[1], with microscopy
the standard formats are 2-, 3- and even 4-
or more dimensional arrays or matrices
which can be iterated through quickly using
CPU and GPU code. While still possible in
Scala, there is a great deal more flexibility
for data types allowing anything to be
stored as an image and then processed as
long as basic functions make sense.
[1] Fighting Bit Rot with Types (Experience
Report: Scala Collections), M Odersky,
FSTTCS 2009, December 2009
What is an image?
A collection of positions and values, maybe
more (not an array of double). Arrays are
efficient for storing in computer memory,
but often a poor way of expressing scientific
ideas and analyses.
Filter Noise?
combine information from nearby
pixels
Find objects
determine groups of pixels
which are very similar to
desired result](https://image.slidesharecdn.com/spark-east-2015-151026164211-lva1-app6891/75/Interactive-Scientific-Image-Analysis-using-Spark-14-2048.jpg)
![SUMMIT EAST
Making Coding Simpler with Types
trait BasicMathSupport[T] extends Serializable {
def plus(a: T, b: T): T
def times(a: T, b: T): T
def scale(a: T, b: Double): T
def negate(a: T): T = scale(a,-1)
def invert(a: T): T
def abs(a: T): T
def minus(a: T, b: T): T = plus(a, negate(b))
def divide(a: T, b: T): T = times(a, invert(b))
def compare(a: T, b: T): Int
}](https://image.slidesharecdn.com/spark-east-2015-151026164211-lva1-app6891/75/Interactive-Scientific-Image-Analysis-using-Spark-15-2048.jpg)
(implicit val wst: BasicMathSupport[T]) = {
val width: Double = 1
kernel = (pos: D3int,value: T) => value * exp(-
(pos.mag/width)**2)
kernelReduce = (ptA,ptB) => (ptA + ptB) * 0.5
runFilter(inImage,kernel,kernelReduce)
}
implicit val SpectraBMS = new BasicMathSupport[Array[Double]] {
def plus(a: Array[Double], b: Array[Double]) =
a.zip(b).map(_ + _)
...
def scale(a: Array[Double], b: Double) =
a.map(_*b)](https://image.slidesharecdn.com/spark-east-2015-151026164211-lva1-app6891/75/Interactive-Scientific-Image-Analysis-using-Spark-16-2048.jpg)
![SUMMIT EAST
Science Problems: Big Stitching
Images : RDD[((x, y, z), Img[Double])] =
[( , Img), …]x⃗
dispField = Images.
cartesian(Images).map{
case ((xA,ImA), (xB,ImB)) =>
xcorr(ImA,ImB,in=xB-xA)
}](https://image.slidesharecdn.com/spark-east-2015-151026164211-lva1-app6891/75/Interactive-Scientific-Image-Analysis-using-Spark-23-2048.jpg)
Uploaded byKevin Mader
Interactive Scientific Image Analysis using Spark
The document outlines a presentation on interactive scientific image analysis using Apache Spark, highlighting techniques such as x-ray tomographic microscopy and hyperspectral imaging. It discusses the challenges posed by large datasets, the evolution of computing, and the necessity for robust frameworks like Spark for efficient analysis. Additionally, it emphasizes the importance of developing user-friendly tools to facilitate image processing and data exploration across various scientific domains.
![Coded Agents – with UiPath SDK + LangGraph [Virtual Hands-on Workshop]](https://cdn.slidesharecdn.com/ss_thumbnails/codedagentsdeck-251215155422-5497c599-thumbnail.jpg?width=640&height=640&fit=bounds)
Interactive Scientific Image Analysis using Spark
- 1.
- 2.
- 3. SUMMIT EAST Synchrotron-based X-Ray Tomographic Microscopy The only technique which can do all peer deep into large samples achieve isotropic spatial resolution with 1.8mm field of view achieve >10 Hz temporal resolution 8GB/s of images [1] Mokso et al., J. Phys. D, 46(49),2013 <1μm Courtesy of M. Pistone at U. Bristol
- 4. SUMMIT EAST Image Science in 2015: More and faster X-Ray Swiss Light Source (SRXTM) images at (>1000fps) 8GB/s, diffraction patterns (cSAXS) at 30GB/s Nanoscopium (Soleil), 10TB/day, 10- 500GB file sizes, very heterogenous data Optical Light-sheet microscopy (see of Jeremy Freeman) produces images 500MB/s High-speed confocal images at (>200fps) 78Mb/s Geospatial New satellite projects (Skybox, etc) will measure hundreds of terabytes to petabytes of images a year → talk → → Personal GoPro 4 Black - 60MB/s (3840 x 2160 x 30fps) for $600 - 400MB/s (640 x 480 x 840 fps) for $400 fps1000
- 5. SUMMIT EAST How much is a TB, really? If you looked at one 1000 x 1000 sized image every second It would take you 139 hours to browse through a terabyte of data. Year Time to 1 TB Man power to keep up Salary Costs / Month 20004096 min 2 people 25 kCHF 2008 1092 min 8 people 95 kCHF 2014 32 min 260 people 3255 kCHF 2016 2 min 3906 people 48828 kCHF
- 6. SUMMIT EAST Computing has changed: Parallel Moores Law Based on data from Transistors ∝2T/(18 months) https://gist.github.com/humberto- ortiz/de4b3a621602b78bf90d There are now many more transistors inside a single computer but the processing speed hasn't increased. How can this be? Multiple Core Many machines have multiple cores for each processor which can perform tasks independently Multiple CPUs More than one chip is commonly present New modalities GPUs provide many cores which operate at slow speed Parallel Code is important
- 7.
- 8.
- 9.
- 10.
- 11. SUMMIT EAST Spark -> Microscopy? These frameworks are really cool and Spark has a big vocabulary, but flatMap, filter, aggregate, join, groupBy, and fold still do not sound like anything I want to do to an image. I want to filter out noise, segment, choose regions of interest contour, component label measure, count, and analyze … Spark Image Layer Developed at , , and The Spark Image Layer is a Domain Specific Language for Microscopy for Spark. It converts common imaging tasks into coarse-grained Spark operations 4Quant ETH Zurich Paul Scherrer Institut
- 12.
- 13.
- 14. SUMMIT EAST Flexibility through Types Developing in Scala brings additional flexibility through types[1], with microscopy the standard formats are 2-, 3- and even 4- or more dimensional arrays or matrices which can be iterated through quickly using CPU and GPU code. While still possible in Scala, there is a great deal more flexibility for data types allowing anything to be stored as an image and then processed as long as basic functions make sense. [1] Fighting Bit Rot with Types (Experience Report: Scala Collections), M Odersky, FSTTCS 2009, December 2009 What is an image? A collection of positions and values, maybe more (not an array of double). Arrays are efficient for storing in computer memory, but often a poor way of expressing scientific ideas and analyses. Filter Noise? combine informationfrom nearby pixels Find objects determine groups of pixels which are very similar to desired result
- 15. SUMMIT EAST Making Coding Simpler with Types trait BasicMathSupport[T]extends Serializable { def plus(a: T, b: T): T def times(a: T, b: T): T def scale(a: T, b: Double): T def negate(a: T): T = scale(a,-1) def invert(a: T): T def abs(a: T): T def minus(a: T, b: T): T = plus(a, negate(b)) def divide(a: T, b: T): T = times(a, invert(b)) def compare(a: T, b: T): Int }
- 16. SUMMIT EAST Continuing with Types Simple filter implementation Spectra as well supported types def SimpleFilter[T](inImage:Image[T]) (implicit val wst: BasicMathSupport[T]) = { val width: Double = 1 kernel = (pos: D3int,value: T) => value * exp(- (pos.mag/width)**2) kernelReduce = (ptA,ptB) => (ptA + ptB) * 0.5 runFilter(inImage,kernel,kernelReduce) } implicit val SpectraBMS = new BasicMathSupport[Array[Double]] { def plus(a: Array[Double], b: Array[Double]) = a.zip(b).map(_ + _) ... def scale(a: Array[Double], b: Double) = a.map(_*b)
- 17.
- 18.
- 19.
- 20. SUMMIT EAST Genetic Studies using Spark Image Layer Analysis could be completed in several months (instead of 120 years, could now be completed in days in the cloud) Data can be freely explored and analyzed val bones= sc.loadImages("work/f2_bones/*/bone.tif") Segment hard and soft tissues Label cells Export results val hardTissue = bones.threshold(OTSU) val softTissue = hardTissue.invert val cells = hardTissue.componentLabel. filter(c=>c.size>100 & c.size<1000) cells.shapeAnalysis.WriteOutput("lacuna.csv")
- 21. SUMMIT EAST Parallel Tools for Image and Quantitative Analysis val cells= sqlContext.csvFile("work/f2_bones/*/cells.csv") val avgVol = sqlContext.sql("select SAMPLE,AVG(VOLUME) FROM cells GROUP BY SAMPLE") Collaborators / Competitors can verify results and extend on analyses Combine Images with Results avgVol.filter(_._2>1000).map(sampleToPath).joinByKey(bones) See immediately in datasets of terabytes which image had the largest cells New hypotheses and analyses can be done in seconds / minutes Task Single Core Time Spark Time (40 cores) Load and Preprocess 360 minutes 10 minutes Single Column Average 4.6s 400ms 1 K-means Iteration 2 minutes 1s
- 22. SUMMIT EAST Science Problems: Full Brain Imaging Collaboration with A. Astolfo and A. Patera Measure a full mouse brain (1cm ) with cellular resolution (1 m) 10 x 10 x 10 scans at 2560 x 2560 x 2160 14 TVoxels 0.000004% of the entire dataset 3 μ → 14TVoxels = 56TB Each scan needs to be registered and aligned together There are no computers with 56TB of memory Even multithreaded approachs are not feasible and require many logistics Analysis of the stitched data is also of interest (segmentation, vessel analysis, distribution and network connectivity)
- 23. SUMMIT EAST Science Problems: Big Stitching Images :RDD[((x, y, z), Img[Double])] = [( , Img), …]x⃗ dispField = Images. cartesian(Images).map{ case ((xA,ImA), (xB,ImB)) => xcorr(ImA,ImB,in=xB-xA) }
- 24. SUMMIT EAST From Matching to Stitching From the updated information provided by the cross correlations and by applying appropriate smoothing criteria (if necessary). The stitching itself, rather than rewriting the original data can be done in a lazy fashion as certain regions of the image are read. This also ensures the original data is left unaltered and all analysis is reversible. def getView(tPos,tSize)= stImgs. filter(x=>abs(x-tPos) <img.size). map { case (x,img) => val oImg = new Image(tSize) oImg.copy(img,x,tPos) }.addImages(AVG)
- 25.
- 26.
- 27. SUMMIT EAST Streaming Analysis Real-time Webcam Processing Filter images Create a background image val wr= new WebcamReceiver() val ssc = sc.toStreaming(strTime) val imgList = ssc.receiverStream(wr) val filtImgs = allImgs.mapValues(_.run("Median...","radius=3")) val totImgs = inImages.count() val bgImage = inImages.reduce(_ add _).multiply(1.0/totImgs)
- 28. SUMMIT EAST Identify Outliers in Streams Remove the background image and find the mean value Show the outliers val eventImages= filtImgs. transform{ inImages => val corImage = inImages.map { case (inTime,inImage) => val corImage = inImage.subtract(bgImage) (corImage.getImageStatistics().mean, (inTime,corImage)) } corImage } eventImages.filter(iv => Math.abs(iv._1)>20). foreachRDD(showResultsStr("outlier",_))
- 29.
- 30.
- 31. SUMMIT EAST A basic image filtering operation Thanks to Spark, it is cached, in memory, approximate, cloud-ready Thanks to Map-Reduce it is fault-tolerant, parallel, distributed Thanks to Java, it is hardware agnostic But it is also not really so readable def spread_voxels(pvec:((Int,Int),Double), windSize: Int = 1) = { val wind=(-windSize to windSize) val pos=pvec._1 val scalevalue=pvec._2/(wind.length*wind.length) for(x<-wind; y<-wind) yield ((pos._1+x,pos._2+y),scalevalue) } val filtImg=roiImg. flatMap(cvec => spread_voxels(cvec)). filter(roiFun).reduceByKey(_ + _)
- 32.
- 33. SUMMIT EAST Reality Check Spark is not performant dedicated, optimized CPU and GPU codes will perform slightly to much much better when evaulated by pixels per second per processing power unit these codes will be wildly outperformed by dedicated hardware / FPGA solutions Serialization overhead and network congestion are not neglible for large datasets →But Scala / Python in Spark is substantially easier to write and test Highly optimized codes are very inflexible Human time is 400x more expensive than AWS time Mistakes due to poor testing can be fatal Spark scales smoothly to enormous datasets GPUs rarely have more than a few gigabytes Writing code that pages to disk is painful Spark is hardware agnostic (no drivers or vendor lock-in)
- 34.
- 35.
- 36. SUMMIT EAST Feature Vectors A pairing between spatial information (position) and some other kind of information (value). We are used to seeing images in a grid format where the position indicates the row and column in the grid and the intensity (absorption, reflection, tip deflection, etc) is shown as a different color →x⃗ f⃗ The alternative form for this image is as a list of positions and a corresponding value x y Intensity 1 1 12 2 1 68 3 1 81 4 1 89 5 1 87 1 2 40 This representation can be called the feature vector and in this case it only has Intensity = ( , )I^ x⃗f ⃗
- 37.
- 38. SUMMIT EAST Adding a new feature In this case we add a very simple feature to the image, the distance from the center of the image (distance). x yIntensity Distance -10 -10 0.9350683 14.14214 -10 -9 0.7957197 13.45362 -10 -8 0.6045178 12.80625 -10 -7 0.3876575 12.20656 -10 -6 0.1692429 11.66190 We now have a more complicated image, which we can't as easily visualize, but we can incorporate these two pieces of information together.
- 39.
- 40. SUMMIT EAST Common Features The distance while illustrative is not a commonly used features, more common various filters applied to the image Gaussian Filter (information on the values of the surrounding pixels) Sobel / Canny Edge Detection (information on edges in the vicinity) Entroy (information on variability in vicinity) x yIntensity Sobel Gaussian 1 1 0.94 0.32 0.53 1 10 0.48 0.50 0.45 1 11 0.50 0.50 0.46 1 12 0.48 0.64 0.46 1 13 0.43 0.78 0.45 1 14 0.33 0.94 0.42
- 41.
- 42. SUMMIT EAST Using Machine Learning Now that the images are stored as feature vectors, they can be easily analyzed with standard Machine Learning tools. It is also much easier to combine with training information. x yAbsorb Scatter Training 700 4 0.3706262 0.9683849 0.0100140 704 4 0.3694059 0.9648784 0.0100140 692 8 0.3706371 0.9047878 0.0183156 696 8 0.3712537 0.9341989 0.0334994 700 8 0.3666887 0.9826912 0.0453049 704 8 0.3686623 0.8728824 0.0453049 Want to predict Training from x,y, Absorb, and Scatter MLLib: Logistic Regression, Random Forest, K- Nearest Neighbors, … →
- 43. SUMMIT EAST Beyond Image Processing For many datasets processing, segmentation, and morphological analysis is all the information needed to be extracted. For many systems like bone tissue, cellular tissues, cellular materials and many others, the structure is just the beginning and the most interesting results come from the application to physical, chemical, or biological rules inside of these structures. = m ∑ j F⃗ijx¨i Such systems can be easily represented by a graph, and analyzed using GraphX in a distributed, fault tolerant manner.
- 44. SUMMIT EAST Hadoop Filesystem (HDFS not HDF5) Bottleneck is filesystem connection, many nodes (10+) reading in parallel brings even GPFS-based infiniband system to a crawl One of the central tenants of MapReduce™ is data-centric computation instead of data to computation, move the computation to the data. Use fast local storage for storing everything redundantly less transfer and fault-tolerance Largest file size: 512 yottabytes, Yahoo has 14 petabyte filesystem in use → →