Download as PDF, PPTX
Uploaded byMartin Svik
Power AI for Earth Observation Data
The document discusses using IBM's PowerAI platform and deep learning techniques to solve an object detection challenge involving satellite imagery. The initial approach uses a convolutional neural network trained on real and synthetic images. Accuracy improves from 44% to 64% by adding layers, dropout, and synthetic images. Residual and inception networks provide unstable gains. Further innovation allows non-differentiable features to boost accuracy above 90%. PowerAI is concluded to provide strong performance for deep learning and computer vision tasks.
Power AI for Earth Observation Data
- 1. Speaker Name :Dr. Martin Svik, MBA. Speaker Title : Executive IT Architect, SPEED Program WW Strategy Watson iLAB Leader Power AI and Earth Observation Data 7-8 June 2018, European GNSS Agency, Janovského 438/2, Prague 7
- 2. Agenda AI – Machine Learning/Deep Learning Evolution IBM Power for AI Challenge– Earth Observation Data
- 3.
- 4. IBM Power for AI • PowerAIthe fastest platform for Artificial Intelligence providing both the best of breed machine learning libraries • OpenCV image processing on PowerAI for Image data https://opencv.org/ P100 GPU POWER8 CPU GPU Memory System Memory P100 GPU 80 GB/s NVLink 115 GB/s P100 GPU POWER8 CPU System Memory P100 GPU 80 GB/s NVLink 115 GB/s 80 GB/s GPU Memory GPU Memory GPU Memory
- 5.
- 6.
- 7. French Military Intelligence Public Challenge Purpose: Detect objects for military intelligence Approach: • Speed dating to establish a partnership with universities, small companies, large companies • Free to select technical approach for detection Provided Data: •Unclassified Commercial Grade Satellite 15 x 15 miles geotiff 2GB images • .50cm per pixel (a car is 10 pixels) • 8,000 training thumbnails 81 x 81 pixels • 4 categories cars, trucks, mix and nothing Below images similar to the samples http://www.intelligencecampus.com/defis.html
- 8. Object Recognition Challenge implications Recognize anywhere, any time of day, any weather, any vehicle implies a minimum of 32 million combinations with usual classification hardly obtainable by a manual selection: · 9+1: 9 most popular vehicle colors 1 for all other colors · 24 vehicle models in the official classification · 10 landscape: city, coastal, mountain, snowy, desert, tropical, rocky, riverside, forest, farm field · 14lighting conditions under 5 latitudes, 12 hours sun light orientation plus night street light, partially shaded vehicles · 6 Weather conditions: sunny, fog, rain, dust wind, wind alone, snow · 4 traffic conditions · 8 or more view angles
- 9. Object Recognition Challenge Vehicles under shadow Original imageAfter histogram equalization of absolute difference of a normalized box filter 9*9 blur and gaussian 3*3 blur Typical problem on an aerial view with similar resolution: locate vehicles under shadow
- 10.
- 11. Proposed Solution • Creating high volume of synthetic images for complementing training data with our partner OKTAL-SE http://www.oktal-se.fr/website/ • Use PowerAIthe fastest platform for Artificial Intelligence providing both the best of breed machine learning libraries • Use OpenCV image processing on PowerAI https://opencv.org/ P100 GPU POWER8 CPU GPU Memory System Memory P100 GPU 80 GB/s NVLink 115 GB/s P100 GPU POWER8 CPU System Memory P100 GPU 80 GB/s NVLink 115 GB/s 80 GB/s GPU Memory GPU Memory GPU Memory Satellite Spectral response
- 12. Approach presentation : technical architecture Data volume, Learning Power and Performance Caffe NVCaffe TorchIBMCaffe Distributed TensorFlowTensorFlowTheano Open PowerAI Plateforms, Deep learning Chainer Learning metric assessment Creation of synthetic training image set (> million) Adjustment of Convolutional neuron network Implement on best Power AI learning platform Iteration for residuals mistakes adaptation Deployment models in production on PowerAI Accuracy, speed and fluidity Satellite Geotiff Image Scanning Call Model for classification Shapefile creation for display location on map Post ML validation Images 81x81 Local zone times N convolutions on 81 * 81 pixels M synthetic feature computation on image (contours, lines, contrast,..) M Convolution of synthetic features (angles, shadow, …) Classification in 4 categories Image Processing Neural Network Model
- 13.
- 14. Start with an approaching neural network •Based on MNIST database is a large database of handwritten digits used for training and testing machine learning models28x28 pixels panchromatic images •Keras is a high-level neural networks API, written in Python and capable of running on top of TensorFlow, CNTK, or Theano •This Keras Tutorial uses shows how to train a classifier for handwritten digits that boasts over 99% accuracy on the famous MNIST dataset with 60,000 images •Training images provided by customer are 81x81 but vehicles range from 10 pixels (cars) to 35 pixels (trucks) •Ratio between training images is 4:1 a minimum of 320,000 images is expected for training •Images set split in 60% Training, 20% Cross validation and 20% Test https://elitedatascience.com/keras-tutorial-deep-learning-in-python#step-4 MNIST Grayscale images 28x28 pixels 60,000 images in set https://en.wikipedia.org/wiki/MNIST_database KERAS Tutorial KERAS Python library https://keras.io/ batch = 1024 steps = 100 epochs = 100 Several hours Accuracy 44% Train & Test: model = Sequential() model.add(Convolution2D(32, 3, 3, activation='relu', input_shape=(1,81,81))) model.add(Convolution2D(32, 3, 3, activation='relu')) model.add(MaxPooling2D(pool_size=(2,2))) model.add(Dropout(0.25)) model.add(Flatten()) model.add(Dense(128, activation='relu')) model.add(Dropout(0.5)) model.add(Dense(4, activation='softmax')) Code: Adjust tutorial code to 81*81 pixels
- 15. Work on training data and neural network to fit use case •Perform image augmentationwith Generators in Keras. •Technique used to artificially expand the data-set. Thus the images can be blurred, shifted, rotated multiplying by 100 the initial training set size •Tune neural network layers •Tune dropout •Add fully connected layers •Add multithreading (very easy) •Add Keras multi GPU code, initially parallizer.py from Ronald Kemker, failed randomly, until version that supports unequal size of GPU datasets found •. Not tested as not available at the time of project, available since October 11, 2017 Keras support of Multi GPU : https://keras.io/getting-started/faq/#how-can-i-run- a-keras-model-on-multiple-gpus KERAS Image Augmentation with Generators Shift horizontally 20% Shift vertically 20% Flip horizontally IBM HPC and Neural Network experts Montpellier high-performance computing (HPC) team https://www.ibm.com/ibm/clientcenter/montpellier/b-pc.shtml batch = 1024 steps = 100 epochs = 100 One hour Accuracy 54% Train & Test: model = Sequential() model.add(Conv2D(1, (5,5), activation='relu', input_shape=(81,81,3))) model.add(Conv2D(32, (5, 5), activation='relu')) model.add(MaxPooling2D(pool_size=(2,2))) model.add(Conv2D(64, (3, 3), activation='relu')) model.add(Dropout(0.05)) print('conv2D 4') model.add(Conv2D(128, (3, 3), activation='relu')) model.add(Dropout(0.25)) model.add(Flatten()) model.add(Dense(256, activation='relu')) model.add(Dense(1024, activation='relu')) model.add(Dense(4, activation='softmax')) Code
- 16. Add training data and reduce overfitting •Work with partnerOKTAL-SE specialized in synthetic imaging (video and still) for aircraft manufacturers, radar manufacturers. •That company creates 3D landscape with physical characteristics, electromagnetic, visible, infrared etc. •Users of OKTAL-SE can create their own 3D environment or use provided environments, European country side, mountain, desert, maritime •For the project we defined 2 vehicle types, one trajectory is an mixed landscape with country side, houses, bridges, other parked vehicles, 2 hours of day, one dawn, one full daylight •2000+ images generated (before Keras Image Augmentation) •Neural Network code unchanged •Reduction of 15% of errors for the addition of 20% synthetic image OKTAL-SE Images variability controlled based on parameters Massive image generation easy to reach a million. Images produced from trajectories and view points. batch = 1024 steps = 100 epochs = 100 One hour Accuracy 64% Train & Test: Data Source: OKTAL-SE
- 17. Test Residual Networks and other models •Deep Residual Learning for Image Recognition were introduced by Microsoft research. They consist in a repetitive set of layers with an identity connection merge between the entry and the exit of the set avoiding loss of information from the initial data •Inception Neural Networks:compute in parallel and merge different convolutions recall the “histogram equalization of absolute difference of a normalized box filter 9*9 blur and gaussian 3*3 blur” on slide 7, inception is able to find the “difference of a normalized box filter 9*9 blur and gaussian 3*3 blur” part of the filter •Tried with 29 and 50 layers and other model •Did not try the Keras resnet50 and inception not available at time of project • https://keras.io/applications/#resnet50 • https://keras.io/applications/#inceptionv3 Residual Networks batch = 1024 steps = 100 epochs = 100, 200, 300 One hour per 100 Epochs Accuracy unstable 60%-66% depending on Epochs. Train & Test: Code https://arxiv.org/pdf/1512.03385.pdf Using available samples in github such as https://github.com/titu1994/Keras- ResNeXt/blob/master/resnext.py Inception https://www.ibm.com/blogs/research/2018/02/deep-learning-models/
- 18.
- 19. The limitation of Neural Networks 19 • “Convolutional Networks express a single differentiable function from raw image pixel values to class probabilities” https://cs.stanford.edu/people/karpathy/sfmltalk.pdf (except a few singularities). •A neural network cannot learn non-differentiable features e.g. Fractals, discrete data processing with absolute values. e.g. Sigmoid function
- 20. Final Test with non-differentiable features •I invented away to have non-differentiable features. It requires additional memory and CPU •PowerAI seamlessly processed the implied workload •Customer does not allow to give the exact resulting precision on the images (both small images test and satellite images) Preprocess to generate images With Non differentiable features neural network 700,000 images generated from clipped to 64 bits After Horizontal & vertical shift pixel by pixel and horizontal flip Found a way to have non-differentiable features processed in the Neural Network. batch = 1024 steps = 100 epochs 5 Accuracy increased above 90% Train & Test:
- 21.
- 22. Conclusion PowerAI Vision providesthe best of breed solution for most of the usual cases, without requiring neural network knowledge, just a few iterations to get to a satisfactory result See demonstration on https://www.youtube.com/watch?v=0F5w6q0ZpBI There are many industry domains where the classical deep learning techniques cannot reach the levels when clients require recognition as good as humans. Lack of training data can be fixed with synthetic images creation, which is very promising. The IBM patented process can provide an improved recognition. The patent requires a platform like PowerAI with enough compute power and memory to perform efficiently the training and testing.
- 23. Thank you 23 Speaker Name :Dr. Martin Svik, MBA. Speaker Title : Executive IT Architect, SPEED Program WW Strategy Watson iLAB Leader Contacts : martin.svik@cz.ibm.com +420 737 264 389 https://www.linkedin.com/in/martinsvik/