The document discusses various techniques for classifying pictures using neural networks, including convolutional neural networks. It describes how convolutional neural networks can be used to classify images by breaking them into overlapping tiles, applying small neural networks to each tile, and pooling the results. The document also discusses using recurrent neural networks to classify videos by treating them as higher-dimensional tensors.
ABC with data cloning for MLE in state space modelsUmberto Picchini
An application of the "data cloning" method for parameter estimation via MLE aided by Approximate Bayesian Computation. The relevant paper is http://arxiv.org/abs/1505.06318
Soft computing is likely to play aprogressively important role in many applications including image enhancement. The paradigm for soft computing is the human mind. The soft computing critique has been particularly strong with fuzzy logic. The fuzzy logic is facts representationas a
rule for management of uncertainty. Inthis paperthe Multi-Dimensional optimized problem is addressed by discussing the optimal thresholding usingfuzzyentropyfor Image enhancement. This technique is compared with bi-level and multi-level thresholding and obtained optimal
thresholding values for different levels of speckle noisy and low contrasted images. The fuzzy entropy method has produced better results compared to bi-level and multi-level thresholding techniques.
ABC with data cloning for MLE in state space modelsUmberto Picchini
An application of the "data cloning" method for parameter estimation via MLE aided by Approximate Bayesian Computation. The relevant paper is http://arxiv.org/abs/1505.06318
Soft computing is likely to play aprogressively important role in many applications including image enhancement. The paradigm for soft computing is the human mind. The soft computing critique has been particularly strong with fuzzy logic. The fuzzy logic is facts representationas a
rule for management of uncertainty. Inthis paperthe Multi-Dimensional optimized problem is addressed by discussing the optimal thresholding usingfuzzyentropyfor Image enhancement. This technique is compared with bi-level and multi-level thresholding and obtained optimal
thresholding values for different levels of speckle noisy and low contrasted images. The fuzzy entropy method has produced better results compared to bi-level and multi-level thresholding techniques.
Provides a brief overview of what machine learning is, how it works (theory), how to prepare data for a machine learning problem, an example case study, and additional resources.
Camp IT: Making the World More Efficient Using AI & Machine LearningKrzysztof Kowalczyk
Slides from the introductory lecture I gave for students at Camp IT 2019. I tried to cover artificial inteligence, machine learning, most popular algorithms and their applications to business as broadly as possible - for in-depth materials on the given topics, see links and references in the presentation.
Background Estimation Using Principal Component Analysis Based on Limited Mem...IJECEIAES
Given a video of 푀 frames of size ℎ × 푤. Background components of a video are the elements matrix which relative constant over 푀 frames. In PCA (principal component analysis) method these elements are referred as “principal components”. In video processing, background subtraction means excision of background component from the video. PCA method is used to get the background component. This method transforms 3 dimensions video (ℎ × 푤 × 푀) into 2 dimensions one (푁 × 푀), where 푁 is a linear array of size ℎ × 푤 . The principal components are the dominant eigenvectors which are the basis of an eigenspace. The limited memory block Krylov subspace optimization then is proposed to improve performance the computation. Background estimation is obtained as the projection each input image (the first frame at each sequence image) onto space expanded principal component. The procedure was run for the standard dataset namely SBI (Scene Background Initialization) dataset consisting of 8 videos with interval resolution [146 150, 352 240], total frame [258,500]. The performances are shown with 8 metrics, especially (in average for 8 videos) percentage of error pixels (0.24%), the percentage of clustered error pixels (0.21%), multiscale structural similarity index (0.88 form maximum 1), and running time (61.68 seconds).
U-Net is a convolutional neural network (CNN) architecture designed for semantic segmentation tasks, especially in the field of medical image analysis. It was introduced by Olaf Ronneberger, Philipp Fischer, and Thomas Brox in 2015. The name "U-Net" comes from its U-shaped architecture.
Key features of the U-Net architecture:
U-Shaped Design: U-Net consists of a contracting path (downsampling) and an expansive path (upsampling). The architecture resembles the letter "U" when visualized.
Contracting Path (Encoder):
The contracting path involves a series of convolutional and pooling layers.
Each convolutional layer is followed by a rectified linear unit (ReLU) activation function and possibly other normalization or activation functions.
Pooling layers (usually max pooling) reduce spatial dimensions, capturing high-level features.
Expansive Path (Decoder):
The expansive path involves a series of upsampling and convolutional layers.
Upsampling is achieved using transposed convolution (also known as deconvolution or convolutional transpose).
Skip connections are established between corresponding layers in the contracting and expansive paths. These connections help retain fine-grained spatial information during the upsampling process.
Skip Connections:
Skip connections concatenate feature maps from the contracting path to the corresponding layers in the expansive path.
These connections facilitate the fusion of low-level and high-level features, aiding in precise localization.
Final Layer:
The final layer typically uses a convolutional layer with a softmax activation function for multi-class segmentation tasks, providing probability scores for each class.
U-Net's architecture and skip connections help address the challenge of segmenting objects with varying sizes and shapes, which is often encountered in medical image analysis. Its success in this domain has led to its application in other areas of computer vision as well.
The U-Net architecture has also been extended and modified in various ways, leading to improvements like the U-Net++ architecture and variations with attention mechanisms, which further enhance the segmentation performance.
U-Net's intuitive design and effectiveness in semantic segmentation tasks have made it a cornerstone in the field of medical image analysis and an influential architecture for researchers working on segmentation challenges.
Fuzzy Entropy Based Optimal Thresholding Technique for Image Enhancement ijsc
Soft computing is likely to play aprogressively important role in many applications including image enhancement. The paradigm for soft computing is the human mind. The soft computing critique has been particularly strong with fuzzy logic. The fuzzy logic is facts representationas a rule for management of uncertainty. Inthis paperthe Multi-Dimensional optimized problem is addressed by discussing the optimal thresholding usingfuzzyentropyfor Image enhancement. This technique is compared with bi-level and multi-level thresholding and obtained optimal thresholding values for different levels of speckle noisy and low contrasted images. The fuzzy entropy method has produced better results compared to bi-level and multi-level thresholding techniques.
Deep Learning: concepts and use cases (October 2018)Julien SIMON
An introduction to Deep Learning theory
Neurons & Neural Networks
The Training Process
Backpropagation
Optimizers
Common network architectures and use cases
Convolutional Neural Networks
Recurrent Neural Networks
Long Short Term Memory Networks
Generative Adversarial Networks
Getting started
HUMAN DETECTION IN HOURS OF DARKNESS USING GAUSSIAN MIXTURE MODEL ALGORITHMijistjournal
Video surveillance systems are very important in our everyday life. Video surveillance applications are used in airports, banks, offices and even our homes to remain us secure. Night vision is the ability to see in low light conditions. Surveillance video may not be seen clearly. Especially under the weak illumination conditions. The details of the image are very poor at night. Most of the previous work has focused on daytime Surveillance. This paper proposes a method of Human detection in hours of darkness (night time) using Gaussian Mixture Model (GMM) in real night environment employing an infrared radiation camera. Infrared video is taken as Input to perform Human detection at night. Then the video was processed using Gaussian mixture model algorithm, it was confirmed that by using this method accurate detection of human at night is possible.
Video surveillance systems are very important in our everyday life. Video surveillance applications are
used in airports, banks, offices and even our homes to remain us secure. Night vision is the ability to see in
low light conditions. Surveillance video may not be seen clearly. Especially under the weak illumination
conditions. The details of the image are very poor at night. Most of the previous work has focused on
daytime Surveillance. This paper proposes a method of Human detection in hours of darkness (night time)
using Gaussian Mixture Model (GMM) in real night environment employing an infrared radiation camera.
Infrared video is taken as Input to perform Human detection at night. Then the video was processed using
Gaussian mixture model algorithm, it was confirmed that by using this method accurate detection of human
at night is possible.
Talk at the modcov19 CNRS workshop, en France, to present our article COVID-19 pandemic control: balancing detection policy and lockdown intervention under ICU sustainability
how to sell pi coins effectively (from 50 - 100k pi)DOT TECH
Anywhere in the world, including Africa, America, and Europe, you can sell Pi Network Coins online and receive cash through online payment options.
Pi has not yet been launched on any exchange because we are currently using the confined Mainnet. The planned launch date for Pi is June 28, 2026.
Reselling to investors who want to hold until the mainnet launch in 2026 is currently the sole way to sell.
Consequently, right now. All you need to do is select the right pi network provider.
Who is a pi merchant?
An individual who buys coins from miners on the pi network and resells them to investors hoping to hang onto them until the mainnet is launched is known as a pi merchant.
debuts.
I'll provide you the Telegram username
@Pi_vendor_247
how to sell pi coins at high rate quickly.DOT TECH
Where can I sell my pi coins at a high rate.
Pi is not launched yet on any exchange. But one can easily sell his or her pi coins to investors who want to hold pi till mainnet launch.
This means crypto whales want to hold pi. And you can get a good rate for selling pi to them. I will leave the telegram contact of my personal pi vendor below.
A vendor is someone who buys from a miner and resell it to a holder or crypto whale.
Here is the telegram contact of my vendor:
@Pi_vendor_247
NO1 Uk Black Magic Specialist Expert In Sahiwal, Okara, Hafizabad, Mandi Bah...Amil Baba Dawood bangali
Contact with Dawood Bhai Just call on +92322-6382012 and we'll help you. We'll solve all your problems within 12 to 24 hours and with 101% guarantee and with astrology systematic. If you want to take any personal or professional advice then also you can call us on +92322-6382012 , ONLINE LOVE PROBLEM & Other all types of Daily Life Problem's.Then CALL or WHATSAPP us on +92322-6382012 and Get all these problems solutions here by Amil Baba DAWOOD BANGALI
#vashikaranspecialist #astrologer #palmistry #amliyaat #taweez #manpasandshadi #horoscope #spiritual #lovelife #lovespell #marriagespell#aamilbabainpakistan #amilbabainkarachi #powerfullblackmagicspell #kalajadumantarspecialist #realamilbaba #AmilbabainPakistan #astrologerincanada #astrologerindubai #lovespellsmaster #kalajaduspecialist #lovespellsthatwork #aamilbabainlahore#blackmagicformarriage #aamilbaba #kalajadu #kalailam #taweez #wazifaexpert #jadumantar #vashikaranspecialist #astrologer #palmistry #amliyaat #taweez #manpasandshadi #horoscope #spiritual #lovelife #lovespell #marriagespell#aamilbabainpakistan #amilbabainkarachi #powerfullblackmagicspell #kalajadumantarspecialist #realamilbaba #AmilbabainPakistan #astrologerincanada #astrologerindubai #lovespellsmaster #kalajaduspecialist #lovespellsthatwork #aamilbabainlahore #blackmagicforlove #blackmagicformarriage #aamilbaba #kalajadu #kalailam #taweez #wazifaexpert #jadumantar #vashikaranspecialist #astrologer #palmistry #amliyaat #taweez #manpasandshadi #horoscope #spiritual #lovelife #lovespell #marriagespell#aamilbabainpakistan #amilbabainkarachi #powerfullblackmagicspell #kalajadumantarspecialist #realamilbaba #AmilbabainPakistan #astrologerincanada #astrologerindubai #lovespellsmaster #kalajaduspecialist #lovespellsthatwork #aamilbabainlahore #Amilbabainuk #amilbabainspain #amilbabaindubai #Amilbabainnorway #amilbabainkrachi #amilbabainlahore #amilbabaingujranwalan #amilbabainislamabad
how to sell pi coins in South Korea profitably.DOT TECH
Yes. You can sell your pi network coins in South Korea or any other country, by finding a verified pi merchant
What is a verified pi merchant?
Since pi network is not launched yet on any exchange, the only way you can sell pi coins is by selling to a verified pi merchant, and this is because pi network is not launched yet on any exchange and no pre-sale or ico offerings Is done on pi.
Since there is no pre-sale, the only way exchanges can get pi is by buying from miners. So a pi merchant facilitates these transactions by acting as a bridge for both transactions.
How can i find a pi vendor/merchant?
Well for those who haven't traded with a pi merchant or who don't already have one. I will leave the telegram id of my personal pi merchant who i trade pi with.
Tele gram: @Pi_vendor_247
#pi #sell #nigeria #pinetwork #picoins #sellpi #Nigerian #tradepi #pinetworkcoins #sellmypi
how to sell pi coins in all Africa Countries.DOT TECH
Yes. You can sell your pi network for other cryptocurrencies like Bitcoin, usdt , Ethereum and other currencies And this is done easily with the help from a pi merchant.
What is a pi merchant ?
Since pi is not launched yet in any exchange. The only way you can sell right now is through merchants.
A verified Pi merchant is someone who buys pi network coins from miners and resell them to investors looking forward to hold massive quantities of pi coins before mainnet launch in 2026.
I will leave the telegram contact of my personal pi merchant to trade with.
@Pi_vendor_247
What price will pi network be listed on exchangesDOT TECH
The rate at which pi will be listed is practically unknown. But due to speculations surrounding it the predicted rate is tends to be from 30$ — 50$.
So if you are interested in selling your pi network coins at a high rate tho. Or you can't wait till the mainnet launch in 2026. You can easily trade your pi coins with a merchant.
A merchant is someone who buys pi coins from miners and resell them to Investors looking forward to hold massive quantities till mainnet launch.
I will leave the telegram contact of my personal pi vendor to trade with.
@Pi_vendor_247
Resume
• Real GDP growth slowed down due to problems with access to electricity caused by the destruction of manoeuvrable electricity generation by Russian drones and missiles.
• Exports and imports continued growing due to better logistics through the Ukrainian sea corridor and road. Polish farmers and drivers stopped blocking borders at the end of April.
• In April, both the Tax and Customs Services over-executed the revenue plan. Moreover, the NBU transferred twice the planned profit to the budget.
• The European side approved the Ukraine Plan, which the government adopted to determine indicators for the Ukraine Facility. That approval will allow Ukraine to receive a EUR 1.9 bn loan from the EU in May. At the same time, the EU provided Ukraine with a EUR 1.5 bn loan in April, as the government fulfilled five indicators under the Ukraine Plan.
• The USA has finally approved an aid package for Ukraine, which includes USD 7.8 bn of budget support; however, the conditions and timing of the assistance are still unknown.
• As in March, annual consumer inflation amounted to 3.2% yoy in April.
• At the April monetary policy meeting, the NBU again reduced the key policy rate from 14.5% to 13.5% per annum.
• Over the past four weeks, the hryvnia exchange rate has stabilized in the UAH 39-40 per USD range.
NO1 Uk Rohani Baba In Karachi Bangali Baba Karachi Online Amil Baba WorldWide...Amil baba
Contact with Dawood Bhai Just call on +92322-6382012 and we'll help you. We'll solve all your problems within 12 to 24 hours and with 101% guarantee and with astrology systematic. If you want to take any personal or professional advice then also you can call us on +92322-6382012 , ONLINE LOVE PROBLEM & Other all types of Daily Life Problem's.Then CALL or WHATSAPP us on +92322-6382012 and Get all these problems solutions here by Amil Baba DAWOOD BANGALI
#vashikaranspecialist #astrologer #palmistry #amliyaat #taweez #manpasandshadi #horoscope #spiritual #lovelife #lovespell #marriagespell#aamilbabainpakistan #amilbabainkarachi #powerfullblackmagicspell #kalajadumantarspecialist #realamilbaba #AmilbabainPakistan #astrologerincanada #astrologerindubai #lovespellsmaster #kalajaduspecialist #lovespellsthatwork #aamilbabainlahore#blackmagicformarriage #aamilbaba #kalajadu #kalailam #taweez #wazifaexpert #jadumantar #vashikaranspecialist #astrologer #palmistry #amliyaat #taweez #manpasandshadi #horoscope #spiritual #lovelife #lovespell #marriagespell#aamilbabainpakistan #amilbabainkarachi #powerfullblackmagicspell #kalajadumantarspecialist #realamilbaba #AmilbabainPakistan #astrologerincanada #astrologerindubai #lovespellsmaster #kalajaduspecialist #lovespellsthatwork #aamilbabainlahore #blackmagicforlove #blackmagicformarriage #aamilbaba #kalajadu #kalailam #taweez #wazifaexpert #jadumantar #vashikaranspecialist #astrologer #palmistry #amliyaat #taweez #manpasandshadi #horoscope #spiritual #lovelife #lovespell #marriagespell#aamilbabainpakistan #amilbabainkarachi #powerfullblackmagicspell #kalajadumantarspecialist #realamilbaba #AmilbabainPakistan #astrologerincanada #astrologerindubai #lovespellsmaster #kalajaduspecialist #lovespellsthatwork #aamilbabainlahore #Amilbabainuk #amilbabainspain #amilbabaindubai #Amilbabainnorway #amilbabainkrachi #amilbabainlahore #amilbabaingujranwalan #amilbabainislamabad
how to sell pi coins on Bitmart crypto exchangeDOT TECH
Yes. Pi network coins can be exchanged but not on bitmart exchange. Because pi network is still in the enclosed mainnet. The only way pioneers are able to trade pi coins is by reselling the pi coins to pi verified merchants.
A verified merchant is someone who buys pi network coins and resell it to exchanges looking forward to hold till mainnet launch.
I will leave the telegram contact of my personal pi merchant to trade with.
@Pi_vendor_247
Introduction to Indian Financial System ()Avanish Goel
The financial system of a country is an important tool for economic development of the country, as it helps in creation of wealth by linking savings with investments.
It facilitates the flow of funds form the households (savers) to business firms (investors) to aid in wealth creation and development of both the parties
What website can I sell pi coins securely.DOT TECH
Currently there are no website or exchange that allow buying or selling of pi coins..
But you can still easily sell pi coins, by reselling it to exchanges/crypto whales interested in holding thousands of pi coins before the mainnet launch.
Who is a pi merchant?
A pi merchant is someone who buys pi coins from miners and resell to these crypto whales and holders of pi..
This is because pi network is not doing any pre-sale. The only way exchanges can get pi is by buying from miners and pi merchants stands in between the miners and the exchanges.
How can I sell my pi coins?
Selling pi coins is really easy, but first you need to migrate to mainnet wallet before you can do that. I will leave the telegram contact of my personal pi merchant to trade with.
Tele-gram.
@Pi_vendor_247
Poonawalla Fincorp and IndusInd Bank Introduce New Co-Branded Credit Cardnickysharmasucks
The unveiling of the IndusInd Bank Poonawalla Fincorp eLITE RuPay Platinum Credit Card marks a notable milestone in the Indian financial landscape, showcasing a successful partnership between two leading institutions, Poonawalla Fincorp and IndusInd Bank. This co-branded credit card not only offers users a plethora of benefits but also reflects a commitment to innovation and adaptation. With a focus on providing value-driven and customer-centric solutions, this launch represents more than just a new product—it signifies a step towards redefining the banking experience for millions. Promising convenience, rewards, and a touch of luxury in everyday financial transactions, this collaboration aims to cater to the evolving needs of customers and set new standards in the industry.
2. Statistics & ML Toolbox : Neural Nets
Very popular for pictures...
Picture xi is
• a n×n matrix in {0, 1}n2
for black & white
• a n × n matrix in [0, 1]n2
for grey-scale
• a 3 × n × n array in ([0, 1]3)n2
for color
• a T ×3×n ×n tensor in (([0, 1]3)T )n2
for
video
y here is the label (“8", “9", “6", etc)
Suppose we want to recognize a “6" on a
picture
m(x) =
+1 if x is a “6"
−1 otherwise
@freakonometrics freakonometrics freakonometrics.hypotheses.org 2 / 57
3. Statistics & ML Toolbox : Neural Nets
Consider some specifics pixels, and associate
weights ω such that
m(x) = sign
i,j
ωi,jxi,j
where
xi,j =
+1 if pixel xi,j is black
−1 if pixel xi,j is white
for some weights ωi,j (that can be nega-
tive...)
@freakonometrics freakonometrics freakonometrics.hypotheses.org 3 / 57
4. Statistics & ML Toolbox : Neural Nets
A deep network is a network with a lot of
layers
m(x) = sign
i
ωi mi (x)
where mi ’s are outputs of previous neural
nets. Those layers can capture shapes in
some areas nonlinearities, cross-dependence,
etc
@freakonometrics freakonometrics freakonometrics.hypotheses.org 4 / 57
5. Detection & Recognition
can be complicated... see Marr (1982, Vision)
@freakonometrics freakonometrics freakonometrics.hypotheses.org 5 / 57
6. Convolutional Network
source: Hastie et al. (2009, The Elements of Statistical Learning)
Neural Nets fails to recognize ’8’ when the digit is not centered
translation, scale and (small) rotation invariances are needed
@freakonometrics freakonometrics freakonometrics.hypotheses.org 6 / 57
7. Rotation, Scaling, Mixing, etc
See http://yann.lecun.com/exdb/lenet/
See also http://scs.ryerson.ca/∼aharley/vis/conv
@freakonometrics freakonometrics freakonometrics.hypotheses.org 7 / 57
8. Convolutional Network
Break the image into over-
lapping image tiles and, feed
each image tile into a small
neural network with the same
weights (and the same activa-
tion function)
1. ConvNets exploit spatially local correlation:
each neuron is locally- connected (to only a
small region of the input volume)
2. Reduce the size of the array, using a pooling
algorithm.
(pooling step reduces the dimensionality of
each feature)
@freakonometrics freakonometrics freakonometrics.hypotheses.org 8 / 57
9. Convolutional Network
pooling:
- makes the input representations smaller and more manageable
- reduces the number of weights and links in the network,
therefore, controlling overfitting
- makes the network invariant to small transformations, distortions
and translations in the input image
- helps us arrive at an almost scale invariant representation of our
image
(via yolo for objects detection)
@freakonometrics freakonometrics freakonometrics.hypotheses.org 9 / 57
11. Convolutional Network
Can be used to detect faces on a picture
via LeCun (2016, Leçon inaugurale)
@freakonometrics freakonometrics freakonometrics.hypotheses.org 11 / 57
12. Convolutional Network
and for object detection (even on a video)
via LeCun (2016, Leçon inaugurale)
@freakonometrics freakonometrics freakonometrics.hypotheses.org 12 / 57
14. Convolutional Network
In some application, need to find scale invariant transformations
via LeCun (2016, Leçon inaugurale)
@freakonometrics freakonometrics freakonometrics.hypotheses.org 14 / 57
15. Recognition with Neural Nets
Mid-2000s caltech101 dataset, with 101 categories, 30 pictures per
category (training sample)
2012 ImageNet dataset, 1.2 million (training) pictures, 20,000
categories see Li Fei-Fei’s page
OverFeat 2012 13.8% error,
GoogLeNet 2014 6.6%
(22 layers),
ResNet 2015 3.6% (152 layers)
see @sidereal’s post
@freakonometrics freakonometrics freakonometrics.hypotheses.org 15 / 57
16. From Pictures to Videos
A picture is a w × h × 3 tensor
A video is a w × h × 3 × t tensor
One can use RNN
recurrent neural nets (or LSTM)
Can also be use to identify objects
see https://towardsdatascience.com
@freakonometrics freakonometrics freakonometrics.hypotheses.org 16 / 57
17. Which Pictures? Satellite
From https://unequalscenes.com/, Hout Bay, about 15km south
of Cape Town, South Africa and Dar es Salaam, Tanzania.
@freakonometrics freakonometrics freakonometrics.hypotheses.org 17 / 57
18. Which Pictures? Satellite
From https://nytimes.com/, Chennai (Madras) in India, 2018 and
2019
@freakonometrics freakonometrics freakonometrics.hypotheses.org 18 / 57
19. Which Pictures? Satellite
via https://geoweb.iau-idf.fr/ (1949 on the left, 2019 on the right)
@freakonometrics freakonometrics freakonometrics.hypotheses.org 19 / 57
20. Which Pictures? Satellite
Possible use for crop insurance:
- To assess crop share (relative proportions) in a large area (no
georeference available of the fields)
- To estimate yield of an specific crop/season in a large area (no
georeference available of the fields)
- To detect and to georeference fields with specific crops
- To detect kind-of-crop info from specific fields
Various sources can be used :
- MODIS (Moderate Resolution Imaging Spectroradiometer)
250m × 250m, 2 images per day
- LANDSAT, 15m × 15m, 1 image every 16 days
(see also https://landsat.visibleearth.nasa.gov )
@freakonometrics freakonometrics freakonometrics.hypotheses.org 20 / 57
21. Which Pictures? Satellite
Moderate Resolution Imaging Spectroradiometer (MODIS) on
NASA’s Aqua satellite of the Camp Fire on November 12, 2018.
[Photo: NASA Earth Observatory vs Nov. 8, 2018, by Landsat 8,
shows short-wave infrared (red), which gives the full extent of the
actively burning area of the Camp Fire
@freakonometrics freakonometrics freakonometrics.hypotheses.org 21 / 57
22. Which Pictures? Satellite
Landsat picture = 17,500px × 14,500, 950Mb
Landsat 8 to cover China : 530 tiles, 23 per year ∼ 11Tb per year
@freakonometrics freakonometrics freakonometrics.hypotheses.org 22 / 57
23. Which Pictures? Satellite
Satellite pictures are
more than visible colors
TIRS (Thermal Infrared Sensor)
Practical problem : clouds
(see https://forestthreats.org/
or https://harrisgeospatial.com/)
@freakonometrics freakonometrics freakonometrics.hypotheses.org 23 / 57
24. Which Pictures? Satellite
Elowitz (2013, What is Imaging Spectroscopy (Hyperspectral
Imaging)?)
Infrareds used to produce (e.g.) some Normalized Difference
Vegetation Index (NDVI)
3d picture : lat, long, index (ndvi)
Practicals with Ewen Gallic on wildfires and natural disasters
@freakonometrics freakonometrics freakonometrics.hypotheses.org 24 / 57
25. Convolutional Network
The light blue grid is called the input feature
map. A kernel (shaded area) of value
0 1 2
2 2 0
0 1 2
slides across the input feature map.
(see http://deeplearning.net/)
@freakonometrics freakonometrics freakonometrics.hypotheses.org 25 / 57
27. Extracting Shapes & Contours
One can also compute gradients to extract contours
@freakonometrics freakonometrics freakonometrics.hypotheses.org 27 / 57
31. Picture Classifier
Still with those handwritten digit pictures, from the mnist dataset.
Here {(yi , xi )} with yi = “3” and xi ∈ [0, 1]28×28
One can define a multilinear principal component analysis
see also Tucker decomposition, from Hitchcock (1927, The
expression of a tensor or a polyadic as a sum of products)
We can use those decompositions to derive a simple classifier.
@freakonometrics freakonometrics freakonometrics.hypotheses.org 31 / 57
34. Picture Classifier
with the (classical) PCA
1 library(factoextra)
2 res.ind <- get_pca_ind(pca)
3 PTS <- res.ind$coord
4 value <- mnist$train$y[idx123 ][1:1000]
Consider a multinomial logistic regression, on the first k
components
5 k <- 10
6 df <- data.frame(y=as.factor(value), x=PTS[,1:k])
7 library(nnet)
8 reg <- multinom(y~.,data=df ,trace=FALSE)
9 df$pred <- predict(reg ,type =" probs ")
@freakonometrics freakonometrics freakonometrics.hypotheses.org 34 / 57
35. Picture Classifier
Consider a dataset with only three labels,
yi ∈ {1, 2, 3} and the k principal
components obtained by PCA,
{x1, · · · , x784} → {˜x1, · · · , ˜xk}
E.g. scatterplot {˜x1,i , ˜x2,i }
with • when yi = 1, • yi = 2 and • yi = 3,
Let mk denote the multinomial logistic regres-
sion
on ˜x1, · · · , ˜xk and visualize the misclassifica-
tion rate
But one can also use some neural network
model,
see Nielsen (2018, Neural Networks and Deep
Learning)
@freakonometrics freakonometrics freakonometrics.hypotheses.org 35 / 57
36. Neural Nets
Consider optimization prob-
lem
min
x∈X
f (x)
solved by gradient descent,
xn+1 = xn − γn f (xn), n ≥ 0
with starting point x0
γn can be call learning rate
Sometime the dimension of the dataset can be really big: we can’t
pass all the data to the computer at once to compute f (x)
we need to divide the data into smaller sizes and give it to our
computer one by one
@freakonometrics freakonometrics freakonometrics.hypotheses.org 36 / 57
37. Neural Nets
Batches
The Batch size is a hyperparameter that defines the num-
ber of sub-samples we use to compute quantities (e.g. the
gradient).
Epoch
One Epoch is when the dataset is passed forward and back-
ward through the neural network (only once).
We can divide the dataset of 2, 000 examples into batches of 400
then it will take 5 iterations to complete 1 epoch.
The number of epochs is the hyperparameter that defines the
number times that the learning algorithm will work through the
entire training dataset.
@freakonometrics freakonometrics freakonometrics.hypotheses.org 37 / 57
38. Picture Classifier
The algorithm is trained on tagged picture, i.e. (xi , yi ) where here
yi is some class of fruits (banana, apple, kiwi, tomatoe, lime,
cherry, pear, lychee, papaya, etc)
The challenge is to provide a new picture (xn+1 and to see the
prediction yn+1)
See fruits-360/Training, e.g. the banana (118)
@freakonometrics freakonometrics freakonometrics.hypotheses.org 38 / 57
40. Picture Classifier
We use here (pre-trained) VGG16 and VGG19 models for Keras,
see Simonyan & Zisserman (2014, Very Deep Convolutional
Networks for Large-Scale Image Recognition)
1 image_prep <- function(x) {
2 arrays <- lapply(x, function(path) {
3 img <- image_load(path , target_size = c(224 ,224))
4 x <- image_to_array(img)
5 x <- array_reshape(x, c(1, dim(x)))
6 x <- imagenet_preprocess_input (x)
7 })
8 do.call(abind ::abind , c(arrays , list(along = 1)))
9 }
10 res <- predict(model , image_prep(img_path))
To create our own model, see Chollet & Allaire (2018, Deep
Learning with R) [github]
@freakonometrics freakonometrics freakonometrics.hypotheses.org 40 / 57
41. Picture Classifier
1 imagenet_decode_predictions (res)
2 [[1]]
3 class_name class_description score
4 1 n07753592 banana 0.9929747581
5 2 n03532672 hook 0.0013420789
6 3 n07747607 orange 0.0010816196
7 4 n07749582 lemon 0.0010625814
8 5 n07716906 spaghetti_squash 0.0009176208
with 99.29% chance, the wikipedia picture of a
banana is a banana
@freakonometrics freakonometrics freakonometrics.hypotheses.org 41 / 57
42. Picture Classifier
With the lime package - Local Interpretable Model-Agnostic
Explanations we can also get some explaination of why
1 model_labels <- readRDS(system.file(’extdata ’, ’
imagenet_labels .rds ’, package = ’lime ’))
2 explainer <- lime(img_path , as_classifier(model ,
model_labels), image_prep)
3 explanation <- explain(img_path , explainer ,
4 n_labels = 2, n_features = 35,
5 n_superpixels = 35, weight = 10,
6 background = "white ")
7 plot_image_explanation (explanation)
(see also the vignette or the kitten example)
@freakonometrics freakonometrics freakonometrics.hypotheses.org 42 / 57
43. Picture Classifier
We can build a class activation map, http://gradcam.cloudcv.org
see Selvaraju et al. (2016, Visual Explanations from Deep
Networks via Gradient-based Localization)
Compute the gradient of the score for class c, yc (before softmax),
with respect to feature maps Ak of convolutional layer,
∂yc
∂Ak
Define the importance weights as an average of gradients
αc,k =
1
nx ny i j
∂yc
∂Ak
i,j
and define
relu
k
αc,kAk
On a 14 × 14 grid, we obtain the picture on
the right
@freakonometrics freakonometrics freakonometrics.hypotheses.org 43 / 57
48. Picture Classifier
Note that (xn+1 can be something that was not in the training
dataset (here yellow zucchini)
or some invented one (here a blue banana)
see also pyimagesearch for food/no food classifier
@freakonometrics freakonometrics freakonometrics.hypotheses.org 48 / 57
54. Picture Classifier
1 hist <- model %>% fit_generator(
2 # training data
3 train_image_array_gen ,
4 # epochs
5 steps_per_epoch = as.integer(train_samples /
batch_size),
6 epochs = epochs ,
7 # validation data
8 validation_data = valid_image_array_gen ,
9 validation_steps = as.integer(valid_samples /
batch_size),
10 # print progress
11 verbose = 2,
12 callbacks = list(
13 # save best model after every epoch
14 callback_model_checkpoint (" fruits_checkpoints .h5",
save_best_only = TRUE),
15 # only needed for visualising with TensorBoard
16 callback_tensorboard (log_dir = "logs ")))
@freakonometrics freakonometrics freakonometrics.hypotheses.org 54 / 57
55. Facial Recognition
For facial recognition, see Schwemmer (2018, facerec: An interface
for face recognition in R), based on kairos
or https://github.com/bnosac/image for various R functions
(Corner Dor Edges etection, etc).
@freakonometrics freakonometrics freakonometrics.hypotheses.org 55 / 57