paper 6

1. See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/331730377
Estimating Compressive Strength of Concrete Using Deep Convolutional Neural
Networks with Digital Microscope Images
Article in Journal of Computing in Civil Engineering · May 2019
DOI: 10.1061/(ASCE)CP.1943-5487.0000837
CITATIONS
25
READS
1,719
3 authors, including:
Some of the authors of this publication are also working on these related projects:
Impact of Social and Psychological factors on workers productivity during Covid-19 Pandemic View project
Modular Construction View project
Youjin Jang
North Dakota State University
19 PUBLICATIONS 176 CITATIONS
SEE PROFILE
Yong Han Ahn
Hanyang University
138 PUBLICATIONS 1,076 CITATIONS
SEE PROFILE
All content following this page was uploaded by Yong Han Ahn on 08 May 2019.
The user has requested enhancement of the downloaded file.

2. Estimating Compressive Strength of Concrete
Using Deep Convolutional Neural Networks
with Digital Microscope Images
Youjin Jang1
; Yonghan Ahn2
; and Ha Young Kim3
Abstract: Compressive strength is a critical indicator of concrete quality for ensuring the safety of existing concrete structures. As an alter-
native to existing nondestructive testing methods, image-based concrete compressive strength estimation models using three deep convolu-
tional neural networks (DCNNs), namely AlexNet, GoogLeNet, and ResNet, were developed for this study. Images of the surfaces of specially
produced specimens were obtained using a portable digital microscope, after which the samples were subjected to destructive tests to evaluate
their compressive strength. The results were used to create a dataset linking the experimentally determined compressive strength with the image
data recorded for each. The results of training, validation, and testing showed that DCNN models largely outperformed the recently proposed
image processing–based ANN model. Overall, the ResNet-based model exhibited greater compressive strength estimation accuracy than either
the AlexNet- or GoogLeNet-based models. These finding indicate that image data obtained using a portable digital microscope contain patterns
that can be correlated with the concrete’s compressive strength, enabling the proposed DCNN models to use these patterns to estimate com-
pressive strength. The results of this study demonstrate the applicability of DCNN models using microstructure images as an auxiliary method
for the nondestructive evaluation of concrete compressive strength. DOI: 10.1061/(ASCE)CP.1943-5487.0000837. © 2019 American Society
of Civil Engineers.
Author keywords: Concrete; Compressive strength; Deep convolutional neural network; Estimation model; Digital microscope image.
Introduction
Concrete is one of the world’s most widely used building materials.
It is obtained by mixing aggregates, cement, water, and any addi-
tives required to achieve the desired properties. Due to its easy
availability, low cost, convenient handling, and the option to shape
it into any desired form, concrete is ubiquitous in the construction
industry, and most buildings today contain RC elements. One of the
main indicators used for evaluating the condition of existing con-
crete structures is the compressive strength of the concrete from
which they are constructed (Tiberti et al. 2015; Baygin et al. 2018).
The compressive strength is generally defined as the failure load of
the concrete under specific loading conditions. Evaluating the com-
pressive strength of concrete is vital for assessing the deterioration
of concrete structures and ensuring their safety (Steenbergen and
Vervuurt 2012).
There are two main approaches to evaluating the compres-
sive strength of concrete: destructive and nondestructive testing.
Destructive test methods measure the compressive strength in a lab-
oratory environment using a concrete core sample obtained from
the actual concrete structure being tested. The compressive strength
corresponds to the nominal stress at which that specimen or con-
crete core fails under uniaxial loading. However, taking concrete
cores is costly and can lead to safety problems because it can
easily damage the concrete structure being tested. Nondestructive
test methods such as the rebound hammer (RH) test, ultrasonic
pulse velocity (UPV) test, and SonReb seek to avoid these
problems by estimating the compressive strength of the concrete
using empirical formulas. However, although these methods give
approximate results for the compressive strength of the concrete
on site without damaging the actual concrete structure, they re-
quire expensive equipment and careful instrument maintenance,
as well as trained and certified personnel with a high degree of
skill and integrity. Recently, a number of image processing–based
methods for the estimation of concrete compressive strength have
been proposed as alternative nondestructive test methods (Başyiğit
et al. 2012; Dogan et al. 2017). The use of images for estimating
the compressive strength of concrete has a number of advantages
because it potentially reduces both the time and cost required
to conduct the tests. Several studies have thus sought to estimate
concrete compressive strength using statistical analysis and artifi-
cial neural networks (ANN) based on data obtained from image
processing and the results of experimental tests conducted using
traditional destructive testing methods (Başyiğit et al. 2012;
Dogan et al. 2017). Unfortunately, as yet these proposed image
processing–based studies suffer from limitations when applied in
situ, and feature engineering is challenging for complex concrete
images.
To address these issues, this study developed a new image-based
model for estimating concrete compressive strength using deep
convolutional neural networks (DCNNs) to analyze images
collected with a portable digital microscope. DCNN is a deep learn-
ing technique that can be used to autonomously extract complex
discriminative features via a learning procedure, thus reducing
1
Postdoctoral Researcher, School of Architecture and Architectural
Engineering, Hanyang Univ., 55 Hanyangdaehak-ro, Sangrok-gu, Ansan-si,
Gyeonggi-do 15588, Republic of Korea. Email: uzjang@gmail.com
2
Associate Professor, School of Architecture and Architectural Engi-
neering, Hanyang Univ., 55 Hanyangdaehak-ro, Sangrok-gu, Ansan-si,
Gyeonggi-do 15588, Republic of Korea. Email: yhahn@hanyang.ac.kr
3
Assistant Professor, Dept. of Financial Engineering, Ajou Univ., 206
Worldcupro, Yeongtong-gu, Suwon, Gyeonggi-do 16499, Republic of
Korea. (corresponding author). Email: hayoungkim@ajou.ac.kr
Note. This manuscript was submitted on July 17, 2018; approved on
November 6, 2018; published online on February 28, 2019. Discussion
period open until July 28, 2019; separate discussions must be submitted
for individual papers. This paper is part of the Journal of Computing
in Civil Engineering, © ASCE, ISSN 0887-3801.
© ASCE 04019018-1 J. Comput. Civ. Eng.
J. Comput. Civ. Eng., 2019, 33(3): 04019018
Downloaded
from
ascelibrary.org
by
HANYANG
UNIVERSITY
on
03/01/19.
Copyright
ASCE.
For
personal
use
only;
all
rights
reserved.

3. the need for human input to identify features of interest. DCNNs
have already achieved highly accurate results in the field of image
recognition for applications such as face recognition (Derman and
Salah 2018), autonomous vehicles (Nugraha et al. 2017), and medi-
cal diagnosis (Sun et al. 2017). In particular, a number of successful
applications of DCNNs have been reported in the field of construc-
tion engineering, including pavement crack detection (Zhang et al.
2017; Gopalakrishnan et al. 2017), concrete crack detection (Cha
et al. 2017), and structural damage detection (Lin et al. 2017).
Building on this earlier work, we therefore applied DCNNs for the
estimation of the compressive strength of concrete based on con-
crete surface images. The performance of DCNN models varies con-
siderably depending on their architecture, with factors such as the
number of layers, units per layer, and size of the convolutional mask
all affecting the results (Ferreira et al. 2018). We therefore used
modified versions of three representative DCNN models, namely
AlexNet, GoogLeNet, and ResNet, to estimate concrete compres-
sive strength and then compared their performance. A perfor-
mance comparison with the image processing–based ANN model
recently proposed by Dogan et al. (2017) was also conducted. The
training, validation, and testing operations were accomplished us-
ing datasets created using concrete specimens prepared in the lab-
oratory. The estimation accuracy achieved by each DCNN model
was evaluated in terms of the value of its coefficient of determi-
nation (R2
), its mean absolute percentage error (MAPE), and its
root-mean-square error (RMSE). The results provide a useful
reference with which to assess the suitability of DCNN as a non-
destructive test method to estimate concrete compressive strength
based on image data.
The remainder of this paper is organized as follows. The next
section reviews the literature on estimating concrete compressive
strength and DCNN models. Then the research methodology is
described including the dataset creation process, performance
evaluation measures, and the experimental settings. Lastly, the ex-
perimental results and discussions are presented, and the paper con-
cludes with a summary of the findings and suggestions for further
study.
Literature Review
Estimation of Concrete Compressive Strength
With regard to the safety management of existing concrete struc-
tures, compressive strength is considered the most critical indicator
of concrete quality (Ju et al. 2017). Due to the complex degradation
mechanisms involved and the multiple factors governing each,
evaluating and estimating the compressive strength of concrete re-
mains a challenging issue. To determine the compressive strength
of concrete, destructive test methods are the most reliable, but it is
not feasible to examine the in situ concrete properties without dam-
aging the structure. As a result, nondestructive test methods offer an
attractive alternative, and researchers are constantly seeking to
develop better nondestructive test methods for estimating the com-
pressive strength of concrete.
Existing nondestructive test methods include the rebound ham-
mer test, ultrasonic pulse velocity test, pull-out test, penetration re-
sistance test, magnetic test, and radioactive test. Among these, the
ultrasonic pulse velocity test, rebound hammer test, and a method
that combines the RH test and the UPV test, known as SonReb, are
the most widely accepted nondestructive test methods, largely due
to their simplicity and effectiveness. To improve their accuracy and
reliability, researchers have attempted to develop better estimation
methods for the RH, UPV, and SonReb tests using regression
analysis, artificial neural networks, and support vector machines
(SVMs) (Trtnik et al. 2009; Atici 2011; Wang et al. 2014; Shih
et al. 2015; Ju et al. 2017; Rashid and Waqas 2017). However,
all these methods require expensive equipment and instrument
maintenance as well as trained and certified personnel with a high
degree of skill.
In recent years, a number of image processing techniques have
been proposed for estimating the compressive strength of concrete
as an alternative to the relatively costly nondestructive test methods
described previously. Images of the surface of the concrete contain
information on the spatial structure and content of its components,
which govern the compressive strength of the concrete. Başyiğit
et al. (2012) performed regression analyses (linear, multilinear, and
nonlinear) to estimate the compressive strength of concrete based
on image processing values obtained from the surface images of
the concrete specimens using a digital camera, whereas Dogan
et al. (2017) estimated concrete compressive strength using image
processing and artificial neural networks. However, both these
studies captured concrete images in ideal laboratory environments
such as photo-shooting tents and cabins. Under a fixed light inten-
sity, the digital camera was held above each sample and an image of
the entire surface of the concrete specimen was captured from the
same height. These very restricted image capture methods clearly
suffer from serious practical limitations when applied to actual
structures on site as an alternative to existing nondestructive test
methods. Moreover, both studies used the statistical properties
(arithmetic mean, standard deviation, and median values) extracted
from a gray-level histogram diagram as inputs. Input features sig-
nificantly influence subsequent estimates of the concrete’s com-
pressive strength and it is very possible that the manually defined
features used in previous studies may lose much of the spatial struc-
ture and component content. In an attempt to address these short-
comings, this study used DCNNs to avoid the need for manual
feature identification because they can be used to extract the fea-
tures from the images directly, thus facilitating concrete compres-
sive strength estimations.
Deep Convolutional Neural Networks
A deep convolutional neural network is a deep learning algorithm
that is designed to process data that comes in the form of multiple
arrays, making it feasible to extract relevant features even in the
presence of noise, shifting, rescaling, and other types of data dis-
tortions (LeCun et al. 1998). DCNNs consist of three types of layers,
namely convolution, pooling, and fully connected layers (Fig. 1).
The general function of a DCNN includes feature extraction, clas-
sification, and regression. For feature extraction, the convolution
and pooling layers are stacked to transform the raw data into a rep-
resentation at a higher level. Fully connected layers are then used to
classify the transformed representation into a specific class. DCNNs
can learn features autonomously by updating the weights of recep-
tive fields (Cha et al. 2017), contributing to major advances in object
detection and recognition in the computer-vision domain. Over the
past few years, several effective DCNN models have been proposed.
Among these, three different DCNN models, namely AlexNet,
GoogLeNet, and ResNet, all of which have won the ImageNet Large
Scale Visual Recognition Challenge (ILSVRC), were selected for
this study.
AlexNet
AlexNet, developed by Krizhevsky et al. (2012), exhibited signifi-
cantly better performance than the other non-deep learning meth-
ods at ILSVRC 2012. The main innovation of AlexNet lies in the
way it uses rectified linear units (ReLU) as the activation function,
which is normally associated with the principle of incentive neuron
© ASCE 04019018-2 J. Comput. Civ. Eng.
J. Comput. Civ. Eng., 2019, 33(3): 04019018
Downloaded
from
ascelibrary.org
by
HANYANG
UNIVERSITY
on
03/01/19.
Copyright
ASCE.
For
personal
use
only;
all
rights
reserved.

4. signaling. Compared with the nonlinear function sigmoid, this sim-
ple linear activation function achieves better and faster training
under large data conditions. Dropout and data augmentation are
used to prevent overfitting, thus reducing the complex interadapta-
tion relationship of neurons and enhancing the robustness of the
model. AlexNet is composed of five convolutional layers (CONV)
and three fully connected layers (FC) (Fig. 2). The first convolu-
tional layer, CONV 1, has 96 kernels of size 11 × 11 × 3; CONV 2
has a size of 55 × 55 × 96, which represents the result of CONV 1,
and contains 356 kernels of size 5 × 5 × 96; CONV 3 is composed
of 38 kernels of size 3 × 3 × 256; and CONV 4 and CONV 5 have
384 and 256 kernels, respectively, of size 3 × 3 × 384. The results
from each convolution layer are expressed in ReLU, and CONV 1,
CONV 2 and CONV 5 have a max 3 × 3 pooling size. CONV 1 and
CONV 2 also apply local response normalization (LRN) to the
result of the max pooling. The FC 6, FC 7, and FC 8 stages follow-
ing the convolution layers have 4,096, 4,096, and 1,000 neurons,
respectively.
GoogLeNet
GoogLeNet, which won ILSVRC 2014, was developed by Szegedy
et al. (2015) and is a 22-layer deep convolutional neural network
architecture based on nine Inception modules (Fig. 3). The salient
feature of GoogLeNet is that it not only increases the depth of the
network, but also broadens the network width without increasing
the amount of computation required. GoogLeNet can extract fea-
tures from different scales at the same time to enhance its learning
ability. The inclusion of the Inception module means that although
GoogLeNet has 12 times fewer parameters than AlexNet, its accu-
racy is higher. The Inception module consists of parallel 1 × 1,
3 × 3, and 5 × 5 convolution layers and a max pooling layer to ex-
tract a variety of features in parallel; 1 × 1 convolution layers are
then added to reduce the parameter quantity and accelerate the cal-
culation. Finally, a filter concatenation layer links the outputs of all
these parallel layers.
ResNet
The residual neural network (ResNet), developed by He et al.
(2016), which won ILSVRC 2015, is a 152-layer deep convolu-
tional neural network. It was inspired by the idea that networks
should perform better as they grow in depth, as demonstrated by
GoogLeNet. ResNet uses a residual network in order to deal with
the degradation problem and uses deeper networks to solve com-
plicated problems. The residual network is composed of residual
learning building blocks (Fig. 4); HðxÞ is the originally expected
mapping output of a certain layer, and x is the input. The use of
shortcut connections means that a self-mapping operation in the
network is equivalent to opening a channel from the input side,
so that the input can go straight to the output. The optimization
target then changes from HðxÞ to HðxÞ − x, and for an opti-
mized mapping its residuals can also be easily optimized to 0. This
means that a residual network solves the degradation problem and
reduces the difficulty of optimizing the parameters of a deep net-
work. Shortcut connections can improve the recognition accuracy,
and the resulting reduction in network complexity is a major
advantage of using a residual network.
Fig. 2. Architecture of AlexNet.
Fig. 1. DCNN architecture.
Fig. 3. Inception module of GoogLeNet.
© ASCE 04019018-3 J. Comput. Civ. Eng.
J. Comput. Civ. Eng., 2019, 33(3): 04019018
Downloaded
from
ascelibrary.org
by
HANYANG
UNIVERSITY
on
03/01/19.
Copyright
ASCE.
For
personal
use
only;
all
rights
reserved.

5. Methodology and Experimental Settings
Dataset Creation
To estimate the compressive strength of concrete using a DCNN, it
is first necessary to create a dataset that can be used to train, val-
idate, and test the DCNN model. A suitable dataset was therefore
created for this study using the four-step process in Fig. 5.
The first step was to produce the concrete specimens. Cylindri-
cal concrete specimens with dimensions of Ø100 × 200 mm were
fabricated by mixing, curing, and polishing the samples in a labo-
ratory environment. The compressive strength of concrete is deter-
mined by its curing age as well as its water:cement ratio (Baygin
et al. 2018). In general, increasing the water:cement ratio reduces
the compressive strength of the concrete. In this study, ordinary
portland cement (OPC) with water:cement ratios of 0.68, 0.50,
and 0.33 was used to provide an appropriate range of concrete com-
pressive strengths (Table 1). A total of 27 specimens were prepared,
with 3 samples aged for 3, 7, and 28 days for each water:cement
ratio.
The second step was to capture the concrete images. We used a
portable digital microscope to capture images of the upper and
lower sides of each concrete specimen, both of which had flat
surfaces. The portable digital microscope used in this study
recorded images composed of approximately 2 million pixels
(1,920 × 1,080) with a resolution of approximately 5,400 dots
per in. (dpi). Because a 9- × 5-mm region can be photographed
by the portable digital microscope, an image of a portion of
the entire concrete specimen surface (Ø100 × 200 mm) was cap-
tured, unlike previous studies in which the entire surface of con-
crete was captured simultaneously using a digital camera. Using a
portable digital microscope allowed us to capture more-detailed
images of the microstructure features associated with the compres-
sive strength of concrete and also enabled us to acquire multiple
different images for each specimen. In particular, the images were
captured without the need for any special environmental settings
such as a photo-shooting tent or cabin to facilitate the proposed
image capture method in a realistic environment such as those
found on site. To increase the robustness of the estimation, con-
crete images were taken under a range of environmental condi-
tions, including different illumination levels (under natural,
direct, and indirect lighting), different photographers, and different
portable digital microscopes, albeit of the same specification.
Between 150 and 200 photos were taken of each specimen, and
a video record of the specimens was made using the same portable
digital microscope and the same settings as those used for the pho-
tographs in order to collect as many images of the concrete spec-
imens as possible.
The third step was to perform a concrete compressive strength
test based on the provisions of KS F 2405 Korean Standard Asso-
ciation (2010) using a 200-ton universal mechanical tester (UMT).
A concrete compressive strength test was conducted for each of the
selected curing ages of 3, 7, and 28 days and for each water:cement
ratio. The test results therefore consisted of 27 performance values
ranging from 8.89 to 41.48 MPa (Table 2), and the values obtained
indicated that the concrete samples constructed for this study con-
sisted of low- to normal-strength concrete.
The final step was to construct a concrete image dataset. DCNN
models require labeled information for all data because they are
supervised learning models. In the labeling process, the images
of the concrete samples collected prior to the compressive test were
used as the input and the results of the compressive strength test
Fig. 4. Residual learning module of ResNet.
Fig. 5. Dataset creation process.
© ASCE 04019018-4 J. Comput. Civ. Eng.
J. Comput. Civ. Eng., 2019, 33(3): 04019018
Downloaded
from
ascelibrary.org
by
HANYANG
UNIVERSITY
on
03/01/19.
Copyright
ASCE.
For
personal
use
only;
all
rights
reserved.

6. were used as the output. Overall, 5,145 concrete image datasets
from the photographs and 299,291 concrete image datasets from
individual frames of the video recording were acquired.
Estimation Model
This study applied DCNN to autonomously extract features in
the hidden layers of deep neural networks, unlike previous stud-
ies using manually extracted features from the concrete images
(e.g., Başyiğit et al. 2012; Dogan et al. 2017). As mentioned pre-
viously, DCNN exhibits good performance in classification and rec-
ognition, especially in the case of images used directly as the input
of the neural network. DCNN has the capacity to learn features
through weight sharing and convolution regardless of the image
coordinates, giving it a robust performance in terms of translation
invariance. It is also well suited to image analysis because it
excludes duplicate values of the same image through the pooling
and convolutional layers and self-trains the features from training
data. In this respect, the proposed estimation model using DCNN
has a decided advantage for detecting features from complex con-
crete surface images.
This study used three different DCNN models, AlexNet, Goog-
LeNet, and ResNet, for estimating concrete compressive strength.
Each DCNN model has different convolution kernel sizes. AlexNet
uses relatively large convolution kernels, such as 11 × 11, 5 × 5,
and 3 × 3, and 8 layers; whereas GoogLeNet uses an Inception
module consisting of parallel 1 × 1, 3 × 3, and 5 × 5 convolution
kernels and 22 layers; and ResNet uses a 3 × 3 convolution kernel
for all 50 layers to ensure that the receptive field provides adequate
cover across the entire input size (Luo et al. 2016). For the ILSVRC
competition, these DCNN models were trained on an ImageNet
dataset containing hundreds of thousands of images with 1,000
classes for image classification. However, because the purpose of
this study was to estimate concrete compressive strength using
concrete surface images, the final output layers of AlexNet, Goog-
LeNet, and ResNet were modified to use a Euclidean loss function
instead of the Softmax function. Weights of three different DCNN
models were learned such that the loss function was minimized by
using a back propagation algorithm.
The three DCNN models in this study all used rectified linear
units for the nonlinear activation functions associated with the input
and output of the convolution layers and the fully connected layers.
Dropout was applied to fully connected layers to minimize over-
fitting, in which the neural network is overly adaptive to the train-
ing dataset and thus cannot properly respond to validation and
testing datasets. Dropout minimizes any overfitting by selecting
and learning a part of the neural network randomly without learning
the entire neural network. This study also used data augmentation
to prevent overfitting. Data augmentation increases the cardinality
of the training set for all the classes, overcomes the problem of
overfitting, and reduces any divergence from the test dataset by
adding random noise or translation to the training dataset because
the training dataset differs from the test dataset in reality. We used
both random cropping and horizontal flipping (Fig. 6). Specifi-
cally, an image with an original size of 1,920 × 1,080 was scaled
to 112 × 112, and a random seed was generated in the 18 × 18
segment in the upper left of the image. An 84 × 84 portion of the
image was then selected using the random seed as the upper-right
Table 1. Mix proportions for concrete samples
Mix
type
Water:
cement
(%)
Sand:
aggregate
(%)
Water
(L=m3)
Cement
(kg=m3)
Fine
aggregate
(kg=m3)
Corse
aggregate
(kg=m3)
Air
content
(%)
Mix 1 68 51 170 250 932 944 4.5
Mix 2 50 48 165 330 852 973 4.5
Mix 3 33 45 160 480 749 965 4.5
Table 2. Compressive strength test results (MPa)
Water:cement
(%)
Experimental compressive strength
3 days 7 days 28 days
68 10.29 10.57 17.09
8.89 10.84 18.83
9.21 10.29 18.70
50 16.67 18.41 28.29
17.29 17.73 27.97
16.77 18.08 20.27
33 25.96 34.99 40.59
26.28 30.58 38.99
27.58 33.83 41.48
Fig. 6. Examples of data augmentation: (a) random cropping; and (b) horizontal flipping.
© ASCE 04019018-5 J. Comput. Civ. Eng.
J. Comput. Civ. Eng., 2019, 33(3): 04019018
Downloaded
from
ascelibrary.org
by
HANYANG
UNIVERSITY
on
03/01/19.
Copyright
ASCE.
For
personal
use
only;
all
rights
reserved.

7. coordinate. The resulting 84 × 84 image could also be rotated 180°
about the x-axis.
Performance Evaluation Measures
The estimation accuracies of the three DCNN models created
for this study were evaluated and compared using three frequently
used performance measurements: R2
, RMSE, and MAPE. The
R-squared coefficient, R2
, is a measure of how well the independent
variables being considered account for the measured dependent
variable; the higher the R-squared value, the better the estimation
power. Root-mean-square error is the square root of the mean
square error and is thus a measure of the average distance of a data
point from the fitted line measured along a vertical line. RMSE rep-
resents the absolute value difference between the experimental com-
pressive strength and the estimated compressive strength (i.e., the
amount of error). The mean absolute percentage error is a statistical
measure of estimation accuracy and expresses the error between the
experimental compressive strength and the estimated compressive
strength as a percentage. MAPE is commonly used in quantitative
forecasting methods because it indicates the relative overall fit. The
three measures are given by the following equations:
R2
¼
ðn
P
i yiy0
i −
P
i y0
i
P
i yiÞ2
ðn
P
i y02
i − ð
P
i y0
i Þ2Þðn
P
i y2
i − ð
P
i yiÞ2Þ
ð1Þ
RMSE ¼
ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi
1
n
X
i
ðyi − y0
i Þ2
s
ð2Þ
MAPE ¼
1
n
X
i
yi − y0
i
yi
× 100 ð3Þ
where yi and y0
i = experimental compressive strength and estimated
compressive strength, respectively; and n = number of data samples.
Experimental Settings
When recording the video of the concrete images with a portable
digital microscope, lens shake tends to produce out-of-focus im-
ages. This was also the case here, with blurred concrete images
being acquired from individual frames of the video recording. If
blurred concrete images are used in training, it may be difficult
to get good accuracy. We therefore used blurred concrete images
for pretraining before conducting the training, validation, and test-
ing. In pretraining, the amount of computation required for learning
may increase, and a degree of overfitting may be generated because
the number of training data images is less than the number of
parameters to be learned. To prevent overfitting, data augmentation
is required in such cases, and here it was conducted via minibatch
learning using a set of 52 images, consisting of a combination of 48
photographic images and 4 images from the video recording, even
where the learning process used photographic images only. In other
words, images from the video recording were added to the photo-
graphic images for the minibatch learning every time to augment
the data.
A total of 5,145 datasets were used for training, validation, and
testing. To present the generalized performance fairly, the DCNN
models were learned as the training dataset and selected as the val-
idation dataset, and the estimation error was measured by the test
dataset. The training, validation, and testing datasets were classified
into 3,601 (70%), 515 (10%), and 1,029 (20%) images, respec-
tively. The initial learning rate, weight decay, parameters of mo-
mentum, and dropout rate were set to 0.01, 0.0005, 0.9, and 0.5,
respectively. The deep learning framework CAFFE version 1.0
(Jia et al. 2014) was used, and the learning was conducted using
a workstation equipped with four GPUs (CPU: Intel Xeon E5-2620
v4 @2.1GHz; RAM: 64GB; and GPU: Nvidia GTX 1080Ti × 4).
Once the three customized DCNN models (AlexNet, GoogLeNet,
and ResNet) had been created, the training operation was repeated
80 million times (number of iterations) in order to obtain the opti-
mum structure. The resulting training loss and validation curves for
each DCNN model all tended to converge, showing that all three
networks achieved an excellent fitting performance for both the
training and validation sets.
Results and Discussion
Experimental Results
DCNN algorithms autonomously learn features through data-
intensive analysis, whereas traditional machine learning algorithms
such as ANN rely on hand-engineered features based on domain
knowledge. Because it is important to construct an appropriate
model architecture that will enable the model itself to learn the fea-
tures well, this study applied AlextNet-, GoogLeNet-, and ResNet-
based models to estimate the compressive strength of the concrete
samples. The resulting dataset was then used to compare the results
of these DCNN models with the results of the image processing–
based ANN model recently proposed by Dogan et al. (2017).
Following Dogan et al. (2017), we extracted the features from the
image by applying appropriate statistical properties (arithmetic
mean, standard deviation, and median) to develop an ANN model
that estimates concrete compressive strength using the extracted
statistical properties as inputs. Table 3 summarizes the R2
, RMSE,
and MAPE results for the three different DCNN models and the
image processing–based ANN model. As noted previously, R2
is a
measure of how well the independent variables approximate the
estimated dependent variable, whereas RMSE and MAPE are used
as a measure of the differences between the values estimated by the
models. High R2
values and low values of RMSE and MAPE are
generally indicative of good performance. The results showed that
three DCNN models largely outperformed the image processing–
based ANN model on all three indicators.
The best model for determining R2 among the DCNN models
was found to be the ResNet-based model (R2
¼ 0.764), whereas
the AlexNet-based model exhibited the worst estimation capabil-
ities (R2
¼ 0.745). This is shown in the plots of the relationships
between the experimental and estimated compressive strength ob-
tained by the three DCNN models (Fig. 7). Table 3 also gives a
direct relationship between R2
and RMSE; the best model for min-
imizing RMSE was again the ResNet-based model (4.46 MPa),
and the worst was the AlexNet-based model (4.64 MPa). However,
the ranking of the three models was not the same for MAPE. Here,
the best result was obtained for the AlexNet-based model (17.67%),
and the worst was the GoogLeNet-based model (18.40%). Figs. 8(a
and b) show the RMSE and MAPE values, respectively, for the
Table 3. Performance comparison of DCNN models and image
processing–based ANN model
Performance
DCNN models Image
processing–based
ANN model
AlexNet GoogLeNet ResNet
R2
0.745 0.748 0.764 0.200
RMSE (MPa) 4.641 4.612 4.463 8.223
MAPE (%) 17.675 18.403 17.765 38.039
© ASCE 04019018-6 J. Comput. Civ. Eng.
J. Comput. Civ. Eng., 2019, 33(3): 04019018
Downloaded
from
ascelibrary.org
by
HANYANG
UNIVERSITY
on
03/01/19.
Copyright
ASCE.
For
personal
use
only;
all
rights
reserved.

8. three DCNN models in terms of the actual experimental compres-
sive strength. In the case of RMSE, the maximum values esti-
mated by the AlexNet-based model, GoogLeNet-based model, and
ResNet-based model were 7,82, 7.03, and 7.75 MPa, respectively,
and the minimum values were 1.70, 2.54, and 1.60 MPa, respec-
tively. For MAPE, the AlexNet-, GoogLeNet-, and ResNet-based
models achieved maximum values of 51.62%, 46.90%, and 45.57%,
respectively, for an experimental compressive strength value of
10.84 MPa. The minimum values estimated were 6.30%, 6.97%,
and 6.67%, respectively.
We also examined the estimation errors, namely the value dif-
ference between the experimental compressive strength and the
estimated strength value. The lowest experimental compressive
strength was estimated to be lower than the value actually measured,
and the highest compressive strength was estimated to be higher
than the observed experimental compressive strength (Fig. 9).
The AlexNet-, GoogLeNet-, and ResNet-based models had maxi-
mum error values of 23.68, 22.48, and 18.4 MPa, respectively,
and minimum error values of −24.59, −21.13, and −17.58 MPa,
respectively. The ResNet model (35.98 MPa) had the smallest differ-
ence between the maximum and minimum errors, and the AlexNet
based model (48.27 MPa) had the largest difference. We also ex-
plored the frequency distribution of the estimation error rate for
the compressive strength (Fig. 10). Here, the error rate was divided
into 5% intervals and the frequency and ratio of the error rate was
analyzed for each of the DCNN models. Overall, an average of 83%
of the total data was estimated to have an error rate of less than 30%.
The model with the lowest error rate was the AlexNet-based model,
whereas the worst performing model was the GoogLeNet-based
model. The AlexNet-based model estimated 83.7% of the total data
with an error rate of less than 30%, and only 3.98% of the total data
had an error rate of over 50%. Even the worst performing model
turned in a reasonable performance: the GoogLeNet-based model
estimated 81.3% of the total data with an error rate of less than
30%, and 6.6% with an error rate of over 50%. The analysis of the
compressive strength estimation accuracy revealed that the estima-
tion accuracy of each of the DCNN models was slightly different
depending on the evaluation criteria applied, although overall the
ResNet-based model had excellent compressive strength estimation
accuracy. Fig. 11 shows examples of good and bad results for the
ResNet-based model.
Discussion
This study investigated the applicability of estimating concrete
compressive strength using microstructure images analyzed with
DCNN. The thickness of the interfacial transition zone (ITZ), and
the existence of microcracks and pores largely determine the com-
pressive strength of concrete structures. This means that there is
a relationship between the concrete microstructure images and
compressive strength. Therefore, this study attempted to estimate
compressive strength by examining images of the concrete micro-
structure. To obtain these images of the concrete surface, we used a
portable digital microscope capable of recording images com-
posed of approximately 2 million pixels (1,920 × 1,080) with a res-
olution of approximately 5,400 dpi. Because each image captured a
Fig. 7. Experimental versus estimated compressive strength: (a) AlexNet-based model; (b) GoogLeNet-based model; and (c) ResNet-based model.
© ASCE 04019018-7 J. Comput. Civ. Eng.
J. Comput. Civ. Eng., 2019, 33(3): 04019018
Downloaded
from
ascelibrary.org
by
HANYANG
UNIVERSITY
on
03/01/19.
Copyright
ASCE.
For
personal
use
only;
all
rights
reserved.

9. 9 × 5 mm region, meaning that each pixel represented 4.66 μm
of the surface, it was possible to observe concrete microstructure
components such as the matrix, aggregate, pores, and ITZ with the
portable digital microscope. However, Because it is difficult to
manually extract features affecting compressive strength, as Fig. 11
demonstrates, this study applied DCNN models to autonomously
extract complex discriminative features through a learning pro-
cedure. The experimental results showed that the proposed DCNN
models were indeed capable of providing good estimates of the
compressive strength using concrete microstructure images. This
indicates that the proposed DCNN model learned the relation-
ship between the concrete microstructure images and compres-
sive strength, thus confirming that the concrete images obtained
using a portable digital microscope include patterns can be used
to estimate the compressive strength of the concrete samples
and that the DCNN models developed for this study can learn these
patterns.
The ultimate goal of this study was to provide a new image-
based method for estimating concrete compressive strength as an
auxiliary method for nondestructive testing. It is therefore impor-
tant to be able to apply the proposed compressive strength estima-
tion method on site to examine an actual structure. The method
proposed here facilitates the capture of images from actual concrete
structures because the images of the concrete surfaces are collected
without any particular constraints on the environment, in contrast to
the methods used in previous studies, which used concrete images
obtained in an ideal laboratory environment. This study used a port-
able digital microscope that enables users to capture images of ac-
tual concrete structures safely and conveniently. A portable digital
Fig. 8. RMSE and MAPE for each experimental compressive strength: (a) RMSE; and (b) MAPE.
Fig. 9. Errors in the estimated compressive strength: (a) AlexNet-
based model; (b) GoogLeNet-based model; and (c) ResNet-based
model.
© ASCE 04019018-8 J. Comput. Civ. Eng.
J. Comput. Civ. Eng., 2019, 33(3): 04019018
Downloaded
from
ascelibrary.org
by
HANYANG
UNIVERSITY
on
03/01/19.
Copyright
ASCE.
For
personal
use
only;
all
rights
reserved.

10. microscope is easy to use in the field because the microscope cam-
era attaches directly to the USB port of a computer. Eyepieces are
not required and the images are shown directly on the monitor.
We acquired multiple microstructure images for the same con-
crete specimen with the same compressive strength because a port-
able digital microscope can only capture a relatively small part of a
large concrete specimen surface in each image. This actually im-
proves the estimation accuracy because DCNN models can be ef-
fectively trained by large datasets (Halevy et al. 2009). However,
the range of the dataset created for this study is relatively small and
the opportunity to learn various types of images is therefore very
limited. If a new image with a compressive strength that is outside
the range of the dataset created for this study is tested, the results
obtained will not be an accurate estimate of the material’s compres-
sive strength. In future studies, datasets that include concrete spec-
imens with a greater range of compressive strength should be
created. Nevertheless, the results of this preliminary study are sig-
nificant in that our findings demonstrate the potential utility of our
proposed method and confirm the applicability of the proposed
model as an auxiliary to existing nondestructive methods.
Fig. 10. Frequency distribution of error rate: (a) AlexNet-based model; (b) GoogLeNet-based model; and (c) ResNet-based model.
Fig. 11. Examples of good and bad results for ResNet-based model: (a) good results; and (b) bad results.
© ASCE 04019018-9 J. Comput. Civ. Eng.
J. Comput. Civ. Eng., 2019, 33(3): 04019018
Downloaded
from
ascelibrary.org
by
HANYANG
UNIVERSITY
on
03/01/19.
Copyright
ASCE.
For
personal
use
only;
all
rights
reserved.

11. Conclusions
Concrete compressive strength, which is considered to determine
the quality of concrete, is a crucial indicator in facility manage-
ment. This study proposed a new model for estimating concrete’s
compressive strength that uses DCNNs to analyze images captured
with a portable digital microscope, and investigated its applicabil-
ity. The experimental results obtained showed that all three DCNN
models developed for this study provide a better performance than
the recently proposed image processing–based ANN model, and
overall, the ResNet-based model in particular demonstrated excel-
lent compressive strength estimation accuracy, outperforming both
the AlexNet- and GoogLeNet-based models. These findings con-
firm that the concrete images obtained using a portable digital mi-
croscope do indeed include patterns that can be used to estimate the
compressive strength of the concrete samples and that the DCNN
model is capable of learning these patterns. Concrete compressive
strength can indeed be estimated by using DCNNs to analyze mi-
crostructure images and the image capture method proposed in this
study is easy to apply to actual concrete structures because it uses a
portable digital microscope that can conveniently capture images of
the concrete surface with few, if any, constraints. This makes our
proposed method an attractive alternative to existing nondestructive
testing methods.
The next step in developing this promising new approach will be
to extend the range of compressive strengths included in the analy-
sis, because the range of the dataset created in this preliminary
study was very limited. We created a dataset consisting of just 27
compressive strength values, ranging from 8.89 to 41.48 MPa,
which correspond to low-normal-strength concrete, for this study.
To improve the applicability of this approach, images of a variety of
high-normal-strength concrete mixes must be included for training
and testing. The dataset created using concrete specimens produced
in a laboratory environment should also be supplemented with mea-
surements of actual concrete structures in the field in order to pro-
vide better estimates of the in situ concrete strength. Using the
method proposed in this study, an extensive dataset should be con-
structed by taking images of actual concrete structures and using the
compressive strength values from samples collected from core drill-
ing. Once a sufficiently large dataset has been constructed, it is ex-
pected that immediate estimates of concrete compressive strength in
the field will become possible simply by capturing images of the
concrete surfaces.
Acknowledgments
This research was supported by a grant (18CATP-C129782-02)
from the Technology Advancement Research Program funded by
the Korean Ministry of Land, Infrastructure and Transport.
References
Atici, U. 2011. “Prediction of the strength of mineral admixture concrete
using multivariable regression analysis and an artificial neural network.”
Expert Syst. Appl. 38 (8): 9609–9618. https://doi.org/10.1016/j.eswa
.2011.01.156.
Başyiğit, C., B. Çomak, S. Kilinçarslan, and I. S. Üncü. 2012. “Assessment
of concrete compressive strength by image processing technique.” Const.
Build. Mater. 37: 526–532. https://doi.org/10.1016/j.conbuildmat.2012
.07.055.
Baygin, M., S. G. Ozkaya, M. A. Ozdemir, and I. Kazaz. 2018. “A new
approach based on image processing for measuring compressive
strength of structures.” Int. J. Intell. Syst. Appl. Eng. 6 (4): 21–25.
Cha, Y., W. Choi, and O. Buyukozturk. 2017. “Deep learning-based crack
damage detection using convolutional neural networks.” Comput.
-Aided Civ. Infrastruct. Eng. 32 (5): 361–378. https://doi.org/10
.1111/mice.12263.
Derman, E., and A. A. Salah. 2018. “Continuous real-time vehicle driver
authentication using convolutional neural network based face recogni-
tion.” In Proc., 13th IEEE Int. Conf. on Automatic Face and Gesture
Recognition, 577–584. Washington, DC: IEEE Computer Society Press.
Dogan, G., M. H. Arslan, and M. Ceylan. 2017. “Concrete compressive
strength detection using image processing based new test method.”
Measurement 109: 137–148. https://doi.org/10.1016/j.measurement
.2017.05.051.
Ferreira, M. D., D. C. Correa, L. G. Nonato, and R. F. de Mello. 2018.
“Designing architectures of convolutional neural networks to solve
practical problems.” Expert Syst. Appl. 94: 205–217. https://doi.org/10
.1016/j.eswa.2017.10.052.
Gopalakrishnan, K., S. K. Khaitan, A. Choudhary, and A. Agrawal. 2017.
“Deep convolutional neural networks with transfer for computer vision-
based data-driven pavement distress detection.” Constr. Build. Mater.
157: 322–330. https://doi.org/10.1016/j.conbuildmat.2017.09.110.
Halevy, A., P. Norvig, and F. Pereira. 2009. “The unreasonable effective-
ness of data.” IEEE Intell. Syst. 24 (2): 8–12.
He, K., X. Zhang, S. Ren, and J. Sun. 2016. “Deep residual learning for image
recognition.” In Proc., IEEE Conf. on Computer Vision and Pattern
Recognition, 770–778. Washington, DC: IEEE Computer Society Press.
Jia, Y., E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick,
S. Guadarrama, and T. Darrell. 2014. “Caffe: Convolutional architecture
for fast feature embedding.” In Proc., 22nd ACM Int. Conf. on
Multimedia, 675–678. New York: ACM.
Ju, M., K. Park, and H. Oh. 2017. “Estimation of compressive strength
of high strength concrete using non-destructive technique and con-
crete core strength.” Appl. Sci. 7 (12): 1249. https://doi.org/10.3390
/app7121249.
Korean Standard Association. 2010. Standard test method for compressive
strength of concrete. KS F 2405. Seoul, South Korea: Korean Standard
Association.
Krizhevsky, A., I. Sutskever, and G. E. Hinton. 2012. “ImageNet classifi-
cation with deep convolutional neural networks.” Adv. Neural Inf.
Process. 25 (2): 1097–1105.
LeCun, Y., L. Bottou, Y. Bengio, and P. Haffner. 1998. “Gradient-based
learning applied to document recognition.” Proc. IEEE 86 (11):
2278–2324. https://doi.org/10.1109/5.726791.
Lin, Y. Z., Z. H. Nie, and H. W. Ma. 2017. “Structural damage detection
with automatic feature-extraction through deep learning.” Comput.
-Aided Civ. Infrastruct. Eng. 32 (12): 1025–1046. https://doi.org/10
.1111/mice.12313.
Luo, W., Y. Li, R. Urtasun, and R. Zemel. 2016. “Understanding the
effective receptive field in deep convolutional neural networks.”
In Proc., 29th Conf. on Neural Information Processing Systems,
4905–4913. Red Hook, NY: Curran Associates.
Nugraha, B. T., S. F. Su, and Fahmizal. 2017. “Towards self-driving car
using convolutional neural network and road lane detector.” In Proc.,
2nd Int. Conf. on Automation, Cognitive Science, Optics, Micro
Electro-Mechanical System, and Information Technology, 65–69.
Red Hook, NY: Curran Associates.
Rashid, K., and R. Waqas. 2017. “Compressive strength evaluation by
non-destructive techniques: An automated approach in construction
industry.” J. Build. Eng. 12: 147–154. https://doi.org/10.1016/j.jobe
.2017.05.010.
Shih, Y. F., Y. R. Wang, K. L. Lin, and C. W. Chen. 2015. “Improving
non-destructive concrete strength tests using support vector machines.”
Materials 8 (10): 7169–7178. https://doi.org/10.3390/ma8105368.
Steenbergen, R. D. J. M., and A. H. J. M. Vervuurt. 2012. “Determining the
in situ concrete strength of existing structures for assessing their struc-
tural safety.” Struct. Concr. 13 (1): 27–31. https://doi.org/10.1002/suco
.201100031.
Sun, W., T. L. Tseng, J. Zhang, and W. Qian. 2017. “Enhancing deep
convolutional neural network scheme for breast cancer diagnosis with
unlabeled data.” Computerized Med. Imaging Graphics 57: 4–9. https://
doi.org/10.1016/j.compmedimag.2016.07.004.
© ASCE 04019018-10 J. Comput. Civ. Eng.
J. Comput. Civ. Eng., 2019, 33(3): 04019018
Downloaded
from
ascelibrary.org
by
HANYANG
UNIVERSITY
on
03/01/19.
Copyright
ASCE.
For
personal
use
only;
all
rights
reserved.

12. Szegedy, C., W. Liu, and Y. Jia. 2015. “Going deeper with convolution.”
In Proc., IEEE Conf. on Computer Vision and Pattern Recognition,
1–9, Washington, DC: IEEE Computer Society Press.
Tiberti, G., F. Minelli, and G. Plizzari. 2015. “Cracking behavior in rein-
forced concrete members with steel fibers: A comprehensive experi-
mental study.” Cem. Concr. Res. 68: 24–34. https://doi.org/10.1016/j
.cemconres.2014.10.011.
Trtnik, G., F. Kavčič, and G. Turk. 2009. “Prediction of concrete
strength using ultrasonic pulse velocity and artificial neural networks.”
Ultrasonics 49 (1): 53–60. https://doi.org/10.1016/j.ultras.2008
.05.001.
Wang, Y. R., W. T. Kuo, S. S. Lu, Y. F. Shih, and S. S. Wei. 2014. “Applying
support vector machines in rebound hammer test.” Adv. Mater. Res.
853: 600–604. https://doi.org/10.4028/www.scientific.net/AMR.853.600.
Zhang, A., et al. 2017. “Automated pixel-level pavement crack detection on
3D asphalt surfaces using a deep-learning network.” Comput. -Aided
Civ. Infrastruct. Eng. 32 (10): 805–819. https://doi.org/10.1111/mice
.12297.
© ASCE 04019018-11 J. Comput. Civ. Eng.
J. Comput. Civ. Eng., 2019, 33(3): 04019018
Downloaded
from
ascelibrary.org
by
HANYANG
UNIVERSITY
on
03/01/19.
Copyright
ASCE.
For
personal
use
only;
all
rights
reserved.
View publication stats
View publication stats