SlideShare a Scribd company logo
1 of 12
Download to read offline
1051-8215 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCSVT.2018.2797897, IEEE
Transactions on Circuits and Systems for Video Technology
Copyright © 2018 IEEE. Personal use of this material is permitted. However, permission to use this material for any other purposes must be
obtained from the IEEE by sending an email to pubs-permissions@ieee.org.

I. INTRODUCTION
eversible data hiding in encrypted images (RDH-EI) is an
emerging technique originated from reversible data hiding
in plaintext images [1-2], which has been investigated by
many researchers [3-21]. In the cloud storage scenario, this
technique is realized by a protocol that contains three parties, an
image owner, a cloud server and an authorized user. The image
owner encrypts the contents before uploading images onto a
cloud storage server. The server hides additional messages into
the encrypted images. The RDH-EI protocol guarantees that the
hidden message can be exactly extracted by the server, and the
original content of the images can be losslessly recovered by
the authorized user.
RDH-EI is especially useful for labeling the ciphertext in
cloud storage, as shown in Fig. 1. When image owners hope to
protect their privacy, RDH-EI first provides a secure encryption
algorithm for the owners to encrypt their images before up-
loading. On the cloud side, RDH-EI allows the server to label
an encrypted image by data hiding, e.g., hiding the identities,
timestamps, and remarks into the ciphertext to generate a
This work was supported by Natural Science Foundation of China (Grant
61572308, Grant U1536108, Grant U1636206, Grant 61525203, Grant
61472235, Grant U1636219, and Grant 61379151), the Shanghai Dawn Scholar
Plan (14SG36), the Shanghai Excellent Academic Leader Plan (Grant No.
16XD1401200), and the Plan for Scientific Innovation Talent of Henan Prov-
ince (Grant No. 2018JR0018).
Z. Qian and H. Xu are with Shanghai Institute for Advanced Communica-
tion and Data Science, School of Communication and Information Engineering,
Shanghai University, Shanghai 200444, China;
X. Luo is with State Key Laboratory of Mathematical Engineering and
Advanced Computing, Zhengzhou 450001, China
X. Zhang is with Shanghai Institute of Intelligent Electronics & Systems,
School of Computer Science, Fudan University, 200433, China.
Corresponding author: Xinpeng Zhang, Phone/Fax: 86-21-66137263; email:
zhangxinpeng@fudan.edu.cn
marked encrypted copy. Therefore, the labels are attached
inside the ciphertext, providing a better management for ad-
ministrators. Meanwhile, storage overheads can also be saved.
On the other hand, when an authorized user downloads the
encrypted image from the cloud, the original content can be
losslessly recovered after image decryption. In traditional sys-
tems of file management, the server constructs a metadata file
to record the information of the uploaded images. The RDH-EI
technique provides an alternative way, which accommodates
additional information of the image inside the encrypted bit-
stream. Therefore, no metadata files are needed anymore for
labeling the uploaded images.
Fig. 1. RDH-EI for cloud storage
Many RDH-EI methods were proposed in the past five years
[3-21]. Most methods were designed for uncompressed images
[3-17], while some works were for JPEG bitstreams [18-21].
Since JPEG images are widely used on Internet, this paper
focuses on RDH-EI in JPEG bitstreams. We propose a novel
RDH-EI framework to make this technique more practical.
Compared with previous works [18-21], two achievements are
Abstract—This paper proposes a novel framework of reversible data hiding in encrypted JPEG bitstream. We first provide a JPEG
encryption algorithm to encipher a JPEG image to a smaller size and keep the format compliant to JPEG decoders. After an image owner
uploads the encrypted JPEG bitstreams to cloud storage, the server embeds additional messages into the ciphertext to construct a marked
encrypted JPEG bitstream. During data hiding, we propose a combined embedding algorithm including two stages, the Huffman code
mapping and the ordered histogram shifting. The embedding procedure is reversible. When an authorized user requires a downloading
operation, the server extracts additional messages from the marked encrypted JPEG bitstream and recovers the original encrypted bit-
stream losslessly. After downloading, the user obtains the original JPEG bitstream by a direct decryption. The proposed framework out-
performs previous works on RDH-EI. First, since the tasks of data embedding/extraction and bitstream recovery are all accomplished by
the server, the image owner and the authorized user are required to implement no extra operations except JPEG encryption or decryption.
Second, the embedding payload is larger than state-of-the-art works.
Index Terms—Reversible data hiding, information hiding, image recovery, JPEG encryption
New Framework of Reversible Data Hiding in
Encrypted JPEG Bitstreams
Zhenxing Qian, Member, IEEE, Haisheng Xu, Xiangyang Luo, Xinpeng Zhang, Member, IEEE
R
1051-8215 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCSVT.2018.2797897, IEEE
Transactions on Circuits and Systems for Video Technology
2
achieved in the proposed method. First, the tasks of data em-
bedding, data extraction and bitstream recovery are all done by
the server. There are no extra computation burdens for own-
er/user except encryption/decryption. Second, we achieve a
larger embedding payload than state-of-the-art works on JPEG
RDH-EI. In the rest of the paper, we first introduce the previous
arts of RDH-EI in Section II. In Sections III, we briefly depict
the new framework. JPEG encryption and decryption algo-
rithms for both the owner and the user are proposed in Section
IV. Section V provides the embedding, extraction and recovery
algorithms. Section VI provides the experimental results and
analyses. The paper is concluded in Section VII.
II.PREVIOUS ARTS
A.RDH-EI for Uncompressed Images
RDH-EI was first realized in uncompressed nature images
[3]. The content owner encrypts the original image using a
stream cipher algorithm and then uploads the encrypted image
onto the cloud. The cloud server embeds additional bits into
ciphertext by flipping three least significant bits (LSB) of half
pixels in each block. On the recipient end, the authorized user
decrypts the marked encrypted image and generates two can-
didates for each block by flipping LSBs again. Since the orig-
inal block of a nature image is smoother than the interfered
block, one hidden bit can be extracted and the original block
can be recovered. This method was improved in [4] using a side
match algorithm to investigate the spatial correlation between
neighboring blocks. The flipping based approaches in [3] and
[4] were improved in [5] to reduce errors by comparing more
neighbor pixels. However, when the pixels in a block have
identical values, data extraction and image recovery in [3-5]
may fail. A swapping and shifting based RDH-EI method was
proposed to overcome this drawback [6]. Machine learning was
also used in RDH-EI to improve the embedding capacity [7].
Although the methods in [2-7] have good embedding and
recovery capabilities, data extraction must be done after image
decryption. This limitation makes the technique less useful in
cloud storage. Separable algorithms were proposed to resolve
the problem [8]. With a pseudo-randomly generated matrix, the
server compresses some LSB-planes of the encrypted pixels to
fewer bits to generate spare rooms for hiding additional mes-
sages. Therefore, the hidden bits can be extracted directly from
the marked encrypted image. On the recipient side, the original
contents are recovered by estimating LSBs using MSBs of
neighboring pixels. This method was improved in [9] by se-
lecting appropriate bitplanes from the encrypted image. Dis-
tributed source coding (DSC) is used to design RDH-EI in [10].
The server compresses some selected bits in the encrypted
image to hide additional messages. A much higher capacity can
be achieved using a Low-Density-Parity-Check (LDPC) based
Slepian-Wolf encoder. This method was further improved in
[11] using a progressive recovery algorithm, which achieves a
better embedding rate. In [12], an RDH-EI with higher em-
bedding capacity was proposed by maintaining some redun-
dancies during image encryption.
Another type of RDH-EI is realized by preprocessing the
original image. Some works require the image owner to vacate
spare rooms in the plaintext image before image encryption.
We name this additional operation of vacating spare rooms in
plaintext as preprocessing. After that, the image owner en-
crypts the processed image and uploads it onto the server. Im-
age encryption should not be accounted as preprocessing, since
encryption is required in all RDH-EI methods. In [13], LSBs of
some pixels are embedded into other pixels using traditional
RDH for plaintext images. Therefore, these LSBs are vacated
as spare rooms. The processed image is then encrypted and
uploaded to the cloud. On cloud, the server embeds additional
bits into the encrypted image using the specified spare rooms,
which provides a very high embedding rate. In [14], some
pixels are used to estimate the rest before encryption. After
encrypting the pixels and estimation errors, a final version of
the encrypted image is formulated. The server realizes data
hiding by modifying the estimation errors. In [15], patch-level
sparse representation is used to explore the correlations of
neighbor pixels. Another RDH-EI approach was realized by
histogram shifting in the spatial permuted images [16]. In [17],
image transformation was proposed to encrypt one image to
another. Additional bits are embedded by RDH in plaintext
images. The preprocessing based methods in [8-17] can achieve
much better embedding rates, but require extra RDH operations
before image encryption.
B.RDH-EI for JPEG Bitstreams
The aforementioned RDH-EI is for uncompressed images.
However, those methods are not useful in many applications
because most images transmitted over Internet are compressed,
e.g., the popular JPEG. As a consequence, some RDH-EI works
were proposed for JPEG bitstreams [18-21].
The first RDH-EI for JPEG bitstreams was proposed in [18].
This scheme begins with a JPEG encryption algorithm, in
which the appended bits of Huffman codes are encrypted by a
stream cipher, and all Huffman codes are kept unchanged. After
encryption, the JPEG file size is preserved and the format is
compliant to JPEG decoders. On cloud, the server selects the
encrypted bitstreams of some blocks as candidates. Additional
bits are encoded by LDPC-based error correction codes (ECC)
and embedded into the useful candidate bitstream by flipping
the LSBs of the encrypted appended bits of the AC coefficients
in each candidate block. After the authorized user downloads
and decrypts the marked encrypted bitstream, LSBs of the
appended bits of useful candidates are flipped again to estimate
the additional bits using a predefined blocking artifact function
and an ECC decoder. Meanwhile, the original bitstream are
losslessly recovered according to the extracted bits. This
method is improved in [19], in which the embedding room was
reserved before bitstream encryption. Although the embedding
capacity is larger, the preprocessing requires the image owner
to do an additional computation. Another solution of reversibly
hiding data in encrypted JPEG bitstream was proposed in [20],
in which image encryption and data embedding are combined
together. By scrambling the JPEG structure, the image is en-
crypted and the additional bits are embedded. The method in
[20] cannot be used in cloud storage since the server is unable
to embed bits into the encrypted bitstream.
1051-8215 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCSVT.2018.2797897, IEEE
Transactions on Circuits and Systems for Video Technology
3
Fig. 2. New Framework of RDH-EI for JPEG
Fig. 3. Traditional Framework of RDH-EI for JPEG
To enhance the security of JPEG encryption and improve the
embedding capability, another RDH-EI for JPEG bitstream was
proposed in [21]. During JPEG bitstream encryption, a new
JPEG bitstream is constructed by selecting a portion of blocks
from the whole image. Bitstreams of the rest blocks are en-
crypted and hidden in the JEPG header. With a compression
algorithm, some appended bits of the encrypted JPEG bitstream
are compressed to accommodate additional bits. On the recip-
ient side, the authorized user uses an iterative algorithm to
recovery the original JPEG bitstream according to the variation
of blocking artifacts. Compared with [18], this method has
larger capacity and better security, and the embedded bits can
be directly extracted by the server.
Although RDH-EI methods for JPEG in [18-21] have good
capabilities in data hiding and image recovery, there exists one
problem that the recipient must do a recovery task after de-
cryption. This recovery burden on the recipient side limits the
real application of RDH-EI techniques, since users always hope
to do nothing except image encryption and decryption. To this
end, this paper proposes a new JPEG RDH-EI framework, in
which no recovery task is required for the owner or the user.
III. NEW FRAMEWORK
The proposed RDH-EI framework focuses on labeling the
encrypted JPEG images on cloud storage. There are three par-
ties, including the image owner, the cloud server and the au-
thorized user. The owner encrypts a JPEG bitstream and up-
loads it to the cloud. The cloud server embeds additional mes-
sages into the encrypted bitstream to generate a marked en-
crypted bitstream. The hidden messages can be extracted from
the marked encrypted bitstream. When an authorized requires a
downloading operation, the server recovers the original en-
crypted bitstream. After decryption, the user obtains the origi-
nal JPEG image. The procedure is shown in Fig. 2.
Fig. 3 shows the traditional framework used in previous
works. The user has to implement the JPEG recovery after
decryption [18][21]. Sometimes the owner is also required to
do preprocessing [19], i.e., generating spare rooms before en-
cryption. In Fig. 2, we propose to accomplish all computation
tasks by the cloud server. Therefore, data extraction and bit-
stream recovery are transparent to the owner or the user.
The traditional framework keeps the length of encrypted
bitstream unchanged after data hiding, which leads to a limited
embedding capacity. To embed more bits into the encrypted
bitstream, the proposed framework allows the increment of
bitstream length. With a condition that the amount of embedded
bits must be larger than the bitstream increment [22], the pro-
posed framework uses a two-phase combined embedding al-
gorithm to increase the embedding payload and eliminate the
bitstream increment.
IV. JPEG ENCRYPTION AND DECRYPTION
In this section, we propose an encryption and decryption
algorithm for JPEG bitstreams. The proposed algorithm aims at
conceal the content of a JPEG image, and keep the encrypted
bitstream compliant to the JPEG decoders that are widely used.
For brevity, we discuss the baseline JPEG for grayscale images.
Fig. 4 shows a simplified syntax of the baseline JPEG. We list a
part of the terms used in JPEG in Table I. In the following
paragraphs, we use these acronyms for brevity discussions.
Table I. Acronyms used in the proposed method
Acronyms Terms
SOI start-of-image
EOI end-of-image
EOB end-of-block
JH JPEG header
ECS entropy-coded segments
DCC code of a DC coefficient
DCH Huffman code in a DCC
DCA appended bits in a DCC
ACC code of an AC coefficient
ACH Huffman code in an ACC
ACA appended bits in an ACC
Data
Hiding
JPEG
Encryption
Bitstream
Recovery
JPEG
Decryption
Data
Extraction
JPEG
Bitstream
Additional
Message
Additional
Message
Original
JPEG
Encryption
Key
Decryption
Key
Owner User
Server
Data
Hiding
JPEG
Encryption
JPEG
Recovery
JPEG
Decryption
Data
Extraction
JPEG
Bitstream
Additional
Message
Additional
Message
Original
JPEG
Encryption
Key
Decryption
Key
Owner User
Server
Preprocessing
(Optional)
1051-8215 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCSVT.2018.2797897, IEEE
Transactions on Circuits and Systems for Video Technology
4
Fig. 4. Simplified Syntax of JPEG Baseline
As shown in Fig. 4, a JPEG bitstream contains a marker of
start-of-image, a JPEG header, the entropy encoded data, and
an end-of-image marker. The section of entropy encoded data is
constructed by entropy-coded segments of all blocks. Each
ECS includes one code of DC coefficient (DCC), several codes
of the AC coefficients (ACC), and an end-of-block marker. The
DCC part contains a DC Huffman code (DCH) and DC ap-
pended bits (DCA), and each ACC part contains AC Huffman
codes (ACH) and AC appended bits (ACA). Therefore, a JPEG
bitstream J can be represented by
J={SOI, JH, ECS1, …, ECSN, EOI}
When an H×
W image is compressed into a JPEG bitstream,
there are N entropy-coded segments, where N=⌈H/8⌉∙⌈W/8⌉. We
denote these segments as ECSi, i=1,2,…,N. According to the
JPEG syntax, each segment can be represented by
ECSi={DCC<i>
, ACC<i, 1>
, ACC<i, 2>
,…, EOB}
={[DCH<i>
, DCA<i>
], [ACH<i, 1>
, ACA<i, 1>
],
[ACH<i, 2>
, ACA<i, 2>
], …, EOB}.
A.JPEG Encryption
Generally, JPEG encryption is required to be compliant to
popular decoders [23]. Therefore, we propose an algorithm to
encrypt the original JPEG bitstream to a new JPEG bitstream
using an encryption key. The encryption is symmetric, and the
secret key must be transmitted to the recipient over a reliable
channel. The encryption algorithm includes four steps.
Fig. 5. New JPEG Bitstream
In Step One, with an encryption key Kenc, the image owner
pseudo-randomly selects n entropy-coded segments from the
original JPEG bitstream J, where 1<n<N. Let these segments
be ECSs(i), i=1, 2, …, n. Correspondingly, the remaining seg-
ments are ECSr(j), j=1, 2, …, N-n. The owner specifies two
positive integers h and w satisfying n=⌈h/8⌉∙⌈w/8⌉. With these
data, the owner constructs a new JPEG frame corresponding to
an h×
w image, as shown in Fig. 5.
In Step Two, the owner uses the encryption key Kenc to en-
crypt the remaining N−n segments by a stream cipher algorithm
such as RC4, SEAL, etc. The encrypted bits, along with the
values of H and W, are then embedded into the reserved ap-
plication segments marked with APPn in JPEG header, using
the same way as [24]. In the modified header, the owner also
specifies the new image size as h×
w.
In Step Three, the owner extracts the DC coefficient from n
selected segments ECSs(i) by decoding the DC Huffman code
and the appended bits, i.e., [DCH<s(i)>
, DCA<s(i)>
], to {ds(1),
ds(2),…, ds(n)}. With the coefficients, the owner generates the
differential values {ds(1)', ds(2)',…, ds(n)'}, where
𝑑𝑠(𝑖)
′
= {
𝑑𝑠(𝑖), 𝑖 = 1
𝑑𝑠(𝑖) − 𝑑𝑠(𝑖−1), 𝑖 = 2, … , 𝑛
(1)
These differential values are further encoded by Huffman codes
to generate [DCH<s(i)>*
, DCA<s(i)>*
].
In Step Four, the owner replaces all ECSs(i) in the new en-
tropy encoded data with ECSs(i)
*
, where ECSs(i)
*
stands for the
encrypted segments,
ECSs(i)
*
={DCC<s(i)>*
, ACC<s(i), 1>
, ACC<s(i), 2>
,…, EOB}
={[DCH<s(i)>*
, DCA<s(i)>*
], [ACH<i, 1>
, ACA<i, 1>
],
[ACH<i, 2>
, ACA<i, 2>
], …, EOB}.
and i=1,2,…,n. Finally, the owner constructs the encrypted
JPEG bitstream J*
,
J*
={SOI, JH*
, ECSs(1)
*
, …, ECSs(n)
*
, EOI}
An example is shown in Fig. 6, in which (a) is the image
decoded from the original JPEG bitstream, and (b) is the en-
crypted image decoded from the encrypted JPEG bitstream.
The original size of (a) is 512×
512. After encryption, the size of
encrypted image (b) is 256×
256. The size of the encrypted
image is specified by the owner.
(a) (b)
Fig. 6. Constructing an encrypted JPEG bitstream, (a) is the original
image Lena, (b) the encrypted image.
B.JPEG Decryption
On the recipient end, the authorized user decrypts the enci-
phered JPEG bitstream J*
using a decryption key Kdec. Gener-
ally, the decryption Kdec is identical to Kenc. The decryption
procedure is reverse to encryption, also containing three steps.
In Step One, the user extracts N-n encrypted segments from
the reserved application APPn in the JPEG header JH*
. With
SOI EOI
JPEG Header Entropy Encoded Data
Appended
bits (DCA)
DC Huffman
code (DCH)
……
Entropy-coded segments of all the blocks (ECS)
Appended
bits (ACA)
AC Huffman
code (ACH)
……
……
EOB
Code of DC coeffi-
cient (DCC)
…
…
Codes of all AC coefficients
(ACC)
…
…
SOI EOI
Modified
JPEG Header
New
Entropy Encoded Data
ECSs(n)
ECSs(2)
ECSs(1) ……
……
……
1051-8215 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCSVT.2018.2797897, IEEE
Transactions on Circuits and Systems for Video Technology
5
the decryption key Kdec, the user uses the stream cipher to de-
crypt these segments back to ECSr(j), where j=1, 2, …, N-n.
In Step Two, n segments ECSs(i)
*
(i=1, 2, …, n) are extracted
from J*
. The user reads [DCH<s(i)>*
, DCA<s(i)>*
] from all seg-
ments ECSs(i)
*
, and decodes them into the values of {ds(1)',
ds(2)',…, ds(n)'}. The original DC coefficients {ds(1), ds(2),…, ds(n)}
of the selected segments are reconstructed by
𝑑𝑠(𝑖) = ∑ 𝑑𝑠(𝑘)′, 𝑖 =
𝑖
𝑘=1 1,2, … , 𝑛 (2)
The user re-encodes these DC coefficients into [DCH<s(i)>
,
DCA<s(i)>
] by Huffman codes, and reconstructs the original
ECSs(i).
In Step Three, the user rearranges all segments of ECSs(i) and
ECSr(j) and reconstructs the original entropy encoded data.
After modifying the JPEG header to specify the original image
size H×W, the original JPEG stream J is finally decrypted.
V.DATA EMBEDDING AND EXTRACTION
After the user uploads an encrypted JPEG bitstream J*
onto
the cloud, the server embeds additional messages into the ci-
phertext to generate a marked encrypted JPEG bitstream. Be-
cause the embedding procedure is reversible, the original en-
crypted JPEG bitstream can be losslessly recovered. The em-
bedding procedure is depicted in Fig. 7, including two stages. In
Stage One, with a Huffman code mapping algorithm, we loss-
lessly embed a part of additional messages into J*
to generate
an intermediate bitstream JM
*
. In Stage Two, the other mes-
sages are reversibly embedded into JM
*
using an ordered em-
bedding algorithm, and a marked encrypted bitstream JE
*
is
finally generated.
A.Stage One: Code Mapping Based Embedding
According to Table K. 5 in JPEG Standard [25], there are 162
Huffman codes for AC coefficients of luminance component.
Each code represents one run/size pair. Hence, there are 162
run/size pairs, including the pairs ranging from 0/1 to 15/10, a
pair 0/0 (EOB), and a zero-run-length (ZRL) pair 15/0. This
information is recorded as the Huffman Table in the JPEG
header. Generally, only a part of the AC Huffman codes are
used during JEPG compression. According to this fact, the
server can employ the unused codes to represent additional bits
to be embedded.
The server first extracts the Huffman table from the header
JH*
of the encrypted JPEG bitstream J*
, and constructs 162 AC
Huffman codes. These codes are separated into two categories,
the frozen codes F and the active codes A. The frozen category
F contains 12 AC Huffman codes corresponding to the run/size
pairs of {0/0, 0/1, …, 0/10, EOB, ZRL}, which are not used
during data embedding. The other 150 codes constitutes the
active category A, corresponding to the run/size pairs of
{1/1, …, 1/10, 2/1, …, 2/10, …, 15/1, …, 15/10}.
The server also extracts the AC Huffman codes {ACH<s(i), 1>
,
ACH<s(i), 2>
, …} from ECSs(i)
*
, where i=1,2,…,n. From these
codes, the server finds all codes that belong to the active cate-
gory A. Assuming there are p kinds of such codes used in J*
, we
denote them as U={u1, u2, …, up}, where U⸦A and 0≤p≤150.
Correspondingly, there must be 150−p AC Huffman codes that
are not used in J*
. Let the category of these codes be N={n1,
n2, …, nq}, where N=A−U and q=150−p.
Next, the server further divides the codes in U into 13 groups
{U4, U5, …, U16}, such that the length of each code in Uk is
equal to k bits, where Uk={u1
(k)
, u2
(k)
, …, upk
(k)
}, k=4,5,…,16,
and ∑ 𝑝𝑘
16
𝑘=4 = 𝑝. In the same way, the server also divides N
into 13 groups {N4, N5, …, N16}, where Nk={n1
(k)
, n2
(k)
, …,
nqk
(k)
} and ∑ 𝑞𝑘
16
𝑘=4 = 𝑞.
With these groups, the server maps the codes in Uk with the
codes in Nk. If pk<qk, each code in Uk is mapped with πk codes in
Nk,
ui
(k)
←{n(i−1)∙πk+1
(k)
, …, ni∙πk
(k)
} (3)
where πk=2⌊log2(qk/pk+1)⌋−1 and i=1, 2, …, pk.
Otherwise, codes in Uk and Nk are mapped by one-to-one
manner,
uj
(k)
←nj
(k)
(4)
where j=1, 2, …, qk. The mapping in (4) is equivalent to a
special case of πk=1 in (3).
Accordingly, the server modifies the Huffman Table in the
JPEG header JH*
to record the mapping relationships. The
run/size values of πk codes in Nk are replaced with the run/size
values of the mapped code in Uk. Therefore, one run/size value
in the modified Huffman Table may have several Huffman
codes. A new JPEG header JH*
' is therefore constructed.
After the mapping relationships are identified in JH*
', each
code in the mapping is used to represent log2(πk+1) additional
bits. Then, according to the additional bits M1 to be embedded,
the server substitutes the used AC Huffman code with the
mapped codes. Thus, a part of the AC Huffman codes in
ECSs(i)
*
are modified to carry additional messages.
For example, in an encrypted JPEG bitstream J*
, an AC
Huffman code “111010” is used, and the code “111011” is not
used. Assuming πk=1 (k=6), a mapping relationship can be
constructed by “111010”←“111011”. After modifying the
Huffman Table in the JPEG header, the run/size value “3/1” has
two Huffman codes “111010” and “111011”. We use “111010”
and “111011” to represent one additional bit “0” and “1”, re-
spectively. Therefore, when embedding a bit “1” into the bit-
stream, the server replaces “111010” with “111011”.
Fig. 7. Two Stages of Data Embedding
Huffman Code
Mapping
J* Bitstream
Substitution
Entropy
Decoding
Ordered
Embedding
Bitstream
Modification
JE
*
Stage One Stage Two
Message Message
JM
*
1051-8215 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCSVT.2018.2797897, IEEE
Transactions on Circuits and Systems for Video Technology
6
After sequentially modifying the AC Huffman codes to hide
the additional bits, a marked encrypted bitstream JM
*
is finally
constructed,
JM
*
={SOI, JH*
', ECSs(1)
*
', …, ECSs(n)
*
', EOI}
where ECSs(i)
*
' (i=1,2,…,n) represents the marked entro-
py-coded segments constructed in Stage One.
The embedding capacity CM of this stage can be evaluated by
𝐶𝑀 = ∑ 𝛾𝑘 ∙ log2(𝜋𝑘 + 1)
16
𝑘=4 (5)
where γk represents the number of used codes in JM
*
satisfying
the mapping relationships.
The marked bitstream JM
*
can still be decoded directly by the
popular JPEG decoders. Meanwhile, the decoded image is
identical to that decoded from J*
, since the embedding proce-
dure is lossless to the content. To make an explicit description,
we summarize the embedding steps in Table II.
Table II. Embedding Steps of Stage One
Input: Encrypted JPEG Bitstream J*
, Additional Message M1
Output: Marked Encrypted Bitstream JM
*
1. Extract JPEG header JH*
and all entropy-coded segments
ECSs(i)
*
from J*
2. Construct 162 AC Huffman codes, and divide them into two
categories: the frozen codes and the active codes.
3. Extract all AC Huffman codes from ECSs(i)
*
, and find the
active codes that are used in J*
.
4. Mapping the used active codes with the unused using (3) and
(4) to represent additional bits.
5. Record the mapping by modifying JH*
to generate a new
JPEG header JH*
'.
6. Date embedding: according to M1, substitute the codes in all
ECSs(i)
*
with the mapped codes to generate ECSs(i)
*
'.
7. Integrate JH*
' with ECSs(i)
*
' to generate the marked JM
*
.
B.Stage Two: Ordered Embedding
In Stage Two, the server embeds more additional bits into the
marked encrypted JPEG bitstream JM
*
generated in Stage One,
using an algorithm of ordered embedding based on histogram
shifting. Generally, there are two goals of histogram shifting
based RDH in JPEG bitstream. The first goal is to embed as
many payloads as possible, while the second the reason is to
reduce the bitstream length increment as far as possible.
In JPEG standard, the zigzag scanned AC coefficients in
each block are turned into (R, V) pairs, where R stands for
zero-run-length and V the value of a non-zero coefficient.
According the value V of each AC coefficient, the size of V is
identified as S. According to the relationships between V and S,
shown in Table III, all (R, V) pairs are turned into (R/S, V). Each
R/S is represented by a Huffman code according to the
pre-defined Huffman Table, i.e., Table K.5 of JPEG Standard.
Correspondingly, the value V is encoded by varia-
ble-length-integer (VLI) codes. The table of VLI codes is also
shown in Table III.
On the aspect of embedding payload, we arbitrarily choose
30 images from the BOSSbase image dataset [26], and com-
press these images by the toolbox of “Independent JPEG Group”
(IJG) [27] using different quality factors (QF). After averaging
the number of (R, V) pairs corresponding to different R in all
bitstreams, more (R, V) pairs in an image when R gets smaller.
The results in Fig. 8 show that the distributions are independent
from the quality factors. Fig. 8 also shows that around 70%
pairs have the run value R=0. As a result, for any quality factor
based bitstream we use pairs with R=0 to carry more bits.
Table III. Categories of different V and the VLI codes
V S VLI Codes
–1, 1 1 0,1
–3, –2, 2, 3 2 00, 01, 10, 11
–7, …, –4, 4, …, 7 3 000, … ,111
–15, …, –8, 8, …, 15 4 0000, … ,1111
–31, …, –16, 16, …, 31 5 00000, … ,11111
–63, …, –32, 32, …, 63 6 000000, … ,111111
–127, …, –64, 64, …,127 7 0000000, … ,1111111
–255, …, –128, 128, …, 255 8 00000000, … ,11111111
–511, …, –256, 256, …, 511 9 000000000, … ,111111111
–1023, …, –512, 512, …, 1023 10 0000000000, … ,1111111111
Fig.8. Average amount of ZRV pairs corresponding to R using dif-
ferent quality factors
When hiding additional data into a JPEG bitstream by his-
togram shifting, the length of the bitstream may increase, in-
cluding the Huffman code increment and the VLI increment.
According to Table I, if we modify the value of a non-zero AC
coefficient to another while keeping S unchanged, the length of
encoded bits does not change. Otherwise, if we modify a coef-
ficient from one size to another, length of the encoded bits will
change. For example, when changing an AC coefficient from –
4 to –5, lengths of the Huffman code and encoded bits keep
unchanged. However, when we change a coefficient from –7 to
–8, lengths of the Huffman code and the VLI code will increase.
The bitstream increment caused by data embedding is also
related to Pi, i.e., the position of the last non-zero coefficient
(using zigzag order) in each block, where i=1,…,n and 2≤Pi≤64.
To evaluate the influence of Pi, we use histogram shifting to
embed bits into the blocks with Pi≤T, where 2≤T≤64. Let the
differential payload be the embedding payload minus the bit-
stream increment. Fig. 9 shows the relationships between the
differential payload and T. The figure indicates that, when
embedding bits into the blocks with small Pi, the differential
payload is positive, meaning that the embedding payload is
larger than bitstream increment. This is the reason why we sorts
all blocks {B1, B2, … Bn} to {Bo(1), Bo(2), … Bo(n)} s.t. Po(1)≤
Po(2)≤…≤Po(n) and embed L bits into wL blocks with small Pi.
1051-8215 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCSVT.2018.2797897, IEEE
Transactions on Circuits and Systems for Video Technology
7
Fig. 9. Differential payload vs. T
Based on these considerations, the server can further embed
as many payloads as possible, while eliminating the bitstream
increment as far as possible.
The server partially decodes the bitstream JM
*
into DCT
blocks {B1, B2, … Bn}. Each block contains one DC coefficient
and 63 AC coefficients. After zigzag scanning these coeffi-
cients in each block, the server obtains a series of (R, V) pairs
for AC coefficients.
In each block, most quantized AC coefficients are equivalent
to 0. The server collects the position of the last non-zero coef-
ficient (using zigzag order) Pi in each block Bi, where i=1,…,n
and 2≤Pi≤64. The server also counts the number of (R, V) pairs
with R=0 and V=±
1 in each block. Denote the pair numbers as
ρi, where 0≤ρi≤63.
The server further sorts all blocks {B1, B2, … Bn} to {Bo(1),
Bo(2), … Bo(n)} satisfying Po(1)≤Po(2)≤…≤Po(n), in which o(∙) is
the sorting operator. In order to hide a message M2 containing L
bits into the bitstream, the server uses the first wL carrier blocks
{Bo(1), Bo(2), … Bo(wL)} to carry these bits, such that
𝐿 ≤ ∑ 𝜌𝑜(𝑘)
𝑤𝐿
𝑘=1
Accordingly, the server constructs a histogram of all (R, V)
pairs with R=0 in carrier blocks Bo(1)~Bo(wL). We denote this
histogram as h(v), in which v∈[–1023, 1023] and h(v) is the
number of (R, V) pairs with R=0 and V=v.
The server embeds L secret bits {b1, b2, …, bL} into these
blocks by shifting the histogram,
𝑣𝑗
′
=
{
𝑣𝑗 − 1 𝑖𝑓 𝑣𝑗 < −1
𝑣𝑗 − 𝑏𝑘 𝑖𝑓 𝑣𝑗 = −1
𝑣𝑗 + 𝑏𝑘 𝑖𝑓 𝑣𝑗 = 1
𝑣𝑗 + 1 𝑖𝑓 𝑣𝑗 > 1
(6)
where vj stands for the j-th AC coefficient with R=0 in the
carrier blocks, and k=1,…,L.
The histogram shifting based embedding is realized by sub-
stituting the entropy code of the pair (0, vj) with the code of (0,
vj') in the bitstream JM
*
. In this way, the server embeds L secret
bits into w segments ECSs(i)
*
' where i=1, 2, …, n. The marked
encrypted bitstream JE
*
is finally generated,
JE
*
={SOI, JH*
', ECSs(1)
*
'', …, ECSs(n)
*
'', EOI}
In which ECSs(i)
*
'' (i=1,2,…,n) represents the entropy-coded
segments constructed in this stage.
The maximal embedding capacity CE of the second stage is
equal to
𝐶𝐸 = ∑ 𝜌𝑖
𝑛
𝑖=1 (7)
Therefore, the maximal embedding capacity of the two stages is
𝐶𝑚𝑎𝑥 = 𝐶𝑀 + 𝐶𝐸 = ∑ 𝛾𝑘 ∙ log2(𝜋𝑘 + 1)
16
𝑘=4 + ∑ 𝜌𝑖
𝑛
𝑖=1 (8)
Meanwhile, the bitstream increment is determined by the em-
bedding payload L in the second stage. The embedding payload
C of the two-stage combined method is equal to
𝐶 = 𝐶𝑀 + 𝐿 (9)
Hence, the bitstream increment can be evaluated by E,
𝐸 ≈ 𝛼1 ∙ 𝐿 2
⁄ + ∑ 𝛼𝑘 ∙
𝑘≥2 𝑛𝑘(𝑤𝐿) (10)
where αk represents the increased code length when modifying
(0/k, ±
(2k
–1)) to (0/k+1, ±
2k
), α1=1, and nk(wL) the number of
(0/k, ±
(2k
–1)) pairs in the histogram of the first wL blocks.
It should be noted that data embedding operations in both
stages do not interfere with each other, which guarantee the
correctness of data extractions. In the first stage, we do not use
the frozen codes {0/0, 0/1, …, 0/10, 15/0} to carry additional
bits, i.e., the AC Huffman codes of (R, V) run/size pairs with
R=0 are not modified. In the second stage, we only modify the
AC Huffman codes with respect to R=0 to carry more bits.
The embedding steps of the second stage are summarized in
Table IV.
Table IV. Embedding Steps of Stage Two
Input: Marked Encrypted Bitstream JM
*
, Additional Message M2
Output: Marked Encrypted Bitstream JE
*
1. Extract all entropy-coded segments ECSs(i)
*
' from JM
*
2. Decode all ECSs(i)
*
' into DCT blocks {B1, B2, … Bn}, and
obtain a series of (R, V) pairs by zigzag scanning.
3. In each Bi, collect the position of the last non-zero coefficient
Pi, and count the number ρi of (R, V) pairs with (0,±
1).
4. Sort all blocks into {Bo(1), …, Bo(n)} s.t. Po(1)≤Po(2) ≤…≤Po(n),
and choose the first wL blocks Bo(1)~Bo(wL) as carriers.
5. Construct a histogram h(v) of all (R, V) pairs with R=0 in
carrier blocks Bo(1)~Bo(wL).
6. Data embedding: implement (6) to hide M2 into Bo(1)~Bo(wL)
by modifying ECSs(i)
*
' to ECSs(1)
*
'',
7. Integrate JH*
' with ECSs(i)
*
'' to generate the marked JE
*
.
C.Data Extraction and Bitstream Recovery
The proposed RDH-EI protocol guarantees that the hidden
messages can be exactly extracted from the marked bitstream
JE
*
, and the original encrypted bitstream J*
can be losslessly
recovered. The procedure of data extraction and bitstream
recovery also includes two stages.
In the first stage, the server extracts L bits from JE
*
using the
reverse histogram shifting and recovers JE
*
to JM
*
. After par-
tially decoding JE
*
into the quantized DCT blocks, the server
sorts all blocks {B1, B2, … Bn} to {Bo(1), Bo(2), … Bo(n)} such
that Po(1)≤Po(2)≤…≤Po(n), using the same algorithm depicted in
Subsection B. According to the payload L, the server identifies
first wL carrier blocks {Bo(1), Bo(2), … Bo(wL)}. From these blocks,
the server sequentially reads all (0, ±
1) pairs, and identifies L
hidden bits {b1, b2, …, bL} by
1051-8215 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCSVT.2018.2797897, IEEE
Transactions on Circuits and Systems for Video Technology
8
𝑏𝑘 = {
0, 𝑖𝑓 |𝑣𝑗′| = 1
1, 𝑖𝑓 |𝑣𝑗′| = 2
(11)
where k=1,2,…,L.
Meanwhile, the original (R, V) pairs with R=0 in wL DCT
blocks are recovered by
𝑣𝑖 =
{
𝑣𝑖
′
+ 1 𝑖𝑓 𝑣𝑖′ < −2
−1 𝑖𝑓 − 2 ≤ 𝑣𝑖′ ≤ −1
1 𝑖𝑓 1 ≤ 𝑣𝑖′ ≤ 2
𝑣𝑖
′
− 1 𝑖𝑓 𝑣𝑖
′
> 2
(12)
The histogram shifting based recovery is realized by substi-
tuting the entropy code of the pair (0, vj') with the code of (0, vj)
in JE
*
. In this way, the marked encrypted bitstream JM
*
is re-
covered,
JM
*
={SOI, JH*
', ECSs(1)
*
', …, ECSs(n)
*
', EOI}
In the second stage, the server extracts the Huffman table
from JH*
', from which the mapping relationships in (3) and (4)
can be constructed. The server sequentially reads the AC
Huffman codes form ECSs(i)
*
' (i=1,2,…,n), and extracts CM bits
according to the mapping relationships. Meanwhile, the server
replaces the modified Huffman table in JH*
' with the original
Huffman Table K.5 defined in JPEG Standard [25]. According
to the recovered Huffman table and the mapping relationships,
the server substitutes the AC Huffman codes that were used to
carry additional bits in JM
*
with the original Huffman codes.
After that, the original encrypted JPEG bitstream J*
can be
recovered,
J*
={SOI, JH*
, ECSs(1)
*
, …, ECSs(n)
*
, EOI}
VI. EXPERIMENTAL RESULTS AND ANALYSIS
A.Performance of JPEG Encryption/Decryption
To verify the proposed method, we use grayscale images
sized 512×
512, and compress them to JPEG bitstreams using
different quality factors. Two groups of experimental results
are shown in Fig. 10, in which JPEG bitstreams of the images
Peppers and Lake are used. Fig. 10(a) shows the original im-
ages decoded from the original JPEG. Quality factors of the
bitstreams are both 80. We encrypt Peppers to a small image
sized 256×
256, and Lake to a smaller image sized 128×
256. Fig.
10(b) shows the images decoded from the encrypted JPEG
bitstreams. With the proposed method, we embed 2863 bits and
798 bits into the encrypted bitstreams of Peppers and Lake,
respectively. The images decoded from the marked encrypted
bitstreams are shown in Fig. 10(c). After decryption, the orig-
inal JPEG bitstreams can be reconstructed. Fig. 10(d) shows the
images of the decrypted bitstreams after data extraction and
bitstream recovery, which are the same as the original JPEG
images in Fig. 10(a).
Fig. 11(a) shows the original JPEG image Lena sized
512×
512. We use the encryption algorithms in [21] to encrypt
the bitstream into 256×
256 sized ciphertext, which is shown
Fig. 11(b). This image looks like an error-decoded image. We
also use the proposed algorithm to do the encryption. Fig. 11(c)
shows the ciphertext JPEG image generated by the proposed
method, which overcomes the drawback in Fig. 11(b) and
achieves a better visual effect.
(a)
(b)
(c)
(d)
Fig. 10. RDH-EI in JPEG bitstreams of Peppers and Lake, (a) the
original JPEG images, (b) the encrypted images with smaller sizes, (c)
the marked encrypted image, (d) the recovered images
(a) (b) (c)
Fig. 11. An example of JPEG bitstream encryption, (a) is the original
image Lena, (b) the encrypted image using [21], and (c) the encrypted
image using the proposed method.
The proposed encryption algorithm is also against attackers.
A representative attack to the format-compliant JPEG encryp-
tion was proposed in [28], which uses the AC Huffman codes in
an encrypted bitstream to analyze the contour of the original
content. We use this attack to verify our method. After en-
crypting the original image Airplane and Baboon into 256×
256
ciphertext images, we use [28] to attack the encrypted bitstream.
Fig. 12(a) shows both images of the attacking results, from
which the attackers cannot achieve the contour information of
1051-8215 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCSVT.2018.2797897, IEEE
Transactions on Circuits and Systems for Video Technology
9
the original image. Fig. 12(b) shows the results of attacking the
encryption algorithm proposed in [18], which reveals the con-
tour of the original images. These experiments indicate that the
proposed method has better security than [18].
(a)
(b)
Fig. 12. Attacking JPEG encrypted bitstream of Airplane and Baboon,
(a) is the attack to our method, and (b) the attack to [18]
The propose encryption algorithm is also secure against the
popular ciphertext-only attack. During JPEG bitstream en-
cryption, we pseudo-randomly select n entropy-coded segment
from the bitstream to construct a new JPEG bitstream, which
can be decoded to smaller sized ciphertext image. Since only a
part of Huffman codes are available an adversary has no in-
formation of the original size. It is difficult for an adversary to
find the original orders of all blocks from 𝐶𝑁
𝑛
∙ 𝑛! possibilities,
as long as the block number N large enough. Therefore the key
space is large enough to ensure security.
B.Performance of Data Hiding
When embedding additional messages into the encrypted
JPEG bitstream, the payloads is related to the sizes of the en-
crypted JPEG images. The original JPEG bitstream are en-
crypted to four kinds of smaller sized images, including
128×
192, 192×
256, 256×
256, and 256×
384. We define the
differential payload as DP to evaluate the gap between em-
bedding payload C and bitstream increment E, i.e. DP=C−E.
Table V shows the embedding payloads corresponding to dif-
ferent sizes and DP values. The results indicate that the em-
bedding payload increases when the size of an encrypted image
gets larger. When a small payload C is embedded into the en-
crypted bitstream, the differential payload DP achieves the best
DPmax. Large embedding payloads can also be achieved when
DP approaches DP0=0 or minimum DPmin.
In Table VI, we provide another group of experimental re-
sults of bitstream sizes using DPmax. The original bitstreams are
generated by compressing 512×
512 images using a quality
factor 80. After encrypting them to ciphertexts corresponding
to 256×
256 encrypted images, sizes of the encrypted bitstreams
are close to the original bitstreams. After embedding the secret
messages into the encrypted bitstreams, there is a slight size
increment in each marked encrypted bitstream.
In Fig. 13(a)~(d), the relationships between the embedding
payload C and the bitstream increment E. The increment values
stand for the increased bits of the entire file, including all parts
of the whole bitstream. We use the images Lena, Peppers, Lake
and Airplane to implement the proposed JPEG RDH-EI. We
encrypt each JEPG to the sizes of 128×
192, 192×
256, 256×
256,
and 256×
384, respectively. Quality factor of these JPEG im-
ages are 80. The dash line on each figure is used as a reference
to show the case when C=E. Results show that the embedding
payload is larger than the bitstream increment in most embed-
ding payloads.
Table V. Payloads Corresponding to Different DP (Bits)
Images
128×
192 192×
256 256×
256 256×
384
DPmax DP0 DPmin DPmax DP0 DPmin DPmax DP0 DPmin DPmax DP0 DPmin
Lena 723 1189 1386 1294 2473 2761 1724 3483 3667 2750 5452 5548
Man 581 1081 1831 612 2015 3671 791 2308 4856 1147 3276 7258
Lake 695 1318 1920 1046 2317 3760 1368 2915 4964 2016 4328 7342
Peppers 660 1494 1621 2123 3193 3193 2863 4253 4253 3949 6262 6262
Baboon 155 626 2825 276 1077 5576 365 1728 7547 546 2800 11271
Airplane 583 1115 1375 1146 2266 2808 1446 3044 3706 2692 4668 5547
Table VI. Size of Files before and after Embedding Data
Images
Original
(Bytes)
Encrypted
(Bytes)
Additional
Message (Bits)
Marked
(Bytes)
Lena 37767 37827 1724 37997
Man 49010 49064 791 49125
Lake 51965 51992 1368 52116
Peppers 39751 39804 2863 40083
Baboon 78506 78442 365 78463
Airplane 38536 38622 1446 38735
1051-8215 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCSVT.2018.2797897, IEEE
Transactions on Circuits and Systems for Video Technology
10
(a) (b)
(c) (d)
Fig. 13. Embedding Payload vs. Bitstream Increment, (a) the original JPEG image Lena, (b) Peppers, (c) Lake, (d) Airplane
We also compare the proposed method with JPEG RDH-EI
methods in [18], [19], and [21]. We use the proposed method to
encrypt a bitstream to ciphertext bitstream corresponding to
256×
256 sized images. Table VII shows that proposed method
can achieve a much larger payload than [18], [19], and [21].
Table VII. Comparison of Embedding Capacity (Bits)
Images [18] [19] [21] Proposed
Lena 750 798 1364 3667
Baboon 750 1555 768 7547
Man 750 1809 1368 4856
Peppers 750 960 1026 4253
Lake 750 1032 1023 4964
The features of these methods are compared in Table VIII.
The proposed method requires the owners and users to do
nothing encryption or decryption, Compared with the
pre-processing or post-processing required in [18], [19] and
[21], the proposed method relaxes the burdens on both the
owner side and the user side.
Table VIII. Comparisons of Features in JPEG RDH-EI Methods
Feature [18] [19] [21] Proposed
Owner’s Pre-processing No Yes No No
User’s Post-processing Yes Yes Yes No
The embedding payload is also related to the quality factor of
the JPEG bitstream. In Fig. 14, we compress the original im-
ages into JPEG bitstream using different quality factors, and
embedding additional bits into the encrypted bitstream. Fig.
13(a)~(c) show the embedding payloads corresponding to
DPmax, DP0, and DPmin, which indicates that the embedding
payloads increases when the quality factor get larger.
(a)
(b)
1051-8215 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCSVT.2018.2797897, IEEE
Transactions on Circuits and Systems for Video Technology
11
(c)
Fig. 14. Embedding Payload vs. Quality Factor, (a) the original
JPEG image Lena, (b) Peppers, (c) Lake
VII. CONCLUSION AND DISCUSSION
This paper proposes a new framework of reversible data
hiding in encrypted JPEG bitstreams. A JPEG encryption and
decryption algorithm is developed to hide the content of the
original image. The encryption is format-compliant to the
popular JPEG decoders. On the cloud side, the server can em-
bed a large amount of additional bits into the encrypted bit-
stream. We propose to do the embedding using a combination
of code mapping and ordered embedding. Once an authorized
user requires a downloading operation, the server extracts the
additional message and recovers the original encrypted bit-
stream. Since the recovery has been done before downloading,
the user obtains an image exactly the same as the original.
The proposed framework outperforms the previous JPEG
RDH-EI frameworks on three aspects. First, the proposed em-
bedding method provides a much larger payload than the other
methods. Second, the proposed framework frees the computa-
tion burdens on both the owner and the user sides. While
pre-processing or post-processing is required in the traditional
works, the propose framework requires the owner or the user to
no extra tasks except encryption or decryption. Finally, the
proposed JPEG encryption algorithm is also secure against the
ciphertext-only attack.
ACKNOWLEDGEMENT
The authors are grateful to the anonymous reviewers for their
constructive comments.
REFERENCES
[1] Z. Ni, Y. -Q. Shi, N. Ansari, and W. Su, “Reversible Data Hid-
ing,” IEEE Transactions on Circuits and Systems for Video
Technology, vol. 16, no. 3, pp. 354-362, 2006.
[2] W. -L. Tai, C. -M. Yeh, and C. -C. Chang, “Reversible data
hiding based on histogram modification of pixel differences,”
IEEE Transactions on Circuits and Systems for Video Tech-
nology, vol. 19, no. 6, pp. 906–910, 2009.
[3] X. Zhang, “Reversible data hiding in encrypted images,” IEEE
Signal Processing Letters, vol. 18, no. 4, pp. 255–258, 2011.
[4] W. Hong, T. Chen, and H. Wu, “An improved reversible data
hiding in encrypted images using side match,” IEEE Signal
Processing Letters, vol. 19, no. 4, pp. 199–202, 2012.
[5] X. Liao, and C. Shu, “Reversible data hiding in encrypted im-
ages based on absolute mean difference of multiple neighboring
pixels,” Journal of Visual Communication and Image Repre-
sentation, vol. 28, pp. 21–27, 2015.
[6] Z. Qian, S. Dai, F. Jiang, and X. Zhang, “Improved joint re-
versible data hiding in encrypted images”, Journal of Visual
Communication and Image Representation, vol. 40, pp. 732-738,
2016.
[7] J. Zhou, W. Sun, L. Dong, et al. “Secure reversible image data
hiding over encrypted domain via key modulation,” IEEE
Transactions on Circuits and Systems for Video Technology, vol.
26, no. 3, pp. 441-452, 2016.
[8] X. Zhang, “Separable reversible data hiding in encrypted image,”
IEEE Transactions on Information Forensics Security, vol. 7, no.
2, pp. 826–832, 2012.
[9] X. Wu, and W. Sun, “High-capacity reversible data hiding in
encrypted images by prediction error,” Signal processing, vol.
104, pp. 387-400, 2014.
[10] Z. Qian, and X. Zhang, “Reversible data hiding in encrypted
image with distributed source encoding,” IEEE Transactions on
Circuits and Systems for Video Technology, vol. 26, no. 4, pp.
636-646, 2016.
[11] Z. Qian, X. Zhang, and G. Feng, “Reversible data hiding in
encrypted images based on progressive recovery”, IEEE Signal
Processing Letters, vol. 23, no. 11, pp. 1672-1676, 2016.
[12] F. Huang, J. Huang, and Y. Q. Shi, “New framework for re-
versible data hiding in encrypted domain”. IEEE Transactions
on Information Forensics and Security, vol. 11, no. 12, pp.
2777-2789, 2016.
[13] K. Ma, W. Zhang, X. Zhao, et al. “Reversible data hiding in
encrypted images by reserving room before encryption,” IEEE
Transactions on Information Forensics Security, vol. 8, no. 3, pp.
553-562, 2013.
[14] W. Zhang, K. Ma and N. Yu, “Reversibility improved data
hiding in encrypted images,” Signal Processing, vol. 94, pp.
118–127, 2014.
[15] X. Cao, L. Du, X. Wei, et al. “High capacity reversible data
hiding in encrypted images by patch-level sparse representation,”
IEEE Transactions on Cybernetics, vol. 46, no. 5, pp. 1132-1143,
2016.
[16] M. Fujiyoshi, “Separable reversible data hiding in encrypted
images with histogram permutation,” in Proceeding IEEE In-
ternational Conference on Multimedia and Expo, pp. 1-4, 2013.
[17] W. Zhang, H. Wang, D. Hou, and N. Yu, “Reversible data hiding
in encrypted images by reversible image transformation,” IEEE
Transactions on Multimedia, vol. 18, no. 8, pp. 1469-1479,
2016.
[18] Z. Qian, X. Zhang, and S. Wang, “Reversible data hiding in
encrypted JPEG bitstream,” IEEE Transactions on Multimedia,
vol. 16, no. 5, pp. 1486-1491, 2014.
[19] J. C. Chang, Y. Z. Lu, and H. L. Wu, “A separable reversible
data hiding scheme for encrypted JPEG bitstreams,” Signal
Processing, vol. 133, pp. 135-143, 2017.
[20] S. Y. Ong, K. S. Wong and K. Tanaka, “Scrambling-embedding
for JPEG compressed image,” Signal Processing, vol. 109, pp.
56-68, 2015.
[21] Z. Qian, H. Zhou, X. Zhang, and W. Zhang, “Separable re-
versible data hiding in encrypted JPEG bitstreams,” IEEE
Transactions on Dependable and Secure Computing, doi:
10.1109 /TDSC.2016.2634161.
[22] F. Huang, X. Qu, H. J. Kim, and J. Huang, “Reversible Data
Hiding in JPEG Images,” IEEE Trans. Circuits and Systems for
Video Technology, vol. 26, no. 9, pp. 1610-1621, 2015.
[23] A. Massoudi, F. Lefebvre, C. De Vleeschouwer, et al. “Over-
view on selective encryption of image and video: challenges and
perspectives,” EURASIP Journal on Information Security, vol. 5,
pp. 1-18, 2008.
[24] T. Richter, A. Artusi, and T. Ebrahimi, “JPEG XT: A New
Family of JPEG Backward-Compatible Standards,” IEEE Mul-
1051-8215 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCSVT.2018.2797897, IEEE
Transactions on Circuits and Systems for Video Technology
12
timedia, vol. 23, no. 3, pp. 80-88, 2016.
[25] Int. Tele. Union, CCITT Recommendation T.81, “Information
technology-digital compression and coding of continuous-tone
Still Images - Requirements and Guidelines,” 1992.
[26] Bossbase Database, http://www.agents.cz/boss/BOSSFinal/
[27] Independent JPEG Group, http://www.ijg.org/
[28] S. Y. Ong, K. S. Wong, X. Qi, et al. “Beyond format-compliant
encryption for JPEG image,” Signal Processing: Image Com-
munication, vol. 31, pp. 47-60, 2015.
Zhenxing Qian (M’12) received both the B.S.
and the Ph.D. degrees from University of Sci-
ence and Technology of China (USTC) in 2003
and 2007, respectively. Since 2009, he has
been with the faculty of the School of Com-
munication and Information Engineering,
Shanghai University, where he is currently a
Professor. His research interests include in-
formation hiding and multimedia security.
Haisheng Xu received the B.S. degree in
School of Communication and Information
Engineering, Shanghai University. He is cur-
rently working toward the Master degree at
Shanghai University. His research interests
include information hiding and image pro-
cessing.
Xiangyang Luo received the M.S. degree and
the Ph.D. degree from Zhengzhou Information
Science and Technology Institute, Zhengzhou,
China in 2004 and 2010, respectively. He has
published more than 100 refereed international
journal and conference papers. He is currently
an associate professor and Ph.D. supervisor at
State Key Laboratory of Mathematical Engi-
neering and Advanced Computing. His re-
search interests include multimedia security.
Xinpeng Zhang (M’11) received the B.S.
degree in computational mathematics from
Jilin University, China, in 1995, and the M.E.
and Ph.D. degrees in communication and
information system from Shanghai University,
China, in 2001 and 2004, respectively. Since
2004, he has been with the faculty of the
School of Communication and Information
Engineering, Shanghai University, where he is
currently a Professor. His research interests
include information hiding, image processing
and digital forensics.

More Related Content

Similar to New Framework of Reversible Data Hiding in Encrypted JPEG Bitstreams.pdf

IRJET- High Capacity Reversible Data Hiding in Encrypted Images by MSB Predic...
IRJET- High Capacity Reversible Data Hiding in Encrypted Images by MSB Predic...IRJET- High Capacity Reversible Data Hiding in Encrypted Images by MSB Predic...
IRJET- High Capacity Reversible Data Hiding in Encrypted Images by MSB Predic...IRJET Journal
 
Reversible encrypted data concealment in images by reserving room approach
Reversible encrypted data concealment in images by reserving room approachReversible encrypted data concealment in images by reserving room approach
Reversible encrypted data concealment in images by reserving room approachIAEME Publication
 
Reversible Data Hiding In Encrypted Images And Its Application To Secure Miss...
Reversible Data Hiding In Encrypted Images And Its Application To Secure Miss...Reversible Data Hiding In Encrypted Images And Its Application To Secure Miss...
Reversible Data Hiding In Encrypted Images And Its Application To Secure Miss...CSCJournals
 
IRJET- Secure Data Deduplication for Cloud Server using HMAC Algorithm
IRJET- Secure Data Deduplication for Cloud Server using HMAC AlgorithmIRJET- Secure Data Deduplication for Cloud Server using HMAC Algorithm
IRJET- Secure Data Deduplication for Cloud Server using HMAC AlgorithmIRJET Journal
 
IRJET- Reversible Image Data Hiding in an Encrypted Domain with High Level of...
IRJET- Reversible Image Data Hiding in an Encrypted Domain with High Level of...IRJET- Reversible Image Data Hiding in an Encrypted Domain with High Level of...
IRJET- Reversible Image Data Hiding in an Encrypted Domain with High Level of...IRJET Journal
 
Non-Separable Histogram Based Reversible Data Hiding Approach Using Inverse S...
Non-Separable Histogram Based Reversible Data Hiding Approach Using Inverse S...Non-Separable Histogram Based Reversible Data Hiding Approach Using Inverse S...
Non-Separable Histogram Based Reversible Data Hiding Approach Using Inverse S...IJCSIS Research Publications
 
IRJET- An Integrity Auditing &Data Dedupe withEffective Bandwidth in Cloud St...
IRJET- An Integrity Auditing &Data Dedupe withEffective Bandwidth in Cloud St...IRJET- An Integrity Auditing &Data Dedupe withEffective Bandwidth in Cloud St...
IRJET- An Integrity Auditing &Data Dedupe withEffective Bandwidth in Cloud St...IRJET Journal
 
Data integrity proof techniques in cloud storage
Data integrity proof techniques in cloud storageData integrity proof techniques in cloud storage
Data integrity proof techniques in cloud storageIAEME Publication
 
International Journal of Computational Engineering Research(IJCER)
International Journal of Computational Engineering Research(IJCER)International Journal of Computational Engineering Research(IJCER)
International Journal of Computational Engineering Research(IJCER)ijceronline
 
IRJET- Data Hiding in Video Stream by Efficient Data Embedding
IRJET- Data Hiding in Video Stream by Efficient Data EmbeddingIRJET- Data Hiding in Video Stream by Efficient Data Embedding
IRJET- Data Hiding in Video Stream by Efficient Data EmbeddingIRJET Journal
 
Data encryption using LSB matching algorithm and Reserving Room before Encryp...
Data encryption using LSB matching algorithm and Reserving Room before Encryp...Data encryption using LSB matching algorithm and Reserving Room before Encryp...
Data encryption using LSB matching algorithm and Reserving Room before Encryp...IJERA Editor
 
Metaphorical Analysis of diseases in Tomato leaves using Deep Learning Algori...
Metaphorical Analysis of diseases in Tomato leaves using Deep Learning Algori...Metaphorical Analysis of diseases in Tomato leaves using Deep Learning Algori...
Metaphorical Analysis of diseases in Tomato leaves using Deep Learning Algori...IRJET Journal
 
IRJET- Mosaic Image Creation in Video for Secure Transmission
IRJET- Mosaic Image Creation in Video for Secure TransmissionIRJET- Mosaic Image Creation in Video for Secure Transmission
IRJET- Mosaic Image Creation in Video for Secure TransmissionIRJET Journal
 
Seed block algorithm
Seed block algorithmSeed block algorithm
Seed block algorithmDipak Badhe
 
Optimized WES-System with Image Bit Embedding for Enhancing the Security of H...
Optimized WES-System with Image Bit Embedding for Enhancing the Security of H...Optimized WES-System with Image Bit Embedding for Enhancing the Security of H...
Optimized WES-System with Image Bit Embedding for Enhancing the Security of H...IRJET Journal
 
IRJET - A Secure Access Policies based on Data Deduplication System
IRJET - A Secure Access Policies based on Data Deduplication SystemIRJET - A Secure Access Policies based on Data Deduplication System
IRJET - A Secure Access Policies based on Data Deduplication SystemIRJET Journal
 

Similar to New Framework of Reversible Data Hiding in Encrypted JPEG Bitstreams.pdf (20)

REAL TIME DATA TRANSFER VIA VIDEO USING REVERSIBLE DATA HIDING TECHNIQUE
REAL TIME DATA TRANSFER VIA VIDEO USING REVERSIBLE DATA HIDING TECHNIQUEREAL TIME DATA TRANSFER VIA VIDEO USING REVERSIBLE DATA HIDING TECHNIQUE
REAL TIME DATA TRANSFER VIA VIDEO USING REVERSIBLE DATA HIDING TECHNIQUE
 
K041068072
K041068072K041068072
K041068072
 
IRJET- High Capacity Reversible Data Hiding in Encrypted Images by MSB Predic...
IRJET- High Capacity Reversible Data Hiding in Encrypted Images by MSB Predic...IRJET- High Capacity Reversible Data Hiding in Encrypted Images by MSB Predic...
IRJET- High Capacity Reversible Data Hiding in Encrypted Images by MSB Predic...
 
Reversible encrypted data concealment in images by reserving room approach
Reversible encrypted data concealment in images by reserving room approachReversible encrypted data concealment in images by reserving room approach
Reversible encrypted data concealment in images by reserving room approach
 
A LITERATURE SURVEY ON SECURE JOINT DATA HIDING AND COMPRESSION SCHEME TO STO...
A LITERATURE SURVEY ON SECURE JOINT DATA HIDING AND COMPRESSION SCHEME TO STO...A LITERATURE SURVEY ON SECURE JOINT DATA HIDING AND COMPRESSION SCHEME TO STO...
A LITERATURE SURVEY ON SECURE JOINT DATA HIDING AND COMPRESSION SCHEME TO STO...
 
Reversible Data Hiding In Encrypted Images And Its Application To Secure Miss...
Reversible Data Hiding In Encrypted Images And Its Application To Secure Miss...Reversible Data Hiding In Encrypted Images And Its Application To Secure Miss...
Reversible Data Hiding In Encrypted Images And Its Application To Secure Miss...
 
IRJET- Secure Data Deduplication for Cloud Server using HMAC Algorithm
IRJET- Secure Data Deduplication for Cloud Server using HMAC AlgorithmIRJET- Secure Data Deduplication for Cloud Server using HMAC Algorithm
IRJET- Secure Data Deduplication for Cloud Server using HMAC Algorithm
 
IRJET- Reversible Image Data Hiding in an Encrypted Domain with High Level of...
IRJET- Reversible Image Data Hiding in an Encrypted Domain with High Level of...IRJET- Reversible Image Data Hiding in an Encrypted Domain with High Level of...
IRJET- Reversible Image Data Hiding in an Encrypted Domain with High Level of...
 
Non-Separable Histogram Based Reversible Data Hiding Approach Using Inverse S...
Non-Separable Histogram Based Reversible Data Hiding Approach Using Inverse S...Non-Separable Histogram Based Reversible Data Hiding Approach Using Inverse S...
Non-Separable Histogram Based Reversible Data Hiding Approach Using Inverse S...
 
IRJET- An Integrity Auditing &Data Dedupe withEffective Bandwidth in Cloud St...
IRJET- An Integrity Auditing &Data Dedupe withEffective Bandwidth in Cloud St...IRJET- An Integrity Auditing &Data Dedupe withEffective Bandwidth in Cloud St...
IRJET- An Integrity Auditing &Data Dedupe withEffective Bandwidth in Cloud St...
 
Data integrity proof techniques in cloud storage
Data integrity proof techniques in cloud storageData integrity proof techniques in cloud storage
Data integrity proof techniques in cloud storage
 
International Journal of Computational Engineering Research(IJCER)
International Journal of Computational Engineering Research(IJCER)International Journal of Computational Engineering Research(IJCER)
International Journal of Computational Engineering Research(IJCER)
 
IRJET- Data Hiding in Video Stream by Efficient Data Embedding
IRJET- Data Hiding in Video Stream by Efficient Data EmbeddingIRJET- Data Hiding in Video Stream by Efficient Data Embedding
IRJET- Data Hiding in Video Stream by Efficient Data Embedding
 
It3116411644
It3116411644It3116411644
It3116411644
 
Data encryption using LSB matching algorithm and Reserving Room before Encryp...
Data encryption using LSB matching algorithm and Reserving Room before Encryp...Data encryption using LSB matching algorithm and Reserving Room before Encryp...
Data encryption using LSB matching algorithm and Reserving Room before Encryp...
 
Metaphorical Analysis of diseases in Tomato leaves using Deep Learning Algori...
Metaphorical Analysis of diseases in Tomato leaves using Deep Learning Algori...Metaphorical Analysis of diseases in Tomato leaves using Deep Learning Algori...
Metaphorical Analysis of diseases in Tomato leaves using Deep Learning Algori...
 
IRJET- Mosaic Image Creation in Video for Secure Transmission
IRJET- Mosaic Image Creation in Video for Secure TransmissionIRJET- Mosaic Image Creation in Video for Secure Transmission
IRJET- Mosaic Image Creation in Video for Secure Transmission
 
Seed block algorithm
Seed block algorithmSeed block algorithm
Seed block algorithm
 
Optimized WES-System with Image Bit Embedding for Enhancing the Security of H...
Optimized WES-System with Image Bit Embedding for Enhancing the Security of H...Optimized WES-System with Image Bit Embedding for Enhancing the Security of H...
Optimized WES-System with Image Bit Embedding for Enhancing the Security of H...
 
IRJET - A Secure Access Policies based on Data Deduplication System
IRJET - A Secure Access Policies based on Data Deduplication SystemIRJET - A Secure Access Policies based on Data Deduplication System
IRJET - A Secure Access Policies based on Data Deduplication System
 

Recently uploaded

DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamUiPathCommunity
 
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)Samir Dash
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Victor Rentea
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MIND CTI
 
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...TrustArc
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businesspanagenda
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Victor Rentea
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FMESafe Software
 
Less Is More: Utilizing Ballerina to Architect a Cloud Data Platform
Less Is More: Utilizing Ballerina to Architect a Cloud Data PlatformLess Is More: Utilizing Ballerina to Architect a Cloud Data Platform
Less Is More: Utilizing Ballerina to Architect a Cloud Data PlatformWSO2
 
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....rightmanforbloodline
 
Decarbonising Commercial Real Estate: The Role of Operational Performance
Decarbonising Commercial Real Estate: The Role of Operational PerformanceDecarbonising Commercial Real Estate: The Role of Operational Performance
Decarbonising Commercial Real Estate: The Role of Operational PerformanceIES VE
 
الأمن السيبراني - ما لا يسع للمستخدم جهله
الأمن السيبراني - ما لا يسع للمستخدم جهلهالأمن السيبراني - ما لا يسع للمستخدم جهله
الأمن السيبراني - ما لا يسع للمستخدم جهلهMohamed Sweelam
 
Simplifying Mobile A11y Presentation.pptx
Simplifying Mobile A11y Presentation.pptxSimplifying Mobile A11y Presentation.pptx
Simplifying Mobile A11y Presentation.pptxMarkSteadman7
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingEdi Saputra
 
Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Zilliz
 
Top 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development CompaniesTop 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development CompaniesTopCSSGallery
 
API Governance and Monetization - The evolution of API governance
API Governance and Monetization -  The evolution of API governanceAPI Governance and Monetization -  The evolution of API governance
API Governance and Monetization - The evolution of API governanceWSO2
 
Six Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal OntologySix Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal Ontologyjohnbeverley2021
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDropbox
 
Design Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptxDesign Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptxFIDO Alliance
 

Recently uploaded (20)

DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
 
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024Finding Java's Hidden Performance Traps @ DevoxxUK 2024
Finding Java's Hidden Performance Traps @ DevoxxUK 2024
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Less Is More: Utilizing Ballerina to Architect a Cloud Data Platform
Less Is More: Utilizing Ballerina to Architect a Cloud Data PlatformLess Is More: Utilizing Ballerina to Architect a Cloud Data Platform
Less Is More: Utilizing Ballerina to Architect a Cloud Data Platform
 
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
TEST BANK For Principles of Anatomy and Physiology, 16th Edition by Gerard J....
 
Decarbonising Commercial Real Estate: The Role of Operational Performance
Decarbonising Commercial Real Estate: The Role of Operational PerformanceDecarbonising Commercial Real Estate: The Role of Operational Performance
Decarbonising Commercial Real Estate: The Role of Operational Performance
 
الأمن السيبراني - ما لا يسع للمستخدم جهله
الأمن السيبراني - ما لا يسع للمستخدم جهلهالأمن السيبراني - ما لا يسع للمستخدم جهله
الأمن السيبراني - ما لا يسع للمستخدم جهله
 
Simplifying Mobile A11y Presentation.pptx
Simplifying Mobile A11y Presentation.pptxSimplifying Mobile A11y Presentation.pptx
Simplifying Mobile A11y Presentation.pptx
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)
 
Top 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development CompaniesTop 10 CodeIgniter Development Companies
Top 10 CodeIgniter Development Companies
 
API Governance and Monetization - The evolution of API governance
API Governance and Monetization -  The evolution of API governanceAPI Governance and Monetization -  The evolution of API governance
API Governance and Monetization - The evolution of API governance
 
Six Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal OntologySix Myths about Ontologies: The Basics of Formal Ontology
Six Myths about Ontologies: The Basics of Formal Ontology
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
 
Design Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptxDesign Guidelines for Passkeys 2024.pptx
Design Guidelines for Passkeys 2024.pptx
 

New Framework of Reversible Data Hiding in Encrypted JPEG Bitstreams.pdf

  • 1. 1051-8215 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCSVT.2018.2797897, IEEE Transactions on Circuits and Systems for Video Technology Copyright © 2018 IEEE. Personal use of this material is permitted. However, permission to use this material for any other purposes must be obtained from the IEEE by sending an email to pubs-permissions@ieee.org.  I. INTRODUCTION eversible data hiding in encrypted images (RDH-EI) is an emerging technique originated from reversible data hiding in plaintext images [1-2], which has been investigated by many researchers [3-21]. In the cloud storage scenario, this technique is realized by a protocol that contains three parties, an image owner, a cloud server and an authorized user. The image owner encrypts the contents before uploading images onto a cloud storage server. The server hides additional messages into the encrypted images. The RDH-EI protocol guarantees that the hidden message can be exactly extracted by the server, and the original content of the images can be losslessly recovered by the authorized user. RDH-EI is especially useful for labeling the ciphertext in cloud storage, as shown in Fig. 1. When image owners hope to protect their privacy, RDH-EI first provides a secure encryption algorithm for the owners to encrypt their images before up- loading. On the cloud side, RDH-EI allows the server to label an encrypted image by data hiding, e.g., hiding the identities, timestamps, and remarks into the ciphertext to generate a This work was supported by Natural Science Foundation of China (Grant 61572308, Grant U1536108, Grant U1636206, Grant 61525203, Grant 61472235, Grant U1636219, and Grant 61379151), the Shanghai Dawn Scholar Plan (14SG36), the Shanghai Excellent Academic Leader Plan (Grant No. 16XD1401200), and the Plan for Scientific Innovation Talent of Henan Prov- ince (Grant No. 2018JR0018). Z. Qian and H. Xu are with Shanghai Institute for Advanced Communica- tion and Data Science, School of Communication and Information Engineering, Shanghai University, Shanghai 200444, China; X. Luo is with State Key Laboratory of Mathematical Engineering and Advanced Computing, Zhengzhou 450001, China X. Zhang is with Shanghai Institute of Intelligent Electronics & Systems, School of Computer Science, Fudan University, 200433, China. Corresponding author: Xinpeng Zhang, Phone/Fax: 86-21-66137263; email: zhangxinpeng@fudan.edu.cn marked encrypted copy. Therefore, the labels are attached inside the ciphertext, providing a better management for ad- ministrators. Meanwhile, storage overheads can also be saved. On the other hand, when an authorized user downloads the encrypted image from the cloud, the original content can be losslessly recovered after image decryption. In traditional sys- tems of file management, the server constructs a metadata file to record the information of the uploaded images. The RDH-EI technique provides an alternative way, which accommodates additional information of the image inside the encrypted bit- stream. Therefore, no metadata files are needed anymore for labeling the uploaded images. Fig. 1. RDH-EI for cloud storage Many RDH-EI methods were proposed in the past five years [3-21]. Most methods were designed for uncompressed images [3-17], while some works were for JPEG bitstreams [18-21]. Since JPEG images are widely used on Internet, this paper focuses on RDH-EI in JPEG bitstreams. We propose a novel RDH-EI framework to make this technique more practical. Compared with previous works [18-21], two achievements are Abstract—This paper proposes a novel framework of reversible data hiding in encrypted JPEG bitstream. We first provide a JPEG encryption algorithm to encipher a JPEG image to a smaller size and keep the format compliant to JPEG decoders. After an image owner uploads the encrypted JPEG bitstreams to cloud storage, the server embeds additional messages into the ciphertext to construct a marked encrypted JPEG bitstream. During data hiding, we propose a combined embedding algorithm including two stages, the Huffman code mapping and the ordered histogram shifting. The embedding procedure is reversible. When an authorized user requires a downloading operation, the server extracts additional messages from the marked encrypted JPEG bitstream and recovers the original encrypted bit- stream losslessly. After downloading, the user obtains the original JPEG bitstream by a direct decryption. The proposed framework out- performs previous works on RDH-EI. First, since the tasks of data embedding/extraction and bitstream recovery are all accomplished by the server, the image owner and the authorized user are required to implement no extra operations except JPEG encryption or decryption. Second, the embedding payload is larger than state-of-the-art works. Index Terms—Reversible data hiding, information hiding, image recovery, JPEG encryption New Framework of Reversible Data Hiding in Encrypted JPEG Bitstreams Zhenxing Qian, Member, IEEE, Haisheng Xu, Xiangyang Luo, Xinpeng Zhang, Member, IEEE R
  • 2. 1051-8215 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCSVT.2018.2797897, IEEE Transactions on Circuits and Systems for Video Technology 2 achieved in the proposed method. First, the tasks of data em- bedding, data extraction and bitstream recovery are all done by the server. There are no extra computation burdens for own- er/user except encryption/decryption. Second, we achieve a larger embedding payload than state-of-the-art works on JPEG RDH-EI. In the rest of the paper, we first introduce the previous arts of RDH-EI in Section II. In Sections III, we briefly depict the new framework. JPEG encryption and decryption algo- rithms for both the owner and the user are proposed in Section IV. Section V provides the embedding, extraction and recovery algorithms. Section VI provides the experimental results and analyses. The paper is concluded in Section VII. II.PREVIOUS ARTS A.RDH-EI for Uncompressed Images RDH-EI was first realized in uncompressed nature images [3]. The content owner encrypts the original image using a stream cipher algorithm and then uploads the encrypted image onto the cloud. The cloud server embeds additional bits into ciphertext by flipping three least significant bits (LSB) of half pixels in each block. On the recipient end, the authorized user decrypts the marked encrypted image and generates two can- didates for each block by flipping LSBs again. Since the orig- inal block of a nature image is smoother than the interfered block, one hidden bit can be extracted and the original block can be recovered. This method was improved in [4] using a side match algorithm to investigate the spatial correlation between neighboring blocks. The flipping based approaches in [3] and [4] were improved in [5] to reduce errors by comparing more neighbor pixels. However, when the pixels in a block have identical values, data extraction and image recovery in [3-5] may fail. A swapping and shifting based RDH-EI method was proposed to overcome this drawback [6]. Machine learning was also used in RDH-EI to improve the embedding capacity [7]. Although the methods in [2-7] have good embedding and recovery capabilities, data extraction must be done after image decryption. This limitation makes the technique less useful in cloud storage. Separable algorithms were proposed to resolve the problem [8]. With a pseudo-randomly generated matrix, the server compresses some LSB-planes of the encrypted pixels to fewer bits to generate spare rooms for hiding additional mes- sages. Therefore, the hidden bits can be extracted directly from the marked encrypted image. On the recipient side, the original contents are recovered by estimating LSBs using MSBs of neighboring pixels. This method was improved in [9] by se- lecting appropriate bitplanes from the encrypted image. Dis- tributed source coding (DSC) is used to design RDH-EI in [10]. The server compresses some selected bits in the encrypted image to hide additional messages. A much higher capacity can be achieved using a Low-Density-Parity-Check (LDPC) based Slepian-Wolf encoder. This method was further improved in [11] using a progressive recovery algorithm, which achieves a better embedding rate. In [12], an RDH-EI with higher em- bedding capacity was proposed by maintaining some redun- dancies during image encryption. Another type of RDH-EI is realized by preprocessing the original image. Some works require the image owner to vacate spare rooms in the plaintext image before image encryption. We name this additional operation of vacating spare rooms in plaintext as preprocessing. After that, the image owner en- crypts the processed image and uploads it onto the server. Im- age encryption should not be accounted as preprocessing, since encryption is required in all RDH-EI methods. In [13], LSBs of some pixels are embedded into other pixels using traditional RDH for plaintext images. Therefore, these LSBs are vacated as spare rooms. The processed image is then encrypted and uploaded to the cloud. On cloud, the server embeds additional bits into the encrypted image using the specified spare rooms, which provides a very high embedding rate. In [14], some pixels are used to estimate the rest before encryption. After encrypting the pixels and estimation errors, a final version of the encrypted image is formulated. The server realizes data hiding by modifying the estimation errors. In [15], patch-level sparse representation is used to explore the correlations of neighbor pixels. Another RDH-EI approach was realized by histogram shifting in the spatial permuted images [16]. In [17], image transformation was proposed to encrypt one image to another. Additional bits are embedded by RDH in plaintext images. The preprocessing based methods in [8-17] can achieve much better embedding rates, but require extra RDH operations before image encryption. B.RDH-EI for JPEG Bitstreams The aforementioned RDH-EI is for uncompressed images. However, those methods are not useful in many applications because most images transmitted over Internet are compressed, e.g., the popular JPEG. As a consequence, some RDH-EI works were proposed for JPEG bitstreams [18-21]. The first RDH-EI for JPEG bitstreams was proposed in [18]. This scheme begins with a JPEG encryption algorithm, in which the appended bits of Huffman codes are encrypted by a stream cipher, and all Huffman codes are kept unchanged. After encryption, the JPEG file size is preserved and the format is compliant to JPEG decoders. On cloud, the server selects the encrypted bitstreams of some blocks as candidates. Additional bits are encoded by LDPC-based error correction codes (ECC) and embedded into the useful candidate bitstream by flipping the LSBs of the encrypted appended bits of the AC coefficients in each candidate block. After the authorized user downloads and decrypts the marked encrypted bitstream, LSBs of the appended bits of useful candidates are flipped again to estimate the additional bits using a predefined blocking artifact function and an ECC decoder. Meanwhile, the original bitstream are losslessly recovered according to the extracted bits. This method is improved in [19], in which the embedding room was reserved before bitstream encryption. Although the embedding capacity is larger, the preprocessing requires the image owner to do an additional computation. Another solution of reversibly hiding data in encrypted JPEG bitstream was proposed in [20], in which image encryption and data embedding are combined together. By scrambling the JPEG structure, the image is en- crypted and the additional bits are embedded. The method in [20] cannot be used in cloud storage since the server is unable to embed bits into the encrypted bitstream.
  • 3. 1051-8215 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCSVT.2018.2797897, IEEE Transactions on Circuits and Systems for Video Technology 3 Fig. 2. New Framework of RDH-EI for JPEG Fig. 3. Traditional Framework of RDH-EI for JPEG To enhance the security of JPEG encryption and improve the embedding capability, another RDH-EI for JPEG bitstream was proposed in [21]. During JPEG bitstream encryption, a new JPEG bitstream is constructed by selecting a portion of blocks from the whole image. Bitstreams of the rest blocks are en- crypted and hidden in the JEPG header. With a compression algorithm, some appended bits of the encrypted JPEG bitstream are compressed to accommodate additional bits. On the recip- ient side, the authorized user uses an iterative algorithm to recovery the original JPEG bitstream according to the variation of blocking artifacts. Compared with [18], this method has larger capacity and better security, and the embedded bits can be directly extracted by the server. Although RDH-EI methods for JPEG in [18-21] have good capabilities in data hiding and image recovery, there exists one problem that the recipient must do a recovery task after de- cryption. This recovery burden on the recipient side limits the real application of RDH-EI techniques, since users always hope to do nothing except image encryption and decryption. To this end, this paper proposes a new JPEG RDH-EI framework, in which no recovery task is required for the owner or the user. III. NEW FRAMEWORK The proposed RDH-EI framework focuses on labeling the encrypted JPEG images on cloud storage. There are three par- ties, including the image owner, the cloud server and the au- thorized user. The owner encrypts a JPEG bitstream and up- loads it to the cloud. The cloud server embeds additional mes- sages into the encrypted bitstream to generate a marked en- crypted bitstream. The hidden messages can be extracted from the marked encrypted bitstream. When an authorized requires a downloading operation, the server recovers the original en- crypted bitstream. After decryption, the user obtains the origi- nal JPEG image. The procedure is shown in Fig. 2. Fig. 3 shows the traditional framework used in previous works. The user has to implement the JPEG recovery after decryption [18][21]. Sometimes the owner is also required to do preprocessing [19], i.e., generating spare rooms before en- cryption. In Fig. 2, we propose to accomplish all computation tasks by the cloud server. Therefore, data extraction and bit- stream recovery are transparent to the owner or the user. The traditional framework keeps the length of encrypted bitstream unchanged after data hiding, which leads to a limited embedding capacity. To embed more bits into the encrypted bitstream, the proposed framework allows the increment of bitstream length. With a condition that the amount of embedded bits must be larger than the bitstream increment [22], the pro- posed framework uses a two-phase combined embedding al- gorithm to increase the embedding payload and eliminate the bitstream increment. IV. JPEG ENCRYPTION AND DECRYPTION In this section, we propose an encryption and decryption algorithm for JPEG bitstreams. The proposed algorithm aims at conceal the content of a JPEG image, and keep the encrypted bitstream compliant to the JPEG decoders that are widely used. For brevity, we discuss the baseline JPEG for grayscale images. Fig. 4 shows a simplified syntax of the baseline JPEG. We list a part of the terms used in JPEG in Table I. In the following paragraphs, we use these acronyms for brevity discussions. Table I. Acronyms used in the proposed method Acronyms Terms SOI start-of-image EOI end-of-image EOB end-of-block JH JPEG header ECS entropy-coded segments DCC code of a DC coefficient DCH Huffman code in a DCC DCA appended bits in a DCC ACC code of an AC coefficient ACH Huffman code in an ACC ACA appended bits in an ACC Data Hiding JPEG Encryption Bitstream Recovery JPEG Decryption Data Extraction JPEG Bitstream Additional Message Additional Message Original JPEG Encryption Key Decryption Key Owner User Server Data Hiding JPEG Encryption JPEG Recovery JPEG Decryption Data Extraction JPEG Bitstream Additional Message Additional Message Original JPEG Encryption Key Decryption Key Owner User Server Preprocessing (Optional)
  • 4. 1051-8215 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCSVT.2018.2797897, IEEE Transactions on Circuits and Systems for Video Technology 4 Fig. 4. Simplified Syntax of JPEG Baseline As shown in Fig. 4, a JPEG bitstream contains a marker of start-of-image, a JPEG header, the entropy encoded data, and an end-of-image marker. The section of entropy encoded data is constructed by entropy-coded segments of all blocks. Each ECS includes one code of DC coefficient (DCC), several codes of the AC coefficients (ACC), and an end-of-block marker. The DCC part contains a DC Huffman code (DCH) and DC ap- pended bits (DCA), and each ACC part contains AC Huffman codes (ACH) and AC appended bits (ACA). Therefore, a JPEG bitstream J can be represented by J={SOI, JH, ECS1, …, ECSN, EOI} When an H× W image is compressed into a JPEG bitstream, there are N entropy-coded segments, where N=⌈H/8⌉∙⌈W/8⌉. We denote these segments as ECSi, i=1,2,…,N. According to the JPEG syntax, each segment can be represented by ECSi={DCC<i> , ACC<i, 1> , ACC<i, 2> ,…, EOB} ={[DCH<i> , DCA<i> ], [ACH<i, 1> , ACA<i, 1> ], [ACH<i, 2> , ACA<i, 2> ], …, EOB}. A.JPEG Encryption Generally, JPEG encryption is required to be compliant to popular decoders [23]. Therefore, we propose an algorithm to encrypt the original JPEG bitstream to a new JPEG bitstream using an encryption key. The encryption is symmetric, and the secret key must be transmitted to the recipient over a reliable channel. The encryption algorithm includes four steps. Fig. 5. New JPEG Bitstream In Step One, with an encryption key Kenc, the image owner pseudo-randomly selects n entropy-coded segments from the original JPEG bitstream J, where 1<n<N. Let these segments be ECSs(i), i=1, 2, …, n. Correspondingly, the remaining seg- ments are ECSr(j), j=1, 2, …, N-n. The owner specifies two positive integers h and w satisfying n=⌈h/8⌉∙⌈w/8⌉. With these data, the owner constructs a new JPEG frame corresponding to an h× w image, as shown in Fig. 5. In Step Two, the owner uses the encryption key Kenc to en- crypt the remaining N−n segments by a stream cipher algorithm such as RC4, SEAL, etc. The encrypted bits, along with the values of H and W, are then embedded into the reserved ap- plication segments marked with APPn in JPEG header, using the same way as [24]. In the modified header, the owner also specifies the new image size as h× w. In Step Three, the owner extracts the DC coefficient from n selected segments ECSs(i) by decoding the DC Huffman code and the appended bits, i.e., [DCH<s(i)> , DCA<s(i)> ], to {ds(1), ds(2),…, ds(n)}. With the coefficients, the owner generates the differential values {ds(1)', ds(2)',…, ds(n)'}, where 𝑑𝑠(𝑖) ′ = { 𝑑𝑠(𝑖), 𝑖 = 1 𝑑𝑠(𝑖) − 𝑑𝑠(𝑖−1), 𝑖 = 2, … , 𝑛 (1) These differential values are further encoded by Huffman codes to generate [DCH<s(i)>* , DCA<s(i)>* ]. In Step Four, the owner replaces all ECSs(i) in the new en- tropy encoded data with ECSs(i) * , where ECSs(i) * stands for the encrypted segments, ECSs(i) * ={DCC<s(i)>* , ACC<s(i), 1> , ACC<s(i), 2> ,…, EOB} ={[DCH<s(i)>* , DCA<s(i)>* ], [ACH<i, 1> , ACA<i, 1> ], [ACH<i, 2> , ACA<i, 2> ], …, EOB}. and i=1,2,…,n. Finally, the owner constructs the encrypted JPEG bitstream J* , J* ={SOI, JH* , ECSs(1) * , …, ECSs(n) * , EOI} An example is shown in Fig. 6, in which (a) is the image decoded from the original JPEG bitstream, and (b) is the en- crypted image decoded from the encrypted JPEG bitstream. The original size of (a) is 512× 512. After encryption, the size of encrypted image (b) is 256× 256. The size of the encrypted image is specified by the owner. (a) (b) Fig. 6. Constructing an encrypted JPEG bitstream, (a) is the original image Lena, (b) the encrypted image. B.JPEG Decryption On the recipient end, the authorized user decrypts the enci- phered JPEG bitstream J* using a decryption key Kdec. Gener- ally, the decryption Kdec is identical to Kenc. The decryption procedure is reverse to encryption, also containing three steps. In Step One, the user extracts N-n encrypted segments from the reserved application APPn in the JPEG header JH* . With SOI EOI JPEG Header Entropy Encoded Data Appended bits (DCA) DC Huffman code (DCH) …… Entropy-coded segments of all the blocks (ECS) Appended bits (ACA) AC Huffman code (ACH) …… …… EOB Code of DC coeffi- cient (DCC) … … Codes of all AC coefficients (ACC) … … SOI EOI Modified JPEG Header New Entropy Encoded Data ECSs(n) ECSs(2) ECSs(1) …… …… ……
  • 5. 1051-8215 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCSVT.2018.2797897, IEEE Transactions on Circuits and Systems for Video Technology 5 the decryption key Kdec, the user uses the stream cipher to de- crypt these segments back to ECSr(j), where j=1, 2, …, N-n. In Step Two, n segments ECSs(i) * (i=1, 2, …, n) are extracted from J* . The user reads [DCH<s(i)>* , DCA<s(i)>* ] from all seg- ments ECSs(i) * , and decodes them into the values of {ds(1)', ds(2)',…, ds(n)'}. The original DC coefficients {ds(1), ds(2),…, ds(n)} of the selected segments are reconstructed by 𝑑𝑠(𝑖) = ∑ 𝑑𝑠(𝑘)′, 𝑖 = 𝑖 𝑘=1 1,2, … , 𝑛 (2) The user re-encodes these DC coefficients into [DCH<s(i)> , DCA<s(i)> ] by Huffman codes, and reconstructs the original ECSs(i). In Step Three, the user rearranges all segments of ECSs(i) and ECSr(j) and reconstructs the original entropy encoded data. After modifying the JPEG header to specify the original image size H×W, the original JPEG stream J is finally decrypted. V.DATA EMBEDDING AND EXTRACTION After the user uploads an encrypted JPEG bitstream J* onto the cloud, the server embeds additional messages into the ci- phertext to generate a marked encrypted JPEG bitstream. Be- cause the embedding procedure is reversible, the original en- crypted JPEG bitstream can be losslessly recovered. The em- bedding procedure is depicted in Fig. 7, including two stages. In Stage One, with a Huffman code mapping algorithm, we loss- lessly embed a part of additional messages into J* to generate an intermediate bitstream JM * . In Stage Two, the other mes- sages are reversibly embedded into JM * using an ordered em- bedding algorithm, and a marked encrypted bitstream JE * is finally generated. A.Stage One: Code Mapping Based Embedding According to Table K. 5 in JPEG Standard [25], there are 162 Huffman codes for AC coefficients of luminance component. Each code represents one run/size pair. Hence, there are 162 run/size pairs, including the pairs ranging from 0/1 to 15/10, a pair 0/0 (EOB), and a zero-run-length (ZRL) pair 15/0. This information is recorded as the Huffman Table in the JPEG header. Generally, only a part of the AC Huffman codes are used during JEPG compression. According to this fact, the server can employ the unused codes to represent additional bits to be embedded. The server first extracts the Huffman table from the header JH* of the encrypted JPEG bitstream J* , and constructs 162 AC Huffman codes. These codes are separated into two categories, the frozen codes F and the active codes A. The frozen category F contains 12 AC Huffman codes corresponding to the run/size pairs of {0/0, 0/1, …, 0/10, EOB, ZRL}, which are not used during data embedding. The other 150 codes constitutes the active category A, corresponding to the run/size pairs of {1/1, …, 1/10, 2/1, …, 2/10, …, 15/1, …, 15/10}. The server also extracts the AC Huffman codes {ACH<s(i), 1> , ACH<s(i), 2> , …} from ECSs(i) * , where i=1,2,…,n. From these codes, the server finds all codes that belong to the active cate- gory A. Assuming there are p kinds of such codes used in J* , we denote them as U={u1, u2, …, up}, where U⸦A and 0≤p≤150. Correspondingly, there must be 150−p AC Huffman codes that are not used in J* . Let the category of these codes be N={n1, n2, …, nq}, where N=A−U and q=150−p. Next, the server further divides the codes in U into 13 groups {U4, U5, …, U16}, such that the length of each code in Uk is equal to k bits, where Uk={u1 (k) , u2 (k) , …, upk (k) }, k=4,5,…,16, and ∑ 𝑝𝑘 16 𝑘=4 = 𝑝. In the same way, the server also divides N into 13 groups {N4, N5, …, N16}, where Nk={n1 (k) , n2 (k) , …, nqk (k) } and ∑ 𝑞𝑘 16 𝑘=4 = 𝑞. With these groups, the server maps the codes in Uk with the codes in Nk. If pk<qk, each code in Uk is mapped with πk codes in Nk, ui (k) ←{n(i−1)∙πk+1 (k) , …, ni∙πk (k) } (3) where πk=2⌊log2(qk/pk+1)⌋−1 and i=1, 2, …, pk. Otherwise, codes in Uk and Nk are mapped by one-to-one manner, uj (k) ←nj (k) (4) where j=1, 2, …, qk. The mapping in (4) is equivalent to a special case of πk=1 in (3). Accordingly, the server modifies the Huffman Table in the JPEG header JH* to record the mapping relationships. The run/size values of πk codes in Nk are replaced with the run/size values of the mapped code in Uk. Therefore, one run/size value in the modified Huffman Table may have several Huffman codes. A new JPEG header JH* ' is therefore constructed. After the mapping relationships are identified in JH* ', each code in the mapping is used to represent log2(πk+1) additional bits. Then, according to the additional bits M1 to be embedded, the server substitutes the used AC Huffman code with the mapped codes. Thus, a part of the AC Huffman codes in ECSs(i) * are modified to carry additional messages. For example, in an encrypted JPEG bitstream J* , an AC Huffman code “111010” is used, and the code “111011” is not used. Assuming πk=1 (k=6), a mapping relationship can be constructed by “111010”←“111011”. After modifying the Huffman Table in the JPEG header, the run/size value “3/1” has two Huffman codes “111010” and “111011”. We use “111010” and “111011” to represent one additional bit “0” and “1”, re- spectively. Therefore, when embedding a bit “1” into the bit- stream, the server replaces “111010” with “111011”. Fig. 7. Two Stages of Data Embedding Huffman Code Mapping J* Bitstream Substitution Entropy Decoding Ordered Embedding Bitstream Modification JE * Stage One Stage Two Message Message JM *
  • 6. 1051-8215 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCSVT.2018.2797897, IEEE Transactions on Circuits and Systems for Video Technology 6 After sequentially modifying the AC Huffman codes to hide the additional bits, a marked encrypted bitstream JM * is finally constructed, JM * ={SOI, JH* ', ECSs(1) * ', …, ECSs(n) * ', EOI} where ECSs(i) * ' (i=1,2,…,n) represents the marked entro- py-coded segments constructed in Stage One. The embedding capacity CM of this stage can be evaluated by 𝐶𝑀 = ∑ 𝛾𝑘 ∙ log2(𝜋𝑘 + 1) 16 𝑘=4 (5) where γk represents the number of used codes in JM * satisfying the mapping relationships. The marked bitstream JM * can still be decoded directly by the popular JPEG decoders. Meanwhile, the decoded image is identical to that decoded from J* , since the embedding proce- dure is lossless to the content. To make an explicit description, we summarize the embedding steps in Table II. Table II. Embedding Steps of Stage One Input: Encrypted JPEG Bitstream J* , Additional Message M1 Output: Marked Encrypted Bitstream JM * 1. Extract JPEG header JH* and all entropy-coded segments ECSs(i) * from J* 2. Construct 162 AC Huffman codes, and divide them into two categories: the frozen codes and the active codes. 3. Extract all AC Huffman codes from ECSs(i) * , and find the active codes that are used in J* . 4. Mapping the used active codes with the unused using (3) and (4) to represent additional bits. 5. Record the mapping by modifying JH* to generate a new JPEG header JH* '. 6. Date embedding: according to M1, substitute the codes in all ECSs(i) * with the mapped codes to generate ECSs(i) * '. 7. Integrate JH* ' with ECSs(i) * ' to generate the marked JM * . B.Stage Two: Ordered Embedding In Stage Two, the server embeds more additional bits into the marked encrypted JPEG bitstream JM * generated in Stage One, using an algorithm of ordered embedding based on histogram shifting. Generally, there are two goals of histogram shifting based RDH in JPEG bitstream. The first goal is to embed as many payloads as possible, while the second the reason is to reduce the bitstream length increment as far as possible. In JPEG standard, the zigzag scanned AC coefficients in each block are turned into (R, V) pairs, where R stands for zero-run-length and V the value of a non-zero coefficient. According the value V of each AC coefficient, the size of V is identified as S. According to the relationships between V and S, shown in Table III, all (R, V) pairs are turned into (R/S, V). Each R/S is represented by a Huffman code according to the pre-defined Huffman Table, i.e., Table K.5 of JPEG Standard. Correspondingly, the value V is encoded by varia- ble-length-integer (VLI) codes. The table of VLI codes is also shown in Table III. On the aspect of embedding payload, we arbitrarily choose 30 images from the BOSSbase image dataset [26], and com- press these images by the toolbox of “Independent JPEG Group” (IJG) [27] using different quality factors (QF). After averaging the number of (R, V) pairs corresponding to different R in all bitstreams, more (R, V) pairs in an image when R gets smaller. The results in Fig. 8 show that the distributions are independent from the quality factors. Fig. 8 also shows that around 70% pairs have the run value R=0. As a result, for any quality factor based bitstream we use pairs with R=0 to carry more bits. Table III. Categories of different V and the VLI codes V S VLI Codes –1, 1 1 0,1 –3, –2, 2, 3 2 00, 01, 10, 11 –7, …, –4, 4, …, 7 3 000, … ,111 –15, …, –8, 8, …, 15 4 0000, … ,1111 –31, …, –16, 16, …, 31 5 00000, … ,11111 –63, …, –32, 32, …, 63 6 000000, … ,111111 –127, …, –64, 64, …,127 7 0000000, … ,1111111 –255, …, –128, 128, …, 255 8 00000000, … ,11111111 –511, …, –256, 256, …, 511 9 000000000, … ,111111111 –1023, …, –512, 512, …, 1023 10 0000000000, … ,1111111111 Fig.8. Average amount of ZRV pairs corresponding to R using dif- ferent quality factors When hiding additional data into a JPEG bitstream by his- togram shifting, the length of the bitstream may increase, in- cluding the Huffman code increment and the VLI increment. According to Table I, if we modify the value of a non-zero AC coefficient to another while keeping S unchanged, the length of encoded bits does not change. Otherwise, if we modify a coef- ficient from one size to another, length of the encoded bits will change. For example, when changing an AC coefficient from – 4 to –5, lengths of the Huffman code and encoded bits keep unchanged. However, when we change a coefficient from –7 to –8, lengths of the Huffman code and the VLI code will increase. The bitstream increment caused by data embedding is also related to Pi, i.e., the position of the last non-zero coefficient (using zigzag order) in each block, where i=1,…,n and 2≤Pi≤64. To evaluate the influence of Pi, we use histogram shifting to embed bits into the blocks with Pi≤T, where 2≤T≤64. Let the differential payload be the embedding payload minus the bit- stream increment. Fig. 9 shows the relationships between the differential payload and T. The figure indicates that, when embedding bits into the blocks with small Pi, the differential payload is positive, meaning that the embedding payload is larger than bitstream increment. This is the reason why we sorts all blocks {B1, B2, … Bn} to {Bo(1), Bo(2), … Bo(n)} s.t. Po(1)≤ Po(2)≤…≤Po(n) and embed L bits into wL blocks with small Pi.
  • 7. 1051-8215 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCSVT.2018.2797897, IEEE Transactions on Circuits and Systems for Video Technology 7 Fig. 9. Differential payload vs. T Based on these considerations, the server can further embed as many payloads as possible, while eliminating the bitstream increment as far as possible. The server partially decodes the bitstream JM * into DCT blocks {B1, B2, … Bn}. Each block contains one DC coefficient and 63 AC coefficients. After zigzag scanning these coeffi- cients in each block, the server obtains a series of (R, V) pairs for AC coefficients. In each block, most quantized AC coefficients are equivalent to 0. The server collects the position of the last non-zero coef- ficient (using zigzag order) Pi in each block Bi, where i=1,…,n and 2≤Pi≤64. The server also counts the number of (R, V) pairs with R=0 and V=± 1 in each block. Denote the pair numbers as ρi, where 0≤ρi≤63. The server further sorts all blocks {B1, B2, … Bn} to {Bo(1), Bo(2), … Bo(n)} satisfying Po(1)≤Po(2)≤…≤Po(n), in which o(∙) is the sorting operator. In order to hide a message M2 containing L bits into the bitstream, the server uses the first wL carrier blocks {Bo(1), Bo(2), … Bo(wL)} to carry these bits, such that 𝐿 ≤ ∑ 𝜌𝑜(𝑘) 𝑤𝐿 𝑘=1 Accordingly, the server constructs a histogram of all (R, V) pairs with R=0 in carrier blocks Bo(1)~Bo(wL). We denote this histogram as h(v), in which v∈[–1023, 1023] and h(v) is the number of (R, V) pairs with R=0 and V=v. The server embeds L secret bits {b1, b2, …, bL} into these blocks by shifting the histogram, 𝑣𝑗 ′ = { 𝑣𝑗 − 1 𝑖𝑓 𝑣𝑗 < −1 𝑣𝑗 − 𝑏𝑘 𝑖𝑓 𝑣𝑗 = −1 𝑣𝑗 + 𝑏𝑘 𝑖𝑓 𝑣𝑗 = 1 𝑣𝑗 + 1 𝑖𝑓 𝑣𝑗 > 1 (6) where vj stands for the j-th AC coefficient with R=0 in the carrier blocks, and k=1,…,L. The histogram shifting based embedding is realized by sub- stituting the entropy code of the pair (0, vj) with the code of (0, vj') in the bitstream JM * . In this way, the server embeds L secret bits into w segments ECSs(i) * ' where i=1, 2, …, n. The marked encrypted bitstream JE * is finally generated, JE * ={SOI, JH* ', ECSs(1) * '', …, ECSs(n) * '', EOI} In which ECSs(i) * '' (i=1,2,…,n) represents the entropy-coded segments constructed in this stage. The maximal embedding capacity CE of the second stage is equal to 𝐶𝐸 = ∑ 𝜌𝑖 𝑛 𝑖=1 (7) Therefore, the maximal embedding capacity of the two stages is 𝐶𝑚𝑎𝑥 = 𝐶𝑀 + 𝐶𝐸 = ∑ 𝛾𝑘 ∙ log2(𝜋𝑘 + 1) 16 𝑘=4 + ∑ 𝜌𝑖 𝑛 𝑖=1 (8) Meanwhile, the bitstream increment is determined by the em- bedding payload L in the second stage. The embedding payload C of the two-stage combined method is equal to 𝐶 = 𝐶𝑀 + 𝐿 (9) Hence, the bitstream increment can be evaluated by E, 𝐸 ≈ 𝛼1 ∙ 𝐿 2 ⁄ + ∑ 𝛼𝑘 ∙ 𝑘≥2 𝑛𝑘(𝑤𝐿) (10) where αk represents the increased code length when modifying (0/k, ± (2k –1)) to (0/k+1, ± 2k ), α1=1, and nk(wL) the number of (0/k, ± (2k –1)) pairs in the histogram of the first wL blocks. It should be noted that data embedding operations in both stages do not interfere with each other, which guarantee the correctness of data extractions. In the first stage, we do not use the frozen codes {0/0, 0/1, …, 0/10, 15/0} to carry additional bits, i.e., the AC Huffman codes of (R, V) run/size pairs with R=0 are not modified. In the second stage, we only modify the AC Huffman codes with respect to R=0 to carry more bits. The embedding steps of the second stage are summarized in Table IV. Table IV. Embedding Steps of Stage Two Input: Marked Encrypted Bitstream JM * , Additional Message M2 Output: Marked Encrypted Bitstream JE * 1. Extract all entropy-coded segments ECSs(i) * ' from JM * 2. Decode all ECSs(i) * ' into DCT blocks {B1, B2, … Bn}, and obtain a series of (R, V) pairs by zigzag scanning. 3. In each Bi, collect the position of the last non-zero coefficient Pi, and count the number ρi of (R, V) pairs with (0,± 1). 4. Sort all blocks into {Bo(1), …, Bo(n)} s.t. Po(1)≤Po(2) ≤…≤Po(n), and choose the first wL blocks Bo(1)~Bo(wL) as carriers. 5. Construct a histogram h(v) of all (R, V) pairs with R=0 in carrier blocks Bo(1)~Bo(wL). 6. Data embedding: implement (6) to hide M2 into Bo(1)~Bo(wL) by modifying ECSs(i) * ' to ECSs(1) * '', 7. Integrate JH* ' with ECSs(i) * '' to generate the marked JE * . C.Data Extraction and Bitstream Recovery The proposed RDH-EI protocol guarantees that the hidden messages can be exactly extracted from the marked bitstream JE * , and the original encrypted bitstream J* can be losslessly recovered. The procedure of data extraction and bitstream recovery also includes two stages. In the first stage, the server extracts L bits from JE * using the reverse histogram shifting and recovers JE * to JM * . After par- tially decoding JE * into the quantized DCT blocks, the server sorts all blocks {B1, B2, … Bn} to {Bo(1), Bo(2), … Bo(n)} such that Po(1)≤Po(2)≤…≤Po(n), using the same algorithm depicted in Subsection B. According to the payload L, the server identifies first wL carrier blocks {Bo(1), Bo(2), … Bo(wL)}. From these blocks, the server sequentially reads all (0, ± 1) pairs, and identifies L hidden bits {b1, b2, …, bL} by
  • 8. 1051-8215 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCSVT.2018.2797897, IEEE Transactions on Circuits and Systems for Video Technology 8 𝑏𝑘 = { 0, 𝑖𝑓 |𝑣𝑗′| = 1 1, 𝑖𝑓 |𝑣𝑗′| = 2 (11) where k=1,2,…,L. Meanwhile, the original (R, V) pairs with R=0 in wL DCT blocks are recovered by 𝑣𝑖 = { 𝑣𝑖 ′ + 1 𝑖𝑓 𝑣𝑖′ < −2 −1 𝑖𝑓 − 2 ≤ 𝑣𝑖′ ≤ −1 1 𝑖𝑓 1 ≤ 𝑣𝑖′ ≤ 2 𝑣𝑖 ′ − 1 𝑖𝑓 𝑣𝑖 ′ > 2 (12) The histogram shifting based recovery is realized by substi- tuting the entropy code of the pair (0, vj') with the code of (0, vj) in JE * . In this way, the marked encrypted bitstream JM * is re- covered, JM * ={SOI, JH* ', ECSs(1) * ', …, ECSs(n) * ', EOI} In the second stage, the server extracts the Huffman table from JH* ', from which the mapping relationships in (3) and (4) can be constructed. The server sequentially reads the AC Huffman codes form ECSs(i) * ' (i=1,2,…,n), and extracts CM bits according to the mapping relationships. Meanwhile, the server replaces the modified Huffman table in JH* ' with the original Huffman Table K.5 defined in JPEG Standard [25]. According to the recovered Huffman table and the mapping relationships, the server substitutes the AC Huffman codes that were used to carry additional bits in JM * with the original Huffman codes. After that, the original encrypted JPEG bitstream J* can be recovered, J* ={SOI, JH* , ECSs(1) * , …, ECSs(n) * , EOI} VI. EXPERIMENTAL RESULTS AND ANALYSIS A.Performance of JPEG Encryption/Decryption To verify the proposed method, we use grayscale images sized 512× 512, and compress them to JPEG bitstreams using different quality factors. Two groups of experimental results are shown in Fig. 10, in which JPEG bitstreams of the images Peppers and Lake are used. Fig. 10(a) shows the original im- ages decoded from the original JPEG. Quality factors of the bitstreams are both 80. We encrypt Peppers to a small image sized 256× 256, and Lake to a smaller image sized 128× 256. Fig. 10(b) shows the images decoded from the encrypted JPEG bitstreams. With the proposed method, we embed 2863 bits and 798 bits into the encrypted bitstreams of Peppers and Lake, respectively. The images decoded from the marked encrypted bitstreams are shown in Fig. 10(c). After decryption, the orig- inal JPEG bitstreams can be reconstructed. Fig. 10(d) shows the images of the decrypted bitstreams after data extraction and bitstream recovery, which are the same as the original JPEG images in Fig. 10(a). Fig. 11(a) shows the original JPEG image Lena sized 512× 512. We use the encryption algorithms in [21] to encrypt the bitstream into 256× 256 sized ciphertext, which is shown Fig. 11(b). This image looks like an error-decoded image. We also use the proposed algorithm to do the encryption. Fig. 11(c) shows the ciphertext JPEG image generated by the proposed method, which overcomes the drawback in Fig. 11(b) and achieves a better visual effect. (a) (b) (c) (d) Fig. 10. RDH-EI in JPEG bitstreams of Peppers and Lake, (a) the original JPEG images, (b) the encrypted images with smaller sizes, (c) the marked encrypted image, (d) the recovered images (a) (b) (c) Fig. 11. An example of JPEG bitstream encryption, (a) is the original image Lena, (b) the encrypted image using [21], and (c) the encrypted image using the proposed method. The proposed encryption algorithm is also against attackers. A representative attack to the format-compliant JPEG encryp- tion was proposed in [28], which uses the AC Huffman codes in an encrypted bitstream to analyze the contour of the original content. We use this attack to verify our method. After en- crypting the original image Airplane and Baboon into 256× 256 ciphertext images, we use [28] to attack the encrypted bitstream. Fig. 12(a) shows both images of the attacking results, from which the attackers cannot achieve the contour information of
  • 9. 1051-8215 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCSVT.2018.2797897, IEEE Transactions on Circuits and Systems for Video Technology 9 the original image. Fig. 12(b) shows the results of attacking the encryption algorithm proposed in [18], which reveals the con- tour of the original images. These experiments indicate that the proposed method has better security than [18]. (a) (b) Fig. 12. Attacking JPEG encrypted bitstream of Airplane and Baboon, (a) is the attack to our method, and (b) the attack to [18] The propose encryption algorithm is also secure against the popular ciphertext-only attack. During JPEG bitstream en- cryption, we pseudo-randomly select n entropy-coded segment from the bitstream to construct a new JPEG bitstream, which can be decoded to smaller sized ciphertext image. Since only a part of Huffman codes are available an adversary has no in- formation of the original size. It is difficult for an adversary to find the original orders of all blocks from 𝐶𝑁 𝑛 ∙ 𝑛! possibilities, as long as the block number N large enough. Therefore the key space is large enough to ensure security. B.Performance of Data Hiding When embedding additional messages into the encrypted JPEG bitstream, the payloads is related to the sizes of the en- crypted JPEG images. The original JPEG bitstream are en- crypted to four kinds of smaller sized images, including 128× 192, 192× 256, 256× 256, and 256× 384. We define the differential payload as DP to evaluate the gap between em- bedding payload C and bitstream increment E, i.e. DP=C−E. Table V shows the embedding payloads corresponding to dif- ferent sizes and DP values. The results indicate that the em- bedding payload increases when the size of an encrypted image gets larger. When a small payload C is embedded into the en- crypted bitstream, the differential payload DP achieves the best DPmax. Large embedding payloads can also be achieved when DP approaches DP0=0 or minimum DPmin. In Table VI, we provide another group of experimental re- sults of bitstream sizes using DPmax. The original bitstreams are generated by compressing 512× 512 images using a quality factor 80. After encrypting them to ciphertexts corresponding to 256× 256 encrypted images, sizes of the encrypted bitstreams are close to the original bitstreams. After embedding the secret messages into the encrypted bitstreams, there is a slight size increment in each marked encrypted bitstream. In Fig. 13(a)~(d), the relationships between the embedding payload C and the bitstream increment E. The increment values stand for the increased bits of the entire file, including all parts of the whole bitstream. We use the images Lena, Peppers, Lake and Airplane to implement the proposed JPEG RDH-EI. We encrypt each JEPG to the sizes of 128× 192, 192× 256, 256× 256, and 256× 384, respectively. Quality factor of these JPEG im- ages are 80. The dash line on each figure is used as a reference to show the case when C=E. Results show that the embedding payload is larger than the bitstream increment in most embed- ding payloads. Table V. Payloads Corresponding to Different DP (Bits) Images 128× 192 192× 256 256× 256 256× 384 DPmax DP0 DPmin DPmax DP0 DPmin DPmax DP0 DPmin DPmax DP0 DPmin Lena 723 1189 1386 1294 2473 2761 1724 3483 3667 2750 5452 5548 Man 581 1081 1831 612 2015 3671 791 2308 4856 1147 3276 7258 Lake 695 1318 1920 1046 2317 3760 1368 2915 4964 2016 4328 7342 Peppers 660 1494 1621 2123 3193 3193 2863 4253 4253 3949 6262 6262 Baboon 155 626 2825 276 1077 5576 365 1728 7547 546 2800 11271 Airplane 583 1115 1375 1146 2266 2808 1446 3044 3706 2692 4668 5547 Table VI. Size of Files before and after Embedding Data Images Original (Bytes) Encrypted (Bytes) Additional Message (Bits) Marked (Bytes) Lena 37767 37827 1724 37997 Man 49010 49064 791 49125 Lake 51965 51992 1368 52116 Peppers 39751 39804 2863 40083 Baboon 78506 78442 365 78463 Airplane 38536 38622 1446 38735
  • 10. 1051-8215 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCSVT.2018.2797897, IEEE Transactions on Circuits and Systems for Video Technology 10 (a) (b) (c) (d) Fig. 13. Embedding Payload vs. Bitstream Increment, (a) the original JPEG image Lena, (b) Peppers, (c) Lake, (d) Airplane We also compare the proposed method with JPEG RDH-EI methods in [18], [19], and [21]. We use the proposed method to encrypt a bitstream to ciphertext bitstream corresponding to 256× 256 sized images. Table VII shows that proposed method can achieve a much larger payload than [18], [19], and [21]. Table VII. Comparison of Embedding Capacity (Bits) Images [18] [19] [21] Proposed Lena 750 798 1364 3667 Baboon 750 1555 768 7547 Man 750 1809 1368 4856 Peppers 750 960 1026 4253 Lake 750 1032 1023 4964 The features of these methods are compared in Table VIII. The proposed method requires the owners and users to do nothing encryption or decryption, Compared with the pre-processing or post-processing required in [18], [19] and [21], the proposed method relaxes the burdens on both the owner side and the user side. Table VIII. Comparisons of Features in JPEG RDH-EI Methods Feature [18] [19] [21] Proposed Owner’s Pre-processing No Yes No No User’s Post-processing Yes Yes Yes No The embedding payload is also related to the quality factor of the JPEG bitstream. In Fig. 14, we compress the original im- ages into JPEG bitstream using different quality factors, and embedding additional bits into the encrypted bitstream. Fig. 13(a)~(c) show the embedding payloads corresponding to DPmax, DP0, and DPmin, which indicates that the embedding payloads increases when the quality factor get larger. (a) (b)
  • 11. 1051-8215 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCSVT.2018.2797897, IEEE Transactions on Circuits and Systems for Video Technology 11 (c) Fig. 14. Embedding Payload vs. Quality Factor, (a) the original JPEG image Lena, (b) Peppers, (c) Lake VII. CONCLUSION AND DISCUSSION This paper proposes a new framework of reversible data hiding in encrypted JPEG bitstreams. A JPEG encryption and decryption algorithm is developed to hide the content of the original image. The encryption is format-compliant to the popular JPEG decoders. On the cloud side, the server can em- bed a large amount of additional bits into the encrypted bit- stream. We propose to do the embedding using a combination of code mapping and ordered embedding. Once an authorized user requires a downloading operation, the server extracts the additional message and recovers the original encrypted bit- stream. Since the recovery has been done before downloading, the user obtains an image exactly the same as the original. The proposed framework outperforms the previous JPEG RDH-EI frameworks on three aspects. First, the proposed em- bedding method provides a much larger payload than the other methods. Second, the proposed framework frees the computa- tion burdens on both the owner and the user sides. While pre-processing or post-processing is required in the traditional works, the propose framework requires the owner or the user to no extra tasks except encryption or decryption. Finally, the proposed JPEG encryption algorithm is also secure against the ciphertext-only attack. ACKNOWLEDGEMENT The authors are grateful to the anonymous reviewers for their constructive comments. REFERENCES [1] Z. Ni, Y. -Q. Shi, N. Ansari, and W. Su, “Reversible Data Hid- ing,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 16, no. 3, pp. 354-362, 2006. [2] W. -L. Tai, C. -M. Yeh, and C. -C. Chang, “Reversible data hiding based on histogram modification of pixel differences,” IEEE Transactions on Circuits and Systems for Video Tech- nology, vol. 19, no. 6, pp. 906–910, 2009. [3] X. Zhang, “Reversible data hiding in encrypted images,” IEEE Signal Processing Letters, vol. 18, no. 4, pp. 255–258, 2011. [4] W. Hong, T. Chen, and H. Wu, “An improved reversible data hiding in encrypted images using side match,” IEEE Signal Processing Letters, vol. 19, no. 4, pp. 199–202, 2012. [5] X. Liao, and C. Shu, “Reversible data hiding in encrypted im- ages based on absolute mean difference of multiple neighboring pixels,” Journal of Visual Communication and Image Repre- sentation, vol. 28, pp. 21–27, 2015. [6] Z. Qian, S. Dai, F. Jiang, and X. Zhang, “Improved joint re- versible data hiding in encrypted images”, Journal of Visual Communication and Image Representation, vol. 40, pp. 732-738, 2016. [7] J. Zhou, W. Sun, L. Dong, et al. “Secure reversible image data hiding over encrypted domain via key modulation,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 26, no. 3, pp. 441-452, 2016. [8] X. Zhang, “Separable reversible data hiding in encrypted image,” IEEE Transactions on Information Forensics Security, vol. 7, no. 2, pp. 826–832, 2012. [9] X. Wu, and W. Sun, “High-capacity reversible data hiding in encrypted images by prediction error,” Signal processing, vol. 104, pp. 387-400, 2014. [10] Z. Qian, and X. Zhang, “Reversible data hiding in encrypted image with distributed source encoding,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 26, no. 4, pp. 636-646, 2016. [11] Z. Qian, X. Zhang, and G. Feng, “Reversible data hiding in encrypted images based on progressive recovery”, IEEE Signal Processing Letters, vol. 23, no. 11, pp. 1672-1676, 2016. [12] F. Huang, J. Huang, and Y. Q. Shi, “New framework for re- versible data hiding in encrypted domain”. IEEE Transactions on Information Forensics and Security, vol. 11, no. 12, pp. 2777-2789, 2016. [13] K. Ma, W. Zhang, X. Zhao, et al. “Reversible data hiding in encrypted images by reserving room before encryption,” IEEE Transactions on Information Forensics Security, vol. 8, no. 3, pp. 553-562, 2013. [14] W. Zhang, K. Ma and N. Yu, “Reversibility improved data hiding in encrypted images,” Signal Processing, vol. 94, pp. 118–127, 2014. [15] X. Cao, L. Du, X. Wei, et al. “High capacity reversible data hiding in encrypted images by patch-level sparse representation,” IEEE Transactions on Cybernetics, vol. 46, no. 5, pp. 1132-1143, 2016. [16] M. Fujiyoshi, “Separable reversible data hiding in encrypted images with histogram permutation,” in Proceeding IEEE In- ternational Conference on Multimedia and Expo, pp. 1-4, 2013. [17] W. Zhang, H. Wang, D. Hou, and N. Yu, “Reversible data hiding in encrypted images by reversible image transformation,” IEEE Transactions on Multimedia, vol. 18, no. 8, pp. 1469-1479, 2016. [18] Z. Qian, X. Zhang, and S. Wang, “Reversible data hiding in encrypted JPEG bitstream,” IEEE Transactions on Multimedia, vol. 16, no. 5, pp. 1486-1491, 2014. [19] J. C. Chang, Y. Z. Lu, and H. L. Wu, “A separable reversible data hiding scheme for encrypted JPEG bitstreams,” Signal Processing, vol. 133, pp. 135-143, 2017. [20] S. Y. Ong, K. S. Wong and K. Tanaka, “Scrambling-embedding for JPEG compressed image,” Signal Processing, vol. 109, pp. 56-68, 2015. [21] Z. Qian, H. Zhou, X. Zhang, and W. Zhang, “Separable re- versible data hiding in encrypted JPEG bitstreams,” IEEE Transactions on Dependable and Secure Computing, doi: 10.1109 /TDSC.2016.2634161. [22] F. Huang, X. Qu, H. J. Kim, and J. Huang, “Reversible Data Hiding in JPEG Images,” IEEE Trans. Circuits and Systems for Video Technology, vol. 26, no. 9, pp. 1610-1621, 2015. [23] A. Massoudi, F. Lefebvre, C. De Vleeschouwer, et al. “Over- view on selective encryption of image and video: challenges and perspectives,” EURASIP Journal on Information Security, vol. 5, pp. 1-18, 2008. [24] T. Richter, A. Artusi, and T. Ebrahimi, “JPEG XT: A New Family of JPEG Backward-Compatible Standards,” IEEE Mul-
  • 12. 1051-8215 (c) 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission. See http://www.ieee.org/publications_standards/publications/rights/index.html for more information. This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/TCSVT.2018.2797897, IEEE Transactions on Circuits and Systems for Video Technology 12 timedia, vol. 23, no. 3, pp. 80-88, 2016. [25] Int. Tele. Union, CCITT Recommendation T.81, “Information technology-digital compression and coding of continuous-tone Still Images - Requirements and Guidelines,” 1992. [26] Bossbase Database, http://www.agents.cz/boss/BOSSFinal/ [27] Independent JPEG Group, http://www.ijg.org/ [28] S. Y. Ong, K. S. Wong, X. Qi, et al. “Beyond format-compliant encryption for JPEG image,” Signal Processing: Image Com- munication, vol. 31, pp. 47-60, 2015. Zhenxing Qian (M’12) received both the B.S. and the Ph.D. degrees from University of Sci- ence and Technology of China (USTC) in 2003 and 2007, respectively. Since 2009, he has been with the faculty of the School of Com- munication and Information Engineering, Shanghai University, where he is currently a Professor. His research interests include in- formation hiding and multimedia security. Haisheng Xu received the B.S. degree in School of Communication and Information Engineering, Shanghai University. He is cur- rently working toward the Master degree at Shanghai University. His research interests include information hiding and image pro- cessing. Xiangyang Luo received the M.S. degree and the Ph.D. degree from Zhengzhou Information Science and Technology Institute, Zhengzhou, China in 2004 and 2010, respectively. He has published more than 100 refereed international journal and conference papers. He is currently an associate professor and Ph.D. supervisor at State Key Laboratory of Mathematical Engi- neering and Advanced Computing. His re- search interests include multimedia security. Xinpeng Zhang (M’11) received the B.S. degree in computational mathematics from Jilin University, China, in 1995, and the M.E. and Ph.D. degrees in communication and information system from Shanghai University, China, in 2001 and 2004, respectively. Since 2004, he has been with the faculty of the School of Communication and Information Engineering, Shanghai University, where he is currently a Professor. His research interests include information hiding, image processing and digital forensics.