Uploaded byMasayuki Tanaka
Learnable Image Encryption
This document proposes a method called learnable image encryption that allows deep learning to be performed on encrypted images while protecting privacy. It works by applying weak block-wise encryption to images before training deep learning models. The models can still learn meaningful patterns from the encrypted images. This approach could help shopping malls analyze customer behavior from security camera footage or allow companies to develop AI systems using encrypted data without compromising privacy. The method achieves comparable accuracy to training on plain images while keeping the encrypted images unintelligible to humans. Code and details on the block-wise encryption and decryption algorithms are available online.
More Related Content
Similar to Learnable Image Encryption
Learnable Image Encryption
- 1.
- 2. Big data fordeep learning 1 Deep network The deep learning is a very powerful tool. Efficient algorithms High performance computer Big data
- 3. Big Data includesa Big Privacy Issue 2 Surveillance camera Big data We cannot use those data for learning. Privacy protection Only owner and police can access with limited purpose. Motivation: How can we take advantage of the big data with privacy protection?
- 4. Practical Case 3 Shopping mallowner: They want to analyze customer’s behavior. They have a big data, but they don’t have enough knowledge to develop software. They have to protect customer’s privacy. Engineering company: They want to develop analysis system. They have enough knowledge to develop software, but they don’t have a big data. They face privacy issue. Police: They don’t care about customer’s behavior.
- 5. Image Encryption 4 Image encryptionhelps to protect the privacy. Original data Encrypted data Key Encryption Decryption Image encryption can protect the privacy. But, we cannot use the encrypted data for the learning. Human understandable Machine learnable Human non-understandable Machine non-learnable
- 6. Homomorphic Encryption 5 𝐸𝑛𝑐𝑋 + 𝐸𝑛𝑐 𝑌 = 𝐸𝑛𝑐(𝑋 + 𝑌) Homomorphic Encryption will solve that problem. Original data Encrypted data Homomorphic encryption + − × ÷ + − × ÷ Calculation after encryption Encryption after calculation = Human non-understandable Machine calculable Still developing Very limited calculation Very heavy computation
- 7. Learnable Image Encryption 6 Wepropose a novel concept of a learnable image encryption. Original data Relatively weak encryption Powerful calculation (Deep learning) Encrypted data Human non-understandable Machine learnable This combination allows us the deep learning with the encrypted data. It means that we can avoid the privacy issue.
- 8. Summary of learningframeworks 7 Traditional learning framework Naive combination of image encryption and learning Proposed learnable image encryption Training with human understandable data Training with human understandable data Training with humannon-understandable data
- 9. -1 -1 -1-1 1 2 3 4 1 2 3 4 Algorithm of learnable image encryption 8 Input original image Block-wise operation 1 2 3 41 2 3 4 1 2 3 4 1 2 3 4 Rearrange intensity values in a row Shuffling 12 34 1 234 12 3 4 12 34 1 234 12 3 4 Reverse several values 3 2 3 1 1 1 4 42 4 2 3 Rearrange intensity values as an RGB image Block-wise operation Output encrypted image Block-size BxB Key
- 10. -1 -1 -1-1 1 2 3 4 1 2 3 4 Image decryption 9 Output original image Block-wise operation 1 2 3 41 2 3 4 1 2 3 4 1 2 3 4 Rearrange intensity values in a row Inverse shuffling 12 34 1 234 12 3 4 12 34 1 234 12 3 4 Reverse several values 3 2 3 1 1 1 4 42 4 2 3 Rearrange intensity values as an RGB image Block-wise operation Input encrypted image Block-size BxB Key
- 11. Network for encryptedimages 10 Block-size BxB Encrypted image Conv2D BxB sized filter with BxB stride Upsampling network BxB factor Classical network
- 12. Experimental results 11 Proposed High accuracy,but human understandable Plain image Comparable high accuracy, and human non-understandable
- 13. Summary 12 Proposed learnable imageencryption Training with humannon-understandable data We have proposed a novel concept of learnable image encryption. It has potential to learn with encrypted image data. Keys: Block-wise relatively weak image encryption. Powerful calculation with the deep learning. Code available: http://www.ok.sc.e.titech.ac.jp/~mtanaka/proj/imagescramble/ Google by “learnable image encryption” Human non-understandable and machine learnable
Editor's Notes
- #2 I’m Masayuki Tanaka from Japan. Today, I will introduce a new concept of a learnable image encryption.
- #3 Now a day, a big data based learning or the deep learning is know as very powerful tool. We can find the deep learning in everywhere. Here is three key components for the deep learning: Big data, high performance computer, and efficient algorithm. For the high performance computer, many companies are investing a lot of money to get many many GPUs. We also need the efficient algorithms, in other words, we need talented and smart young researchers. OK, those two components are, of course, important. However, in this presentation, I will be focusing on the big data.
- #4 We can get billions or trillions of mages everyday. However, we cannot utilize those billions of images for the deep learning. There are several challenges. But, one of those challenges is privacy issue. I can say big data includes big privacy issue. For example, let’s consider the surveillance camera case. As I said, each surveillance camera produces a lot of images everyday. However, almost all images are just threw away without using for the deep learning. The reason is the privacy protection. Usually, we cannot access those data. Only surveillance camera owner or the police can access those data, but with very limited purpose. So, my motivation is: How can we take advantage of the big data with the privacy protection.
- #5 Let’s consider more practical case. First is the shopping mall owner. They want to analyze customer’s behavior. They already have a big data, but they don’t have enough knowledge to develop software which analyze customer’s behavior. Of course, they have to protect customer’s privacy. Second is engineering company. Usually, we are in those categories. They want to develop analysis system for business. They have enough knowledge to develop software, but they don’t have a big data. They have tried to access the big data which the shopping mall owner has. However, they faced privacy issue. Third is police or other authorized association. It is very simple. They might be able to access any kind of data, may be. But, they don’t care about customer’s behavior and to develop the software.
- #6 Now, I will move to technical aspect. Image encryption is very helpful to protect the privacy. There are already a lot of encryption algorithms. Here, I’d like to introduce very simple case. The original data is encrypted with the special key. Then, only the person who knows that key can reconstruct or decrypted to the original image. We have good news in terms of the privacy. The encrypted data is human non-understandable. It means the privacy can be protected. However, nobody can utilized those encrypted data for the deep learning.
- #7 To overcome this problem. Homomorphic encryption algorithm has been intensively researched. Homomorphic encryption has very good properties. We can apply arithmetic operation or we can calculate in encrypted domain. Namely, it is guaranteed that Calculation after Encryption equals Encryption after Calculation. It is very very good property. I can say the encrypted data by the homomorphic encryption is human non-understandable and machine calculable. That is great properties and many people want to use. However, this technique is still developing, and very very limited calculation and very heavy computation. I hope that this homomorphic encryption will work soon, however, right now, we can not use this homomorphic encryption is very challenge to apply the deep learning. 演算 Arithmetic operation
- #8 Then, in this presentation, I’d like to propose a new concept of the learnable image encryption. The learnable image encryption consists of two components: relatively weak encryption and powerful calculation like deep learning. Even if the encryption is relatively weak, human cannot understand the encrypted data. In addition, the powerful calculation algorithm like the deep learning can lean the encrypted data. This is the proposed concept. If we can find that kind of balance, this combination allows us the deep learning with the encrypted data. Again, human cannot understand the encrypted data. So, we can avoid the privacy issue.
- #9 Here is summary of learning frameworks with privacy issues. First is the traditional learning framework without considering the privacy issue. Plain images are directly used for the network learning. It is normal case, but in terms of the privacy, it is not good because every data is human understandable. Second is the naive combination of the image encryption and the learning. It apply image encryption. In this sense, the stored data is human non-understandable. However, of course, original data or the plain images should be reconstructed for the network learning. Actually, it is same as the traditional learning framework. Third, this is my proposal, is the learnable image encryption. We can learn with the encrypted and human non-understandable data. So, once we apply the learnable image encryption, we are free from the privacy issue.
- #10 Now, I will explain processing pipeline of the proposed encryption algorithm. First and it is very important, the proposed algorithm is block-wise operation. We apply following operation block-by-block. I picked two-by-two bock for example. Those intensity data is rearrange into a single row. Then, apply the shuffling and reverse several values. This process corresponds to the key. Then, we have the encrypted single row data. This single row data is rearrange into the RGB image block. This encrypted image block is stored back. Again, the point is the block wise operation. Even if we apply the shuffling and the reverse operation to each block, each block still have the original information. It is the point.
- #11 The image decryption is simple reverse process of the image encryption. Each block of encrypted image is rearranged into a single row data. Then, apply the reverse operation and the inverse shuffling. This operation is associated the key. If the person knows the shuffling algorithm and the position of the reverse, that person easily can reconstruct data. However, if the person don’t know those information, it is difficult to reconstruct the data.
- #12 Next, I will show the network structure for the encrypted data. The image encryption is performed block-by-block. As I told you, each block includes original information. So, first, the network extracts those information block-wise operation manner. Then, the extracted features are upsampled to obtain the same size of input data. After that, we can apply the classical network. For this reason, the block-wise operation is very important and the key of this algorithm.
- #13 I will show kind of the preliminary results. We have evaluated with cifar-10 and cifar-100 dataset. There are four approach, first, training with the plain image, it is the classical approach, second, is existing image shuffling algorithm, third, is the naive block wise pixel shuffling algorithm, fourth, is the proposed algorithm. As you can see, the plain image is human understandable, it is bad in terms of the privacy. However, the validation accuracy of the plain image is very good. Exsiting image shuffling algorithm, in terms of the privacy, it is done pretty good job. Human cannot understand the data. But, in the same time, machine cannot learn. As a result, we can get very lower validation accuracies compared with the results of the plain image. Naive block wise pixel shuffling, the performance or the validation accuracy of this is comparable to results of the plain image case. However, human can get some information from shuffled image. Compared to those encrypted image, I think that human cannot get meaningfull information from the data encrypted by the proposed algorithm. However, the machine can learn with those data. Actually, the performance is very comparable to the results of the plain image.
- #14 Finally, I’d like to conclude as follows: We have propose the novel concept of the learning image encryption. Key components of the proposed algorithm are block-wise relatively weak image encryption and powerful calculation with the deep learning. The important properties of the proposed framework is human non-understandable and machine learnable. The code if available online, if you have interests, please try by yourself. Thanks,