1. A tensor is a multi-dimensional array that can represent various types of data like scalars, vectors, matrices, and higher dimensional data. 2. The document discusses tensors in the context of deep learning, explaining concepts like tensor dimensions, shape, and how real world data like images, video, and time series data can be represented as tensors. 3. Examples of tensor representations are shown for different types of data like vector data as a 2D tensor, time series data as a 3D tensor, image data as a 4D tensor, and video data as a 5D tensor.

1. Tomomi Research Inc.
Tensor
: Data representation for deep learning
2018/09/21 (Fri)
Dr. Seong-Hun Choe

2. Tomomi Research Inc.
Agenda
9/22/2018 2©2017 Western Digital Corporation or its affiliates. All rights reserved. Confidential.
1. Tensor : Dimension
2. Tensor : Shape
3. Real world data : Which tensor?

3. Tomomi Research Inc.
Tensor
WTF is a tensor?
Remember that it is a just container for data.
the data are almost numerical data.
So, Tensor is a container for numbers.
9/22/2018 3©2017 Western Digital Corporation or its affiliates. All rights reserved. Confidential.
Various explanations,
but very hard to understand its concept
?
?

4. Tomomi Research Inc.
Keywords
9/22/2018 4©2017 Western Digital Corporation or its affiliates. All rights reserved. Confidential.
1. Tensor is a container of numbers.
2. Tensor is a generalization of matrices to an arbitrary number of dimensions.
3. In tensor, dimension is often called axis.
4. number of dimension (=axis) is called ranks.

5. Tomomi Research Inc.
1. Scalar (0D tensor)
A tensor that contains only one number is called a scalar
9/22/2018 5©2017 Western Digital Corporation or its affiliates. All rights reserved. Confidential.
• example : 12
dimension can be shown with ndim method.

6. Tomomi Research Inc.
2. Vector (1D tensor)
An array of numbers is called a vector, or 1D tensor
9/22/2018 6©2017 Western Digital Corporation or its affiliates. All rights reserved. Confidential.
• example : [12,3,6,14]

7. Tomomi Research Inc.
3. Matrix (2D tensor)
An array of vectors is a matrix, or 2D tensor
9/22/2018 7©2017 Western Digital Corporation or its affiliates. All rights reserved. Confidential.
• example : [[1, 3, 5, 7],
[2, 4, 6, 8],
[3 ,6, 9,12]]

8. Tomomi Research Inc.
4. 3D tensor
It is just nD tensor from 3D tensor.
9/22/2018 8©2017 Western Digital Corporation or its affiliates. All rights reserved. Confidential.
• example :[[[1, 3, 5, 7],
[2, 4, 6, 8],
[3 ,6, 9,12]],
[[1, 3, 5, 7],
[2, 4, 6, 8],
[3 ,6, 9,12]],
[[1, 3, 5, 7],
[2, 4, 6, 8],
[3 ,6, 9,12]]]

9. Tomomi Research Inc.
Agenda
9/22/2018 9©2017 Western Digital Corporation or its affiliates. All rights reserved. Confidential.
1. Tensor : Dimension
2. Tensor : Shape
3. Real world data : Which tensor?

10. Tomomi Research Inc.
Key attributes
9/22/2018 10©2017 Western Digital Corporation or its affiliates. All rights reserved. Confidential.
1. Number of dimension(=axes)
2. Shape : how many dimensions in the tensor has along each axis.
3. Data type : dtype in python, (float32, float64, unit8 and so on.)
Shape is very important in deep learning programming.

11. Tomomi Research Inc.
2.1. Scalar (0D tensor)
Scalar has empty shape.
9/22/2018 11©2017 Western Digital Corporation or its affiliates. All rights reserved. Confidential.
• example : 12

12. Tomomi Research Inc.
2.2 Vector (1D tensor)
1D tensor has a shape with a single element, such as (4,)
9/22/2018 12©2017 Western Digital Corporation or its affiliates. All rights reserved. Confidential.
• example : [12,3,6,14]

13. Tomomi Research Inc.
2.3. Matrix (2D tensor)
2D tensor has a shape such as (3,4). it is familiar with matrix
representation.
9/22/2018 13©2017 Western Digital Corporation or its affiliates. All rights reserved. Confidential.
• example : [[1, 3, 5, 7],
[2, 4, 6, 8],
[3 ,6, 9,12]]

14. Tomomi Research Inc.
2.4. 3D tensor
3D tensor has a shape (3, 3, 4)
9/22/2018 14©2017 Western Digital Corporation or its affiliates. All rights reserved. Confidential.
• example :[[[1, 3, 5, 7],
[2, 4, 6, 8],
[3 ,6, 9,12]],
[[1, 3, 5, 7],
[2, 4, 6, 8],
[3 ,6, 9,12]],
[[1, 3, 5, 7],
[2, 4, 6, 8],
[3 ,6, 9,12]]]

15. Tomomi Research Inc.
2.5. MNIST example
Keras & Tensorflow
9/22/2018 15©2017 Western Digital Corporation or its affiliates. All rights reserved. Confidential.
• loading the mnist dataset:
• dimension of train images
-> 3D tensor
• shape of train images
-> 3D tensor여서(m,n,p)의 형태
내용은、(28,28)の어레이가 60000개 있습니다. 라는
의미
• data type of train images
-> 8 bit integer

16. Tomomi Research Inc.
Summary (1) : Tensor
9/22/2018 16©2017 Western Digital Corporation or its affiliates. All rights reserved. Confidential.
dimension 0 1 2 3 4
Name Scalar Vector Matrix 3D tensor 4D tensor
Another name 0D tensor 1D tensor 2D tensor 3D tensor 4D tensor
Example 12 [12,3,6,14] …
Shape
()
empty
(4,) (3,4) (3,3,4) (5,3,3,4)

17. Tomomi Research Inc.
Agenda
9/22/2018 17©2017 Western Digital Corporation or its affiliates. All rights reserved. Confidential.
1. Tensor : Dimension
2. Tensor : Shape
3. Real world data : Which tensor?

18. Tomomi Research Inc.
3. Real-World examples of data as tensors
9/22/2018 18©2017 Western Digital Corporation or its affiliates. All rights reserved. Confidential.
Name Tensor Shape
Vector data* 2D tensor (samples, feature)
Timeseries data or
sequence data
3D tensor (samples, timesteps, features)
Images 4D tensor (samples, height, width, channels)
Video 5D tensor (samples, frames, height, width, channels)
Vector data is different with vector. vector is 1D tensor.

19. Tomomi Research Inc.
3.1. Vector data
2D tensor
9/22/2018 19©2017 Western Digital Corporation or its affiliates. All rights reserved. Confidential.
Samples Age ZIP code Income
1 12 123-324 10k
2 34 234-567 13k
3 12 349-874 20k
…
9,999 45 874-988 30k
10,000 56 888-234 12k
Example : Actual personal data,
3
10,000
Shape : (samples, feature)
= (10,000, 3)
• Numpy array
[[12, 123-324,10k],
[34,234-567,13k],
…
[56 ,888-234, 12k]]

20. Tomomi Research Inc.
time
(min.)
current price the highest
price
the lowest
price
0
1
2
…
390
time
(min.)
current price the highest
price
the lowest
price
0
1
2
…
390
time
(min.)
current price the highest
price
the lowest
price
0
1
2
…
390
3.2. Timeseries data or sequence data
3D tensor
9/22/2018 20©2017 Western Digital Corporation or its affiliates. All rights reserved. Confidential.
time
(min.)
current price the highest
price
the lowest
price
0
1
2
…
390
Example: stock price dataset (1 year)
3
390
min.
250 days
Shape : (samples, timesteps, features)
= (250, 390, 3)

21. Tomomi Research Inc.
3.3 Image data
4D tensor
9/22/2018 21©2017 Western Digital Corporation or its affiliates. All rights reserved. Confidential.
Example : A batch of 128 color images of size 256 * 256
Shape : (samples, height, width, channels)
= (128, 256, 256, 3)

22. Tomomi Research Inc.
3.3 Video data
5D tensor
9/22/2018 22©2017 Western Digital Corporation or its affiliates. All rights reserved. Confidential.
Example : 60 second, 144X 156 Youtube video clip sample at 4 fps would be 240 frames.
A batch of 4 such video clips
Shape : (samples, frames, height, width, channels)
= (4, 240, 144, 156, 3)
total = 4 * 240 * 144 * 156 * 3 = 106,168,320
if dtype of the tensor is float32,
total memory will be 405 MB!

23. Tomomi Research Inc.
Summary (2) : Real data
9/22/2018 23©2017 Western Digital Corporation or its affiliates. All rights reserved. Confidential.
Vector data
Timeseries data or
sequence data
Image data Video data
Tensor dimension 2D 3D 4D 5D
Example personal data annual stock data batch of color image batch of video frame
Example
Shape
(samples, feature)
= (10,000, 3)
(samples, timesteps, features)
= (250, 390, 3)
(samples, height, width,
channels)
= (128, 256, 256, 3)
(samples, frames, height, width,
channels)
=(4, 240, 144, 156, 3)

24. Tomomi Research Inc.
Tensor at Sony neural network console
9/22/2018 24©2017 Western Digital Corporation or its affiliates. All rights reserved. Confidential.
(samples, channels, height, width)
= (128, 1, 28, 28)

25. Tomomi Research Inc.
Tensor at Tensorflow
주로Placeholder를 이용해 input data를 준비할때
9/22/2018 25©2017 Western Digital Corporation or its affiliates. All rights reserved. Confidential.
(samples, timesteps, features)
= (?, 25, 1)
Sample수를 placeholder안에
지정하지 않을때는 、None으로 기입.

26. Tomomi Research Inc.
Tensor at Keras
9/22/2018 26©2017 Western Digital Corporation or its affiliates. All rights reserved. Confidential.
(samples, time_setp, features)
= (176, 25,1)