Have you ever wondered how information is represented within a neural network?
All the inputs, outputs, and transformations are represented by data structures, referred to as tensors. Now you probably are wondering, what are really tensors?
The term “tensor” is more a mathematical generalization of many concepts rather than one specific thing.
If you’ve programmed before, you are most likely familiar with the terms “number”, “array/list”, “2D array”.
If you have studied math, you probably have heard the terms “scalar”, “vector”, and “matrix”.
Well, what do all these terms have in common? A number is the same thing as a scalar, an array or list is the same thing as a vector, and a 2D array is the same thing as a matrix. Another similarity is that all these terms refer to different types of tensors.
However, once we move to machine learning, we kind of throw out these computer science/mathematical terms because we can use one term to refer to different dimensions of tensors. Instead, we refer to tensors as an ND array or ND tensor, ND standing for the number of dimensions. Note that in neural network programming, tensors are mainly multi-dimensional arrays.
Essentially:
a number is a 0-dimensional array/tensor
an array is a 1-dimensional array /tensor
and a 2D array is a 2-dimensional array/tensor
Something important to note is that our array/tensor dimensions do not tell us how many elements exist within them. Just because you have a 2D array doesn’t mean you can only have 2 values.
x = [[2, 3]] #2D arrayy = [[2, 3, 4, 5], [3, 4, 5, 6]] #also a 2D array
After this explanation, you might be wondering what an actual application of a tensor would be.
Some common things stored in tensors include:
3D objects → Time series data
4D objects → Images
5D objects → Videos
3D tensors are a great way to store time-series data. Looking at the Apple stock, store some data points in a 3D tensor such as the low price per day, high price per day, and final price per minute
4D tensors are great at storing data for images. For example, the MNIST data set contains thousands of images of handwritten digits. We could use a tensor to store the width and height of the images, the colour depth of the images, and the fourth parameter would describe the quantity of the set of pictures.
Lastly, 5D tensors can store data for videos. We can store information about a series of clips by putting the height and width of the frames, the colour depth, the quantity of the set of pictures, and the frame rate.
Contact me for any inquiries 🚀
Hi, I’m Ashley, a 16-year-old coding nerd, and an A.I. and neuroscience enthusiast!
I hope you enjoyed reading my article, and if you did, feel free to check out some of my other pieces on Medium :)
The repository for this code is on my GitHub under my “Pytorch” repository.
Articles you will like if you read this one:
💫 An Overview on Convolutional Neural Networks
If you have any questions, would like to learn more about me, or want resources for anything A.I. or programming related, you can contact me by:
💫Email: ashleycinquires@gmail.com
