Tensorflow || What is tensors?

Definition
Tensorflow is a free and open-source software library for machine learning and artificial intelligence.

Features

• AutoDifferentiation
• Eager execution
• Distribute
• Losses
• Metrics
• TF.nn
• Optimizers

Applications
TensorFlow is available on 64-bit Linux, macOS, Windows, and mobile computing platforms including Android and iOS.

Before staring this session, you can import tensorflow and check current version at the first time.

import tensorflow as tf
print(tf.__version__)

What is ‘tensors’?

• Scalar: a single number
• Vecotr: a number with direction ( e.g. wind speed and direction)
• Matrix: a 2-dimentional array of numbers
• Tensor : a n-dimentional array of numbers (when n can be any number, a 0-dimentional tensor is a scalar, a 1-dimentional tensor is a vector)

#Create tensors with tf.constant()

scalar = tf.constant(7)

#Create tensors with tf.constant()

scalar = tf.constant(7)
scalar

check the number of dimensions of a tensor (ndim stands for number of dimensions)

scalar.ndim

Create a vector

vector = tf.constant([10, 10])
vector

Create the dimension of our vector

vector.ndim

Create a matrix (has more than 1 dimension)

matrix = tf.constant([[10, 7], [7, 10]])
matrix

matrix.ndim

Create another matrix

another_matrix = tf.constant([[10., 7.], [3., 2.],[8., 9.]], dtype = tf.float16) # specify the data type with dtype parameter

another_matrix

another_matrix.ndim

Let’s create a tensor

tensor = tf.constant([[[1, 2, 3],
[4, 5, 6]],
[[7, 8, 9],
[10, 11, 12]],
[[13, 14, 15],
[16, 17, 18]]])
tensor

tensor.ndim

What we’ve created so far:

• Scalar: a single number
• Vecotr: a number with direction ( e.g. wind speed and direction)
• Matrix: a 2-dimentional array of numbers
• Tensor: a n-dimentional array of numbers (when n can be any number, a 0-dimentional tensor is a scalar, a 1-dimentional tensor is a vector)

Creating tensors with tf.Variable

tf.Variable

Create the same tensor with tf.Variable() as above

import tensorflow as tf
changeable_tensor = tf.Variable([10, 7])
unchangeable_tensor = tf.constant([10, 7])

changeable_tensor, unchangeable_tensor

Let’s try change one of the elements in our changeable tensor

changeable_tensor[1].assign(8)
changeable_tensor

Let’s try unchangeable tensor

unchangeable_tensor[0].assign(8)
unchangeable_tensor

Create random tensor

Random tensors are tensors of some abitrary size which contain random numbers.

Create two random (but the same) tensors

random_1 = tf.random.Generator.from_seed(5) # set seed for reproducibility
random_1 = random_1.normal(shape=(3, 2))#
random_2 = tf.random.Generator.from_seed(5) # random.normal = random w/ normal distribution
random_2 = random_2.normal(shape = (3, 2))

random_1, random_2

are they equal

random_1, random_2, random_1 == random_2

Shuffle the order of elements in a tensor

Shuffle a tensor (valuable for when you want to shuffle your data so the inherent order doesn’t effect learning)

not_shuffled = tf.constant([[10, 7],
[3, 4],
[2, 7]])

Shuffle our non-shuffled tensors

tf.random.shuffle(not_shuffled)

Shuffle our non-shuffled tensors

tf.random.set_seed(42) # global level seed
tf.random.shuffle(not_shuffled, seed = 42) # operation level seed

if both global and operation seed are set: Both seeds are used in conjunction to determine the random sequence

Exercise: Read through Tensorflow doc on random seed generation:
https://www.tensorflow.org/api_docs/python/tf/random/set_seed

tf.random.set_seed(1234)
print(tf.random.uniform([1], seed = 42)) # generates ‘A1’
print(tf.random.uniform([1])) # generates ‘A2’

tf.random.set_seed(1234)

@tf.function
def f():
a = tf.random.uniform([1])
b = tf.random.uniform([1])
return a, b

@tf.function
def g():
a = tf.random.uniform([1])
b = tf.random.uniform([1])
return a, b

print(f()) # prints ‘(A1, A2)’
print(g()) # prints ‘(A1, A2)’

import tensorflow as tf

not_shuffled = tf.constant([[10, 7],
[3, 4],
[2, 7]])
tf.random.set_seed(42) # if we set global random seed, we cam the same order at every time
tf.random.shuffle(not_shuffled, seed = 42) # if we set operation-level random seed, we get different order every time.

Other ways to get tensors

tf.ones

tf.ones([10, 7])

Create a tensor w/ all zeros

tf.zeros([10, 7])

You can also turn numPy arrays into tensors

Turn NumPy arrays into tensors

the main difference between Numpy and tensorflow tensors is that tensors can be run on a GPU computing.

import numpy as np
numpy_A = np.arange(1, 25, dtype = np.int32) # Create a NumPy array between 1 and 25
numpy_A

X = tf.constant(some_matrix) # capital for matrix

Y = tf.constant(vector) # non-capital for vector

A = tf.constant(numpy_A, shape=(2 ,3, 4))
B = tf.constant(numpy_A)
C = tf.constant(numpy_A, shape=(3, 8))
D = tf.constant(numpy_A, shape=(2,12))
A, B, C
A.ndim, B.ndim, C.ndim, D.ndim
tf.size(A)

Getting information from tensors

• Shape: tensor.shape
• Rank: tensor.ndim
• Axis or dimenson: tensor[0], tensor[:, 1]
• Size: tf.size(tensor)

Create a rank 4 tensor

rank_4_tensor = tf.zeros(shape=[2, 3, 4, 5])
rank_4_tensor[0]

rank_4_tensor.ndim, tf.size(rank_4_tensor), rank_4_tensor.shape # numpy = 234*5=120

Get various attributes of our tensor

print(“Datatype of every element:”, rank_4_tensor.dtype)
print(“Number of dimensions (rank):”, rank_4_tensor.ndim)
print(“Shape of tensor:”, rank_4_tensor.shape)
print(“Elements along the 0 axis:”, rank_4_tensor.shape[0])
print(“Elements along the last axis:”, rank_4_tensor.shape[-1]) #last axis
print(“Total number of elements in our tensor:”, tf.size(rank_4_tensor))
print(“Total number of elements in our tensor:”, tf.size(rank_4_tensor).numpy())
print(“Total number of elements in our tensor:”, tf.size(rank_4_tensor).dtype)

Indexing tensors

tensors can be indexed just like python lists.

Get the first two elements of each dimensions

rank_4_tensor[:2, :2, :2, :2]

Get the first element from each dimension from each index except for the final one

rank_4_tensor[:1, :1, :1, :]
rank_4_tensor[:, :1, :1, :1]

Create a rank 2 tensors

rank_2_tensor = tf.constant([[10, 7],
[2, 3]])
rank_2_tensor, rank_2_tensor.shape, rank_2_tensor[:,-1]

Add dimensions (tf.newaxis, tf.expand_dims)

rank_3_tensor = rank_2_tensor[…, tf.newaxis]
rank_3_tensor, tf.expand_dims(rank_2_tensor, axis = -1) # “-1” means expand the final axis

Manipulagting tensors(tensor operations)

** Basic operations **

+, -, *, /

tensor = tf.constant([[10, 7],[3, 4]])
tensor + 10, tensor * 2 # tf.multiply(tensor, 2)

Matrix multiplication

In ML, matriox multiplication is one of the most common tensor operations.

** Resource: matrix multiplication: https://www.mathsisfun.com/algebra/matrix-multiplying.html

• There are two rules our tensors (or matrices) need to fulfil if we’re going to matrix multiply them:

1. the inner dimensions must match
2. the resulting matrix has the shape of the outer dimensions

tensor, tensor @ tensor, tensor * tensor

Try to matrix multiply tensors of same shape

import tensorflow as tf
X = tf.constant([[10, 7],
[3, 4]])
Y = tf.constant([[10, 7],
[3, 4]])

tf.matmul(X, Y), X @ Y

Let’s change shapoe (tf.reshape), can do the same with transpose (tf.transpose)

import tensorflow as tf
Z = tf.constant([[10,7,4],[2,3,10],[1, 2, 3]])

X = tf.constant([[1,2],
[3,4],
[5,6]])
X_reshaped = tf.reshape(X, shape=(2, 3)) # witih same amount of values
X_transposed = tf.transpose(X)

X_reshaped @ Z, tf.transpose(X)

** The dot product**

Matrix multiplication is also referred to as the dot product.

You can perform matrix multiplication using:

• tf.matmul()
• tf.tensordot()

Changing a new tensor with default datatype (float32)

B = tf.constant([1.7, 7.4, 8.6])
C = tf.constant([1.7])
D = tf.constant([1, 7])
E = tf.constant([1., 2.])
B, C, D, E

Change from float32 to float16 (tf.cast)

B = tf.cast(B, dtype=tf.float16)
B

B.dtype

Aggregating tensors

Aggregating tensors = condensing then from multiple values down to a smaller amount of values

get an absolute values (tf.abs)

F = tf.constant([-3, -5])
tf.abs(F)

Create a random tensors with values between 0 and 100 of size 50

import numpy as np
G = tf.constant(np.random.randint(0, 100, size=50))
G

tf.size(G), G.shape, G.ndim

Find min, max, mean, sum

tf.reduce_min(G), tf.reduce_max(G), tf.reduce_mean(G), tf.reduce_sum(G)

Find (tfp.stats.variance, tf.math.reduce_std) - To define variance of our tensor, we need access to tensorflow_probability

import tensorflow_probability as tfp
tfp.stats.variance(G), tf.math.reduce_std(tf.cast(G, dtype = tf.float32)) # define standard deviation

Find the position maximum and minimum

Create a new tensor for finding positional min and max

import tensorflow as tf
tf.random.set_seed(42)
F = tf.random.uniform(shape=[50])
F, tf.argmax(F)

Index on our largest value position

F[tf.argmax(F)]

Find the max value

tf.reduce_max(F)

Check for equality

F[tf.argmax(F)] == tf.reduce_max(F)

Find the positional min, min value

tf.argmin(F), F[tf.argmin(F)]