Linear algebra for data science

Why Learn It:
Linear algebra is a very fundamental area in mathematics. But we use it wide range of application with respect to machine learning.
We use it to solve some specific types of problems in machine learning algorithm.

What is scaler quantity:
Scalar quantity is defined as the physical quantity with magnitude and no direction.

Some physical quantities can be described just by their numerical value (with their respective units) without directions (they don’t
have any direction). The addition of these physical quantities follows the simple rules of the algebra. Here, only their magnitudes are
added.
Example: A bike is moving with the speed of 5Km/h is a scaler quantity because it has only magnitude.

What is vector quantity:
A vector quantity is defined as the physical quantity that has both direction as well as magnitude.
Example: A bike is moving with the speed of 5Km/h in east direction is a vector quantity because it has both magnitude and direction
as well.


Represent vector graphically:
To represent vector graphically. We only care about the magnitude in the direction. We do not necessarily not care about Where we
start and where we place it.
In [1]:
import matplotlib.pyplot as plt
plt.plot([0,5],[1,1])
#vector v1
plt.plot([1,6],[2,2])
#vector v2
plt.plot([8,-8],[0,0],'k--')
plt.plot([0,0],[8,-8],'k--')
plt.grid()
plt.axis((-8,8,-8,8))

plt.show()

Explanation: As you can see we have v1 and v2 vector. we do not care about where to start. we just care about the magnitude in the
direction. Here v1 and v2 are same because both have same magnitude and direction is "Horizontal" direction.

Real coordinate space:
Coordinate Geometry is even more useful in space than in the plane, since it is much harder to draw figures in 3D for accurate
visualization.
Notation:
Vectors will be written either as rows or columns when only vector operations are concerned. If matrices enter the picture, then there
is a difference between row vectors (row matrices) and column vectors (column matrices).
1.
2.
3.
4.

In general, one writes O or sometimes 0 for the zero vector.
In 2-space one often writes O or 0 for (0, 0), I or i for (1, 0) and J or j for (0, 1).
In 3-space one oftern writes O or 0 for (0, 0, 0), I or i for (1, 0, 0) and J or j for (0, 1, 0), K or k for (0, 0, 1).
In n-space, even when n = 1, 2 or 3, one writes vectors e1, ..., en, where ek is the vector in n-space with all entries = 0 except for
1 in the k entry.

Addition of vectors:
The process of adding two or more vectors is called vector addition.


Consider a vector A and B

“When two vectors are represented by two sides of a triangle in magnitude and direction were taken in the same order then the third
side of that triangle represents in magnitude and direction the resultant of the vectors.”

In [2]:
import numpy as np
import matplotlib.pyplot as plt
v1=np.array([1,2])
v2=np.array([3,4])
v3=v1+v2
print("v3=",v3)
plt.plot([0,v1[0]],[0,v1[1]],'r',label='v1')
plt.plot([0,v2[0]],[0,v2[1]],'b',label='v2')


plt.plot([0,v2[0]],[0,v2[1]],'b',label='v2')
plt.plot([0,v3[0]],[0,v3[1]],'g',label='v3')
plt.plot([8,-8],[0,0],'k--')
plt.plot([0,0],[8,-8],'k--')
plt.grid()
plt.axis((-8,8,-8,8))
plt.legend()
plt.show()
v3= [4 6]

Multiply a vector by a scalar:
When a vector is multiplied by a scalar quantity, then the magnitude of the vector changes in accordance with the magnitude of the
scalar but the direction of the vector remains unchanged.
Suppose we have a vector a, then if this vector is multiplied by a scalar quantity k then we get a new vector with magnitude as
|ka|and the direction remains same as the vector a if k is positive and if k is negative then the direction of k becomes just opposite of
the direction of vector a.

Now let us understand visually the scalar multiplication of the vector

Let us take the values of ‘k ‘to be = 2,3,-3,−12 and so on.

In [3]:
v1=np.array([3,4])
a=0.5
v2=v1*a
plt.plot([0,v1[0]],[0,v1[1]],'r-',label='v1')
plt.plot([0,v2[0]],[0,v2[1]],'b--',label='v2')
plt.plot([8,-8],[0,0],'k--')
plt.plot([0,0],[8,-8],'k--')
plt.grid()


plt.grid()
plt.axis((-8,8,-8,8))
plt.legend()
plt.show()

In [4]:
v1=np.array([3,4])
a=-0.5
v2=v1*a
plt.plot([0,v1[0]],[0,v1[1]],'r-',label='v1')
plt.plot([0,v2[0]],[0,v2[1]],'b--',label='v2')
plt.plot([8,-8],[0,0],'k--')
plt.plot([0,0],[8,-8],'k--')
plt.grid()
plt.axis((-8,8,-8,8))
plt.legend()
plt.show()

Length of a vector:
The length of the directed segment determines the numerical value of the vector is called the length of vector AB.
The magnitude of a vector is the length of the vector.


In [5]:
v1=np.array([1,2,3,4,5])
length=np.sqrt(np.dot(v1,v1))
print("The length of vector v1 is: ",length)
The length of vector v1 is:

7.416198487095663

Dot product:
A vector has magnitude (how long it is) and direction:

Here are two vectors:


We will multiply these two vectors by using dot product:

And the result is a number (called a "scalar" so we know it is not a vector).

Angle between two vector:


In [6]:
v1 = np.array([8,4])
v2 = np.array([-4,8])

ang = np.rad2deg(np.arccos( np.dot(v1,v2) / (np.linalg.norm(v1)*np.linalg.norm(v2)) ))
plt.plot([0,v1[0]] , [0,v1[1]] , 'r' , label = 'v1')
plt.plot([0,v2[0]]+v1[0] , [0,v2[1]]+v1[1], 'b' , label = 'v2')
plt.plot([16,-16] , [0,0] , 'k--')
plt.plot([0,0] , [16,-16] , 'k--')
plt.grid()
plt.axis((-16, 16, -16, 16))
plt.legend()
plt.title('Angle between Vectors - %s' %ang)
plt.show()

Distance of a point from origin


Distance between two points:


Projection of vectors:


Equation of a line (2-D)/plane (3-D)/Hyperplane (n-D)



Equation of circle(2-D),sphere(3-D),hypersphere(n-D):


Equation of an elipse(2-D),ellipsoid(3-D) and hyperellipsoid(nD)

Row vector:
row matrix is a 1 × m matrix, that is, a matrix consisting of a single row of m elements.

Column vector:
column matrix is an m × 1 matrix, that is, a matrix consisting of a single column of m elements,

Matrix:
A matrix is a rectangular arrangement of numbers into rows and columns.
In [7]:
A = np.array([[1,2,3,4] , [5,6,7,8] , [10 , 11 , 12 ,13] , [14,15,16,17]])
A


A
Out[7]:
array([[ 1, 2, 3, 4],
[ 5, 6, 7, 8],
[10, 11, 12, 13],
[14, 15, 16, 17]])

Zero matrix:
A matrix is known as a zero or null matrix if all of its elements are zero.
In [8]:
A=np.zeros(9).reshape(3,3)
A
Out[8]:
array([[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.]])

Ones matrix:
A matrix is known as a ones matrix if all of its elements are one.
In [9]:
A=np.ones(9).reshape(3,3)
A
Out[9]:
array([[1., 1., 1.],
[1., 1., 1.],
[1., 1., 1.]])

Identity matrix
In linear algebra, the identity matrix of size n is the n × n square matrix with ones on the main diagonal and zeros elsewhere. It is
denoted by Iₙ.
In [10]:
A=np.eye(9)
A
Out[10]:
array([[1.,
[0.,
[0.,
[0.,
[0.,
[0.,
[0.,
[0.,
[0.,

0.,
1.,
0.,

0.,
0.,
0.,
0.,
0.,
0.,

0.,
0.,
1.,
0.,
0.,
0.,
0.,
0.,
0.,

0.,
0.,
0.,
1.,
0.,
0.,
0.,
0.,
0.,

Diagonal Matrix

0.,

0.,
0.,
0.,
1.,
0.,
0.,
0.,
0.,

0.,
0.,
0.,
0.,
0.,
1.,
0.,
0.,
0.,

0.,
0.,
0.,
0.,
0.,
0.,
1.,
0.,
0.,

0.,

0.,
0.,
0.,
0.,
0.,
0.,
1.,
0.,

0.],
0.],
0.],
0.],
0.],
0.],
0.],
0.],
1.]])


a diagonal matrix is a matrix in which the entries outside the main diagonal are all zero; the term usually refers to square matrices
In [11]:
A=np.diag([1,2,3,4,5,6,7,8])
A
Out[11]:
array([[1,
[0,
[0,
[0,
[0,

[0,
[0,
[0,

0,
2,
0,
0,
0,
0,
0,
0,

0,
0,
3,
0,
0,
0,
0,
0,

0,
0,
0,
4,
0,
0,
0,
0,

0,
0,
0,
0,
5,
0,
0,
0,

0,
0,
0,
0,
0,
6,
0,
0,

0,
0,
0,
0,
0,
0,
7,
0,

0],
0],

0],
0],
0],
0],
0],
8]])

Matrix Addition
Matrix addition is the operation of adding two matrices by adding the corresponding entries together.
In [12]:
M = np.array([[1,2,3],[4,-3,6],[7,8,0]])
N = np.array([[1,1,1],[2,2,2],[3,3,3]])
print("\n First Matrix (M): \n", M)
print("\n Second Matrix (N): \n", N)
C = M+N
print("\n Matrix Addition (M+N): \n", C)
First Matrix (M):
[[ 1 2 3]
[ 4 -3 6]
[ 7 8 0]]
Second Matrix (N):
[[1 1 1]
[2 2 2]
[3 3 3]]
Matrix Addition (M+N):
[[ 2 3 4]
[ 6 -1 8]
[10 11 3]]

Matrix subtraction

Subtraction of matrix is possible by subtracting the element of another matrix if they have the same order. This new matrix is the
difference between the above two matrices. So, the difference between two matrices is obtained by subtracting the corresponding
elements of the given matrices.
In [13]:
M = np.array([[1,2,3],[4,-3,6],[7,8,0]])
N = np.array([[1,1,1],[2,2,2],[3,3,3]])
print("\n First Matrix (M): \n", M)
print("\n Second Matrix (N): \n", N)
C = M-N
print("\n Matrix Subtraction (M-N): \n", C)
First Matrix (M):
[[ 1 2 3]
[ 4 -3 6]
[ 7 8 0]]
Second Matrix (N):


Second Matrix (N):
[[1 1 1]
[2 2 2]
[3 3 3]]
Matrix Subtraction (M-N):
[[ 0 1 2]
[ 2 -5 4]
[ 4 5 -3]]

Matrices Scalar Multiplication
The term scalar multiplication refers to the product of a real number and a matrix. In scalar multiplication, each entry in the matrix is
multiplied by the given scalar.
In [14]:

M = np.array([[1,2,3],[4,-3,6],[7,8,0]])
C = 10
print("\n Matrix (M): \n", M)
print("\nMatrices Scalar Multiplication: \n", C*M)
Matrix (M):
[[ 1 2 3]
[ 4 -3 6]
[ 7 8 0]]
Matrices Scalar Multiplication:
[[ 10 20 30]
[ 40 -30 60]
[ 70 80
0]]

Transpose of a matrix
The transpose of a matrix is an operator which flips a matrix over its diagonal; that is, it switches the row and column indices of the
matrix A by producing another matrix, often denoted by Aᵀ
In [15]:
M = np.array([[1,2,3],[4,-3,6],[7,8,0]])
print("\n Matrix (M): \n", M)
print("\nTranspose of M: \n", np.transpose(M))
Matrix (M):
[[ 1 2 3]
[ 4 -3 6]
[ 7 8 0]]
Transpose of M:
[[ 1 4 7]
[ 2 -3 8]
[ 3 6 0]]

Determinant of a matrix
In [16]:
M = np.array([[1,2,3],[4,-3,6],[7,8,0]])
print("\n Matrix (M): \n", M)
print("\nDeterminant of M: ", np.linalg.det(M))
Matrix (M):
[[ 1 2 3]
[ 4 -3 6]
[ 7 8 0]]


Determinant of M:

195.0

Rank of a matrix
In [17]:
M = np.array([[1,2,3],[4,-3,6],[7,8,0]])
print("\n Matrix (M): \n", M)
print("\nRank of M: ", np.linalg.matrix_rank(M))
Matrix (M):
[[ 1 2 3]
[ 4 -3 6]
[ 7 8 0]]
Rank of M:

3

Inverse of matrix A
In [18]:

M = np.array([[1,2,3],[4,-3,6],[7,8,0]])
print("\n Matrix (M): \n", M)
print("\nInverse of M: \n", np.linalg.inv(M))
Matrix (M):
[[ 1 2 3]
[ 4 -3 6]
[ 7 8 0]]
Inverse of M:
[[-0.24615385 0.12307692 0.10769231]
[ 0.21538462 -0.10769231 0.03076923]
[ 0.27179487 0.03076923 -0.05641026]]

Matrix Multiplication (pointwise multiplication)
In [19]:
M = np.array([[1,2,3],[4,-3,6],[7,8,0]])
N = np.array([[1,1,1],[2,2,2],[3,3,3]])
print("\n First Matrix (M): \n", M)
print("\n Second Matrix (N): \n", N)
print("\n Point-Wise Multiplication of M & N: \n", M*N)
First Matrix (M):
[[ 1 2 3]
[ 4 -3 6]
[ 7 8 0]]
Second Matrix (N):
[[1 1 1]
[2 2 2]
[3 3 3]]
Point-Wise Multiplication of M & N:
[[ 1 2 3]
[ 8 -6 12]

[21 24 0]]

Sum of all elements in a matrix
In [20]:


N = np.array([[1,1,1],[2,2,2],[3,3,3]])
print("\n Matrix (N) ==> \n", N)
print ("\nSum of all elements in a Matrix: ")
print (np.sum(N))
Matrix (N) ==>
[[1 1 1]
[2 2 2]
[3 3 3]]
Sum of all elements in a Matrix:
18

Column-Wise Addition
In [21]:
N = np.array([[1,1,1],[2,2,2],[3,3,3]])
print("\n Matrix (N): \n", N)
print ("\nColumn-Wise summation: ")
print (np.sum(N,axis=0))
Matrix (N):
[[1 1 1]
[2 2 2]
[3 3 3]]
Column-Wise summation:
[6 6 6]

Row-Wise Addition
In [22]:
N = np.array([[1,1,1],[2,2,2],[3,3,3]])
print("\n Matrix (N) ==> \n", N)
print ("\nRow-Wise summation ==>")
print (np.sum(N,axis=1))
Matrix (N) ==>
[[1 1 1]
[2 2 2]
[3 3 3]]
Row-Wise summation ==>
[3 6 9]