Linear algebra học AI từ cơ bản đến nâng cao

AI VIETNAM
AI Course 2022

Linear Algebra and
Applications
Quang-Vinh Dinh
PhD in Computer Science


Outline







Project Description
Matrix Operations
Delving into Numpy
Least Squares Estimation
Cosine Similarity
Singular Value Decomposition


Pipeline


Binary BoW: 1 for word appearing
0 for word non-appearing

Text Representation

BoW: frequency of each word occurring

❖ Bag-of-Words (BoW)

vocabulary_size = 9

Corpus
doc1 = “deep learning book”
doc2 = “machine learning algorithm”

[‘deep’, ‘learning’, ‘book’]

Tokenization

[‘machine’, ‘learning’, ‘algorithm’]

doc3 = “learning ai from scratch”

[‘learning’, ‘ai’, ‘from’, ‘scratch’]

doc4 = “ai vietnam”

[‘ai’, ‘vietnam’]

Vocabulary =

deep

learning

book

machine

algorithm

ai

from

scratch

vietnam

same size

👉 Given a string = “vietnam machine learning deep learning book”
deep

learning

book

machine

algorithm

ai

from

scratch

vietnam

BoW

1

2

1

1

0

0

0

0

1

Binary BoW

1

1

1

1

0

0

0

0

1

2


Outline







Project Description
Matrix Operations

Delving into Numpy
Least Squares Estimation
Cosine Similarity
Singular Value Decomposition


AI VIETNAM
AI Course 2022

Vector & Matrix
Matrix

Vector
𝑛 is a natural number

Matrix A has the shape of rectangle

ℛ is a set of real numbers

Has 𝑚 rows and n columns

𝑣Ԧ has a length of n and contain real numbers

Use capital letter

𝑣Ԧ ∈ ℛ 𝑛

A ∈ ℛ 𝑚×𝑛

𝑣1

ℛ
𝑣Ԧ = 𝑣2 ∈ ℛ = ℛ 3
𝑣3
ℛ

𝑎11 𝑎12
ℛ ℛ
A = 𝑎21 𝑎22 ∈ ℛ ℛ = ℛ 3×2
𝑎31 𝑎32
ℛ ℛ

3


Vector Addition
𝑣1
𝑣Ԧ = …
𝑣3

𝑢1
𝑢= …
𝑢3

𝑣1
𝑢1
𝑣1 + 𝑢1
…
𝑣Ԧ + 𝑢 = … + … =
𝑣3
𝑢3

𝑣3 + 𝑢3
data x

data y

result

1

5

6

2

6

8

+

=

3

7

10

4

8

12

4


AI VIETNAM
AI Course 2022

Vector Operations

Vector subtraction

𝑣1
𝑣Ԧ = …
𝑣𝑛

𝑢1
𝑢= …
𝑢𝑛

𝑣1
𝑢1
𝑣1 − 𝑢1
…
𝑣Ԧ + 𝑢 = … − … =
𝑣𝑛
𝑢𝑛

𝑣𝑛 − 𝑢𝑛

data x

data y

result

5

1

4

6

2

4

-

=

7

3

4

8

4

4
5


Vector Operations
Multiply with a number
𝑢1
𝛼𝑢1
𝛼𝑢 = 𝛼 … = …
𝑢𝑛
𝛼𝑢𝑛

data

result

1

2

2
3

*

2

=

4
6

Length of a vector
𝑢 =

𝑢12 + ⋯ + 𝑢𝑛2

6


AI VIETNAM
AI Course 2022

Matrix Operations

𝑎11 … 𝑎1𝑛
A = … … ...
𝑎𝑚1 … 𝑎𝑚𝑛

𝑏11 … 𝑏1𝑛
B = … … ...
𝑏𝑚1 … 𝑏𝑚𝑛

Addition
(𝑎11 + 𝑏11 ) … (𝑎1𝑛 + 𝑏1𝑛 )
…

…
...
A+B=
(𝑎𝑚1 + 𝑏𝑚1 ) … (𝑎𝑚𝑛 + 𝑏𝑚𝑛 )

A

4

2

9

8

Subtraction
A−B=

𝑎11 − 𝑏11 … (𝑎1𝑛 − 𝑏1𝑛 )
…
…
...
𝑎𝑚1 − 𝑏𝑚1 … (𝑎𝑚𝑛 − 𝑏𝑚𝑛 )

C

B

+

1

2

3

4

A

=

5

4

12

12
C

B

4

2

9

8

_

1

2

3

4

=

3

0

6

4
7


Vector Operations
Dot product
𝑣1
𝑣Ԧ = …
𝑣𝑛

𝑢1

𝑢= …
𝑢𝑛

𝑣Ԧ ∙ 𝑢 = 𝑣1 × 𝑢1 + ⋯ + 𝑣𝑛 × 𝑢𝑛
w

v

1

2

2

result

=

8

3
8


Matrix Operations
Matrix-vector multiplication
𝑎11 … 𝑎1𝑛
A = … … ...
𝑎𝑚1 … 𝑎𝑚𝑛

𝑥1
𝒙= …
𝑥𝑛

A ∈ ℛ 𝑚×𝑛
𝑛
C = A𝒙 where 𝑐𝑖 = σ𝑙=1 𝑎𝑖𝑙 𝑥𝑙

X

X

result

v

v

1

2

1

3

4

2

=

5
11

1

2

1

2

3

4

result

=

7

10
9


AI VIETNAM
AI Course 2022

Matrix Operations

Matrix-matrix multiplication

𝑎11 … 𝑎1𝑛
A = … … ...
𝑎𝑚1 … 𝑎𝑚𝑛

𝑏11 … 𝑏1𝑘
B = … … ...
𝑏𝑛1 … 𝑏𝑛𝑘

A ∈ ℛ 𝑚×𝑛

B ∈ ℛ 𝑛×𝑘

𝑎11 … 𝑎1𝑛
AB = … … . . .
𝑎𝑚1 … 𝑎𝑚𝑛

𝑏11 … 𝑏1𝑘
… … ... =
𝑏𝑛1 … 𝑏𝑛𝑘

C = AB
𝑛

𝑐𝑖𝑗 = ෍ 𝑎𝑖𝑙 𝑏𝑙𝑗
𝑙=1

(𝑎11 𝑏11 + ⋯ + 𝑎1𝑛 𝑏𝑛1 ) … (𝑎1𝑛 𝑏1𝑘 + ⋯ + 𝑎1𝑛 𝑏𝑛𝑘 )
…
…
...
𝑎𝑚1 𝑏11 + ⋯ + 𝑎𝑚𝑛 𝑏𝑛1 … (𝑎𝑚1 𝑏1𝑘 + ⋯ + 𝑎𝑚𝑛 𝑏𝑛𝑘 )
10


Matrix Operations
Matrix-matrix multiplication
𝑎11 … 𝑎1𝑛
A = … … ...
𝑎𝑚1 … 𝑎𝑚𝑛

𝑏11 … 𝑏1𝑘
B = … … ...
𝑏𝑛1 … 𝑏𝑛𝑘

A ∈ ℛ 𝑚×𝑛

B ∈ ℛ 𝑛×𝑘

C = AB
𝑛

𝑐𝑖𝑗 = ෍ 𝑎𝑖𝑙 𝑏𝑙𝑗
𝑙=1

11
4

Matrix-matrix multiplication
𝑎11 … 𝑎1𝑛
A = … … ...
𝑎𝑚1 … 𝑎𝑚𝑛

𝑏11 … 𝑏1𝑘
B = … … ...
𝑏𝑛1 … 𝑏𝑛𝑘

A ∈ ℛ 𝑚×𝑛

B ∈ ℛ 𝑛×𝑘
C = AB
𝑛

𝑐𝑖𝑗 = ෍ 𝑎𝑖𝑙 𝑏𝑙𝑗
𝑙=1

X

result

Y

1

2

3

4

2

3

2

1

=

X

Y

6

5

14

13

2

3

2

1

result

1

2

3

4

=

11

16

5

8
12
4


Hadamard product
𝑣1
𝑣Ԧ = …

𝑣𝑛

𝑢1
𝑢= …
𝑢𝑛

𝑣1
𝑢1
𝑣1 × 𝑢1
…
𝑣⨀𝑢
Ԧ
= … ⨀ … =
𝑣𝑛
𝑢𝑛
𝑣𝑛 × 𝑢𝑛
data x

data y

result

1

5

5

2

6

12
=

*
3

7

21

4

8

32
13


AI VIETNAM
AI Course 2022

Numpy Array Operations

 Division

data x

data y

result

1

5

0.2

2

6

0.33

/

=

3

7

0.42

4

8

0.5

14
4


AI VIETNAM
AI Course 2022

Matrix Operations

Transpose
𝑎11 … 𝑎1𝑛
A = … … ...
𝑎𝑚1 … 𝑎𝑚𝑛

𝑎11 … 𝑎𝑚1
A𝑇 = … … . . .
𝑎1𝑛 … 𝑎𝑚𝑛

Example
𝑎11
A = 𝑎21
𝑎31

𝑎12
𝑎22
𝑎32

A𝑇

𝑎11 𝑎21 𝑎31
= 𝑎
12 𝑎22 𝑎32

T
1

2

1

3

3

4

2

4

15


AI VIETNAM
AI Course 2022

Numpy

 Some important functions

reshape() function
data_rs

data
1

2

3

4

5

6

1

2

3

4

5

6

16
4

AI VIETNAM
AI Course 2022

data

Numpy

1

2

sum(data)

10

Axis 0

4

3

Summation

Square root
data

result

1

1.0

2

Axis 1

sum(data, axis=0)

4

6

sum(data, axis=1)

3

7

1.4

sqrt(data) =
3

1.7

4

2.0

17


numpy.hstack()

arr_1

1

2

3

arr_2

4

5

6

numpy.vstack()

vstack((arr_1, arr_2 ))
result

1

2

3

4

5

6

18


AI VIETNAM
AI Course 2022

Vector-matrix multiplication
𝑎11 𝑎12
𝑥1
𝐴= 𝑎
𝒙= 𝑥
21 𝑎22
2

Matrix and Vector
𝒃=

𝑏1
𝑏2

𝐴𝒙 = 𝒃

𝑎11 𝑎12 𝑥1 𝑎11 𝑥1 + 𝑎12 𝑥2
𝑏1
=
=
𝑎21 𝑎22 𝑥2 𝑎21 𝑥1 + 𝑎22 𝑥2
𝑏2

𝒙

multiply by A
𝒃

Matrix A transforms x to b
Each element of b is a linear combination
of all the elements of x
𝑏1 = 𝑎11 𝑥1 + 𝑎12 𝑥2
𝑏2 = 𝑎21 𝑥1 + 𝑎22 𝑥2
19


AI VIETNAM
AI Course 2022

Matrix and Vector

Matrix A translates 𝒙 symmetrically with
respect to x1

𝑥2 𝐴 = 01 − 01

Flipping an image vertically
values (red, green, blue) of the pixel p

𝑏1 =1𝑥1 + 0𝑥2 = 𝑥1
𝑏2 =0𝑥1 − 1𝑥2 = -𝑥2

All the element of the pixel p(𝑥1 , 𝑥2 , r, g, b)

𝐮(𝟔, 𝟒)

coordinate of p

Translate a pixel (𝑥1 , 𝑥2 ) by 𝑨 =

𝐩(𝟑, 𝟐)

1
0
0 −1

𝑥1
𝐪(3,-2)

𝐯(6,-4)

Original Image

Output Image
20

AI VIETNAM
AI Course 2022

Matrix and Vector

Matrix A translates 𝒙 symmetrically with
respect to x2
−1 0
𝐴=
0 1

Flipping an image horizontally
values (red, green, blue) of the pixel p

𝑏1 =−1𝑥1 + 0𝑥2 =−𝑥1
𝑏2 =0𝑥1 + 1𝑥2 = 𝑥2

All the element of the pixel p(𝑥1 , 𝑥2 , r, g, b)

𝑥2

coordinate of p

𝐮(𝟒, 𝟒)

𝐯(-4,4)
𝐪(-1,2)

Translate a pixel (𝑥1 , 𝑥2 ) by 𝑨 =

−1 0
0 1

𝐩(𝟏, 𝟐)
𝑥1

Original Image

Output Image
21


AI VIETNAM
AI Course 2022

Background Subtraction
Background

Year 2022

coming image

22