MIDTERM REPORT DIGITAL IMAGE PROCESSING canny algorithm and code

VIETNAM GENERAL CONFEDERATION OF
LABOR TON DUC THANG UNIVERSITY FACULTY
OF INFORMATION TECHNOLOGY

MIDTERM REPORT
DIGITAL IMAGE PROCESSING

Canny Algorithm and Code

Instructor: PHD. PHAM VAN HUY
Authors: NGUYENMINHNHUT– 518H0545
TRAN NHAN TAI – 518H0560
Class: 18H50203 – 18H50204
Course: 22

HO CHI MINH CITY, 2021


VIETNAM GENERAL CONFEDERATION OF LABOUR
TON DUC THANG UNIVERSITY
FACULTY OF INFORMATION TECHNOLOGY

MOBILE APP DEVELOPMENT
DIGITAL IMAGE PROCESSING

Canny Algorithm and Code

Instructor: PHD. PHAM VAN HUY
Authors: NGUYENMINHNHUT– 518H0545
TRAN NHAN TAI – 518H0560
Class: 18H50203 – 18H50204

Course: 22

HO CHI MINH CITY, 2021


i

Appreciation Letter
Firstly, this should be an honor to send my regards to the Faculty of
Information Technology, lecturers and staff from all departments of Ton Duc
Thang University. I would like to express my sincere thanks for the support and
assistance during the implementation of the statistics and probability report.
I would like to express my gratitude to PHD. Pham Van Huy - teacher who
directly instructed and supervised me to complete this essay.
I sincerely thank my friends and classmates who are studying and working
at Ton Duc Thang University and the family has encouraged, facilitated and
helped me during the process.
Due to the fact that my actual ability is still weak, I ensure that I still have
many shortcomings, so I hope my supervisor and the other professors will ignore
it. At the same time, I hope to receive many comments from many sources to
help me accumulate more experience to complete the upcoming graduation
report to achieve better results.


ii

THIS REPORT HAS BEEN CONDUCTED IN TON DUC
THANG UNIVERSITY
I assure that this is my own product and has been guided by PhD. Pham
Van Huy. The research contents, results in this topic are all about honesty. The

data in the tables for analysis, comments and evaluation are collected by the me
from various sources in the reference section.
In addition, comments and assessments as well as data from other authors
or organizations are also used in the essay but with references and annotations.
If there is any fraud is detected, I ensure my complete responsibility for
the contents of my work. Ton Duc Thang University is not related to violations of
authority and copyright caused by me during my work process (if any).
th

Ho Chi Minh City, Saturday, 5 December,
2021
Authors
(Sign and provide full name)

Nguyen Minh Nhut

Tran Nhan Tai


iii

VERIFICATION AND EVALUATION FROM LECTURER
Supervisor’s evaluation

_________________________________________________________
_________________________________________________________
_________________________________________________________
_________________________________________________________
_________________________________________________________
_________________________________________________________

_________________________________________________________
Ho Chi Minh city, date:
(Sign and provide full name)

Marking lecturer’s evaluation

_________________________________________________________
_________________________________________________________
_________________________________________________________
_________________________________________________________
_________________________________________________________
_________________________________________________________
_________________________________________________________
Ho Chi Minh city, date:
(Sign and provide full name)


iv

ABSTRACT
Today life is blessed with the presence of technology and valuable inventions,
one of them is camera, this one gadget is thing that you cannot go out without. Then,
images are things that you are able to modify and have good time with.

When images become extremely popular, the analysis of images has been
a growing field of science and application for several decades. Therefore, the
ability to analyze images has been increasing.
One of the most well-known, classical techniques is edge detection. There are
various solutions for this topic such as Prewitt, Robert, Sobel, Marr Hildrith and
Canny but this report we will focus on Canny edge detector and this algorithm is

reacted as an efficient detector for edge extraction. To make it easier to view but not
try to rebuild the rest of the algorithm since “please do not re-invent the wheel when
it is already there for you”, we will go through all these headers below:

PART 1: CANNY ALGORITHM THEORY
Chapter 1: What is Edge detection?
Chapter 2: Canny edge and Canny edge detection algorithm
PART 2: CANNY ALGORITHM SCRIPTING
Chapter 3: Canny algorithm scripting


1

TABLE OF CONTENTS
MIDTERM REPORT...................................................................................................................... 1
DIGITAL IMAGE PROCESSING............................................................................................... 1
Canny Algorithm and Code...................................................................................................... 1
Instructor: PHD. PHAM VAN HUY.......................................................................................... 1
Authors: NGUYENMINHNHUT– 518H0545.........................................................................1
VIETNAM GENERAL CONFEDERATION OF LABOUR...................................................2
TON DUC THANG UNIVERSITY.............................................................................................. 2
FACULTY OF INFORMATION TECHNOLOGY...................................................................2
MOBILE APP DEVELOPMENT................................................................................................. 2
DIGITAL IMAGE PROCESSING............................................................................................... 2
Canny Algorithm and Code...................................................................................................... 2
Instructor: PHD. PHAM VAN HUY.......................................................................................... 2
Authors: NGUYENMINHNHUT– 518H0545.........................................................................2
Appreciation Letter....................................................................................................................... i
VERIFICATION AND EVALUATION FROM LECTURER................................................iii
Supervisor’s evaluation............................................................................................................... iii

Marking lecturer’s evaluation..................................................................................................... iii
ABSTRACT.................................................................................................................................... iv
TABLE OF CONTENTS............................................................................................................... 1
LIST OFTABLES AND ILLUSTRATIONS..............................................................................3
LIST OF ILLUSTRATION............................................................................................................ 3
PART 1: CANNY ALGORITHM THEORY.............................................................................. 4
CHAPTER 1: WHAT IS EDGE DETECTION?......................................................................... 4
1.1 What is an edge?................................................................................................................... 4
Edges extraction from a sign................................................................................................. 4
Image intensity graph............................................................................................................ 5
1.1.1 Contour and Edge:............................................................................................................. 6
Edge detection....................................................................................................................... 6
Contour detection................................................................................................................... 6
1.2 What is edge detection?......................................................................................................... 7
Types of edge detections....................................................................................................... 8
CHAPTER 2: Canny edge and Canny edge detection algorithm..................................... 9
2.1 What is gradient and gradient based operator?....................................................................... 9


2

2.1.1 What is gradient?................................................................................................................... 9
2.1.2 What is image gradient.......................................................................................................... 9
Image gradient properties.................................................................................................... 10
Gradient direction and edge.............................................................................................. . 11
2.2 Who is Canny?....................................................................................................................... 12
John F. Canny portrait......................................................................................................... 12
2.3 Canny Edge Detection algorithm............................................................................................ 12
2.2.1 Smoothing (remove noise):.................................................................................................. 13
2.3 Finding gradients:.................................................................................................................... 15

2.4 Non-maximum suppression.................................................................................................... 16
2.5 Double thresholding................................................................................................................ 17
2.5 Edge tracking by Hysteresis................................................................................................... 17

PART 2: CANNY ALGORITHM SCRIPTING......................................................................19
CHAPTER 3: Canny algorithm scripting.............................................................................. 19

REFERENCES............................................................................................................................. 25
Reading section......................................................................................................................... 26
Watching section....................................................................................................................... 26


3

LIST OFTABLES AND ILLUSTRATIONS
LIST OF ILLUSTRATION
Picture 1.1: Flutter’s log (source) ...................................................................
Picture 1.2: Flutter codebase example ............................................................
Code snippet 1.1: Text sample .......................................................................
Picture 1.3: Text sample’ result ......................................................................
Picture 2.1: ToDoNhut’s starting screen ........................................................
Picture 2.2: Step 1 to create task ....................................................................
Picture 2.3: Step 2 to create task ....................................................................
Picture 2.4: Finish creating task .....................................................................
Picture 2.5: Task infor screen .........................................................................
Picture 2.6: Edit Task screen ..........................................................................
Picture 2.7: Mark as complete ........................................................................
Picture 2.8: Delete a task ................................................................................
Picture 2.9: Delete a task ................................................................................
Picture 2.10: Delete a task ..............................................................................

Picture 3.1: ToDoNhut Github repo ..............................................................


4

PART 1: CANNY ALGORITHM THEORY
CHAPTER 1: WHAT IS EDGE DETECTION?
Shapes within an image are usually defined by their boundaries, and these boundaries can be
detected or enhanced with edge detection or enhancement algorithms. Some texture measures
are also reliant on the ability to detect edges. Since Edge detection is a well-developed field on its
own within image processing. Edge detection is basically image segmentation technique, divides
spatial domain, on which the image is defined, into meaningful parts or regions.

1.1 What is an edge?
In low level image processing, we are about to “talk grayscale” and in
grayscale images, edges are generally defined as large or abrupt changes in
intensity along a line or curve.
These changes would be evident as large values in the first derivative of a
signal. Since an image has at least two dimensions, the derivative has a
slightly more complicated definition than in the one-dimensional case. A hard
edge exists when the change is very abrupt perhaps only 2 or 3 pixels wide. A
soft edge exists when the transition from bright to dark takes several pixels.

Then, in simpler way to define edge is that “Edges are discontinuities in intensity”.

Edges extraction from a sign
As we can notice, in the sign, from red to white and reverse, those lines stand among
them are called edge. And here, imagine this sign as a diagram, you will see there is an
area that the color red will span its weight to a point and then, it stops doing that and the
color white kicks in so the breakpoint between them is an edge, but not just one

5

breakpoint but many of them form up a line or curve will turn into a fully edge,
that is why it is discontinuities in intensity.
The other sources define edges correspond to abrupt changes in image
intensity. For us to say, this term is easier to get to know to. We did mention a
diagram with some dramatically uncertain changes will make up the formats
of the edges, for the graph below there:

Image intensity graph

You can notice some uplifting and down filtering things going on here and we
can assure that there are some edges. The only question here is that how we
can estimate the rate of change in this graph or function is applied to.
Let consider an image is a[ x] where x is a vector that span the space in which the
image exists. So, when we perform edge enhancement, we tend to apply a particular
model Em on a[ x] so we can receive a new image which called b[ x] Em a[ x] .

-

Moreover, you can take a look at the reference [1] - Slide Lecture 05. Edge detection
for a more detailed view in what is called an edge but we can also define edge is the
border line between 2 different regions which can be some of those below:

o Surface normal discontinuity
o Depth discontinuity
o Surface color discontinuity
o Illumination discontinuity

6

1.1.1 Contour and Edge:
And there is still one more thing that you can be confused of and this can be the
differentiation of contour and edge, these two things can be sometimes similar but
totally not the same. Since we are discussing about edge and edge detection in this
report so we keep it short on the comparison here. First of all, in detection, edge
detection and contour detection are used for determine structural outlines of any
object but edge may not form a closed shape meanwhile the contour will, it needs to
come up with the closed shape to make a boundary around the object. Take a look at
the two figures below so you will have a better overview of those two things here

Edge detection

Contour detection


7

1.2 What is edge detection?
Edge detection is a basic tool used in image processing, basically for feature
detection and extraction, which aim to identify points in a digital image where
brightness of image changes sharply and find discontinuities.
The purpose of edge detection is significantly reducing the amount of data in an
image and preserves the structural properties for further image processing.

In a grey level image, the edge is a local feature that, with in a neighborhood
separates regions in each of which the gray level is more or less uniform with

in different values on the two sides of the edge. For a noisy image it is difficult
to detect edges as both edge and noise contains high frequency contents
which results in blurred and distorted result.
So, the input of our detector is an image and the output is basically an image
but it is binary and it is called edge map.
What are some applications of edge detection? There are some:
o Finger print: edge detection helps enhance the recognition of the finger prints
o

and there are some papers which talks about this problem too
Satellite imaging: for location or place recognition, shapes detection and more,

edge map as the output can help us find out the results easier
o Robotic vision: as autopilot, we do need to detect lanes to help our
robot decide when and where to steer its wheel to or we can say lane’s
edges to wheel angle conversion
o Medical science: we can simply convert a complex X-ray image of
any body part into an edge map so we can easily detect pathological
objects such as tumors or cancer cells


8

According to a paper called “ALGORITHM AND TECHNIQUE ON VARIOUS
EDGE DETECTION: A SURVEY”, they give us a very clear view for types of
edge detection, we can take a look at diagram below:

Types of edge detections
As Canny edge detection is put in first order – gradient based operator which
functions based on thing called gradient, we are about to figure it out in Chapter 2.

9

CHAPTER 2: Canny edge and Canny edge detection algorithm
2.1 What is gradient and gradient based operator?
2.1.1 What is gradient?
Gradient is also called a slope which shows how steep a line is, it can be
calculated by dividing the change in height by the change in horizontal
distance and its formula can be depicted by
dy

grad
where dy is change in y and dx is change in x from Cartesian coordinate system.
In vector calculus, the gradient of a scalar-valued differentiable function f of several
variables is the vector field (or vector-valued function) signed as f whose value at
point p is the vector whose components are the partial derivatives of f at p ,
then
f
f
f

(

,...,

).

x1
xn

The gradient vector can be interpreted as the "direction and rate of fastest
increase". If the gradient of a function is non-zero at a point p, the direction of
the gradient is the direction in which the function increases most quickly from
p, and the magnitude of the gradient is the rate of increase in that direction.

2.1.2 What is image gradient
-

-

For the simplest way to define what image gradient is, we can say gradient
of an image can be measure of change in image function in x direction
and y direction, so as the slope, the “how steep it is” or the rate of fastest
increase and it points in the direction of most rapid change in intensity
For that we have the change in image in horizontal and vertical direction but
we are now discussing about pixel level which means we are trying to look for
the edge strength and its direction at the very location ( x, y) of the image so
the we can come up with this:
f

grad ( f ) g

-

This equation here is denoted for the gradient of the image f at point ( x, y) of the
image and it because it points to the greatest change in intensity, we can now commit
it as a vector and this vector points to the greatest change of f at point ( x, y) .

-

To make it look better, we should break this equation down a bit, because it points to
somewhere and it is considered as the point of the greatest then it should be general
so it can be defined by 2 directions x and y. By the x, the gradient can be


10

f

[g
x

, 0]
x

when you put them together we have the equation and it can be looked up by
finding the and this angle defined how steep it is but in the sake of edge, it
shows you the way where the edge is heading to, you can take a look at the
image below then we all can have the clearest view of the components:

Image gradient properties

-

From this point, we now can discuss about the 2 most important things which
called edge strength and direction. As we have defined above, edge strength
here can be defined as the magnitude of our gradient vector and the edge
direction is just that arctan( )

-

The magnitude(length) of the vector f , this term has the same scenario with length of

a normal vector which is | v | x

y2 , denoted as || f ||then we have:

|| f || mag( f )
This magnitude is the value of the rate of change in the direction of our gradient
vector then in edge detection, we shall call it edge strength. Sometimes, the edge
strength can be mounted to the point of Manhattan distance formula:

|| f ||
-

Then the direction of the gradient is given by the angle :


11

( x, y) tan

-

-

1

Come back to the image of properties above, we can see our vector will point
to the direction in which color changes the most greatly then the edge is in

between them then we want to get the edge out, the edge’s direction must be
orthogonal to the gradient direction which means you just make it 90
degrees and you get edge direction.
To make it look better, we should take a look at the figure below here:

Gradient direction and edge

-

In the above figure, edge normal here is actually our gradient direction and
the edge direction is definitely orthogonal to the normal and reverse.


12

2.2 Who is Canny?
John Francis Canny (born in 1958) is an Australian computer scientist from MIT and Adelaide
University, and Paul E Jacobs and Stacy Jacobs Distinguished Professor of Engineering in
the Computer Science Department of the University of California, Berkeley. He has made
significant contributions in various areas of computer science and mathematics including
artificial intelligence, robotics, computer graphics, human-computer interaction, computer
security, computational algebra, and computational geometry.

John F. Canny portrait (source)
Of course, The Canny edge detector was developed by John F. Canny in 1986. Canny also
produced a computational theory of edge detection explaining why the technique works.

2.3 Canny Edge Detection algorithm
Canny’s edge approach is based on 3 basic objectives
-

Low error rate. All edges should be found, and there should be no spurious responses.
That is, the edges detected must be as close as possible to the true edges.

-

Edge points should be well localized. The edges located must be as close as
possible to the true edges. That is, the distance between a point marked as an edge
by the detector and the center of the true edge should be minimum.

-

Single edge point response. The detector should return only one point for
each true edge point. That is, the number of local maxima around the true
edge should be minimum. This means that the detector should not identify
multiple edge pixels where only a single edge point exists.


13

To get started, we need to discuss about steps that the algorithm follows. After a
period of time, I notice that there is no available standard writing resource that can
describe steps of this algorithm concertedly until now but the Canny edge detection
algorithm still will generally go through these steps below:

Algorithm Flowchart


14

2.3.1 Smoothing (remove noise) input image:
Since all images which taken from a camera cannot avoid to have some
amount of noise. Because of that, the noise must be reduced before
implementing Canny edge detection algorithm.
Then, you may ask: “If I do not remove noises from my input image, can I still
detect edges correctly?”. The answer is relative but you should remove noises
before applying Canny first order derivative edge detecting technic since the
technic itself is very sensitive to noises, imagine that every detail on an image
is represented by a pixel then you have to calculate derivative on that pixel
and unfortunately, it is a noisy area, even you still get your extremum but that
should be a wrong result after going through that area.
Since Canny assumes that noises in image do follow the rules of Gaussian
Normal Distribution which can be depicted as:

f(x)
Where f ( x ) is probability density function,
deviation and
is mean.

is standard

Gaussian filter is a technique which will apply smoothing to our images which
will just obey the Normal Distribution rules.
In 1-D, we can assume that the distribution has a mean of zero where = 0
then the equation has the form of:
f(x)
As we have discussed above, image can be observed as 2-D union in two
dimensions’ space such as images, the formula of Gaussian functions is:

G(x, y)

Where is the distance from the origin in the horizontal axis, similar is a distance
in vertical axis and is a standard deviation of Gaussian filter.


15

Bonus, when we discuss about n-direction the equation can be like this:

G( x, )
Then, we will not hesitate to convolve the G and f to create a smoothed
image using Gaussian filter:

fs ( x, y ) G( x, y )

f ( x , y)

2.3.2 Finding gradients:
As we have discussed above quite clearly, in image processing, gradient is a variation
between pixels in the image which is in the grayscale intensity space. The Canny edge
detection algorithm based on finding where the gradient translation to the maxima.
By using the Sobel – operator, we can find the gradients at each pixel (first derivative) in

the smoothed image. We can apply the convolution operator in two dimensions’ space of
image. Two kernels arefor horizontal axis and

By =using these∗ two kernels with convolution operator, we can have

=

∗

and

with and are the gradients in the x- and y-directions. Then we can
compute the gradient magnitudes (edge strengths) as Euclidean distance measure

equation:
2

||=√ +
Or apply the Manhattan distance measure:
| |=|
|+|

2

|


16
By implement that technics, we will have two-dimension matrix which shows
the grayscale intensive transform in per pixel based on original image.
Similarly, we can have angle matrix with the same size as original image
which will show the gradient direction in per pixel.
( , ) = arctan (

)

−

( , ) , (−90°, 90 °)
Because the value domain of arctan is 2 2

, so we can group the

gradient direction (edge 0°)normal) into 4 groups:
horizontal edge (

vertical edge (asymptotic45°) to −90°or 90°)
wing up left (−45°)
wing up right (

Then we can conclude the≈ edge0° direction based on edge normal:
vertical edge (
≈ 90°)
≈ −90°
horizontal edge (≈ 45°
)
wing up left (
≈ −45°)

2.3.4 Non-maximum suppression
The purpose of this step is to convert the “blurred” edges into “sharp” edges in
the image of the gradient magnitudes. Basically, this process will keep all the
maxima in the gradient image and deleting everything else by compare per pixel
with their neighbors by edge strength value.


17

Round the gradient direction to nearest 45°, corresponding to the use
of an 8-connected neighborhood.
Compare the edge strength of the current pixel with the edge strength of the
pixel in the positive and negative gradient direction. Example: if the gradient
direction is north ( ≈ 90° ≈ −90° ) the we compare it with 2 neighborhood pixels:
above and below the current pixel. Similarity with 3-other direction.

If the edge strength of the current pixel is largest, preserve the value
else remove it.

2.3.5 Double thresholding:
The edge-pixels remaining after the non-maximum suppression step are
marked with their strength. Some of them may be caused by noise or color
variations for instance due to rough surfaces. Because of that, we can apply
the simplest way to discern between noise and edges by using a threshold,
which will only preserve the strong edges. With two thresholds, we can use as:
Edge pixels which stronger than high threshold will be marked as
strong Edge pixels weaker than the low threshold will be remove.

Edge pixels between two thresholds are marked as weak.

2.3.6 Edge tracking by Hysteresis
A result after step 4 (Non-maximum suppression) still have strong and weak
edges. By using Hysteresis for edge tracking, we can reduce an amount of
trivial edges. The logic is that noise and other small variations are usually to
result weak edges. The strong edges almost be a true edge in the image. The
weak edges can be true edges or noise/ color variations.
For example, we have 3 different edge units: A, B, C with A is a strong edge
(edge strength higher than a high threshold which known as “maxVal”), B and C

are weak edges (edge strengths between high threshold and low threshold)


18

After apply hysteresis A will be preserved because it is a strong edge. C is a
weak edge but have a connection with a strong edge (A) so it will be preserved
too. B is a weak edge and standing alone so it need to be suppressed.
The edge tracking can be implement by using BLOB-analysis (Binary Large
Object). Follow that technic, we divide 8-connected neighborhood with per
BLOB unit. BLOB unit which contain at least one strong edge pixel will be
preserved, and suppressed it if not.


19

PART 2: CANNY ALGORITHM SCRIPTING
CHAPTER 3: Canny algorithm scripting
In OpenCV, there is a Canny function with structure as:
void cv::Canny( InputArray
OutputArray
double
double
int
bool
)

With parameters:

image

edges
same size as image
threshold1
threshold2
apertureSize
L2gradient
√(
2

(L2gradient = true), or whether the default 1 = |

/ |+|

/ |

(L2gradient = false)
In Canny function, we don’t have to attend where is the higher threshold
parameters in procedure prepareThresh

template <bool L2gradient>

inline void prepareThresh(f64 low_thresh, f64 high_thresh, s32
&low, s32 &high)

{
if (low_thresh > high_thresh)
std::swap(low_thresh, high_thresh);
#if defined __GNUC__
low = (s32)low_thresh;
high = (s32)high_thresh;

low -= (low > low_thresh);
high -= (high > high_thresh);
#else
low = internal::round(low_thresh);

ℎ