Image pattern recognition with neural network

h ê đề dà
dành
h cho
h lớ
lớp cao học
h
Chuyên

IMAGE PATTERN RECOGNITION
WITH NEURAL NETWORK
Dr. Lê Dũng
g
School of Electronics and Telecommunications
Hanoi University of Science and Technology

Hà nội
ộ 11/2011


C

huyên đề dành cho lớp cao học

2

TABLE OF CONTENT
Part I:

Design a real application using image pattern
recognition with neural network
1. Automatic Envelopes Classification System in the post office

2. Skin color detector with Neural Network

Part II:

Image Pattern Recognition
+ Digital Image and Image Acquisition
+ Image Enhancement
+ Image
I
Segmentation
S
t ti
+ Image Pattern Recognition

Part III:

Recognition with Neural Network
+ Theory of Neural Network
+ Using Neural Network for Pattern Recognition

HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY

SCHOOL OF ELECTRONICS AND TELECOMMUNICATION


C

huyên đề dành cho lớp cao học

3

MỤC TIÊU CỦA CHUYÊN ĐỀ
™ Hiểu thế nào là “Mẫu
Mẫu ảnh
ảnh” (Image Pattern) và nguyên
lý nhận dạng mẫu ảnh (Image Pattern Recognition).
™ Nguyên tắc tạo và chuẩn hóa mẫu ảnh phù hợp
cho việc nhận dạng bằng mạng Nơron.
Nơron
™ Nắm được lý thuyết cơ bản về mạng Nơron trong
nhận dạng mẫu ảnh.
Ỵ Định
Đị h hướng
h ớ
cho
h việc
iệ ứng
ứ
d
dụng
kỹ thuật
th ật nhận
hậ
dạng mẫu ảnh bằng mạng nơron.
HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY

SCHOOL OF ELECTRONICS AND TELECOMMUNICATION


C

huyên đề dành cho lớp cao học

4

TÀI LIÊU THAM KHẢO CHÍNH
‰ “Digital Image Processing”
Barnd Jähne
Spring Verlag 1995

‰ “Neural Networks for Pattern Recognition”
Bishop, C.M.
Oxford University Press, 1995.

‰ “Neural Network Design”
Martin T. Hagan, Howard B. Demuth, Mark Beale
Thomson Learning, 1996.

‰ “Gradient-based learning applied to document recognition”
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P
Proceedings of the IEEE,
IEEE vol.
vol 86,
86 1998.
1998
HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY

SCHOOL OF ELECTRONICS AND TELECOMMUNICATION

C

Part I : Design a real application

huyên đề dành cho lớp cao học

AUTOMATIC ENVELOPES CLASSIFICATION SYSTEM
(Project : 010-2001-TCT-RDP-BC-26 at the Research Institute of Post and Telecommunication 2000-2001)
Collected envelopes

Control & Operation System Centre
control

Recognition result
(postcode)

Classification of
envelope types

Automatic
Handwritten Postcode
Recognition
Image 640x480

Standard
envelopes
CCD
camera
Separate,
p

, Index &
Put each of
envelopes
on conveyer belt

Light

My
design

control

Mechanical
System to do
classification

Conveyer belt

HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY

SCHOOL OF ELECTRONICS AND TELECOMMUNICATION

5


C

Part I : Design a real application

huyên đề dành cho lớp cao học

DESIGN THE PROCESS OF FUNCTIONS
Error (reject)

Preprocessing

Segmentation

(Neural network)

Image WxH
CCD
Camera

Light

xxxxx

Recognition

28117

Classification OK
Index i

Envelope

Multi-layers Feed-Forward
Neural Network
Conveyer belt

‹ Preprocessing

‹ Segmentation

‹ Recognition

- Grayscale convert
- Segment the postcode area
- Enhancement
- Segment
g
5 areas for
- Binary convert
each code number
- Subtraction
- Segment 5 number patterns
- Noise filter
- Find envelope
p frame
- Rotate & Crop Postcode area
HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY

- Patterns normalization
- Recognition using
a neural network.
- Check result
(OK or reject)

SCHOOL OF ELECTRONICS AND TELECOMMUNICATION

6


C

Part I : Design a real application

huyên đề dành cho lớp cao học

7

NEURAL NETWORK FOR PATTERN RECOGNITION
16x16
Input layer

Input layer : 256
Normalized
Pattern 16x16

Layer 2
16

Activation function
7
Log-Sigmoid

a=

5x5

7x7
3x3

Layer 2 : 106
(Feature map)

16

3x3

Layer 3
7

1
1 + e − n net

10

0

1

HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY

Output layer

2

3

4

5

6

7

8

9

Result

Layer 3 : 68
(Sub-sampling)
Output layer : 10
(classification)
Total : 440 neurons

SCHOOL OF ELECTRONICS AND TELECOMMUNICATION


C

huyên đề dành cho lớp cao học

Part I : Design a real application

SOFTWARE MODULE (3) : RECOGNIZING POSTCODE

HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY

SCHOOL OF ELECTRONICS AND TELECOMMUNICATION

8


C

Part I : Design a real application

huyên đề dành cho lớp cao học

SKIN COLOR DETECTOR
FOR A REAL-TIME HAND GESTURE RECOGNITION SYSTEM
2D Color
Camera

Skin color
detector

Skin color
detection
image

Hand feature extraction

Hand gesture recognition

Hand tracking
g
Human-Robot interaction
Human-Computer interaction

HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY

SCHOOL OF ELECTRONICS AND TELECOMMUNICATION

9


C

Part I : Design a real application

huyên đề dành cho lớp cao học

THE OVERVIEW OF MY COMBINING METHOD
Step 1

Step 2

Image frame

Result
PRNN

Collecting

The
statistical
t ti ti l
skin color
model

Training
Building

Training data set
+ Skin color patterns
+ Non-skin color patterns

The originality of the method:

Step 1: Using the statistical skin color model
Ỉ Fast detection with not high accuracy (coarse work)
Ỉ Check all pi
pixels
els of the image frame (coarse data)
Step 2: Using the pattern recognition neural network

Elliptical
p
skin color
model
Multi-layers FeedForward Neural
Network

ỈDetect with high accuracy (fine work)

ỈL
ỈLearn
tto correctt th
the ddetection
t ti errors off the
th statistical
t ti ti l skin
ki color
l model.
d l
HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY

SCHOOL OF ELECTRONICS AND TELECOMMUNICATION

10


C

Part I : Design a real application

huyên đề dành cho lớp cao học

DEFINE A PATTERN FOR SKIN COLOR PRNN*

Non-skin

Skin detection result based on
the color information of a pixel

?

Pattern for PRNN
with 9 pixels
Skin

Skin

Non-skin

Under examination
pixel and its
8 neighborhood pixels

* PRNN – Pattern
P
R
Recognition
i i Neural
N
l Network
N
k
HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY

SCHOOL OF ELECTRONICS AND TELECOMMUNICATION

11

C

Part I : Design a real application

huyên đề dành cho lớp cao học

TỔNG KẾT CÁC KHÁI NIỆM TRONG PHẦN I
¾ Khái niệm
iệ mẫu
ẫ ảnh
ả h trong
t
nhận
hậ dạng.
d
¾ Khái niệm tạo và chuẩn hố mẫu ảnh
¾ Khái niệm
ệ nhận
ậ dạng
ạ g
¾ Khái niệm mạng nơron và huấn luyện mạng nơron
¾ Khái niệm thư viện mẫu ảnh để huấn luyện
¾ Khái niệm huấn luyện mạng nơron

HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY

SCHOOL OF ELECTRONICS AND TELECOMMUNICATION

12