giáo trình môn chuyên ngành máy học

Machine Learning
Machine Learning
Damon Waring
Damon Waring
22 April 2003
22 April 2003
2 of 15
Agenda
Agenda

Problem, Solution, Benefits
Problem, Solution, Benefits

Machine Learning Overview/Basics
Machine Learning Overview/Basics

Face detection, recognition, and demo
Face detection, recognition, and demo

How this applies to us
How this applies to us

Summary
Summary
3 of 15
Problem
Problem
Software frequently requires users or developers to do
Software frequently requires users or developers to do
simple, repetitive tasks
simple, repetitive tasks

4 of 15
Solution
Solution

Machine Learning
Machine Learning

“
“
The study of computer algorithms that
The study of computer algorithms that
improve automatically through
improve automatically through
experience” –Tom Mitchell,
experience” –Tom Mitchell,
Machine
Machine
Learning
Learning

Machine learning uses include:
Machine learning uses include:

Security (Pattern recognition, face recognition)
Security (Pattern recognition, face recognition)

Business (Stocks, user behaviors)
Business (Stocks, user behaviors)

Medical (Research)

Medical (Research)

Ease of Use (Focus of this presentation)
Ease of Use (Focus of this presentation)
Algorithms that execute based on experience
Algorithms that execute based on experience
Algorithms that execute based on experience
Algorithms that execute based on experience
5 of 15
Benefits
Benefits

Makes human-computer interaction easier
Makes human-computer interaction easier

Relatively simple to integrate
Relatively simple to integrate

Will distinguish your product from others
Will distinguish your product from others

Increase customer satisfaction
Increase customer satisfaction

Will improve simple intelligent systems (ex:
Will improve simple intelligent systems (ex:
Microsoft Word’s grammar checker)
Microsoft Word’s grammar checker)
Enhances the user experience
Enhances the user experience

Enhances the user experience
Enhances the user experience
6 of 15
High Level Operation:
High Level Operation:
Recognition Algorithms
Recognition Algorithms

Training Mode
Training Mode

Training Set
Training Set

Iteratively analyze inputs
Iteratively analyze inputs
and refine algorithm
and refine algorithm

Store learned data
Store learned data

Operation Mode
Operation Mode

New input
New input

Process input using
Process input using

learned data
learned data

Produce a decision
Produce a decision
Recognition algorithms are taught and react like humans
Recognition algorithms are taught and react like humans
Recognition algorithms are taught and react like humans
Recognition algorithms are taught and react like humans
“
“
Learn from nature. It has had 4 billion years to develop its
Learn from nature. It has had 4 billion years to develop its
techniques” – My Dad
techniques” – My Dad
7 of 15
Case Study: Artificial Neural
Case Study: Artificial Neural
Network
Network

Takes N inputs
Takes N inputs

Calculates the weight
Calculates the weight
each input has on final
each input has on final
decision
decision


Neuron outputs a 1 if
Neuron outputs a 1 if
the decision is true, 0
the decision is true, 0
if it is false
if it is false

Groups of neurons
Groups of neurons
make up an artificial
make up an artificial
neural network
neural network
Group of weighted input values determine a binary output
Group of weighted input values determine a binary output
Group of weighted input values determine a binary output
Group of weighted input values determine a binary output
8 of 15
Face Detection
Face Detection
1.
1.
Image pyramid used to locate faces of different sizes
Image pyramid used to locate faces of different sizes
2.
2.
Image lighting compensation
Image lighting compensation
3.

3.
Neural Network detects rotation of face candidate
Neural Network detects rotation of face candidate
4.
4.
Final face candidate de-rotated ready for detection
Final face candidate de-rotated ready for detection
9 of 15
Face Detection (Con’t)
Face Detection (Con’t)
5.
5.
Submit image to Neural Network
Submit image to Neural Network
a.
a.
Break image into segments
Break image into segments
b.
b.
Each segment is a unique input to the network
Each segment is a unique input to the network
c.
c.
Each segment looks for certain patterns (eyes, mouth,
Each segment looks for certain patterns (eyes, mouth,
etc)
etc)
6.
6.

Output is likelihood of a face
Output is likelihood of a face
10 of 15
Face Recognition and demo
Face Recognition and demo

Demo: Hidden Markov Model Face Recognition
Demo: Hidden Markov Model Face Recognition

Observes location of facial features with respect to
Observes location of facial features with respect to
each other
each other

Person is found through unique “fingerprint”
Person is found through unique “fingerprint”
created by distances between features
created by distances between features

Demo is from OpenCV – Intel’s open source
Demo is from OpenCV – Intel’s open source
computer vision library
computer vision library
Implementations vary widely and have different success rates
Implementations vary widely and have different success rates
Implementations vary widely and have different success rates
Implementations vary widely and have different success rates
11 of 15
Adobe Photoshop Album
Adobe Photoshop Album


Software that organizes digital pictures
Software that organizes digital pictures

Tags are dragged to each photo to categorize it
Tags are dragged to each photo to categorize it

Tagging 100’s of photos is tedious
Tagging 100’s of photos is tedious

Face recognition could automatically tag photos or
Face recognition could automatically tag photos or
replace tags altogether
replace tags altogether
Machine learning can be used to make everyday apps easier
Machine learning can be used to make everyday apps easier
Machine learning can be used to make everyday apps easier
Machine learning can be used to make everyday apps easier
12 of 15
Current Uses of ML
Current Uses of ML

DivX – Face detection
DivX – Face detection

POV-Ray – Neural Net learns memory accesses
POV-Ray – Neural Net learns memory accesses

Ancestry.com – Uses Optical Character
Ancestry.com – Uses Optical Character

Recognition to digitize newspapers
Recognition to digitize newspapers

Deep Blue Junior – Less powerful than Deep Blue,
Deep Blue Junior – Less powerful than Deep Blue,
but smarter because of Neural Networks
but smarter because of Neural Networks
13 of 15
Other Areas
Other Areas

Artificial Intelligence (AI)
Artificial Intelligence (AI)

Data Mining
Data Mining

Fuzzy Logic
Fuzzy Logic

Optical Character Recognition (OCR)
Optical Character Recognition (OCR)
14 of 15
Summary
Summary

Machine learning is possible today
Machine learning is possible today

Large amounts of research are available

Large amounts of research are available

Quality open source code available in some areas
Quality open source code available in some areas

Will require time and creativity to implement
Will require time and creativity to implement

Why do it? Makes human-computer interface
Why do it? Makes human-computer interface
simpler
simpler
15 of 15
References
References

Books
Books

Machine Learning
Machine Learning
by Tom Mitchell (
by Tom Mitchell (
/> />)
)

Web sites
Web sites

Hidden Markov Models

Hidden Markov Models
/> />
Links recommended by PCAI
Links recommended by PCAI
/> />
CMU’s research areas (scroll down): />CMU’s research areas (scroll down): />
MIT’s Media Lab: />MIT’s Media Lab: />
Computer vision links: />Computer vision links: />
Open source computer vision library (OpenCV): />Open source computer vision library (OpenCV): />
Journals
Journals

PCAI (a great industry magazine, web site is bad)-
PCAI (a great industry magazine, web site is bad)-

ScienceDirect () “Computer Vision and Image Understanding,” “Artificial
ScienceDirect () “Computer Vision and Image Understanding,” “Artificial
Intelligence,” “Neural Networks”
Intelligence,” “Neural Networks”

IEEE Proceedings () “Pattern Analysis and Machine Intelligence,” “Image Processing”
IEEE Proceedings () “Pattern Analysis and Machine Intelligence,” “Image Processing”

IEEE Papers/Proceedings referenced in this presentation
IEEE Papers/Proceedings referenced in this presentation

Hidden Markov Models (used in OpenCV Demo) “Maximum likelihood training of the embedded HMM for
Hidden Markov Models (used in OpenCV Demo) “Maximum likelihood training of the embedded HMM for
face detection and recognition.” Nefian, A.V.; Hayes, M.H. III; Image Processing, 2000. Proceedings. 2000
face detection and recognition.” Nefian, A.V.; Hayes, M.H. III; Image Processing, 2000. Proceedings. 2000

International Conference on, Volume: 1, Pages 33-36.
International Conference on, Volume: 1, Pages 33-36.

“
“
Neural network-based face detection.” Rowley, H.A.; Baluja, S.; Kanade, T; Pattern Analysis and Machine
Neural network-based face detection.” Rowley, H.A.; Baluja, S.; Kanade, T; Pattern Analysis and Machine
Intelligence, IEEE Transactions on, Volume 20 Issue 1, Jan 1998. Pages 23-38. (Paper posted at:
Intelligence, IEEE Transactions on, Volume 20 Issue 1, Jan 1998. Pages 23-38. (Paper posted at:
/> />)
)

“
“
Rotation Invariant Neural Network-Based Face Detection”
Rotation Invariant Neural Network-Based Face Detection”
/> />