mobile image processing part3

Mobile image processing – Part 3
 Mobile application development on Android

“
Hello World
”
project

Hello

World

project
 “Viewfinder EE368” project
 Ima
g
e processin
g
librar
y
ggy
 Mobile application examples
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no. 1
Scan Life
2D barcode recognition
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no. 2

Spring 2006 Project:
Visual Code Marker Recognition
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no. 3
Image processing library for Android

 Jon’s Java Imaging Library (JJIL)
http://code google com/p/jjil/
http://code
.
google
.
com/p/jjil/
 Collection of common image processing functions
 Lookin
g
for EE368 students to au
g
ment this librar
y
with new
g
gy
image processing functions and algorithms
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no. 4
Importing JJIL as external JAR (1)
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no. 5
Importing JJIL as external JAR (2)
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no. 6
Android and JJIL image formats
 Image formats
 “Bitma
p
” class in Android
p
 “Image” class in JJIL: subclasses RgbImage, Gray8Image,

Gray16Image, Gray32Image

Conversion example

Conversion

example
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no. 7
JJIL sequences
 Images are processed through sequences (pipelines)

Sequence acts like a queue where the first image inserted

Sequence

acts

like

a

queue
,
where

the

first

image

inserted

is the first image retrieved
 Image downsampling example
Sequence
RbI
Sequence.Push
(RgbImage)
Sequence.GetFront()
Rgb
3x3Average
Rgb
SubSample
Sequence
R
g
bI
mage
RgbImage
3x3Average
SubSample
Sequence.Add
Sequence.Add
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no. 8
Sequence.Add
(new Rgb3x3Average)
Sequence.Add
(new RgbSubSample)
3x3 averaging filter (1)

Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no. 9
3x3 averaging filter (2)
Original Image
Processed Image
Original

Image
Processed

Image
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no. 10
Horizontal differences (1)
RGB to
grayscale
conversion
conversion
Horizontal
differences
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no. 11
Horizontal differences (2)
Original Image
Processed Image
Original

Image
Processed

Image
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no. 12
Vertical differences (1)

1. Transpose the image
2 Find horizontal differences on the transposed image
2
.
Find

horizontal

differences

on

the

transposed

image
3. Transpose the resulting image
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no. 13
Vertical differences (2)
Original Image
Processed Image
Original

Image
Processed

Image
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no. 14
JJIL functions

ApplyMaskRgb
BinaryHeap
Complex32Gray32
Gray8ConnComp
Gray8Crop
Gray8DeblurHorizHalftone
Gray8Peak3x3
Gray8Reduce
Gray8Shrink
Complex32Gray32
Complex32IFFT
Fft1d
Gray16Gray8
G 16Li C b
Gray8DeblurHorizHalftone
Gray8DetectHaarMultiScale
Gray8Fft
Gray8GaussDeblurHoriz
G8G Hi
Gray8Shrink
LinefitHough
Rgb3x3Average
RgbAvgGray
R bCli
G
ray
16Li
n
C
om

b
Gray16Threshold
Gray32Div
Gra
y
32Gra
y
8
G
ray
8G
auss
H
or
i
z
Gray8GaussSmoothVert
Gray8Gray32
Gra
y
8Hist
R
g
bCli
p
RgbCrop
RgbDimMask
R
g
bHorizGaussSmooth

yy
Gray32Scale2Gray8
Gray32Threshold
Gray3Bands2Rgb
Gray82Gray32
y
Gray8HistEq
Gray8HistMatch
Gray8HorizSimpleEdge
Gray8HorizSum
g
RgbHsv
RgbMaskedAbsDiff
RgbMaxContrast2Gray
RgbMaxDiff
Gray82Gray32
Gray8Abs
Gray8Add
Gray8And
Gray8HorizSum
Gray8HorizVar
Gray8HorizVertContrast
Gray8InverseFilter
RgbMaxDiff
RgbMeanStdDev
RgbMinDiff
RgbSelectGray
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no. 15
Gray8CannyHoriz Gray8LinComb RgbSubSample
…

Communications between phone and server
Server
Phone
HTTP Post (e.g., Image, Features )
Wir
e
l
ess

HTTP Response (e g Identification)
eess
Network
HTTP

Response

(e
.
g
.,
Identification)
Apache
PHP
MATLAB / C
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no. 16
HTTP file upload
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no. 17
Virtual campus tour guide
Landmark recognition
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no. 18

/>Real-time object tracking
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no. 19
/>CD/DVD/Book cover recognition
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no. 20
V3V0UP4afUk
Spring 2008 Project:
CD Cover Recognition
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no. 21
Retrieving music for recognized images
Phone
Server
HTTP Post (e.g., Image, Features )
Phone
Wireless
Network
HTTP Response (e.g., Identification)
Network
Sample MP3 files
from the web can be
sent to the phone
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no. 22
Playing music files on the phone
Context appContext = getApplicationContext();
MediaPlayer player
=
MediaPlayer.create(appContext, uri);
Uri uri = Uri.parse(“ />MediaPlayer

player

MediaPlayer.create(appContext,

uri);

player.start();
player.pause();
l kT (2000) //
p
l
ayer.see
kT
o
(2000)
;
//
msec
player.stop();
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no. 23
Retrieving videos for recognized frames
Phone
Server
HTTP Post (e.g., Image, Features )
Phone
Wireless
Network
HTTP Response (e.g., Identification)
Network
Sample video files
from the web can be
streamed to the phone

Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no. 24
Playing videos on the phone
Context appContext = getApplicationContext();
VideoView player = new VideoView(appContext);
l tVid P th(“htt // it / it / l 4”)
p
l
ayer.se
tVid
eo
P
a
th(“htt
p:
//
somes
it
e.com
/
repos
it
ory
/
samp
l
e.mp
4”)
;
player.start();
player.pause();

player.seekTo(2000); // msec
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 3 no. 25
player.stopPlayback();