mobile image processing part1

Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 1 no. 1
Mobile image processing – Part 1
 Mobile application development on Android
 Project 1: “Hello Image”, reading/writing/modifying images
 Project 2: “Viewfinder EE368”, processing viewfinder frames
 Project 3: “CVCamera”, utilizing OpenCV functions
 Mobile application examples
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 1 no. 2
/>Mobile landmark recognition
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 1 no. 3
/>Mobile product recognition
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 1 no. 4
Popular mobile platforms
Symbian
Apple iOS
Windows 7
Nokia N8 HTC Arrive Samsung Focus
iPhone
iPad Motorola Droid
Nokia N97
Android
Samsung
Galaxy Tab
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 1 no. 5
Spotlight on Android
 Open source mobile platform developed by Google
 Supported by Open Handset Alliance
 13 mobile operators
 18 handset manufacturers
 20 semiconductor companies
 16 software makers

 Uses an open-source, customizable Linux kernel and a
virtual machine designed for mobile hardware
 Largest share of smartphone market in the US
 Android Market now contains over 250k apps
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 1 no. 6
Android software stack
[ Professional
Android
Application
Development ]
Linux Kernel
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 1 no. 7
Programming in Java
 Android encourages high-level development
 Android uses Java as the main programming language
 Inherits basic classes from conventional Java
 String, Container, Math, IO, Network
 Adds new classes specific to mobile devices
 Camera, Telephony, Map, GPS, Speech
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 1 no. 8
Android programming with Eclipse
Eclipse
IDE
Android
SDK
Java
Runtime
program installation
and execution
system

messages
programmer
inputs
data and
messages
Internet
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 1 no. 9
Class tutorials for Android (1)
 Android tutorials designed specifically for mobile image
processing in the EE368 class
 Tutorial #1: Basic Android Setup
 Image processing-oriented introduction to Android
 Explains how to download and install the different software
packages (JDK, SDK, Eclipse) on your own computer
 Shows how to build and run a viewfinder app that computes color
histograms in real time
 Tutorial #2: OpenCV for Android Setup
 Builds on core skills developed in Tutorial #1
 Explains how to download and install OpenCV for Android
 Shows how to build a more advanced viewfinder app that computes
various image feature keypoints supported by OpenCV
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 1 no. 10
Class tutorials for Android (2)
/>Support for Windows, Macintosh,
and Linux in both tutorials
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 1 no. 11
Eclipse IDE
Project files
Class
hierarchy

Text editor
Errors and
warnings
Different
perspectives
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 1 no. 12
Structure of an Android Project in Eclipse
Programmer-defined Java files
Program new functions here
Auto-generated Java files
Don’t edit anything here
Android library
Don’t edit anything here
Resource files
Edit layout, define constants,
import external media files,
change program permissions
Java Native Interface files
C/C++ code
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 1 no. 13
“Hello Image” project
 Goals of this project
 Learn how to create a new Android project
 Learn how to read/write images from/to SD card
 Learn how to access and modify image bitmaps
 Full source available on class website:
 />Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 1 no. 14
Recommended project settings (1)
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 1 no. 15
Recommended project settings (2)

Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 1 no. 16
HelloImage Activity class (1)
public class HelloImage extends Activity {
Bitmap myBitmap;
ImageView myImageView;
public void onCreate(Bundle savedInstanceState) { … }
private void processImage() { … }
private void brighten(int offset) { … }
}
Subclass of Activity class
will be visible on the phone
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 1 no. 17
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.main);
String imageFileName = “/sdcard/test_vga.jpg”;
File imageFile = new File(imageFileName);
if (imageFile.exists()) {
// Load the image from file
myBitmap = BitmapFactory.decodeFile(imageFileName);
// Display the image in the image viewer
myImageView = (ImageView)findViewById(R.id.my_image_view);
if (myImageView != null) {
myImageView.setImageBitmap(myBitmap);
}
}
}
HelloImage Activity class (2)
onCreate called when the
application first starts

Load image
from SD card
Display image
in viewer
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 1 no. 18
private void processImage() {
brighten(50);
try {
String outputPath = "/sdcard/test_vga_output.jpg";
int quality = 75;
FileOutputStream fileOutStr = new FileOutputStream(outputPath);
BufferedOutputStream bufOutStr =
new BufferedOutputStream(fileOutStr);
myBitmap.compress(CompressFormat.JPEG, quality, bufOutStr);
bufOutStr.flush();
bufOutStr.close();
} catch (FileNotFoundException exception) {
Log.e("debug_log", exception.toString());
} catch (IOException exception) {
Log.e("debug_log", exception.toString());
}
myImageView.setImageBitmap(myBitmap);
}
HelloImage Activity class (3)
Apply some image processing
operation on the stored bitmap
Save the processed
image to the SD card
Catch exceptions
Show the image on the display

Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 1 no. 19
private void brighten(int brightenOffset) {
// Create new array
int width = myBitmap.getWidth();
int height = myBitmap.getHeight();
int[] pix = new int[width * height];
myBitmap.getPixels(pix, 0, width, 0, 0, width, height);
// Apply pixel-by-pixel change
int index = 0;
for (int y = 0; y < height; y++) { for (int x = 0; x < width; x++) {
int r = (pix[index] >> 16) & 0xff;
int g = (pix[index] >> 8) & 0xff;
int b = pix[index] & 0xff;
r = Math.max(0, Math.min(255, r + brightenOffset));
g = Math.max(0, Math.min(255, g + brightenOffset));
b = Math.max(0, Math.min(255, b + brightenOffset));
pix[index++] = 0xff000000 | (r << 16) | (g << 8) | b;
} // x } // y
// Change bitmap to use new array
myBitmap = Bitmap.createBitmap(width, height, Bitmap.Config.RGB_565);
myBitmap.setPixels(pix, 0, width, 0, 0, width, height);
}
HelloImage Activity class (4)
Extract RGB
Values
Pack RGB
Values
Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 1 no. 20
Running the program on an Android device
0123456789ABC

Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 1 no. 21
“Hello Image” application running on device
/>Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 1 no. 22
Class Project: Plant Leaf Classification
D. Knight, J. Painter, and M. Potter, Spring 2010
/>Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 1 no. 23
Class Project: Jigsaw Puzzle Solver
L. Liang and Z. Liu, Spring 2010
/>Bernd Girod, David Chen: EE368 Mobile Image Processing – Part 1 no. 24
Mobile image processing – Part 1
 Mobile application development on Android
 Project 1: “Hello Image”, reading/writing/modifying images
 Project 2: “Viewfinder EE368”, processing viewfinder frames
 Project 3: “CVCamera”, utilizing OpenCV functions
 Mobile application examples