The effective transmission and processing of mobile multimedia

THE EFFECTIVE
TRANSMISSION AND PROCESSING
OF MOBILE MULTIMEDIA
MA HAIYANG
B.E., WUHAN UNIVERSITY, CHINA
A THESIS SUBMITTED
FOR THE DEGREE OF DOCTOR OF
PHILOSOPHY
Graduate School for Integrative Sciences and
Engineering
NATIONAL UNIVERSITY OF SINGAPORE
2014
Declaration
I hereby declare that this thesis is my original work and it has been
written by me in its entirety. I have duly acknowledg ed all the sources of
information which have been used in the thesis.
This thesis has also not been submitted for any degree in any university
previously.
a
c
 2014
MA Ha iyang
All Rights Reserved
Dedication
This thesis is dedicated to
my beloved parents,
Ma Wenke and Liu Qiaoyun,
who raised me and keep supporting me throughout my whole life.
c
Acknowledgements
This thesis is the outcome of five years of research work during which I

have been accompanied and suppo r ted by many people. Without them,
the completion of my thesis would not b e possible. I am honored to take
this opportunity to thank them.
First, I would like to express my sincere gratitude to Prof. Roger Zim-
mermann for his consist ent support and illuminating guidance during my
PhD study. His rigorous attitude on research helped me develop a sci-
entific and systematic thinking which is critical to problem-solving. His
wholehearted encourag ement helps me overcome many obstacles I o nce felt
insurmountable. I feel extremely proud to have started and spent my PhD
study under his supervision.
My heartfelt thanks go to Dr. Deepak Gangadharan, Dr. Hao Jia and
Wang Guanfeng with whom I have collaborated during my PhD research.
I have benefited a lot from their technical insights, as they help me to
analyze and solve a problem from different perspectives.
I would also like to tha nk NGS, the Graduate School for Integrative
Sciences and Engineering of National University of Sing apore for pr oviding
me the opportunity to do docto ral research in a distinguished university
with financial support. The PhD study in NUS has opened up a new door
in my life.
In t he end, I want to express my appreciation t o the companion from
my dear colleagues: Liang Ke, Hao Jia, Ma He, Shen Zhij ie, Zhang Ying,
Zhang Lingyan, Fang Shunkai, Cui Weiwei, Wang Guanfeng and Yin Yifang
in the Media Management Research Lab.
d
Publications
Peer Reviewed
• Deepak Gangadharan, Haiyang Ma, Samarjit Chakraborty, Roger
Zimmermann. Video Quality Driven Buffer Dimensioning via Prio r i-
tized Frame Drops. In IEEE International C onference on Com puter
Design (ICCD), October 2011.

• Haiyang Ma, Deepak Gang adha r an, Nalini Venkatasubramanian, Roger
Zimmermann. Energy-aware Complexity Adaptation for Mobile Video
Calls. In Proceedings of the 19
th
annual ACM International Confer-
ence on Multimedia (ACM MM), November 2011.
• Guanf eng Wa ng, Haiya ng Ma, Beomjoo Seo, Roger Zimmermann.
Sensor-Assisted Camera Motion Analysis and Motion Estimation Im-
provement for H.264/AVC Video Encoding. In ACM Workshop on
Network and Ope rating Systems Support for Digital Audio and Video
(NOSSDAV), June 2012.
• Haiyang Ma
, Roger Zimmermann. Adaptive Coding with Energy
Conservation for Mobile Video Calls. In IEEE International Conf er-
ence on Multimedia and Expo (ICME), July 2012.
• Haiyang Ma
, Roger Zimmermann. Energy Conservation in 802.1 1
WLAN for Mobile Video Calls. In IEEE International Symposium
on Multimedia (ISM), December 2012.
• Jia Hao, Roger Zimmermann, Haiyang Ma
. GTube: Geo-Predictive
Video Streaming over HTTP in Mobile Environments. In the 5
th
AC M Multimedia Systems Confe rence (ACM MMSys), March 2014.
• Haiyang Ma
, Jia Hao, Roger Zimmermann. Access Point Centric
Scheduling for DASH Streaming in Multirate 802.11 Wireless Net-
work. In IEEE International Conference on Multi media and Expo
(ICME), July 2014.
e

CONTENTS
Summary vi
List of Figures ix
List of Tables xii
1 Introduction 1
1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Motivation: An Energy Aware and Bandwidth Efficient Mul-
timedia System . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.3 Research Work and Contributions . . . . . . . . . . . . . . . 5
1.3.1 Workload Complexity Reduction of MPEG-4 on Mo-
bile Platforms . . . . . . . . . . . . . . . . . . . . . . 5
1.3.2 Energy Efficient Mobile Call Framework with Ada p-
tive Coding of H.2 64 . . . . . . . . . . . . . . . . . . 6
1.3.3 Adaptive Packet Transmission Scheme for Mobile Video
Calls . . . . . . . . . . . . . . . . . . . . . . . . . . .
7
1.3.4 Access Point Centric DASH Scheduling in Multirate
802.11 Wireless Networks . . . . . . . . . . . . . . .
9
1.4 Organization . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2 Literature Review 11
2.1 Video Coding Scalability . . . . . . . . . . . . . . . . . . . . 11
i
CONTENTS
2.1.1 Decoding Workload Adaptation . . . . . . . . . . . . 13
2.1.2 Encoding Workload Adaptation . . . . . . . . . . . . 14
2.1.3 Hardware-Assisted Coding . . . . . . . . . . . . . . . 16
2.1.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . 17
2.2 Hardware Energy Conservation . . . . . . . . . . . . . . . . 17
2.2.1 CPU . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.2.2 Network Interface Card . . . . . . . . . . . . . . . . . 19
2.2.3 Graphical Display . . . . . . . . . . . . . . . . . . . . 20
2.2.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . 22
2.3 Energy-Optimized Multimedia Systems . . . . . . . . . . . . 22
2.3.1 Cross-Layer Adaptive Coding Framework . . . . . . . 22
2.3.2 Computation Offloading to the Cloud . . . . . . . . . 23
2.3.3 Server and Middleware-Assisted Rat e Adaptation . . 24
2.3.4 Error-Resilient Coding and Transmission . . . . . . . 24
2.3.5 Power-Aware 802.11 WLAN Design . . . . . . . . . . 26
2.3.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . 26
2.4 Quality Adaptation in HTTP Streaming . . . . . . . . . . . 27
2.4.1 Architecture of DASH Streaming . . . . . . . . . . . 27
2.4.2 Client-side Approaches . . . . . . . . . . . . . . . . . 28
2.4.3 Server-side Approaches . . . . . . . . . . . . . . . . . 30
2.4.4 Intermediary Approaches . . . . . . . . . . . . . . . . 30
2.4.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . 31
3 Workload Complexity Reduction of MPEG-4 on Mobile
Platforms
33
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.2 System Overview . . . . . . . . . . . . . . . . . . . . . . . . 34
3.3 Complexity Scalability of MPEG-4 . . . . . . . . . . . . . . 35
3.3.1 Profiling Environment . . . . . . . . . . . . . . . . . 3 5
3.3.2 Encoder Adaptation . . . . . . . . . . . . . . . . . . 36
3.3.3 Decoder Adaptation . . . . . . . . . . . . . . . . . . 40
3.4 Metrics for System QoS and Power Model . . . . . . . . . . 42
3.4.1 Methodology for Real-time Performance Monitoring . 43
3.4.2 Power Model . . . . . . . . . . . . . . . . . . . . . . 45
3.5 Algorithms for Adaptive System . . . . . . . . . . . . . . . . 45
ii

CONTENTS
3.5.1 Coding Module . . . . . . . . . . . . . . . . . . . . . 45
3.5.2 Feedback Module . . . . . . . . . . . . . . . . . . . . 46
3.6 Exp erimental Evaluation . . . . . . . . . . . . . . . . . . . . 4 9
3.6.1 Exp erimental Setup . . . . . . . . . . . . . . . . . . . 49
3.6.2 Parameter Calculation for Energy Model . . . . . . . 50
3.6.3 Exp erimental Result . . . . . . . . . . . . . . . . . . 50
3.7 Overhead Computation . . . . . . . . . . . . . . . . . . . . . 53
3.8 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
4 An Energy Efficient Mobile Call Framework with Adaptive
Coding of H.264
55
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
4.2 Complexity Scalability of H.264 . . . . . . . . . . . . . . . . 56
4.2.1 Complexity Adaptation of H.264 Encoder . . . . . . 57
4.2.2 Complexity Adaptation of H.264 Decoder . . . . . . 63
4.3 System Desig n . . . . . . . . . . . . . . . . . . . . . . . . . . 64
4.3.1 Derivation of Buffer Limit . . . . . . . . . . . . . . . 64
4.3.2 Adaptation Workflow . . . . . . . . . . . . . . . . . . 65
4.4 Exp eriments . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
4.4.1 Exp erimental Setup . . . . . . . . . . . . . . . . . . . 66
4.4.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . 66
4.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
4.5.1 Hardware-assisted Coding . . . . . . . . . . . . . . . 67
4.5.2 Applicability to Other Codecs . . . . . . . . . . . . . 68
4.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
5 Adaptive Packet Transmission Scheme for Mobile Video
Calls 70
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
5.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

5.2.1 Power Save Mode . . . . . . . . . . . . . . . . . . . . 71
5.2.2 IEEE 802.11e . . . . . . . . . . . . . . . . . . . . . . 72
5.3 Transmission Analysis . . . . . . . . . . . . . . . . . . . . . 72
5.3.1 State Transitions under Dynamic PSM . . . . . . . . 72
5.3.2 Delay In Video Calling . . . . . . . . . . . . . . . . . 74
5.4 Transmission Schedule Design . . . . . . . . . . . . . . . . . 77
iii
CONTENTS
5.4.1 Session Establishment . . . . . . . . . . . . . . . . . 77
5.4.2 Exchange of Execution Condition . . . . . . . . . . . 77
5.4.3 Estimation of Network L atency . . . . . . . . . . . . 78
5.4.4 Making Transmission Decisions . . . . . . . . . . . . 79
5.5 Exp erimental Evaluation . . . . . . . . . . . . . . . . . . . . 8 2
5.5.1 Exp erimental Setup . . . . . . . . . . . . . . . . . . . 82
5.5.2 Processing of Video Packets . . . . . . . . . . . . . . 83
5.5.3 Evaluation Criteria . . . . . . . . . . . . . . . . . . . 84
5.5.4 Exp erimental Results . . . . . . . . . . . . . . . . . . 8 5
5.5.5 Overhead Measurement . . . . . . . . . . . . . . . . . 89
5.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
6 Access Point Centric Scheduling for HTTP Streaming in
Multirate 802.11 Wireless Networks
91
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
6.2 Fair Queuing in Wireless Network . . . . . . . . . . . . . . . 92
6.3 System Desig n . . . . . . . . . . . . . . . . . . . . . . . . . . 93
6.3.1 Info Collector . . . . . . . . . . . . . . . . . . . . . . 93
6.3.2 Packet Scheduler . . . . . . . . . . . . . . . . . . . . 94
6.3.3 URL Redirector . . . . . . . . . . . . . . . . . . . . . 94
6.4 Exp eriments . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
6.4.1 Exp erimental Setup . . . . . . . . . . . . . . . . . . . 100

6.4.2 Evaluation Metrics . . . . . . . . . . . . . . . . . . . 102
6.4.3 Type of DASH Clients . . . . . . . . . . . . . . . . . 103
6.4.4 Exp erimental Results . . . . . . . . . . . . . . . . . . 104
6.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
6.5.1 Layer ing Principle . . . . . . . . . . . . . . . . . . . 113
6.5.2 End-to-end Principle . . . . . . . . . . . . . . . . . . 114
6.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
7 Conclusions 116
7.1 Summary of Research Techniques . . . . . . . . . . . . . . . 116
7.2 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . 118
7.3 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
7.4 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
iv
CONTENTS
Bibliography 123
v
CONTENTS
Summary
In recent years we have witnessed an explosively increasing presence of mo-
bile devices in people’s daily lives. The progress in fabrication techniques
of mobile chips and the widespread deployment of advanced wireless trans-
mission technologies have successfully transformed a mobile device into a
portable mult imedia entertainment hub. Killer applications such as HD
video streaming and video calling, once considered only possible on PCs
due to their demanding processing requirement, now have become very
popular on mobile platforms such as smartphones and tablets, etc. More-
over, in recent years mo bile devices are increasingly utilized as a “second
screen” to provide an enhanced viewing experience to content on TV and
monitors.
However, mobile platforms are severely constrained by two factors, en-

ergy and bandwidth. Over the recent decades, the capacity of batteries
did not enjoy a growth rate proportional to the ra pid improvement of the
processing capabilities of the mobile devices. What is worse, a battery
can drain very fast for mobile video calls, a s they r equire the simultaneous
running of both an encoder and a decoder that entails a nearly full-speed
execution of many power-hungry hardware components such as graphical
display, CPU and network interface card, etc. As a consequence, the user
satisfaction will be severely degraded as a result of a limited service dura-
tion.
Mobile platforms also suffer from the scarcity and fast varying quality
of the network bandwidth and it is more challenging to ensure a sustained
quality of service in a wireless environment, compared to a wired one. It
has long been a heated research area on the efficient utilization and fair
allocation of the limited bandwidth resources in a wireless network, and the
popularity of HTTP streaming in recent years has been requesting for an
effective scheduling solution for the bandwidth distribution among different
types of clients.
To improve the multimedia consumption experience on mobile platforms
in spite of energy and bandwidth constraints, in this thesis we establish
an energy aware and bandwidth efficient multimedia system. Specifically,
we target two popular applications, video calling and HTTP streaming
and propose an effective transmission and processing framework for mo -
vi
CONTENTS
bile multimedia. We investigate and propose several energy conservation
schemes targeting the CPU and wireless network interf ace card to r educe
the power consumption during mobile video calls. We also design an Access
Point centric scheduler for HTTP streaming in mult ir ate WiFi networks to
achieve a fair and efficient bandwidth distribution scheme among streaming
clients. Our work can be roughly categorized as follows:

1) Workload Complexity Reduction of MPEG-4 on Mobile Pl atform s.
We present a detailed offline profiling and analysis for the workload of
MPEG-4 (MPEG-4 Part 2). Based on the analysis, we propose several
discrete coding sets by combining the most efficient coding parameters, in
terms of workload and output quality, for both the encoder and decoder.
A framework has been develop ed that dynamically selects the coding set
and applies Dynamic Voltage and Frequency Scaling (DVFS) to reduce the
energy consumptio n on CPU while ensuring an acceptable coding quality.
2) Energy Efficient Mobile Call Framework with Adaptive Coding of
H.264. For H.264 which has a much higher coding complexity and larger
parameter space than MPEG-4, we utilize the texture similarities between
spatially and temporally adjoining macroblocks for workloa d reduction.
The control of the quality-complexity tradeoff is unified through the tuning
of a single parameter adaptive to the execution environment. To satisfy
the short latency requirement imposed by interactive communications, we
derive a dynamic upper bound for the encoder buffer by feeding back the
execution conditio ns of both calling par ticipants.
3) Adaptive Packet Transmission Scheme for Mobile Video Calls. We
design an RTP packet transmission scheme for mobile video calls with
delay-sensitive multimedia traffic. We utilize the dynamic Power Save
Mode (PSM) widely available in the current WiFi deployments by aggre-
gating the available queuing time for each packet , so t hat considerable
energy can be saved on the WiFi network interface card (NIC).
4) Acces s Point Centric DASH scheduling in Multirate 802.11 Wireless
Networks. We propose the design o f a cross-layer AP (Access Point) centric
streaming scheduler for DASH (Dynamic Adaptive Streaming over HTTP)
in multirate 802.11 wireless networks. Residing at the AP, the scheduler
achieves proport ional fairness at the packet level by implementing weighted
fair queuing. At the request level, the scheduler uses URL redirection to
vii

CONTENTS
modify the bitrate version requested by the client when necessary to reduce
playback freezes and quality fluctuations.
Our work demonstrates that the proposed framework effectively com-
bats the two primary constraints, energy and ba ndwidth, on mobile plat-
forms. It can reduce the p ower consumpt ion of mobile devices and provide a
fair and effective bandwidth allocation scheme in wireless networks. There-
fore users can achieve a hig h level of satisfaction for multimedia services
on mobile platforms.
viii
LIST OF FIGURES
1.1 Estimation of global mobile tra ffic per month by Cisco [22]. . 2
2.1 The processing flow of an H.264 video encoder. . . . . . . . . 11
2.2 The processing flow of an H.264 video decoder. . . . . . . . . 12
2.3 Architecture of the DASH streaming. . . . . . . . . . . . . . 28
3.1 System framework for mobile video calls. The white blocks
show the coding module and the solid arrows show its work-
flow. The blue blocks show the proposed feedback module
with its workflow indicated in dashed arrows. . . . . . . . . .
34
3.2 The adoption percent ages of coding modes a mong all mac-
roblocks with regard to Motion Level (MotionL). The upper
graph is for P frames while the lower graph is for B frames. .
38
3.3 ∆SAD ver sus ∆COUNT relationship for video football in
different encoding sets. Asterisks aligning around the fitting
curve have the highest utility values. . . . . . . . . . . . . .
39
3.4 Workload Reductio n ∆W orkRed and Relat ive Quality ∆Rel Q
for diff erent decoding sets. . . . . . . . . . . . . . . . . . . .

42
3.5 Relative Quality Loss QL
bd
with regard to fr ame size ratio
S
B
+S
D
S
I
due to B frame discard. . . . . . . . . . . . . . . . . .
44
3.6 Comparison of system performance at fixed frequency (Fig-
ures a, b and c are for resolution 640 × 360. Figures d, e and
f are for resolution 640 × 480. . . . . . . . . . . . . . . . . .
51
ix
LIST OF FIGURES
3.7 Dynamic energy consumption of CPU with DVFS. . . . . . . 53
4.1 Example of a ma cro block partition in H.264. . . . . . . . . . 59
4.2 BTC of the 6th frame of video football. . . . . . . . . . . . . 59
4.3 Illustration of weights fo r neighboring macroblocks. . . . . . 61
4.4 Motion Estima t ion Workload and PSNR Loss as a function
of α. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
4.5 Encoder frame drop rate during the video call. . . . . . . . . 67
4.6 PSNR of encoded videos during the video call. . . . . . . . . 68
4.7 Encoder queuing time during the video call. . . . . . . . . . 68
4.8 Overall energy consumption of CPU during the video call. . 69
5.1 WiFi interface state transitions in adaptive PSM. . . . . . . 73
5.2 Delay components of video call frames under dynamic PSM. 74

5.3 RTT calculations in WiFi CAM (a) and PSM (b) states. . . 78
5.4 Flow chart of transmission decisions. . . . . . . . . . . . . . 80
5.5 Transmission and reception time of audio RTP packets from
10 s to 12 s. . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
5.6 Cumulative Distribution Function (CDF) of sleep duration
with 100 ms timeo ut. . . . . . . . . . . . . . . . . . . . . . . 87
5.7 Packet loss rate, miss rate and play jitter wit h variable timeout. 88
6.1 Architecture of AP centric scheduling in a DASH streaming
system. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
6.2 Workflow of the URL Redirector. . . . . . . . . . . . . . . . 96
6.3 Initial positions of the AP and the clients. . . . . . . . . . . 102
6.4 Downloaded chunk bitrates of each client in Scena rio 1: dif-
ferent starting time and st ationary clients. . . . . . . . . . .
105
6.5 Average received bitrates for each DASH client in Scenario 1. 106
6.6 Total playback freeze for clients in Scenario 1. . . . . . . . . 108
6.7 MAC-layer throughput of AP in Scenario 1. . . . . . . . . . 108
6.8 Average received bitrates for each DASH client in Scenario 2. 109
6.9 Downloaded chunk bitrates of each client in Scenario 2: sta-
tionary and moving clients. . . . . . . . . . . . . . . . . . . . 110
6.10 To t al playback freeze for clients in Scena r io 2. . . . . . . . . 112
6.11 MAC-layer throughput of AP in Scenario 2. . . . . . . . . . 112
x
LIST OF FIGURES
6.12 Download history of each client in Scenario 3: mixed traffic. 114
6.13 MAC-layer throughput of AP in Scenario 3. . . . . . . . . . 115
xi
LIST OF TABLES
2.1 Energy consumption for different parts of Nokia N95 [92]. . . 18
3.1 Available encoding options. . . . . . . . . . . . . . . . . . . 39

3.2 Selected encoding sets for adaptation. . . . . . . . . . . . . . 40
3.3 Dynamic power para meters for a T9600 CPU [18][32]. . . . 50
3.4 Performance comparison at resolution 640 × 360 with fixed
frequency. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
52
3.5 Performance comparison at resolution 640 × 480 with fixed
frequency. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
3.6 Performance comparison at 400kbps with adaptive frequency. 54
4.1 Partition Level, Coding Modes and Base Texture Complexity. 58
4.2 Comparison of performance metrics between non-adaptive
(N) a nd adaptive (A) approaches. . . . . . . . . . . . . . . .
67
5.1 Parameters adopted for transmission. . . . . . . . . . . . . . 83
5.2 Specificatio ns of AR5008. . . . . . . . . . . . . . . . . . . . . 83
5.3 Parameters adopted for audio and video codec. . . . . . . . . 83
5.4 Performance comparison between adaptive a nd normal trans-
mission with a 100 ms timeo ut. . . . . . . . . . . . . . . . .
86
5.5 Sleeping conditions and energy consumptions between dif-
ferent timeout values. . . . . . . . . . . . . . . . . . . . . . .
88
6.1 Parameters in the simulation for DASH streaming. . . . . . 101
xii
LIST OF TABLES
6.2 Bitrate versions (Mb/ s) of the media resources. . . . . . . . 101
6.3 Jain’s Index for bandwidth allocation in Scenario 1. . . . . . 106
6.4 Performance Table for clients in Scenario 1. . . . . . . . . . 107
6.5 Performance Table for clients in Scenario 2. . . . . . . . . . 111
6.6 Performance Table for DASH clients in Scenario 3. . . . . . 113
xiii

CHAPTER 1
Introduction
1.1 Background
Mobile devices are emerging as portable multimedia entertainment hubs
that have completely revolutionized people’s daily lives. There is an ob-
served trend that the annual shipment of mobile devices, including smart-
phones, tablets, etc., keeps increasing at a steady rate while the market
share for the traditional PC shrinks, as demonstrated in Gartner’s pre-
diction [
21]. There are several contributing reasons for this trend. First,
the progress in t he chip design and the fabrication techniques, together
with the maturity of highly integrated system on chip (SoC) solutions,
has greatly lowered the manuf acturing cost of mobile devices. Second,
the widespread deployment of advanced wireless transmission technologies,
such as WiFi, 3G, 4G, etc., have established a seamless global communi-
cation network that enables people to stay connected on the go. Third,
the latest generation of mobile operating systems such as iOS and An-
droid pr ovide consumers with a huge collection of mobile apps. These apps
give consumers unique experiences unheard of in the PC era, by taking
advantage of touch-based interaction and various hardware components in-
tegrated into the mobile devices. Figure
1.1 illustrates the prediction of
global mobile data traffic per month by Cisco [
22] over a time span of 5
years, which clearly shows t hat mobile traffic is expected to increase nearly
1
CHAPTER 1. INTRODUCTION
eleven fold between 2013 to 2018.
Figure 1.1: Estimation of globa l mobile tra ffic per month by Cisco [22].
Video calls, as a convenient communication means, was tradit ionally

considered a “killer” application only suitable for PCs or executed with
dedicated hardware components, because of its demanding requirement
on bandwidth and processing capability. Thanks to the technological ad-
vances, it is gaining great popularity o n mobile platfo r ms in recent years,
where is termed “Mobile VoIP” or “Mobile Video Calls”. Various apps
featuring mobile video call, such as Apple’s Facetime, Google’s Hangouts,
Microsoft’s Skype, etc., have successfully established a considerable user
base. In a report published by Juniper Research [
20], mobile video calling
users are expected to exceed 130 million by 2016 .
However, mo bile devices are powered by batteries, and the growth rate
of battery capacity per volume (weight) has been far lagging behind the
expansion of the mobile platform processing capabilities over the years.
What is worse, compared to other applications, the constrained capacity
has a more undesirable influence on mobile video call, as the latter requires
the simultaneous running of both an encoder and a decoder that entails a
nearly full-speed execution of many power-hungry hardware modules such
as camera, graphical display, CPU and network interface card, etc. As a
consequence, a user engaged in a video call would possibly be forced to
terminate halfway through as the battery drains very fast and the user
finally gets frustrated by the limited service duration. Recent trends show
that mor e and more mobile devices are designed with a slim body and a
2
CHAPTER 1. INTRODUCTION
large display screen. This leaves only space for a tiny and compact battery,
aggrava t ing the incompatibility between the growing mobile processing de-
mands and the constraint of battery capacity in the coming years.
On-demand video streaming, similar to video calls, is another domi-
nant and pervasive application on mobile platforms. According to a Cisco
report [

22] it is estimated that over two-thirds of t he world’s mobile data
traffic will be video by 2018. Of the various video streaming formats
and specificatio ns, MPEG-DASH [
23] (Dynamic Adaptive Streaming over
HTTP) streaming has been gaining great po pula r ity in r ecent years and var-
ious co mmercial standards and implementations have been launched, such
as Apple’s HTTP Live Streaming ( HLS)
1
, Micro sof t ’s Smooth Streaming
2
and Adobe’s HTTP Dynamic Streaming
3
. Companies like Hulu and Net-
flix are also using DASH for over- t he- t op (OTT) streaming services, which
refers to the delivery of multimedia content over the internet wit hout the
invo lvement of an operator for the cable or satellite broadcast television
system.
Mobile video streaming gained popularity with the founding of the
video-sharing website YouTube in 2005 and the introduction of the iPhone
in 2007, which at that time only supported 2G networks. Tra ditional mul-
timedia streaming services require the deployment of specifically designed
streaming servers and client players, combined with application layer trans-
mission protocols (Real-time Transport Pr otocol (RTP) [
14], Real Time
Messaging Protocol (RTMP) [
19] for Flash video
4
, etc.) as well as control
protocols (Real Time Streaming Protocol (RTSP) [9], Session Description
Protocol (SDP) [

17], etc). DASH, on the other hand, encapsulates multi-
media content into HTTP segments and transmits them using the HTTP
protocol. With a simple configuration to existing HTTP servers, DASH
enjoys extremely easy deployment and firewall tr aversal. Like usual web
pages, the DASH traffic can be replicated at content delivery networks
(CDN) and ca ched at gateways for fa ster access. Because of the trans-
mission reliance guaranteed by TCP, DASH clients are less vulnerable to
complex packet loss handling overhead and DASH streaming is expected
1
/>2
http:/ /www.microsoft.com/silverlight/smoothstrea ming
3
http:/ /www.adobe.com/ products/hds-dyna mic-streaming.html
4
http:/ /www.adobe.com/ de vnet/video.html
3
CHAPTER 1. INTRODUCTION
to be a primary traffic pattern in the foreseeable future.
However, for years in the industry the quality instability o f video
streaming has long been a daunting problem, which is intimately corre-
lated to the quality instability of the servicing network. In par ticular,
mobile video streaming is a severe sufferer of this problem due to t he fre-
quent instability of the wireless network environments. It has long been a
heated research area to improve the efficient utilization and fair a llocation
of the limited bandwidth resources in a wireless network.
Another issue wor t h noting is that, multimedia packets are inherently
delay-sensitive but error-tolerant to some degree. DASH, on the other
hand, is transmitting them on top of TCP, which assumes the payload to
be delay-tolerant and error-sensitive. As a consequence of this mismatch,
DASH clients are vulner able to playba ck freezes as they are forced to wait

for re-transmissions or late-arriving packets and this can ea sily result in
playback freezes due to buffer underruns. Thus a large playout buffer and
long initial buffering time are required for smooth playing as TCP provides
no delay guarantees.
Even mo re challenging, DASH depends on the estimation of the throug h-
put at the application layer for bandwidth estimation and bitrate adjust-
ment at the client or server side, which is laid over TCP and subject t o
various network congestion control mechanisms. As a result, the estimation
can be over-sensitive or sluggish, and may not truly reflect the underlying
bandwidth chang es in the wireless network, which is subject to fading and
interference, etc. Furthermore, as is observed by Akhshabi et al. [
25] in
wired networks, if multiple DASH clients are sharing a LAN network, the
router or the gateway becomes the bottleneck and the bitrate allocated to
each client can be seriously affected by the stream st arting time as well
as the interplay between the different rate adaptation logics adopted by
each client. As a result, it becomes a non-trivial task to ensure a fair
bandwidth allocat ion and achieve a good QoE (Quality of Experience) for
DASH clients in a wireless network.
4
CHAPTER 1. INTRODUCTION
1.2 Motivation: An Energy Aware and Band-
width Effic ient Multimedi a System
Following the introduction, it can be observed that energy and bandwidth
have become two significant resource constraints that greatly influence the
user experience on mobile platforms. Our work in this thesis is therefore
aimed at providing a solution, an energy aware and bandwidth efficient
multimedia system, given the limited capacity of batteries and the unfair
distribution of bandwidth for several different mobile applications. Specif-
ically, we t arget two applications scenarios, video calling and streaming.

Given the limit ed battery capacity that powers mobile devices, video call-
ing on mobile plat forms requires an effective energy conservation solution
so as to extend the servicing time. The popularity of HTTP streaming in
recent years has been the driving force for the investigation of an effective
scheduling solution for the bandwidth allocation and distribution among
different types of clients in a wireless network.
1.3 Research Work and Co ntributions
In this thesis we focus on the coding and transmission of video streams on
wireless mobile platforms. We establish an energy efficient video calling
framework on mobile pla t forms primarily t hr ough reducing energy con-
sumption of the CPU and wireless network interface card (WNIC). For
HTTP streaming in multir ate 802.11 wireless networks, we present an Ac-
cess Point (AP) centric scheduler that schedules packet tra nsmissions and
distributes available bandwidth to each DASH client. We present a more
detailed explanation of our contributions in the following subsections.
1.3.1 Workload Complexity Reduction of MPEG-4
on Mobile Platforms
Of the various embedded hardwar e components, the CPU plays an essential
and pivotal role for the normal functioning of a mobile device and therefore
is a primary energy consumer, esp ecially during the execution of intensive
workloads such as video coding. As a power ma nagement technique, Dy-
5
CHAPTER 1. INTRODUCTION
namic Voltage and Frequency Scaling (DVFS) is usually applied to adjust
the frequency and voltage of a CPU and we can take advantage of DVFS to
reduce CPU power consumption. However, this comes with reduced work
that can be completed in unit time, which could have a detrimental impact
on the quality of t ime-sensitive video coding jobs.
To solve this problem, we present a detailed offline profiling and com-
plexity analysis for the coding workload of a popular video codec, MPEG-4

(MPEG-4 Part 2) on different videos [79]. Based on the analysis, we pro-
pose several discrete coding sets for videos of different motion levels, by
combining the most utility aware coding parameters for both the encoder
and decoder. A framework has been developed that dynamically selects t he
coding set and a pplies DVFS to reduce the energy consumption on CPU
while ensuring an accept able coding quality.
The contributions of this work can be listed as follows:
• Realtime Adaptive Video Processing. We propose a realtime
adaption framewo r k for MPEG-4 video processing. First we select
the most efficient encoding and decoding parameters for videos of
different motion levels through extensive offline profiling and analysis.
Then we desig n a feedback algorithm to adaptively apply different
coding parameters while monitoring the system performance online
during a video call to meet the computation requirements.
• CPU Parameter Tuning through Execution Feedback. We
design a feedback mechanism that integrates energy saving techniques
through the tuning of hardware parameters. Specifically we utilize
Dynamic Vo lt age and Frequency Scaling (DVFS) to control the power
consumption on CPU. In this way graceful quality loss can be traded
for ma ximal reduction of energy consumption.
1.3.2 Energy Efficient Mobile Call Framework with
Adaptive Coding of H.264
Currently H.264 is one of the most used compression standards for video
streaming and conferencing. Compared to its predecessor MPEG-4, H.264
has a much improved coding efficiency by achieving similar qualities with
6