Các phương pháp tiết kiệm năng lượng sử dụng công nghệ mạng điều khiển bằng phần mềm trong môi trường điện toán đám mây

MINISTRY OF EDUCATION AND TRAINING
HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY

TRAN MANH NAM

CÁC PHƯƠNG PHÁP TIẾT KIỆM NĂNG LƯỢNG SỬ DỤNG
CÔNG NGHỆ MẠNG ĐIỀU KHIỂN BẰNG PHẦN MỀM TRONG
MÔI TRƯỜNG ĐIỆN TOÁN ĐÁM MÂY
SDN-BASED ENERGY-EFFICIENT NETWORKING IN
CLOUD COMPUTING ENVIRONMENTS

DOCTORAL THESIS OF TELECOMMUNICATIONS ENGINEERING

HANOI - 2018


MINISTRY OF EDUCATION AND TRAINING
HANOI UNIVERSITY OF SCIENCE AND TECHNOLOGY

TRAN MANH NAM

CÁC PHƯƠNG PHÁP TIẾT KIỆM NĂNG LƯỢNG SỬ DỤNG
CÔNG NGHỆ MẠNG ĐIỀU KHIỂN BẰNG PHẦN MỀM TRONG MÔI
TRƯỜNG ĐIỆN TOÁN ĐÁM MÂY
SDN-BASED ENERGY-EFFICIENT NETWORKING IN CLOUD
COMPUTING ENVIRONMENTS
Specialization: Telecommunications Engineering
Code No: 62520208

DOCTORAL THESIS OF TELECOMMUNICATIONS ENGINEERING
Supervisor: Assoc.Prof. Nguyen Huu Thanh

HANOI - 2018


PREFACE

I hereby assure that the results presented in this dissertation are my work under the
guidance of my supervisor. The data and results presented in the dissertation are completely
honest and have not been disclosed in any previous works. The references have been fully
cited and in accordance with the regulations.

Tôi xin cam đoan các kết quả trình bày trong luận án là cơng trình nghiên cứu của tơi
dưới sự hướng dẫn của giáo viên hướng dẫn. Các số liệu, kết quả trình bày trong luận án là
hoàn toàn trung thực và chưa được cơng bố trong bất kỳ cơng trình nào trước đây. Các kết
quả sử dụng tham khảo đều đã được trích dẫn đầy đủ theo đúng quy định.
Hà Nội, Ngày … tháng … năm
Tác giả

Trần Mạnh Nam

ii


ACKNOWLEDGEMENTS
First and foremost, I would like to thank my advisor, Associate Prof. Dr. Nguyen Huu
Thanh, for providing an excellent researching atmosphere, for his valuable comments,
constant support and motivation. His guidance helped me in all the time and also in writing
this dissertation. I could not have thought of having a better advisor and mentor for my PhD.
Moreover, I would like to thank Associate Prof. Dr. Pham Ngoc Nam, Dr. Truong Thu
Huong for their advices and feedbacks, also for many educational and inspiring discussions.

My sincere gratitude goes to the members (present and former) of the Future Internet Lab,
School of `Electronics and Telecommunications, Hanoi University of Science and
Technology. Without their support and friendship it would have been difficult for me to
complete my PhD studies.
Finally, I would like to express my deepest gratitude to my family. They are always
supporting me and encouraging me with their best wishes, standing by me throughout my
life.
Hanoi, ……………………….

iii


CONTENTS

LIST OF FIGURES ............................................................................. ix
LIST OF TABLES .............................................................................. xi
INTRODUCTION ................................................................................. 1
CHAPTER 1.AN
OVERVIEW
OF
ENERGY-EFFICIENT
APPROACHES IN CLOUD COMPUTING ENVIRONMENTS ............. 6
1.1

Today's Internet .................................................................................. 6

1.1.1

Cloud Computing Services and Infrastructures ............................. 6

1.1.2

Energy consumption problems ...................................................... 6

1.2

An Overview of Energy-Efficient Approaches ................................. 8

1.2.1

Energy consumption characteristics .............................................. 8

1.2.2

Energy-Efficient Approaches' Classification .................................. 9

1.3

Software-defined Networking (SDN) technology ........................... 10

1.3.1

SDN Architecture......................................................................... 10

1.3.2

SDN Southbound API - OpenFlow Protocol ................................ 11

1.3.3

SDN Controllers .......................................................................... 12

1.4

Difficulties on Network Energy Efficiency and Motivations.......... 13

1.5

Dissertation’s Contributions ........................................................... 14

1.5.1 Proposing an energy-aware and flexible data center network that is
based on the SDN technology. .................................................................... 14
1.5.2 Proposing energy-efficient approaches in a network virtualization for
cloud environments. ..................................................................................... 14
1.5.3 Proposing an energy-aware data center virtualization for cloud
environments. .............................................................................................. 15

CHAPTER 2.SDN-BASED ENERGY-AWARE
DATA CENTER
NETWORK ........................................................................................ 16
2.1

Background Technologies .............................................................. 16

2.1.1

DCN technique and architecture ................................................. 16

2.1.2

Existing system ........................................................................... 22

2.2

Power-Control System of a DC Network ........................................ 23

2.2.1

Energy modeling of a network ..................................................... 23

iv


2.2.2

The Diagram of the Power-Control System ................................. 26

2.3 Energy-Aware Routing based on Power Profile of Devices in Data
Center Networks using SDN.......................................................................... 30
2.3.1

Energy-Aware Routing and Topology Optimization Algorithm ..... 31

2.3.2

Performance evaluation .............................................................. 37

2.4 Green Data Center using centralized Power-control of the Network
and servers ..................................................................................................... 40
2.4.1

Extended Power-Control System ................................................ 41

2.4.2

Use case ..................................................................................... 42

2.4.3

Topology-aware VM migration algorithm ..................................... 44

2.4.4

VM Migration cost and Power modeling of a Server .................... 46

2.4.5

Results ........................................................................................ 46

2.5

Conclusion ........................................................................................ 49

CHAPTER 3.ENERGY-EFFICIENT NETWORK VIRTUALIZATION
FOR CLOUD ENVIRONMENTS........................................................ 50
3.1

Network Virtualization and Virtual Network Embedding ............... 52

3.2 Constructing Energy-Aware SDN-based Network Virtualization

System 52
3.2.1

System’s Diagram ....................................................................... 53

3.2.2

System’s workflow ....................................................................... 54

3.3

Modeling and Problem Formulation ............................................... 55

3.3.1

VNE Modeling ............................................................................. 55

3.3.2

Objective and Constraints ........................................................... 56

3.3.3

Time-based Embedding Strategies ............................................. 58

3.4

Energy-efficient VNE algorithms ..................................................... 59

3.4.1

Energy-cost Coefficient of Capacity ............................................ 59

3.4.2

Virtual Node Mapping algorithms ................................................ 60

3.4.3

Virtual Link Mapping (VLiM) Algorithm ........................................ 63

3.5

Performance Evaluation .................................................................. 64

3.6

Conclusion ........................................................................................ 68

CHAPTER 4.AN
ENERGY-AWARE
DATA
CENTER
VIRTUALIZATION FOR CLOUD ENVIRONMENTS ......................... 69
v


4.1

Virtual DC Technologies .................................................................. 70

4.1.1

Virtual data center embedding..................................................... 70

4.1.2

Virtual machine migration and server consolidation .................... 72

4.1.3

Discussion ................................................................................... 72

4.2

Design Objectives ............................................................................ 74

4.3

Problem Formulation ....................................................................... 75

4.3.1

Data Center Modeling ................................................................. 75

4.3.2

Energy Modeling of DC Components .......................................... 76

4.3.3

Energy-Efficient Problem Formulation ......................................... 77

4.4

A New Concept for VDC Embedding .............................................. 78

4.4.1

Energy-aware VDC architecture .................................................. 78

4.4.2

Energy-aware VDC embedding algorithm ................................... 79

4.4.3

Joint VDC Embedding and VM Migration Algorithms .................. 82

4.5

Performance Evaluation .................................................................. 85

4.5.1

Performance criteria .................................................................... 85

4.5.2

Numerical results......................................................................... 86

4.6

Conclusion ........................................................................................ 92

CHAPTER 5.CONCLUSION
AND
FUTURE
WORK ............................................................................................... 93
5.1

Major contributions .......................................................................... 93

5.2

Future research directions .............................................................. 94

LIST OF PUBLICATIONS ................................................................. 95
REFERENCES .................................................................................. 97

vi


ABBREVIATIONS

APCI
APEX
ASIC
BAU
BFS

CAPEX
DC
DCN
D-ITG
EA-NV
EA-VDC
ECO
FM
FPGA
GH
HEA-E
HEE
IaaS
ICT
ISP
MoA
MST
NaaS
NFV
NV
OLD
OPEX
PaaS
PCS
PM
POD
PSnEP
RMD-EE
SaaS
SDSN

SN

Advanced Configuration & Power Interface
Capital expenditure
Application specific integrated circuits
Business-as-usual
Breadth-first Search
Capital Expenditure
Data center
Data center network
Distributed internet traffic generator
Energy-aware network virtualization
Energy-aware Virtual Data Center
Eco sustainable
Full migration
Field programmable gate arrays
GreenHead
Heuristic Energy-aware VDC Embedding
Heuristic energy-efficient
Infrastructure-as-a-service
Information and communication technologies
Internet service provider
Migrate on arrival
Minimum spanning tree
Network-as-a-service
Network function virtualization
Network virtualization
OpenDayLight
Operating expenses
Platform-as-a-service

Power-Control System
Partial migration
Optimized data centers
Power scaling and energy-profile-aware
Reducing middle node energy efficiency
Software-as-a-service
Software-Defined Substrate Network
SecondNet

vii


SNMP
TCAM
VDC
VDCE
VLiM
VM
VmM
VNE
VNoM
VNR

Simple network management protocol
Ternary content-addressable memory
Virtual data center
Virtual data center embedding
Virtual link mapping
Virtual Machine
Virtual machine mapping

Virtual network embedding
Virtual node mapping
Virtual network requests

viii


LIST OF FIGURES
Figure 1.1: Estimate of the global carbon footprint of ICT (including PCs, telcos’ networks
and devices, printers and datacenters) [15].......................................................................... 7
Figure 1.2: Energy consumption estimation for the European telcos’ network infrastructures
in the”Business-As-Usual” (BAU) and in the Eco-sustainable (ECO) scenarios, and
cumulative energy savings between the two scenarios [16]. ................................................ 7
Figure 1.3: Operating Expenses (OPEX) estimation related to energy costs for the European
telcos’ network infrastructures in the ”Business-As-Usual” (BAU) and in the Ecosustainable (ECO) scenarios, and cumulative savings between the two scenarios [17] ...... 8
Figure 1.4: SDN Architecture ............................................................................................. 11
Figure 1.5: OpenFlow controller and switches................................................................... 12
Figure 2.1: DCN Architecture [43] ..................................................................................... 18
Figure 2.2: Three-tier DCN Architecture [45] ................................................................... 18
Figure 2.3: Fat-tree DCN Topology ................................................................................... 19
Figure 2.4: Dcell DCN Architecture [53] ........................................................................... 19
Figure 2.5: BCube DCN Architecture [54] ......................................................................... 20
Figure 2.6: Fat-tree architecture with k = 4 ....................................................................... 21
Figure 2.7: Diagram of the ElasticTree system [57] .......................................................... 22
Figure 2.8: Energy – Utilization relation of a network [58] .............................................. 23
Figure 2.9: Power-control System of a Network ................................................................. 26
Figure 2.10: Fat-tree topology with Minimum Spanning Tree ........................................... 28
Figure 2.11: Power Scaling Algorithm ............................................................................... 32
Figure 2.12: Power Scaling and Energy-Profile-Aware - PSnEP algorithm (Proposed
Algorithm 1). The flowchart describes the process between Edge and Aggregation switches

............................................................................................................................................. 34
Figure 2.13: use-case with PSnEP algorithm in a DCN ..................................................... 36
Figure 2.14: PSnEP vs Power scaling (PS) with k=6 Fat-tree, mix scenario .................... 39
Figure 2.15: Energy-saving level ratio of the PSnEP algorithm to the PS algorithm in
different sizes ....................................................................................................................... 40
Figure 2.16: Extended Power-Control system (Ext-PCS) ................................................... 41
Figure 2.17: Example .......................................................................................................... 43
Figure 2.18: First-fit Migration [67] Algorithm ................................................................. 43

ix


Figure 2.19: Topology-Aware Placement Algorithm .......................................................... 44
Figure 2.20: K=8, comparison with full mesh scenario ..................................................... 47
Figure 2.21: K=16, comparison with full mesh scenario ................................................... 48
Figure 2.22: K=8, comparison with Honeyguide ............................................................... 48
Figure 2.23: K=16, comparison with Honeyguide ............................................................. 49
Figure 3.1: FlowVisor – Hypervisor-like Network Layer [71] ........................................... 51
Figure 3.2: Example of a virtual network on top of a physical network ............................. 52
Figure 3.3: Energy-Aware Network Virtualization system’s Diagram ............................... 53
Figure 3.4: Online VNE mapping method ........................................................................... 58
Figure 3.5: Online using Time Window method.................................................................. 59
Figure 3.6: The GUI of an Energy-aware network virtualization platform ........................ 65
Figure 3.7 AR– Online ........................................................................................................ 66
Figure 3.8: AR – Online using Time Windows .................................................................... 66
Figure 3.9: Percentage of Power Consumption to Full State in Online Strategy ............... 66
Figure 3.10 Percentage of Power Consumption to Full State in OuTW Strategy ............. 66
Figure 3.11: Comparison of comsumed energy between Online and OuTW strategies ..... 67
Figure 3.12: Comparison of acceptance ratio between Online and OuTW strategies ....... 67
Figure 4.1: Traditional cloud service provider vs NaaS ..................................................... 69

Figure 4.2: Embedding virtual data center requests on a physical data center ................. 71
Figure 4.3: Virtual data center embedding - Static mapping; ............................................ 73
Figure 4.4: Virtual data center embedding - Dynamic mapping ........................................ 73
Figure 4.5: Energy proportional property of energy-aware data centers .......................... 74
Figure 4.6: Energy-Aware VDC Architecture ..................................................................... 79
Figure 4.7: VDC Embedding Flowchart ............................................................................. 80
Figure 4.8: Flowchart of Partial Migration (PM) .............................................................. 84
Figure 4.9: Migration on Arrival ........................................................................................ 85
Figure 4.10: Fluctuation of system utilization (SecondNet)................................................ 87
Figure 4.11: DC Utilization per Load ................................................................................. 88
4.12: Acceptance Ratio per VM ........................................................................................... 88
Figure 4.13: Acceptance Ratio per VDC............................................................................. 89

x


Figure 4.14: Total power consumption of the physical DC ................................................ 89
Figure 4.15: Average consumed power per serving VDC................................................... 90
Figure 4.16: Number of migrations for different strategies ................................................ 91
Figure 4.17: Comparison of embedding - migration strategies .......................................... 91
4.18: Different embedding-magrition strategies: (a) GreenHead, (b) SecondNet, (c) Partial
Migration, (d) Migration on Arrival, (e) Full Migration .................................................... 92

LIST OF TABLES
Table 1.1: The Internet’s users in the world [1] ................................................................... 6
Table 1.2: Estimated power consumption sources in a generic platform of IP router ......... 8
Table 1.3: Classification of energy-efficient approaches of the future Internet [4] ............. 9
Table 2.1: Power Summary For A 48-Port Pronto 3240 .................................................... 30
Table 2.2: Energy consumption of NetFPGA-Based OpenFlow Switch ............................. 32
Table 2.3: Energy-saving ratio of PSnEP to Power scaling algorithm in different topology’s

sizes...................................................................................................................................... 39
Table 2.4: Traffic demand ................................................................................................... 42
Table 2.5: Power profile of server Dell PowerEdge R710.................................................. 47
Table 3.1: Virtual Network Embedding Terminology ......................................................... 55
Table 3.2: Acceptance ratio and power consumption of the system under different window
size in OuTW........................................................................................................................ 68
Table 4.1: Standard deviation of system utilization ............................................................ 87

xi


INTRODUCTION
1. Overview of Network Energy Efficiency in Cloud Computing Environments
The advances in Cloud Computing services as well as Information and Communication
Technologies (ICT) in the last decades have massively influenced economy and societies
around the world. The Internet infrastructure and services are growing day by day and play
a considerable role in all aspects including business, education as well as entertainment. In
the last four years, the percentage of people using Internet witnesses an annual growth of
3.5%, from 39% world population’s percentage in Dec-2013 to 51.7% in June-2017 [1].
To support the demand of cloud network infrastructure and Internet services in the rapid
growth of users, it is necessary for the Internet providers to have a large number of devices,
complex design and architecture that have the capacity to perform increasingly number of
operations for a scalability. Consequently, many huge cloud infrastructures have been
employed by Telcos, Internet Service Providers (ISPs) and enterprises for the exploded
demand of various applications and data cloud-services such as YouTube, Dropbox,
e-learning, cloud office etc. To meet the requirements of these booming services all around
the world, cloud network infrastructures have been built up in a very large scale, even
geographically distributed data centers with a huge number of network devices and servers.
In addition, the maintenance of the systems with high availability and reliability level
requires a notable redundancy of devices such as routers, switches, links etc. As a result,

having such a large infrastructure consumes a huge volume of energy, which leads to
consequent environmental and economic issues:
-

-

Environmentally, the amount of energy consumption and carbon footprint of the
ITC-sector is remarkable. The manufacture of ICT equipment is estimated its use and
disposal account for 2% of global CO2 emissions, which is equivalent to the
contributions from the aviation industry [2]. The networking devices and
components estimate around 37% of the total ICT carbon emission [3];
Economically, the huge consumed power leads to the costs sustained by the
providers/operators to keep the network up and running at the desired service level
and their need to counterbalance ever-increasing cost of energy.

Although network energy efficiency has recently attracted much attention from
communities [4], there are still many issues in realization of the energy-efficient network
including inflexibility and the lack of an energy-aware network. The main difficulties of the
network energy efficiency as well as its research motivations are shortly described as
follows:
-

Inflexible network: first, one important point the network in cloud data centers (DC)
nowadays is the inflexibility issue. For changing the processing algorithm and the
control plane of a network, its administrators should carefully re-design,

1


-

re-configure and migrate the network for a long time. In many cases, there is a
technical challenge for an administrator to apply new approaches and evaluate their
efficiency. Consequently, the flexible and programmable network is strictly
necessary. Secondly, there are difficulties in evaluating the energy-saving levels of
new energy-efficient approaches in a network due to the lack of the centralized
power-control system. This system allows administrators and developers to monitor,
control and managing the working states as well as power consumption of all
network devices in real-time.
Energy-aware networking for virtualization technologies in cloud environments:
cloud computing has emerged in the last few years as a promising paradigm that
facilitates such new service models as Infrastructure-as-a-Service (IaaS), Storageas-a-Service (SaaS), Platform-as-a-Service (PaaS), Network-as-a-Service (NaaS).
For such kinds of cloud services, virtualization techniques including network
virtualization [5] [6] [7] and data center virtualization [8] [9] [10] have quickly
developed and attracted much attention of research and industrial communities.
Currently, research in virtualization technologies mainly focuses on the resource
optimization and resource provisioning approaches [8] [9]. There are very few
works focusing on the energy efficiency of a network. With the benefits of flexible
controlling and resource management of virtualization technologies as well as new
network technologies such as Software-defined Networking (SDN) [11] [12] [13],
researching in network energy efficiency in virtualization is an important and
promising approach.

Additionally, the SDN technology, the emergence of new trends in networking
technology, provides new way to realize and optimize network energy efficiency. Softwaredefined networking [11] aims to change the inflexible state networking, by breaking vertical
integration, separating the network’s control logic from the underlying routers and switches,
promoting (logical) centralization of network control, and introducing the ability to program
the network. Consequently, SDN is an important key for resolving aforementioned
difficulties.

2. Research Scope and Methodology
a) Research Scope
The scope of this research focuses on the network energy efficiency in cloud computing
environments, including: (1) energy efficiency in centralized data center network; (2) energy
efficiency in network virtualization; and (3) energy efficiency in data center virtualization.
The proposed energy-efficient approaches are based on the Software-defined Networking
technology [11] [12] [13].
b) Research Methodology: the research methodology is used following the reference
[14].
2


o Step 1: Problem formulation:
 Interrogative form
 Describe relations among constructs
o Step 2: Hypothesis formulation: answering to problem statements
o Step 3: Research design: building research plan for a research process
including survey, related work and experiments
o Step 4: Sampling and Data Collection
o Step 5: Data analysis
o Step 6: Manuscript Writing

3. Contributions and Structure of the Dissertation
Recently, Software-defined Networking technology [5] is likely an evolutionary step in
Internet technologies that makes networking become more flexible and programmable. SDN
is an important key to resolving the difficulties of energy efficiency. This technology also
can quickly realize the virtualization technologies including network virtualization and data
center virtualization. Consequently, SDN-based energy-efficient networking approaches in
cloud environments are focused on this dissertation with the following contributions:
-

-

-

The SDN technology is used as core technology in this dissertation for proposing
energy-efficient network approaches. The first contribution of this dissertation is
resolving the lack of energy-aware network in a DC by (1) proposing a SDN-based
power-control system (PCS) of a network. The proposed system allows the
administrator of a network to flexibly control and monitor the state of network
devices and the energy consumption of the whole network infrastructure. Thanks to
the flexibility and availability of this PCS system, several energy-efficient
algorithms are proposed and evaluated on it successfully.
The network virtualization (NV) technology in cloud environments becomes more
popular and plays an important role for such cloud services including Network-asa-service (NaaS), Infrastructure-as-a-service (IaaS). The energy-aware NV platform
is necessary for network energy efficiency. Appropriately, (2) the SDN-based
energy-aware network virtualization (EA-NV) platform is proposed in this
dissertation. The platform is aware of power consumption of the network
virtualization environment. Two novel energy-efficient virtual network embedding
algorithms are also proposed and implemented in this platform that focus on
increasing the energy-saving level and maintaining the reasonable resource
optimization of a network.
Virtual data center technology is a concept of network virtualization in cloud
environments that allows creating multiple separated virtual data centers (VDC) on
top of the physical data center [8] [9] [10]. In consequence, (3) an energy-aware
virtual data center platform is deployed. On this system, novel energy-aware
algorithms are also proposed which focus on the following objectives: (1) resource
3

efficiency that deals with efficient mapping of virtual resources on substrate
resources in terms of CPU, memory and network bandwidth; and (2) energy
efficiency that deals with minimizing energy consumption of the virtual data center
while meeting virtual data center mapping demands.
The above contributions of this dissertation are organized as the collection of several
SDN-based network energy-efficient approaches which are presented in five chapters as
follows:
-

-

The first chapter presents an overview of energy-efficient network in cloud
environments and their classification. The difficulties of the network’s energy
efficiency area as well as the background of the Software-defined Networking
technology are also described in details.
In the second chapter, a SDN-based power-control system (PCS) of a data center
network is proposed. Based on this platform, developers can propose, implement
and evaluate several network energy-saving algorithms. Two energy-efficient
approaches, which are applied onto the PCS system, are also proposed with their
results and algorithms published in:
 Tran Manh Nam, Nguyen Huu Thanh, Doan Anh Tuan “Green Data Center
Using Centralized Power-Management Of Network And Servers”, The 15th
international Conference on Electronics, Information, and Communication
(IEEE - ICEIC), Jan 2016, Da Nang, Vietnam
 Tran Manh Nam, Nguyen Huu Thanh, Ngo Quynh Thu and Hoang Trung Hieu,
Stefan Covaci, “Energy-Aware Routing based on Power Profile of Devices in
Data Center Networks using SDN”, the 12th IEEE ECTI-CON conference - 2015,
Hua-Hin, Thailand - Achieved a student Grant of ECTI-CON, Jun, 2015.
 Tran Manh Nam, Truong Thu Huong, Nguyen Huu Thanh, Pham Van Cong, Ngo
Quynh Thu, Pham Ngoc Nam, “A Reliable Analyzer for Energy-Saving

Approaches in Large Data Center Networks”, IEEE ICCE - The International
Conference on Communications and Electronics - 2014, Da Nang, Vietnam
 Tran Manh Nam, Tran Hoang Vu, Vu Quang Trong, Nguyen Huu Thanh, Pham
Ngoc Nam, “Implementing Rate Adaptive Algorithm in Energy-Aware Data
Center Network”, National Conference on Electronics and Communications
(REV2013-KC01)., Hanoi, Vietnam.

-

The third chapter describes an energy-aware network virtualization concept and its
power monitoring and controlling abilities. The proposed concept is SDN-based
which allows developers to implement several energy-efficient virtual network
embedding algorithms. Two energy-efficient embedding algorithms, namely
heuristic energy-efficient node mapping and reducing middle node energy
efficiency, are proposed in this section. The results and algorithms of this chapter
are published in:

4


 Tran Manh Nam, Nguyen Huu Thanh, Nguyen Hong Van, Kim Bao Long,
Nguyen Van Huynh, Nguyen Duc Lam, Nguyen Van Ca, “Constructing EnergyAware Software-Defined Network Virtualization”, Proceedings of Asia-Pacific
Advanced Network Research Workshop (APAN-NRW), August 10th - 14th
2015, Kuala Lumpur, Malaysia - (best student paper award).
 Thanh Nguyen Huu, Anh-Vu Vu, Duc-Lam Nguyen, Van-Huynh Nguyen,
Manh-Nam Tran, Quynh-Thu Ngo, Thu-Huong Truong, Tai-Hung Nguyen,
Thomas Magedanz. “A Generalized Resource Allocation Framework in Support
of Multilayer Virtual Network Embedding based on SDN”, Elsevier - Computer
Networks, 2015.
 Nam T.M., Huynh N.V., Thanh N.H. (2016). “Reducing Middle Nodes Mapping

Algorithm for Energy Efficiency in Network Virtualization”. In: Advances in
Information and Communication Technology, ICTA 2016. Advances in
Intelligent Systems and Computing, vol 538. Springer, Cham.
/>-

SDN-based Energy-aware Virtual Data Center (VDC) approach is presented in the
fourth chapter. The VDC technology and its main problems, namely VDC
embedding problems, are described in details. Three Joint VDC Embedding and VM
migration strategies are successfully proposed and evaluated on top of this SDNbased VDC concept. The experimental results and detailed algorithms of this chapter
are published in:
 Tran Manh Nam, Nguyen Van Huynh, Le Quang Dai, Nguyen Huu Thanh, “An
Energy-Aware Embedding Algorithm for Virtual Data Centers”, ITC28 International Teletraffic Congress, Sep - 2016, Wurzburg, Germany.
 Tran Manh Nam, Nguyen Huu Thanh, Hoang Trung Hieu, Nguyen Tien Manh,
Nguyen Van Huynh, Tuan Hoang. (2017). “Joint Network Embedding and
Server Consolidation for Energy-Efficient Dynamic Data Center Virtualization”,
Elsevier - Computer Networks, 2017 - doi.org/10.1016/j.comnet.2017.06.007

-

In the last chapter, the conclusion of the dissertation and its future work are
presented.

5


CHAPTER 1. AN OVERVIEW OF ENERGY-EFFICIENT
APPROACHES IN CLOUD COMPUTING ENVIRONMENTS
This chapter provides an overview of the Internet status nowadays and the energyefficient approaches in cloud computing environments, on which the networking community
is focusing currently. The chapter also addresses the difficulties and motivations on network
energy efficiency and the future Internet technologies in cloud computing environments

including the Software-Defined Networking technology, network virtualization technology
and data center virtualization technology. In a nutshell, the research approaches and
contributions of this dissertation are summarized in this chapter.

1.1 Today's Internet
1.1.1 Cloud Computing Services and Infrastructures
The advances in Information and Communication Technologies (ICT) in the last decades
have massively influenced economy and societies around the world. The Internet services as
well as cloud computing services are growing day by day and play a considerable role in all
aspects including education, business and entertainment. As we can see in the Table 1.1¸ in
the last four years, the percentage of people using Internet witnesses an annual growth of
3.5%, from 39% world population’s percentage in Dec-2013 to 51.7% in June-2017 [1].
Table 1.1: The Internet’s users in the world [1]

Date
Dec, 2013
Dec, 2014
Dec, 2015
Dec. 2016
June. 2017

Number of users World population’s percentage
2,802 millions
39.0 %
3,079 millions
42.4 %
3,366 millions
46.4 %
3,696 millions
49.5 %

3,885 millions
51.7 %

To meet this booming of cloud services such as IaaS, NaaS, SaaS, cloud computing
environments with their large network infrastructures have been deployed in a very large
scale, even geographically distributed data centers with a huge number of devices. These
large infrastructures consumes the high volume of energy which leads to many
environmental and economical problems.
1.1.2 Energy consumption problems
Although the benefits of having that infrastructure are considerable, such a large system
consumes the high volume of energy and leads to consequent issues:

6


Figure 1.1: Estimate of the global carbon footprint of ICT (including PCs,
telcos’ networks and devices, printers and datacenters) [15].

-

-

Environmentally, the amount of energy consumption and carbon footprint of the
ITC-sector is remarkable (Figure 1.1). Gartner Company, the ICT research and
advisory company, estimates that the manufacture of ICT equipment, its use and
disposal account for 2% of global CO2 emissions, which is equivalent to the
contributions from the aviation industry [2]. The networking devices and
components eliminate around 37% of the total ICT carbon emission [3];
Economically, the huge consumed power leads to the costs sustained by the
providers/operators to keep the network up and running at the desired service level

and leads to their need of counterbalancing ever-increasing cost of energy (Figure
1.2 and Figure 1.3).

Figure 1.2: Energy consumption estimation for the European telcos’ network infrastructures in
the”Business-As-Usual” (BAU) and in the Eco-sustainable (ECO) scenarios, and
cumulative energy savings between the two scenarios [16].

Because of these issues, the requirement of designing a high performance and energyefficient network has become a crucial matter for Telcos and ISPs towards greener cloud
environments.

7


Figure 1.3: Operating Expenses (OPEX) estimation related to energy costs for the European
telcos’ network infrastructures in the ”Business-As-Usual” (BAU) and in the Eco-sustainable
(ECO) scenarios, and cumulative savings between the two scenarios [17]

1.2 An Overview of Energy-Efficient Approaches
In this section, first, the most significant part of energy consumption of network device
is characterized with its existing researches. Secondly, the taxonomy energy-efficient
approaches, which are currently undertaken, is also presented.
1.2.1 Energy consumption characteristics
Table 1.2: Estimated power consumption sources in a generic platform of IP router

Efficient energy use, sometimes simply called energy efficiency concept, is far from
being new in a computing system. To the best of our knowledge, the first support of power
management system was published in 1999, namely “Advanced Configuration & Power
Interface” (ACPI) standard [18]. Thenceforth, more energy-saving mechanisms were
developed and introduced, especially in hardware enhancement with the new CPUs, which
could be more efficient and consumed less energy. Tucker [19] and Neilson [20] estimated

on IP routers that the control plane weighs 11%, data plane for 54% and power and heat
management for 35%. Tucker and Neilson also broke out the energy consumption of data
plane in more detail as described in Table 1.2. From 54% energy consumption of data plane,
the buffer management weighs 5%, the packet processing weighs about 32%; the network
interfaces weigh about 7%; and the switching fabric for about 10%. This estimation work
provides a clear indication for developers in order to increase the energy-saving level of
networks in the further researches.

8


1.2.2 Energy-Efficient Approaches' Classification
From the general point of view, existing approaches are founded on few basic concepts.
As shown in surveys of Raffaele Bolla et al. [4] and Aruna Banzino et al. [21], the largest
part of undertaken energy-efficient concepts is founded on few energy-saving mechanisms
and power management criteria that are already partially available in computing systems.
These approaches, which are depicted in the Table 1.3, are classified as (1) re-engineering;
(2) dynamic adaptation; and (3) smart sleeping [4].
Table 1.3: Classification of energy-efficient approaches of the future Internet [4]

1.2.2.1 Re-Engineering
The re-engineering approaches focus on introducing and designing more energy-efficient
elements inside network equipment architectures. Novel technologies mainly consist of new
silicon (ex: for Application Specific Integrated Circuits (ASICs) [22], Field Programmable
Gate Arrays (FPGAs) [23], etc.) and memory technologies (ex: Ternary ContentAddressable Memory (TCAM), etc.) for packet processing engines, and novel network
media technologies (energy-efficient lasers for fiber channel, etc.). The approaches can be
divided into two sub-approaches as follows: (1) energy-efficient silicon which focuses on
developing new silicon technologies [24]; and (2) complexity reduction which focuses on
reducing equipment complexity in terms of header processing, buffer size, switching fabric
speedup and memory access bandwidth speedup [25] [26].

1.2.2.2 Dynamic Adaptation
The dynamic adaptation approaches of network resources are aimed at modulating
capacities of devices (working speeds, computational capabilities of packet processing…)
according to the current traffic demand [4]. These approaches are founded on two main kinds
of power management capabilities provided by the hardware level, namely power scaling
and idle logic.
Power scaling capabilities allow dynamically reducing the working rate of processing
engines or of link interfaces [27] [28]. This is usually accomplished by tuning the clock
frequency and/or the voltage of processors, or by throttling the CPU clock (i.e., the clock
signal is gated or disabled for some number of cycles at regular intervals). On the other hand,
idle logic allows reducing power consumption by rapidly turning off sub-components when
no activities are performed, and by re-waking them up when the system receives new
activities. In detail, wake-up instants may be triggered by external events in a pre-emptive

9


mode (e.g., “wake-on-packet”), and/or by a system internal scheduling process (e.g., the
system wakes itself up every certain periods, and controls if there are new activities to
process).
1.2.2.3 Sleeping/Standby
Sleeping and standby approaches are founded on power management primitives, which
allow devices or part of them to turn themselves almost completely off, and enter very low
energy states, while all their functionalities are frozen [4]. Thus, sleeping/standby states can
be thought as deeper idle states, characterized by higher energy savings and much larger
wake-up times. In more detail, the applications and services of a device (or its part) stop
working and lose their network connectivity [29] [30] when it goes sleeping. As a result, the
sleeping device loses its network ”presence” since it cannot maintain network connectivity,
and answer to application/service-specific messages. Moreover, when the device wakes up,
it has to re-initialize its applications and services by sending a non-negligible amount of

signaling traffic.

1.3 Software-defined Networking (SDN) technology
Recently, the future Internet technologies in cloud computing environments such as
Software-defined Networking [11]; Network Virtualization (NV) [6] [7]; Network Function
Virtualization (NFV) [31]; Virtual Data Center (VDC) [32] are booming and are strongly
implemented in cloud environments [8] [9] [10]. On the way to realize these technologies
and transfer to the industrial market, the flexible network is mandatory. SDN technology
with its characteristics including programmable, capable of centralized management will
play very important role in the innovation of all other techniques. In this Section, the
overview of the SDN technology is depicted.
1.3.1 SDN Architecture
Software-defined Networking (SDN) [11] is an emerging networking paradigm that gives
hope to change the limitations of current network infrastructures. First, it breaks the vertical
integration by separating the network’s control logic (the control plane) from the underlying
routers and switches that forward the traffic (the data plane) [33]. Second, with the separation
of the control and data planes, network switches become simple forwarding devices and the
control logic is implemented in a logically centralized controller (or network operating
system1), simplifying policy enforcement and network re-configuration and evolution.
A simplified view of this architecture is shown in Figure 1.4. It is important to emphasize
that a logically centralized programmatic model does not postulate a physically centralized
system. In fact, the need to guarantee adequate levels of performance, scalability, and
reliability would preclude such a solution. Instead, production-level SDN network designs
resort to physically distributed control planes. The separation of the control plane and the
data plane can be done by a well-defined programming interface between the switches and

10


the SDN controller. The controller exercises direct control over the state in the data plane

elements via this well-defined application programming interface (API), as depicted in
Figure 1.4. The most notable example of such an API is OpenFlow [34], [35].
Network applications

Northbound API
(ex: RestAPI, )
SDN controller (ex: POX, Floodlight, ODL)
Routing

Centralized
Management

Monitoring

Network OS
Southbound API
(ex: OpenFlow)

SDN Data Forwarding element (ex:
OF switch, OVS)
Figure 1.4: SDN Architecture

1.3.2 SDN Southbound API - OpenFlow Protocol
OpenFlow [34] [35] is the first and also the most widely known SDN protocol for
southbound API, it provides the communication protocol between the control plane on SDN
controller and the forwarding planes on OpenFlow switches. OpenFlow specifies how these
planes communicate and interact with each other since the connection is setup until the end.
The OpenFlow protocol is layered above the Transmission Control Protocol, leveraging the
use of Transport Layer Security (TLS). The default port number for controllers to listen on
is 6653 for switches that want to connect.

An OpenFlow switch has one or more tables of packet (Figure 1.5) handling rules (flow
table). Each rule matches a subset of the traffic and performs certain actions (dropping,
forwarding, modifying, etc.) on the traffic. Depending on the rules installed by a controller
application, an OpenFlow switch can be instructed by the controller behave like a router,
switch, firewall, or perform other roles (e.g., load balancer, traffic shaper, and in general
those of a middlebox). A flow-table contains several flow entries, each flow entry consists
of three main parts:
-

Match rule: this includes various fields to match on a packet: IP source address, IP
destination address, MAC source address, MAC destination address, TCP source
port address, etc. A field can be left empty, which means any packets can match
with this field.

11


-

Action: this action is applied to the match packet. Actions include forwarding packet
to another port, drop packet, etc.
Stats: this part records the number of packet and byte that has matched with this
flow entry. It also records the duration from the starting time until current. This stats
component is usually used for monitoring and in management functions.
SDN
Controller

Global Network View
Centralized management

Openflow switch

Openflow switch

Openflow switch
CONTROL
PATH

Secure
channel

DATA PATH
(Flow Tables, Meter Table)

Figure 1.5: OpenFlow controller and switches

When a packet arrives, it will be paired with the first matching flow entry in the flow
table. If the packet is not matched with any entries, the switch will send an OpenFlow
PacketIn message to the controller which will take appropriate actions afterwards. After that,
the controller will send an OpenFlow FlowMod message back to the switch in order to create
a new entry matching this packet together with some action. That way, if later similar packets
get into the switch, the switch does not need to ask the controller for further action.
1.3.3 SDN Controllers
In Software-defined Networking, SDN Controller does exactly what its name suggests,
controlling the network as the “brain” of network. It has the global view of a network, with
all information about the network topology, flow tables of the OpenFlow switches, etc.
Using this information, the SDN Controller manages OpenFlow switches via southbound
APIs (e.g. OpenFlow) and leads to the deployment of applications and business logic ’above’
via northbound APIs.
The first developed SDN Controller is NOX which was introduced by Natasha Gude et

al. in [36]. Subsequently, other open source controllers were also developed, e.g. POX [37],
Beacon [38], and Floodlight (forked from Beacon) [39]. Later, multiple vendors such as
Cisco, IBM, HPE, VMware and Juniper joined the SDN Controller market and each of them
possessed their own products. From Beacon, HPE, Cisco, and IBM Controllers have moved
towards OpenDaylight (ODL) [40]. Despite being one of the early controllers, and being less
popular than its counterparts, the POX controller, written in Python, is still fully functional,
12


easy to be grasped, installed and configured, that makes it ideal for academic researchers in
their experiment. That also explains why this POX controller is selected in this dissertation
for the SDN architecture.

1.4 Difficulties on Network Energy Efficiency and Motivations
Although the concept of network energy efficiency is not new, there are still issues in
realization of the energy-efficient network due to the inflexibility of a network and the lack
of an energy-aware network. These difficulties are depicted as follows:
-

-

Inflexible network: First, cloud services up-to-date frequently and lead to the change
of network infrastructure. On the contrary, one important point of networks
nowadays is the inflexibility issue. Administrators should plan and prepare well for
any changes in the network, which might require re-designing, re-configuring and
migrating. In many cases, there is a big challenge for any developers to apply any
new approaches and evaluate them. Consequently, the network flexibility is vitally
necessary. Secondly, there are difficulties in evaluating the energy-saving levels of
new energy-efficient approaches in a network due to the lack of the power-control
system of a network. Developers struggle when they propose and evaluate a new

energy-saving approach.
Cloud computing has been blooming in the last few years as a promising paradigm
that facilitates new service models such as Infrastructure-as-a-Service (IaaS),
Platform-as-a-Service (PaaS), Network-as-a-Service (NaaS). On the cloud
computing environments, virtualization techniques such as network virtualization
[5] [6] [7] and data center virtualization [8] [9] [10] have rapidly been developed
and attracted much attention from industrial communities. Currently, virtualization
works mainly focus on the resource optimization and resource provisioning
approaches [7] [41], while there are only few works focusing on the energy
efficiency. One of the main difficulties of network energy efficiency in virtualization
technologies is the lack of energy measurement method of the network infrastructure
in cloud environments. Consequently, the implementation of energy-aware
platforms, which work well for network virtualization and data center virtualization,
is an important and promising approach in the energy efficiency area of the
networking.

Above difficulties as well as the potentials of SDN technology are great motivation for
the construction of SDN-based energy-efficient networking in cloud computing
environments. In this dissertation, several energy-efficient networking approaches are
proposed with specific algorithms and, equally important, experimental results. The detailed
contributions are described in the next section.

13