IPoDWDM deep dive Jusep Unserman

IPoDWDM and
IP NGN Core design
Josef Ungerman

Cisco, CCIE #6167

© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

1


• Motivations for IP NGN
• Trends is IP/MPLS Core Design
• Making Routers Cheaper
• IPoDWDM
• Pre-FEC proactive protection
• 100GE and 40GE
• Future

© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

2


IP Traffic will increase 4x from 2009 to 2014
(34% CAGR)
IP Traffic

IP Traffic grew 45%
in 2009
Mobile Data Traffic
will double every
year

Video will be 66% of
mobile data, and
91% of global traffic
Internet Traffic

© 2010 Cisco and/or its affiliates. All rights reserved.

Source: Cisco Visual Networking Index—Forecast, 2009–2014 [Cisco VNE]

Business IP traffic
will increase 2.6x
(21% CAGR), only
video will grow 10x
Central & Eastern
Europe will be
2.3EB/month (3.6%
of the global
64EB/month)
Cisco Confidential

3

How to move bits cheaper...

...reduce OPEX, CAPEX, and keep reasonable quality?

1)

Reduce the number of networks

2)

Reduce the number of layers

3)

Reduce the number of nodes

4)

Reduce the number of links

5)

Use modern technologies

the largest portion of the traffic will rule the design

the largest portion of the traffic must be as close to fiber as
possible, eliminate overlay
the largest portion of the traffic must pass the lowest possible
number of nodes

the number of changes in the network must be minimal when
adding more capacity; use statistical multiplex - small traffic
follows big traffic
Watch TCO,
Price/Performance ratio, Watt/Gigabit ratio,
investment protection (h/w upgradeability, s/w roadmaps)

© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

4


© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

5


• IP NGN = Single Multiservice Network

Not multiple single-service networks. Key enablers are Virtualization, QoS and Security.

• IP NGN = MPLS + DWDM Optical Transport

Moving bits cheaper. 100GE evolution, Price/Gigabit and Watt/Gigabit reduction.

• IP NGN = optimal balance between Clouds and Circuits

Statistical Multiplex (IP/MPLS, MPLS-TP) and Division Multiplex (DWDM, OTN).

© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

6


Designing End to End IP/MPLS
networks
Hierarchy

Hierarchical Design
(divide & conquer approach)
PE is always connected to P
Simple upgrades
Static Transport Layer

Protection is in the IP domain

© 2010 Cisco and/or its affiliates. All rights reserved.

Router Bypass

• Hierarchical Design
with optical router bypass
• Still simple upgrades

• Cheaper bandwidth
• Quality is kept

Hollow Core idea

Flat Design
(full mesh between IP routers)
Fewer nodes, Much more Links

Complex upgrades, complex QoS
Dynamic Transport Layer
(G.MPLS, VCAT/ODU-FLEX)

Protection in the Optical domain

Cisco Confidential

7


Metro
Aggregation
BNG
(Edge)

The
The Quality
Quality of
of IP
IP NGN

NGN Design
Design
•• Hierarchy
Hierarchy (P
(P is
is connected
connected to
to PE)
PE)
•• One
One network,
network, one
one IGP
IGP (not
(not multiple)
multiple)
•• QoS
QoS and
and Security
Security everywhere
everywhere
•• Scalability
Scalability –– where
where will
will 40/100GE
40/100GE start?
start?
SSN – Single Service Node (eg. BRAS, IGW)
MSN – Multiservice Node (eg. P router)

Core

•SSN-MSN link = ok
Internet Gateways

safe operation

•MSN-MSN link = think twice!

such link needs proper QoS and capacity mgt

•SSN-SSN link = stop!

don’t break the hierarchy, don’t create another net

© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

8


Statistical Multiplexing
Creates an IP hierarchy

DWDM or dark fiber – direct link
(optical layer bypass)
Creates an IP mesh

Too large distances – OTN switch

(transport layer bypass)

© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

9


100GE

IEEE 802.3ba ratified
CRS-3 today, in 2011/12 also ASR9K & Nexus7K

OTN (Optical Transport Network)
OTN Framing

implemented today for Ethernet interfaces – ITU-T G.709
CRS, ASR9000, 7600, CPT, ONS (ODU2e, ODU3, future ODU4)

OTN Aggregation

implemented today for 10GE
ONS15454 Muxponder

ODU3 (future Any-Rate, ODU4, MPLS-TP)

OTN Switching

geographically very large countries, or very dense 10G E-Line networks (G.MPLS)

developed for next generation portfolio

© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

10


100G lambda

100G SR

100G lambdas

OTN

10G lambda
10G and 100G DWDM
Coexistence
10G and 100G lambdas co-exist on same fibre
Packet uses 100G, everything else 10G
Advantages
Only high demand clients upgraded to 100G
Protects existing 10G DWDM investment
Lowest cost per bit (100G TXPs>10 x10G TXPs)
Disadvantages
Need a guard band between 10G and 100G
frequencies
Not appealing in ULH environments

© 2010 Cisco and/or its affiliates. All rights reserved.

OTN

10G SR
OTN Multiplexing
• All lambdas upgraded to 100Gbps

• Sub-100G services provided by OTN OEO

Advantages
All lambdas on a fibre are 100G

Disadvantages
100TXP investment upfront
Need an additional OTN OEO
All 10G TXPs are obsolete

Cisco Confidential

11


No

No

No
No

No
No

© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

12


35bn
domain for OTN

30bn
25bn

domain for MPLS

20bn
15bn
10bn
5bn

Wholesale and retail Ethernet services :
E-Line, E-Tree and E-LAN

~90% of Ethernet market place <1GE in 2013

What is the most efficient way to support these
Ethernet Services? OTN circuits or Packets?

Source Infonetics 2009 and Cisco VNI
© 2010 Cisco and/or its affiliates. All rights reserved.

Circuit
Packet

Cisco Confidential

13


Routers

Presentation_ID

© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

14


NEW

Routers: 23% Cumulative Average $/Gbps Drop per year / fewer ASICs
Optics: $/G stays flat (best case) or increases from one technology to the next
Cisco Core Router Example

10G/40G/100G Networking Ports Biannual Worldwide
and Regional Market Size and Forecasts

May 2010

• Silicon has fundamentally followed Moore’s law
• Optics is fundamentally an analog problem
© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

15


1)

Compact Anatomy

2)

Linecard Architecture

3)

Special Core-facing Linecards

4)

Oversubscribed Cards

5)

Power Consumption

RSP, Route/Switch Processor (instead of RP and FC)
Ethernet-oriented Linecard (non-modular, less memory)
4x 10G NPU (instead of 1x 40G NPU)
one full-duplex NPU shared for rx and tx (instead of 2 dedicated NPU’s)
2x 40G fabric interface (instead of 1x 80G fabric interface)
8/16 queues per port (instead of thousands)
lower-scale NPU (no need for thousands of interfaces)
licenses for features that not everybody uses (IPoDWDM, SyncE, VPN,...)
2:1 ingress overbooking (eg. GPON Aggregation or Intra PoP PE links)
newer chips, fewer chips = 50+% Watt-per-Gigabit savings

© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

16


Cisco CRS

very modular router anatomy
RP (active)

MSC-40
NP
Q

buff.

Q
NP

buff.

Q

buff.

NP’

FP-40

PLIM

buff.

NP

Q
NP

IOS

Q

buff.

FP-140– 140G
buff.

Q

Q

IOS

IOS

Q

IOS

4, 8 or 16 Linecard slots + 2 RP slots

BRKIPM-2012

Switch Fabric Cards
(all 8 active)

MSC-140 –
140G
buff.
NP’ Q
Q

IOS

Q

141G
141G rx
rx
225G
tx
225G tx

© 2010 Cisco Systems, Inc. All rights reserved.

IOS

NP’
NP’ Q

buff.

midplane 140G

100GE

midplane 140G

SPA

midplane 40G

SIP-800
SPA

IOS

midplane 40G

IOS

RP (standby)

14x 10GE

QFA
QFA (Quantum
(Quantum Flow
Flow Array)
Array)
-- 140
140 Gbps,
Gbps, 125
125 Mpps
Mpps NPU
NPU
-- one
one for
for RX,
RX, one
one for
for TX
TX
processing
processing

17


Cisco ASR9000

compact router/switch anatomy

Trident
Trident NPU
NPU

-- 15
15 Gbps,
Gbps, 14
14 Mpps
Mpps (2x)
(2x)
-- shared
for
RX
shared for RX && TX
TX processing
processing
-- more
independent
NPU’s
more independent NPU’s per
per card
card

IOS

Transport LC – 40G
buff. NP
buff. NP
buff. NP

RSP (active)
IOS

RSP
RSP (Route/Switch
(Route/Switch
Processor)
Processor)
• • CPU
CPU ++ Switch
Switch Fabric
Fabric
• • active/active
active/active SF
SF

Transport LC – 8x TGE OS
NP buff.

90G
90G
IOS

IOS

buff. NP

buff. NP
buff. NP
buff. NP
buff. NP

NP buff.
NP buff.

Transport LC – 80G
buff. NP
buff. NP

NP buff.

RSP (fab. active)
IOS

IOS

buff. NP
buff. NP

Transport LC – 16x TGE OS
NP buff.
NP buff.

IOS
IOS

NP buff.
NP buff.
NP buff.
NP buff.
NP buff.
NP buff.

4 or 8 Linecard slots

BRKIPM-2012

© 2010 Cisco Systems, Inc. All rights reserved.

18


Edge-facing Card
CRS-1

EMSE MSC40

$50K

ES+ 4TG

$39K

CRS-3

EMSE MSC140

$29K

7600

ASR9000

A9K-8T-B

$16K

Core-facing Card

Over-subscribed Card

FP40 + 4xTGE

$19K

ES+T 4TG

$16K

FP140 + 14xTGE

$18K

A9K-8T-L

$9.25K

FP40 + 8xTGE

$16K

FP140 + 20xTGE

$15K

-

A9K-8T/4-L

-

$5.75K

Watt per TGE (max.)
CRS-1

MSC40 + 4xTGE

125 W

7600

76-ES+4T

100 W

MSC/FP + 14xTGE

43 W

CRS-1
ASR9000
CRS-3
© 2010 Cisco and/or its affiliates. All rights reserved.

FP40 + 4xTGE

105 W

A9K-8T

78 W

Cisco Confidential

19


IPoDWDM

Presentation_ID

© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

20


OTN OEO SDH/SONET Solution
Router
Transponders
or DWDM I/F

Short Reach
Optics I/F

Cross
Connect
(XC)

Invest in High Capacity SDH/SONET/OTN
10 transponders needed
4-14 Short Reach optics
Every Lambda OEO
Addt’l transponder & SR for each λ
Expensive switch w/active electronics
Continue to Invest in XCs & Transponders
© 2010 Cisco and/or its affiliates. All rights reserved.

IPoDWDM Solution
Router

Tunable
DWDM I/F
ROADMs

Invest in IPoDWDM
0 transponders needed
2 Tunable DWDM interfaces in router
All pass-through traffic stays optical
ROADM full provisioned, no truck rolls
Expensive switch eliminated
Eliminate Unnecessary OEO XC & Transponders
Cisco Confidential

21


IP Layer Management

Metro
Network

Optical Layer Management

Transponders
converting short
reach to λ

Core
Router

Metro
Network

Electrical switching
– OEO conversions

P2P DWDM
Manual patching of
10G connections
© 2010 Cisco and/or its affiliates. All rights reserved.

Electrical XC

Cisco Confidential

22


Common Network Management and Control

Metro
Network

Integrated
transponders

Core
Router

Metro

Network

Photonic
switching –
no OEO
conversions

Mesh
ROADM
© 2010 Cisco and/or its affiliates. All rights reserved.

ROADM
Cisco Confidential

23


• Integration of core routers with optical transport platform
2 layer in one

reduced OPEX

Eliminates need O-E-O modules (transponders) in transport platform

Integration at control plane level (pre-FEC FRR) to improve network resiliency

• Increased rack space and power efficiency

• Possible integration with 3rd party transport equipment

• Improves OSNR, CD and PMD robustness through use of advanced

modulations for high speed channels
ODB, DPSK+ for 40 Gbps

CP-DQPSK planned for 100 Gbps
• Available for CRS, ASR 9000, 7600, and 12000

© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

24


Before

Router

ROADM

DWDM I/F

Router Transponder ROADM

Transponder
Integrated into Router

© 2010 Cisco and/or its affiliates. All rights reserved.

Cisco Confidential

25